github token - still facing rate limit

[lbesnard]

I've been a long time user of obtainium, but lately I've been hit by constant rate limit issues from Github.

I've created a few tokens as usual, as described here https://wiki.obtainium.imranr.dev/sources/#github , did some fine grained ones, and basic ones. They work fine as can be seen in the code below.

However, on obtainium, once added, i still get rate limits. I've tried a few tokens, and ran on my computer the code below, and my token never seem to be used and go above 0.

I'm wondering if there is maybe a bug where obtainium is not using that token or if I'm just plain stupid.

mytoken=....
curl -H "Authorization: token $mytoken" -X GET https://api.github.com/rate_limit

❯ curl -H "Authorization: token $mytoken" -X GET https://api.github.com/rate_limit | grep used
  % Total    % Received % Xferd  Average Speed  Time    Time    Time   Current
                                 Dload  Upload  Total   Spent   Left   Speed
100   1917 100   1917   0      0   4432      0                              0
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
      "used": 0,
    "used": 0,

[Harshul1484]

Your used: 0 result is the key clue. Calls to /rate_limit don't count against the limit, so if Obtainium had sent that token during the current hour, used would be above 0. It isn't sending it.

In the current source (github.dart, getTokenIfAny, and getSourceConfigValues in app_source.dart), three settings make Obtainium ignore the token in Settings:

1. An old per-app token. Any non-empty token saved on an individual app takes priority over the global one ("Legacy per-app github-creds tokens still work", per this commit). The per-app field is only shown when an app uses Override source, so a token saved by an older version can be hidden. If you added apps back when tokens were set per app, removing and re-adding one affected app is the quickest test.
2. The hubproxy setting. If "'sky22333/hubproxy' instance for GitHub requests" has any value, Obtainium drops the token on purpose and sends requests through the proxy.
3. Override source. If an app's source is overridden to a different host, only that app's own token field is used.

There's also a behaviour change since v1.6.14 (29 Aug): if GitHub rejects a token (Bad credentials or Resource not accessible by personal access token), Obtainium now retries the request without a token (added for #3211). So a stale per-app token no longer shows an error. It just drops you to the anonymous limit of 60 requests an hour, which fits "lately" in your post.

To see which case you're in, check App logs in Settings after a refresh:

• Lines like GitHub request for https://api.github.com/... rejected due to token access, retrying without token. mean some token is being sent and rejected. That can't be your global token, which works in curl, so look for a per-app one (1).
• No such lines, but still rate-limited, means no token is sent at all, which points to (2) or (3).

[lbesnard]

Thanks for the detailed answer.

There was nothing in the log message which pointed to (2) or (3). Turned out to be (3) for some apps.

When doing a refresh on the main obtainium page, an error message with rate limit would appear, but only listing a dozen app or so. I opened every single of these apps, clicked on edit, did a change and revert it just to trigger saving the edit, (so basically doing nothing as a used point of view). This seems to have fixed the rate limit for these apps.