fix(llm): cap combined retry budget across transient and schema failures#762
fix(llm): cap combined retry budget across transient and schema failures#762bh462007 wants to merge 1 commit into
Conversation
|
@bh462007 is attempting to deploy a commit to the codersogs-3057's projects Team on Vercel. A member of the Team first needs to authorize it. |
pavsoss
left a comment
There was a problem hiding this comment.
Hey @bh462007 the fix looks solid, but this branch contains unmerged commit from #761 , please rebase the branch for this PR on top of the main branch to isolate the LLM router change
Also it's recommended to first fetch changes from the upstream and then origin a new branch (for the new issues) on upstream/main
|
this branch have conflict with the main, please fix it |
ee33b4a to
e0819e1
Compare
|
Rebased my branch onto I also added the |
Fixes #735
What was wrong
llmCall()maintained two separate retry counters for each provider:retryCountfor transient failures (maxMAX_RETRIES = 2)schemaRetryCountfor schema/JSON failures (max1)Although the loop checked both limits, each error path only considered its own counter before retrying. If a provider alternated between transient and schema failures, it could make 4 total attempts before falling back, exceeding the intended overall retry budget.
What changed
Added a
totalAttemptscounter with a limit ofMAX_TOTAL_ATTEMPTS = MAX_RETRIES + 1(3). Each retry now checks both its individual limit and the overall attempt limit.This only affects mixed failure sequences. Existing behavior remains unchanged for:
I intentionally kept the existing retry counters instead of replacing them with a single shared counter, since that would have changed the schema-only retry behavior and broken an existing test.
Testing
router.test.tscontinue to pass without changes, confirming no regressions for transient-only, schema-only, or timeout scenarios.npx tsc --noEmitwith no new TypeScript errors.Note: I couldn't run the full test suite through the normal
npm testpath becausetests/setup.tsis configured as a globalsetupFilesentry invitest.config.tsand imports@testing-library/jest-dom/vitest, but the required Testing Library packages aren't listed inpackage.json. This prevents every test in the repository from running. To verify this change, I temporarily commented out that single config entry, ran the tests, then restored it before committing (git diffwas clean afterward). I'm happy to open a separate issue for the test setup if that's helpful.Scope
src/lib/llm/router.tssrc/lib/llm/router.test.ts