Skip to content

fix(router): persist and apply retry_policy via update_settings (#31308)#9

Merged
shivshekhar12 merged 1 commit into
litellm_internal_stagingfrom
fix/users/shiv/save-retry-policy
Jul 7, 2026
Merged

fix(router): persist and apply retry_policy via update_settings (#31308)#9
shivshekhar12 merged 1 commit into
litellm_internal_stagingfrom
fix/users/shiv/save-retry-policy

Conversation

@shivshekhar12

Copy link
Copy Markdown
Collaborator

Summary

retry_policy set in the Admin UI / config was silently dropped:

  • It was missing from UpdateRouterConfig, so Pydantic dropped it at parse time before the DB write.
  • It was missing from the Router.update_settings() allowlist, so a value loaded from the DB was never applied to the live router.

This PR:

  • Adds retry_policy: Optional[RetryPolicy] to UpdateRouterConfig (moved RetryPolicy above it so it can be referenced).
  • Adds retry_policy to the update_settings() allowlist.
  • Coerces the on-the-wire dict into a RetryPolicy instance before applying it to the live router.

Test plan

  • test_update_settings_retry_policyRouter.update_settings() accepts a retry_policy dict and coerces it into RetryPolicy.
  • DB-load path applies retry_policy to the live Router on startup.
  • POST /config/update persists retry_policy into the DB merged with existing values.
  • UpdateRouterConfig parses and serializes retry_policy correctly.

Fixes BerriAI#31308

Made with Cursor

@yicongkuang1 yicongkuang1 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTm

@shivshekhar12 shivshekhar12 merged commit a9f7286 into litellm_internal_staging Jul 7, 2026
6 checks passed
@shivshekhar12 shivshekhar12 deleted the fix/users/shiv/save-retry-policy branch July 7, 2026 18:48
shivshekhar12 added a commit that referenced this pull request Jul 7, 2026
…-422

Root cause: the Router Settings "Reliability & Retries" form rendered every field
as a plain text input, so the routing_groups List field was POSTed to
/config/update as the string "[]" instead of the array []. The backend validates
the whole router_settings body at once, so that one malformed field returned a 422
(routing_groups -> "Input should be a valid list") that rejected the entire payload,
dropping every other field (allowed_fails, cooldown_time, timeout, retry_after, ...).
The save handler also didn't await the request, so the 422 was swallowed and a
success toast always showed.

Changes (frontend only, router_settings/index.tsx):
- Parse List/Dictionary fields back into real JSON before sending, driven by the
  field_type metadata from /router/settings, so routing_groups goes out as [].
- await the save call: show success only on success; surface the real backend error
  via NotificationsManager.fromBackend on failure.

Tests:
- Backend unit (test_router_types.py): UpdateRouterConfig rejects "[]", accepts [],
  parses populated entries.
- Backend functional (test_config_update_routing_groups.py): ConfigYAML rejects the
  string; POST /config/update returns 422 for "[]" and 200 for [] with a real list
  persisted; full Reliability & Retries payload round-trips.
- UI functional (router_settings/index.test.tsx): payload sends routing_groups as an
  array; failed save surfaces the error and shows no success toast.
- Suites: targeted UI 51 passed; full UI 3837 passed / 381 files; backend 10 passed.

Stacked on #9 (backend retry_policy persistence)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Router retry_policy set via UI/ POST / Config/update is silently dropped (never persisted or applied)

2 participants