fix(rerank): back off between retries instead of spinning

[gloryfromca]

Summary

All three rerank providers retried 429 / 5xx with a bare continue, so the whole
retry budget was spent within milliseconds of the first rejection. Against a
per-minute quota — which is what hosted rerank endpoints enforce — that is a
guaranteed failure: three attempts burn instantly and the caller gives up while
the window it needed to wait out has barely started.

Surfaced while driving the LoCoMo agentic suite against a hosted rerank endpoint.
Search requests failed with RerankServiceError even though the endpoint was
healthy and merely pacing us.

Adds _errors.backoff_sleep() — exponential, full jitter, capped at 8s — and
wires it into the vLLM, DeepInfra and DashScope retry loops. Jitter matters here
because a batch of concurrent searches trips the limit together and would
otherwise re-collide on every retry.

No behaviour change on a healthy endpoint: the sleep runs only on a retryable
failure that is actually going to be retried. A 4xx that is not 429 still raises
immediately.

Area

• Architecture method
• Benchmark
• Use case
• Documentation
• Developer experience
• CI, build, or release

Verification

uv run pytest tests/unit/test_component/test_rerank -q   -> 52 passed
uv run pytest tests/unit -q                              -> 2144 passed
make lint                                                -> clean, 4 contracts kept

New regression test test_429_retry_waits_between_attempts patches
asyncio.sleep and asserts one wait per failed attempt — it fails if the
backoff is removed.

Field evidence: with the old code an agentic LoCoMo pass logged 84 provider-side
failures on the query path; with backoff plus a larger retry budget the same pass
completed with zero.

Checklist

• I kept the change scoped to the relevant area.
• I am opening this from a separate branch, not pushing directly to main.
• I updated docs, examples, or setup notes when behavior changed.
• I added or updated tests when the change affects behavior.
• I did not commit secrets, .env files, dependency folders, or generated output.
• Active relative links in Markdown files resolve.

No doc update: this is internal retry behaviour with no configuration surface.
max_retries already existed and is unchanged.

Notes for Reviewers

Worth a look at the constants (_BACKOFF_BASE_SECONDS = 0.5, cap 8s). With the
default max_retries = 3 the worst case adds ~3.5s before giving up, averaging
~1.75s with jitter. If that is too patient for a latency-sensitive deployment,
the base is the knob to turn.

DashScope raises RerankError rather than going through upstream_http_error,
so its retry arm is shaped slightly differently from the other two — the backoff
call sits in the same place regardless.

🤖 Generated with Claude Code