Add production-ready SmartRouter for JS SDK deployments

[yanruiyang517]

/claim #286

Closes #286

Summary

• Add SmartRouter deployments for OpenAI, Google, and Cohere.
• Select the available deployment with the lowest tracked token usage under its configured limit.
• Support retries, handler/network timeouts, 429 failover, cross-request rate-limit cooldowns, and Retry-After.
• Support actual Axios stream responses and validate custom-handler async iterables.
• Normalize provider response and usage shapes, including correct Cohere input + output token accounting.
• Sanitize OpenAI request bodies so the router-only prompt field is never forwarded to the provider API.
• Support both plain-object and AxiosHeaders-style Retry-After access.
• Preserve valid zero-valued OpenAI options in the compatibility facade.
• Isolate Sentry/PostHog callbacks so observability failures cannot break routing.
• Validate duplicate IDs, non-finite/negative limits and usage, retries, and invalid timeouts.
• Add config-driven construction suitable for jsonnet-rendered configuration.
• Map Google multi-message roles and automatically select the streaming endpoint.
• Keep the existing OpenAI public class as a compatibility facade while routing its chat paths through SmartRouter.
• Document direct SmartRouter migration for new examples and applications.

Provider and reliability coverage

• OpenAI, Google, and Cohere endpoint/header/payload shapes
• OpenAI internal-field sanitization
• OpenAI compatibility facade request-body sanitization and zero-value preservation
• OpenAI/Google/Cohere response and usage normalization
• Correct Cohere total-token tracking
• Least-usage routing and token-limit exclusion
• Non-429 retry exhaustion
• 429 failover, persistent cooldown, and AxiosHeaders Retry-After
• Custom-handler timeout and fallback
• Streaming pass-through and invalid-stream rejection
• Independent callback delivery and callback failure isolation
• Google multi-message mapping
• Config construction and invalid config rejection

Compatibility decision

The issue text refers to endpoint classes from an older repository layout. In the current JS SDK, removing the exported OpenAI class and its embeddings/function/schema helpers would be an unrelated breaking API change. This PR therefore makes SmartRouter the recommended path for all new chat/stream code while retaining the legacy class as a compatibility facade. The limitation and migration path are documented in lib/router/README.md.

Validation

Clean repository validation completed before the final provider-correctness refresh:

• 4 Jest suites passed
• 28 tests passed, 0 failed
• TypeScript project build passed
• Prettier check passed

Final provider-correctness refresh:

• Added providerPayloadRegression.test.ts with 4 focused tests.
• Updated the existing Cohere expectation and made the OpenAI payload assertion exact.
• Independent runtime harness passed 7 assertions for OpenAI payload sanitization, Cohere token accounting, and both supported Retry-After header forms.
• TypeScript transpile/syntax checks passed for the newly changed code paths.
• Temporary validation files were removed; the PR contains only product code, documentation, fixtures, examples, and tests.

Upstream CI remains the authoritative full-suite result for the final head.

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[yanruiyang517]

I have read the Arakoo CLA Document and I hereby sign the CLA

[yanruiyang517]

recheck

[yanruiyang517]

All checks are passing now. Please let me know if any changes are needed.

[yanruiyang517]

Rechecking the PR status: it is still open and mergeable, with the CLA and JS build checks passing.

The change keeps the existing OpenAI public API while routing chat/stream calls through SmartRouter, and the PR description lists the focused SmartRouter, endpoint, streaming, type build, Prettier, and diff checks that were run. Ready for review when maintainers have time.

[yanruiyang517]

Completed the SmartRouter reliability refresh and revalidated the focused JS SDK paths.

Final validation:

• 4 test suites passed
• 28 tests passed, 0 failed
• TypeScript project build passed
• Prettier check passed
• PR remains mergeable

The refresh adds persistent 429 cooldowns with Retry-After, handler/network timeout failover, streaming validation, callback isolation, config validation, Google multi-message mapping, and updated migration documentation. Ready for maintainer review.

[yanruiyang517]

Final recheck completed. I found and fixed three provider-correctness gaps that were not covered by the earlier validation:

• OpenAI request payloads no longer forward the router-only prompt field.
• Cohere total_tokens now includes both input and output tokens.
• Retry-After now supports AxiosHeaders-style .get() access as well as plain header objects.

I also updated the OpenAI compatibility facade to preserve valid zero-valued options and added four focused regression tests. The existing Cohere expectation was corrected and the OpenAI payload assertion is now exact.

Verification for the final refresh:

• independent runtime harness: 7 assertions passed
• TypeScript transpile/syntax checks for the changed paths: passed
• no temporary validation workflow or result files remain
• PR is open and mergeable

The PR description now accurately separates the previous clean 28-test run from the final provider-correctness regression coverage. Ready for maintainer review; upstream CI should be treated as authoritative for the final head.