Skip to content

[Feature] Add Anthropic and Google Gemini Provider Support #90

Description

@anshul23102

Summary

TokenFirewall handles OpenAI costs only. Adding Anthropic and Gemini adapters makes it a universal cost layer for the three dominant model provider families.

Problem Statement

Teams using multiple LLM providers must maintain separate cost-tracking solutions per provider. A single TokenFirewall instance should be able to enforce budgets across all providers in use.

Proposed Solution

Define a ProviderAdapter interface:

interface ProviderAdapter {
  name: string;
  countTokens(text: string, model: string): Promise<number>;
  estimateCost(inputTokens: number, outputTokens: number, model: string): number;
  interceptRequest(req: Request): LLMRequestMetadata;
  interceptResponse(res: Response): LLMResponseMetadata;
}

Anthropic Adapter

  • Use @anthropic-ai/sdk client.beta.messages.countTokens() for pre-request counting.
  • Parse usage.input_tokens and usage.output_tokens from the response body.
  • Pricing table for Anthropic model tiers (opus, sonnet, haiku), configurable via env vars.

Gemini Adapter

  • Use @google/generative-ai model.countTokens() for estimation.
  • Parse usageMetadata.promptTokenCount and candidatesTokenCount from responses.
  • Support Gemini 1.5 Pro and Flash tiers.

Acceptance Criteria

  • provider: 'anthropic' routes token counting to the Anthropic adapter.
  • provider: 'gemini' routes to the Gemini adapter.
  • Cost estimates use the configured pricing table and can be overridden per model via environment variables.
  • Integration tests use recorded HTTP fixtures (no live API calls).
  • Unknown model names within a known provider fall back to the most expensive tier as a safe default.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions