Which format to support more LLM provider option is better?

[balaboom123]

Question

Which format is the best for further development?

Current Situation

PR #30 implements a simple priority-based approach where OpenRouter is checked first, and if its API key exists, it's used regardless of whether OpenAI's API key is also present.

Option 1: First Available API Key (Current PR #30)

This approach checks for API keys in a predetermined priority order:

if os.getenv("OPENROUTER_API_KEY"):
    llm_instance = ChatOpenRouter(model="openai/gpt-4o")
elif os.getenv("OPENAI_API_KEY"):
    llm_instance = ChatOpenAI(model="gpt-4o")

In this implementation, OpenRouter is always tried first. If both API keys are present in the environment, OpenRouter will always be selected because of the if-else structure.

Option 2: CLI Parameter Approach

This implementation adds a command-line parameter to explicitly select the LLM provider:

# In your CLI parameters section
api_provider = input('Enter your api provider: ')

# In your initialization code
if api_provider == "openrouter" and os.getenv("OPENROUTER_API_KEY"):
    llm_instance = ChatOpenRouter(model="openai/gpt-4o")
elif api_provider == "openai" and os.getenv("OPENAI_API_KEY"):
    llm_instance = ChatOpenAI(model="gpt-4o")

This gives users the ability to specify which provider they want to use at runtime through CLI commands like --api-provider openai.

Option 3: Environment Variable Configuration

This approach adds a new environment variable to select the provider:

# Read from environment variable
api_provider = os.getenv("API_PROVIDER", "auto").lower()

# Then use the same logic as Option 2
if api_provider == "openrouter" and os.getenv("OPENROUTER_API_KEY"):
    llm_instance = ChatOpenRouter(model="openai/gpt-4o")
elif api_provider == "openai" and os.getenv("OPENAI_API_KEY"):
    llm_instance = ChatOpenAI(model="gpt-4o")

For the .env.example:

# Optional: specify preferred API provider (openai, openrouter, auto)
API_PROVIDER=auto

This allows users to set their preference once in their .env file with a setting like API_PROVIDER=openai.

conclusion

I will be really appreciated if you can provide the suggestions about which format is the best for further development?

[sauravpanda]

@gregpr07 I would say using something like litellm would be great, as it will make it easy to switch LLMs and we don't have to internally build the translation mapping for all different LLMs and their responses: https://litellm.vercel.app/docs/completion/input

[balaboom123]

Thank you for your suggestions!