Submitting API Keys is a security vulnerability.
Then perhaps we want to replace the user-submitted Gemini token with just our token. Gemini I believe also allows OAuth, it's kind of like singing in with google where we just need to add another scope for Gemini access, and we can use their free plan when we're making gemini calls. But I worry that will be too much work, we can simply modify the deployment so that for now we provide the Gemini API key which won't be that much since it's not a main use case anyways.
Also we might want to use batch API instead of regular completion API for our Gemini API usage, since that is 50% less cost, should be just 3-4 lines of change when we made those calls in evaluators.
Submitting API Keys is a security vulnerability.
Then perhaps we want to replace the user-submitted Gemini token with just our token. Gemini I believe also allows OAuth, it's kind of like singing in with google where we just need to add another scope for Gemini access, and we can use their free plan when we're making gemini calls. But I worry that will be too much work, we can simply modify the deployment so that for now we provide the Gemini API key which won't be that much since it's not a main use case anyways.
Also we might want to use batch API instead of regular completion API for our Gemini API usage, since that is 50% less cost, should be just 3-4 lines of change when we made those calls in evaluators.