Context
Mellea's ModelOption sentinels (e.g. STOP_SEQUENCES, TEMPERATURE, MAX_NEW_TOKENS) are passed through to backends with no type or value validation. Today, an invalid value typically surfaces as a confusing backend-specific error (or silently misbehaves) far from the user's call site.
This came up concretely in #1112: STOP_SEQUENCES requires list[str] for uniform behavior across backends — a bare str works on OpenAI/LiteLLM but breaks HuggingFace's stop_strings. A per-option validate_stop_sequences helper was prototyped there but pulled out, since validating one option in isolation creates inconsistency with the rest.
What we want
We need a single, consistent strategy for validating ModelOption values.
Context
Mellea's
ModelOptionsentinels (e.g.STOP_SEQUENCES,TEMPERATURE,MAX_NEW_TOKENS) are passed through to backends with no type or value validation. Today, an invalid value typically surfaces as a confusing backend-specific error (or silently misbehaves) far from the user's call site.This came up concretely in #1112:
STOP_SEQUENCESrequireslist[str]for uniform behavior across backends — a barestrworks on OpenAI/LiteLLM but breaks HuggingFace'sstop_strings. A per-optionvalidate_stop_sequenceshelper was prototyped there but pulled out, since validating one option in isolation creates inconsistency with the rest.What we want
We need a single, consistent strategy for validating
ModelOptionvalues.