Skip to content

PR: feat(config): per-model extra_body and thinking overrides - #7774

Open
TheGoddessInari wants to merge 1 commit into
esengine:main-v2from
TheGoddessInari:extra-body-per-model
Open

PR: feat(config): per-model extra_body and thinking overrides#7774
TheGoddessInari wants to merge 1 commit into
esengine:main-v2from
TheGoddessInari:extra-body-per-model

Conversation

@TheGoddessInari

@TheGoddessInari TheGoddessInari commented Aug 6, 2026

Copy link
Copy Markdown

Summary
Multi-model providers like NVIDIA NIM host models with incompatible reasoning wires under a single base URL. Today extra_body and thinking are provider-scoped: every model on the provider receives the same body parameters. This breaks providers where Model A needs chat_template_kwargs.thinking_mode while Model B needs chat_template_kwargs.enable_thinking and Model C needs neither.
Splitting a multi-model provider into N single-model entries works around the limitation but duplicates base URL, API key, and headers — configuration that is identical across models.
This adds two fields to ProviderModelOverride:

  • ExtraBody map[string]any — per-model extra top-level JSON request body fields, merged into (and overriding) the provider-level extra_body for that model only.
  • Thinking string — per-model thinking toggle ("enabled" / "disabled"), overriding the provider-level thinking field for that model only. The merge happens in applyModelOverride(), which already handles every other per-model override. Model-level keys win over provider-level keys; omitted keys inherit the provider value unchanged.
    Files changed
    File Change
    internal/config/config.go Add ExtraBody and Thinking fields to ProviderModelOverride; extend applyModelOverride() with merge + override logic internal/config/render.go Include new fields in renderModelOverride() and renderAnyMap() TOML output; update modelOverrideEmpty() internal/config/render_test.go Round-trip test with nested maps; legacy struct compatibility test; merge semantics test Config example (NVIDIA NIM)
    [[providers]]
    name = "nvidia"
    kind = "openai"
    base_url = "https://integrate.api.nvidia.com/v1"
    models = [
    "minimaxai/minimax-m3",
    "nvidia/nemotron-3-super-120b-a12b",
    "z-ai/glm-5.2",
    "google/gemma-4-31b-it",
    "deepseek-ai/deepseek-v4-flash",
    ]
    api_key_env = "NVIDIA_API_KEY"

[providers.model_overrides]
"minimaxai/minimax-m3" = {
context_window = 262000,
max_output_tokens = 16384,
extra_body = { chat_template_kwargs = { thinking_mode = "enabled" } },
}
"nvidia/nemotron-3-super-120b-a12b" = {
context_window = 1000000,
max_output_tokens = 32768,
extra_body = { chat_template_kwargs = { enable_thinking = true } },
}
"z-ai/glm-5.2" = {
context_window = 1000000,
max_output_tokens = 16384,
thinking = "enabled",
extra_body = { chat_template_kwargs = { enable_thinking = true, clear_thinking = false } },
}
Merge semantics
Model overrides are applied after the provider entry is fully constructed.
For extra_body: the provider-level map is shallow-cloned, then model-level
keys are written on top. A model override with extra_body = {} clears all
provider-level keys for that model. A model without extra_body in its override
inherits the provider value unchanged.
For thinking: a non-empty string replaces the provider-level value. An empty
or omitted string inherits the provider value.
Provider: thinking = "enabled", extra_body = { min_p = 0.01, stream = true }
Override: thinking = "disabled", extra_body = { chat_template_kwargs = { enable_thinking = true } }
Result: thinking = "disabled", extra_body = { min_p = 0.01, stream = true, chat_template_kwargs = { enable_thinking = true } }
What this does NOT do

  • No new provider kind or wire protocol.
  • No new vendor detection heuristics.
  • No changes to buildRequest() — the merged extra_body and resolved
    thinking flow through the existing OpenAI-compatible request serialization.
  • No impact on system prompt, tool schemas, or prefix construction.
    Backward compatibility
    BurntSushi/toml ignores unknown fields when decoding. Older Reasonix releases
    reading a config with the new fields will silently skip them — no parse error.
    The existing TestRenderTOMLModelOverrides legacy compatibility test
    (render_test.go:1127) already validates this pattern for context_window /
    max_output_tokens; the new fields follow the same convention.
    Cache-impact: low
    Only touches request body serialization for providers that use model overrides.
    The prefix (system prompt + tools) is unaffected. Guard: verify that a
    non-overridden model's marshalled request body is byte-identical before and
    after the change.
    Testing
  1. go test ./internal/config/ -run TestRenderModelOverrideExtraBodyAndThinking —
    round-trip TOML encode/decode with nested maps, plus legacy struct
    compatibility
  2. go test ./internal/config/ -run TestApplyModelOverrideMergesExtraBody —
    merge semantics: model wins over provider for conflicting keys, provider
    values inherit for missing keys, thinking override works
  3. go test ./... — full suite regression

Issues

Verification

Documentation impact

Documentation-impact: TODO

For changes to user-visible CLI, Desktop, configuration, provider, permission,
or tool behavior, use one of:

Documentation-impact: updated - added model_overrides section with per-model extra_body/thinking docs to CONFIG_PATHS.md and GUIDE.md
:

Cache impact

Cache-impact: none — changes only apply when a specific model is selected via model_overrides; provider request serialization for non-overridden models is byte-identical before and after.
Cache-guard: existing TestRenderTOMLModelOverrides round-trip test + new TestRenderModelOverrideExtraBodyAndThinking cover the changed surface.
System-prompt-review: N/A

For cache-sensitive changes, fill these lines before requesting review:

  • Cache-impact: none, low, medium, or high, plus the reason.
  • Cache-guard: the focused guard test/command added or run, or why an existing guard covers the change.
  • System-prompt-review: required reviewer/approval note when provider-visible system prompt, memory prefix, output style, or skill index behavior changes.

@github-actions github-actions Bot added v2 Go rewrite (1.x) — main-v2 branch, active development config Configuration & setup (internal/config) labels Aug 6, 2026
Summary
Multi-model providers like NVIDIA NIM host models with incompatible reasoning
wires under a single base URL. Today extra_body and thinking are
provider-scoped: every model on the provider receives the same body parameters.
This breaks providers where Model A needs chat_template_kwargs.thinking_mode
while Model B needs chat_template_kwargs.enable_thinking and Model C needs
neither.
Splitting a multi-model provider into N single-model entries works around the
limitation but duplicates base URL, API key, and headers — configuration that
is identical across models.
This adds two fields to ProviderModelOverride:
- ExtraBody map[string]any — per-model extra top-level JSON request body
fields, merged into (and overriding) the provider-level extra_body for that
model only.
- Thinking string — per-model thinking toggle ("enabled" / "disabled"),
overriding the provider-level thinking field for that model only.
The merge happens in applyModelOverride(), which already handles every other
per-model override. Model-level keys win over provider-level keys; omitted keys
inherit the provider value unchanged.
Files changed
File	Change
internal/config/config.go	Add ExtraBody and Thinking fields to ProviderModelOverride; extend applyModelOverride() with merge + override logic
internal/config/render.go	Include new fields in renderModelOverride() and renderAnyMap() TOML output; update modelOverrideEmpty()
internal/config/render_test.go	Round-trip test with nested maps; legacy struct compatibility test; merge semantics test
Config example (NVIDIA NIM)
[[providers]]
name        = "nvidia"
kind        = "openai"
base_url    = "https://integrate.api.nvidia.com/v1"
models      = [
  "minimaxai/minimax-m3",
  "nvidia/nemotron-3-super-120b-a12b",
  "z-ai/glm-5.2",
  "google/gemma-4-31b-it",
  "deepseek-ai/deepseek-v4-flash",
]
api_key_env = "NVIDIA_API_KEY"

[providers.model_overrides]
"minimaxai/minimax-m3" = {
  context_window     = 262000,
  max_output_tokens  = 16384,
  extra_body         = { chat_template_kwargs = { thinking_mode = "enabled" } },
}
"nvidia/nemotron-3-super-120b-a12b" = {
  context_window     = 1000000,
  max_output_tokens  = 32768,
  extra_body         = { chat_template_kwargs = { enable_thinking = true } },
}
"z-ai/glm-5.2" = {
  context_window     = 1000000,
  max_output_tokens  = 16384,
  thinking           = "enabled",
  extra_body         = { chat_template_kwargs = { enable_thinking = true, clear_thinking = false } },
}
Merge semantics
Model overrides are applied after the provider entry is fully constructed.
For extra_body: the provider-level map is shallow-cloned, then model-level
keys are written on top. A model override with extra_body = {} clears all
provider-level keys for that model. A model without extra_body in its override
inherits the provider value unchanged.
For thinking: a non-empty string replaces the provider-level value. An empty
or omitted string inherits the provider value.
Provider:  thinking = "enabled",  extra_body = { min_p = 0.01, stream = true }
Override:  thinking = "disabled", extra_body = { chat_template_kwargs = { enable_thinking = true } }
Result:    thinking = "disabled", extra_body = { min_p = 0.01, stream = true, chat_template_kwargs = { enable_thinking = true } }
What this does NOT do
- No new provider kind or wire protocol.
- No new vendor detection heuristics.
- No changes to buildRequest() — the merged extra_body and resolved
thinking flow through the existing OpenAI-compatible request serialization.
- No impact on system prompt, tool schemas, or prefix construction.
Backward compatibility
BurntSushi/toml ignores unknown fields when decoding. Older Reasonix releases
reading a config with the new fields will silently skip them — no parse error.
The existing TestRenderTOMLModelOverrides legacy compatibility test
(render_test.go:1127) already validates this pattern for context_window /
max_output_tokens; the new fields follow the same convention.
Cache-impact: low
Only touches request body serialization for providers that use model overrides.
The prefix (system prompt + tools) is unaffected. Guard: verify that a
non-overridden model's marshalled request body is byte-identical before and
after the change.
Testing
1. go test ./internal/config/ -run TestRenderModelOverrideExtraBodyAndThinking —
round-trip TOML encode/decode with nested maps, plus legacy struct
compatibility
2. go test ./internal/config/ -run TestApplyModelOverrideMergesExtraBody —
merge semantics: model wins over provider for conflicting keys, provider
values inherit for missing keys, thinking override works
3. go test ./... — full suite regression

Cache-impact: none — changes only apply when a specific model is selected via model_overrides; provider request serialization for non-overridden models is byte-identical before and after.
Cache-guard: existing TestRenderTOMLModelOverrides round-trip test + new TestRenderModelOverrideExtraBodyAndThinking cover the changed surface.

Documentation-impact: updated - added model_overrides section with per-model extra_body/thinking docs to CONFIG_PATHS.md and GUIDE.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config Configuration & setup (internal/config) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant