PR: feat(config): per-model extra_body and thinking overrides - #7774
Open
TheGoddessInari wants to merge 1 commit into
Open
PR: feat(config): per-model extra_body and thinking overrides#7774TheGoddessInari wants to merge 1 commit into
TheGoddessInari wants to merge 1 commit into
Conversation
Summary
Multi-model providers like NVIDIA NIM host models with incompatible reasoning
wires under a single base URL. Today extra_body and thinking are
provider-scoped: every model on the provider receives the same body parameters.
This breaks providers where Model A needs chat_template_kwargs.thinking_mode
while Model B needs chat_template_kwargs.enable_thinking and Model C needs
neither.
Splitting a multi-model provider into N single-model entries works around the
limitation but duplicates base URL, API key, and headers — configuration that
is identical across models.
This adds two fields to ProviderModelOverride:
- ExtraBody map[string]any — per-model extra top-level JSON request body
fields, merged into (and overriding) the provider-level extra_body for that
model only.
- Thinking string — per-model thinking toggle ("enabled" / "disabled"),
overriding the provider-level thinking field for that model only.
The merge happens in applyModelOverride(), which already handles every other
per-model override. Model-level keys win over provider-level keys; omitted keys
inherit the provider value unchanged.
Files changed
File Change
internal/config/config.go Add ExtraBody and Thinking fields to ProviderModelOverride; extend applyModelOverride() with merge + override logic
internal/config/render.go Include new fields in renderModelOverride() and renderAnyMap() TOML output; update modelOverrideEmpty()
internal/config/render_test.go Round-trip test with nested maps; legacy struct compatibility test; merge semantics test
Config example (NVIDIA NIM)
[[providers]]
name = "nvidia"
kind = "openai"
base_url = "https://integrate.api.nvidia.com/v1"
models = [
"minimaxai/minimax-m3",
"nvidia/nemotron-3-super-120b-a12b",
"z-ai/glm-5.2",
"google/gemma-4-31b-it",
"deepseek-ai/deepseek-v4-flash",
]
api_key_env = "NVIDIA_API_KEY"
[providers.model_overrides]
"minimaxai/minimax-m3" = {
context_window = 262000,
max_output_tokens = 16384,
extra_body = { chat_template_kwargs = { thinking_mode = "enabled" } },
}
"nvidia/nemotron-3-super-120b-a12b" = {
context_window = 1000000,
max_output_tokens = 32768,
extra_body = { chat_template_kwargs = { enable_thinking = true } },
}
"z-ai/glm-5.2" = {
context_window = 1000000,
max_output_tokens = 16384,
thinking = "enabled",
extra_body = { chat_template_kwargs = { enable_thinking = true, clear_thinking = false } },
}
Merge semantics
Model overrides are applied after the provider entry is fully constructed.
For extra_body: the provider-level map is shallow-cloned, then model-level
keys are written on top. A model override with extra_body = {} clears all
provider-level keys for that model. A model without extra_body in its override
inherits the provider value unchanged.
For thinking: a non-empty string replaces the provider-level value. An empty
or omitted string inherits the provider value.
Provider: thinking = "enabled", extra_body = { min_p = 0.01, stream = true }
Override: thinking = "disabled", extra_body = { chat_template_kwargs = { enable_thinking = true } }
Result: thinking = "disabled", extra_body = { min_p = 0.01, stream = true, chat_template_kwargs = { enable_thinking = true } }
What this does NOT do
- No new provider kind or wire protocol.
- No new vendor detection heuristics.
- No changes to buildRequest() — the merged extra_body and resolved
thinking flow through the existing OpenAI-compatible request serialization.
- No impact on system prompt, tool schemas, or prefix construction.
Backward compatibility
BurntSushi/toml ignores unknown fields when decoding. Older Reasonix releases
reading a config with the new fields will silently skip them — no parse error.
The existing TestRenderTOMLModelOverrides legacy compatibility test
(render_test.go:1127) already validates this pattern for context_window /
max_output_tokens; the new fields follow the same convention.
Cache-impact: low
Only touches request body serialization for providers that use model overrides.
The prefix (system prompt + tools) is unaffected. Guard: verify that a
non-overridden model's marshalled request body is byte-identical before and
after the change.
Testing
1. go test ./internal/config/ -run TestRenderModelOverrideExtraBodyAndThinking —
round-trip TOML encode/decode with nested maps, plus legacy struct
compatibility
2. go test ./internal/config/ -run TestApplyModelOverrideMergesExtraBody —
merge semantics: model wins over provider for conflicting keys, provider
values inherit for missing keys, thinking override works
3. go test ./... — full suite regression
Cache-impact: none — changes only apply when a specific model is selected via model_overrides; provider request serialization for non-overridden models is byte-identical before and after.
Cache-guard: existing TestRenderTOMLModelOverrides round-trip test + new TestRenderModelOverrideExtraBodyAndThinking cover the changed surface.
Documentation-impact: updated - added model_overrides section with per-model extra_body/thinking docs to CONFIG_PATHS.md and GUIDE.md
TheGoddessInari
force-pushed
the
extra-body-per-model
branch
from
August 6, 2026 15:06
2617fd2 to
0c26e54
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Multi-model providers like NVIDIA NIM host models with incompatible reasoning wires under a single base URL. Today extra_body and thinking are provider-scoped: every model on the provider receives the same body parameters. This breaks providers where Model A needs chat_template_kwargs.thinking_mode while Model B needs chat_template_kwargs.enable_thinking and Model C needs neither.
Splitting a multi-model provider into N single-model entries works around the limitation but duplicates base URL, API key, and headers — configuration that is identical across models.
This adds two fields to ProviderModelOverride:
Files changed
File Change
internal/config/config.go Add ExtraBody and Thinking fields to ProviderModelOverride; extend applyModelOverride() with merge + override logic internal/config/render.go Include new fields in renderModelOverride() and renderAnyMap() TOML output; update modelOverrideEmpty() internal/config/render_test.go Round-trip test with nested maps; legacy struct compatibility test; merge semantics test Config example (NVIDIA NIM)
[[providers]]
name = "nvidia"
kind = "openai"
base_url = "https://integrate.api.nvidia.com/v1"
models = [
"minimaxai/minimax-m3",
"nvidia/nemotron-3-super-120b-a12b",
"z-ai/glm-5.2",
"google/gemma-4-31b-it",
"deepseek-ai/deepseek-v4-flash",
]
api_key_env = "NVIDIA_API_KEY"
[providers.model_overrides]
"minimaxai/minimax-m3" = {
context_window = 262000,
max_output_tokens = 16384,
extra_body = { chat_template_kwargs = { thinking_mode = "enabled" } },
}
"nvidia/nemotron-3-super-120b-a12b" = {
context_window = 1000000,
max_output_tokens = 32768,
extra_body = { chat_template_kwargs = { enable_thinking = true } },
}
"z-ai/glm-5.2" = {
context_window = 1000000,
max_output_tokens = 16384,
thinking = "enabled",
extra_body = { chat_template_kwargs = { enable_thinking = true, clear_thinking = false } },
}
Merge semantics
Model overrides are applied after the provider entry is fully constructed.
For extra_body: the provider-level map is shallow-cloned, then model-level
keys are written on top. A model override with extra_body = {} clears all
provider-level keys for that model. A model without extra_body in its override
inherits the provider value unchanged.
For thinking: a non-empty string replaces the provider-level value. An empty
or omitted string inherits the provider value.
Provider: thinking = "enabled", extra_body = { min_p = 0.01, stream = true }
Override: thinking = "disabled", extra_body = { chat_template_kwargs = { enable_thinking = true } }
Result: thinking = "disabled", extra_body = { min_p = 0.01, stream = true, chat_template_kwargs = { enable_thinking = true } }
What this does NOT do
thinking flow through the existing OpenAI-compatible request serialization.
Backward compatibility
BurntSushi/toml ignores unknown fields when decoding. Older Reasonix releases
reading a config with the new fields will silently skip them — no parse error.
The existing TestRenderTOMLModelOverrides legacy compatibility test
(render_test.go:1127) already validates this pattern for context_window /
max_output_tokens; the new fields follow the same convention.
Cache-impact: low
Only touches request body serialization for providers that use model overrides.
The prefix (system prompt + tools) is unaffected. Guard: verify that a
non-overridden model's marshalled request body is byte-identical before and
after the change.
Testing
round-trip TOML encode/decode with nested maps, plus legacy struct
compatibility
merge semantics: model wins over provider for conflicting keys, provider
values inherit for missing keys, thinking override works
Issues
Verification
Documentation impact
Documentation-impact: TODO
For changes to user-visible CLI, Desktop, configuration, provider, permission,
or tool behavior, use one of:
Documentation-impact: updated - added model_overrides section with per-model extra_body/thinking docs to CONFIG_PATHS.md and GUIDE.md
:
Cache impact
Cache-impact: none — changes only apply when a specific model is selected via model_overrides; provider request serialization for non-overridden models is byte-identical before and after.
Cache-guard: existing TestRenderTOMLModelOverrides round-trip test + new TestRenderModelOverrideExtraBodyAndThinking cover the changed surface.
System-prompt-review: N/A
For cache-sensitive changes, fill these lines before requesting review:
Cache-impact:none,low,medium, orhigh, plus the reason.Cache-guard: the focused guard test/command added or run, or why an existing guard covers the change.System-prompt-review: required reviewer/approval note when provider-visible system prompt, memory prefix, output style, or skill index behavior changes.