Repository navigation
feat(ingress): map enable_thinking/reasoning_effort onto the thinking object - #437
Conversation
… object (#435) The OpenAI ingress translator silently dropped `enable_thinking`, `chat_template_kwargs.enable_thinking` and `reasoning_effort`, so the reasoning decision fell to the served dialect's policy and the client's ask never reached prepareRequest. Map them onto the house `thinking` object — the seam every dialect already consults: - enable_thinking:false -> {type:"disabled"} (the GLM 37->3 probe shape) - enable_thinking:true -> enabled, 2048 default budget - reasoning_effort -> inverse of openai-api-format's budget ladder (2000/8000/24000/32000), so the o1/o3 round trip is stable An enabled block is always legal (>=1024, < max_tokens): clamps against a client-sent max_tokens; against our own 4096 placeholder it RAISES the placeholder instead — clamping "high" to 4095 would invert the ask to "minimal" downstream. No room -> skip, dialect policy decides. Same fix closes the Qwen openai-wire passthrough, which mapped ANY originalRequest.thinking to enable_thinking:true — the exact inversion of an explicit disabled ask. Measured on the real lane (probe proxy -> hub /v1/messages -> sonnet nominal gc@glm-5.3, "Reponds exactement: ok"): no controls -> 36 completion tokens / 131 chars reasoning; enable_thinking:false -> 3 tokens, reasoning absent — the 2026-08-20 probe shape, end to end. Pins: translator mapping matrix (11), Qwen dialect interaction (4), new GLM dialect suite (4). 51/51 green across the four suites; tsc delta 0 (verified against the pre-change tree). Refs #435 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…elsewhere in the file) Refs #435 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…onerated (#438) #399 measured (07/10, hub + vLLM frognano-4b :5003) reasoning on every hub-served request while direct :5003 never thinks. Investigation (2026-10-11), all measured: - The paired inbound capture req-1-11725 (19:39:33Z) is MINIMAL — no thinking, no tools, no system. The outbound body built for it was captured through the real pipeline via an echo endpoint: byte-identical 198 bytes at 6f732eb (main on 07/10) and af52be5 (current) — model, messages, temperature, stream, stream_options, max_tokens. NO thinking field of any spelling. - frognano-4b matches neither qwen nor alibaba (matchesModelFamily = startsWith | "/fam"), so QwenModelDialect — whose passthrough could emit enable_thinking — never ran for it. The #437-passthrough hypothesis is refuted for this issue. - The flip no longer reproduces via the hub (11/10): the exact r11725 body AND a 26-tools consumer body (req-1-6396) both answer immediately with no reasoning. Same body both eras => the change was server-side (:5003 deployment, refs vllm#70/#71). Pins: the six neutral fields exactly; no thinking field even with an inbound enabled thinking block off the o1/o3 lane; o1 keeps the budget->reasoning_effort mapping (the only emission site). Refs #399 Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Suivi de la note non bloquante du merge — pourquoi le placeholder 32001 ne reçoit PAS de clamp supplémentaire (argument écrit, grain 3 deepqueue 11/10). La note : sans
Conclusion : aucun changement de code — le clamp correct vit déjà à la couche qui possède la connaissance du plafond, le cas sans plafond est indécidable à la couche ingress, et le résidu échoue haut et clair. Si un endpoint réel 400 sur cette forme, le geste est de déclarer |
What
The OpenAI ingress (
/v1/chat/completions) translator silently droppedenable_thinking,chat_template_kwargs.enable_thinkingandreasoning_effort— the reasoning decision fell to the served dialect's policy (CLAUDISH_GLM_THINKINGpassthrough default = GLM thinks;CLAUDISH_QWEN_THINKINGdisabled) and the client's ask never reachedprepareRequest. They now map onto the housethinkingobject, the seam every dialect already consults (originalRequest.thinking).enable_thinking: false{type: "disabled"}enable_thinking: true{type: "enabled", budget_tokens: 2048}reasoning_effort: minimal/low/medium/high{type: "enabled", budget_tokens: 2000/8000/24000/32000}enable_thinkingwins over thechat_template_kwargsspelling; unknown effort value → ignored (no partial mapping); no controls → unchanged behavior.openai-api-format.ts's budget→effort map, so the o1/o3 round trip is stable.budget_tokens ≥ 1024(native Anthropic floor) and< max_tokens(Qwen requirement). Against a client-sentmax_tokensthe budget clamps into it; against our own 4096 placeholder it raises the placeholder tobudget + 1— clamping "high" (32000) to 4095 would invert the ask to "minimal" downstream. No room → mapping skipped, dialect policy decides.Second defect found and fixed in passing: the Qwen openai-wire passthrough mapped ANY
originalRequest.thinkingtoenable_thinking: true— the exact inversion of an explicit disabled ask arriving from this new mapping.Acceptance criteria
AC1 — per-field pin at the translated
thinking: 11 pins inopenai-request-to-anthropic.test.ts(disabled ×2 spellings + precedence, enabled default, effort ladder, placeholder-raise, client-ceiling clamp, no-room skip, unknown-effort, no-controls, non-mutation) + 4 Qwen dialect interaction pins + newglm-model-dialect.test.tssuite (4 pins, with the 2026-08-20 probe of record in its header).AC2 — real-lane control, no budget spent beyond the GLM Coding plan. Probe chain: branch-build proxy (sandboxed config, port 3980) → customEndpoint
hubprobe(anthropic format) → hub/v1/messages→ sonnet nominalgc@glm-5.3— so the hub's GLM dialect emits the field on the openai wire. Prompt"Reponds exactement: ok",max_tokens: 200, 2026-10-11:enable_thinking: falseThe 37→3 shape of the 2026-08-20 probe of record, end to end through the ingress.
AC3 — doc lists the mapped request fields:
docs/reference/openai-ingress.mdgains a "#435 request-side thinking controls" section (table + budget rules + downstream fates per dialect + the measurement), and the request-converter table now lists the three fields.CLAUDE.mdingress section carries the decision-level line.Verification
bun teston the four suites (translator, qwen dialect, glm dialect, route-levelproxy-server-openai-notice.test.ts): 51/51 pass.-p packages/cli/tsconfig.json): 226 pre-change = 226 post-change (verified by stashing the working tree and re-running) — zero new errors; the two TS6133 in dialect files predate this change.Closes #435
🤖 Generated with Claude Code