How to enable reasoning/thinking for custom OpenAI-compatible local models?

[fluxxus]

Hi,

I'm using Zero v0.9.0 on macOS arm64 with a local OpenAI-compatible server (mlx-serve).

Everything works correctly with the custom provider — model discovery, streaming, tools, file access, etc. — but I haven't found a way to enable reasoning/thinking for my local model.

My model is:

ddalcu/Qwen3.8-27B-MLX-Serve-6bit

Provider:

http://127.0.0.1:11234/v1

The model supports OpenAI-style reasoning effort. Other clients can send:

"reasoning_effort": "medium"

and mlx-serve then reports:

thinking=true
reasoning budget 8192: enforced in-stream

With Zero, however, requests arrive at mlx-serve with:

thinking=false

I briefly see effort: auto in the Zero TUI, but because this is a custom OpenAI-compatible model, there doesn't seem to be a way to select low, medium, high, etc.

Is there currently a supported way to enable reasoning for a custom/local OpenAI-compatible model?

Ideally I'd like to configure something like:

reasoning_effort: medium

without needing a proxy or adding the model to Zero's built-in model registry.

Thanks! Zero otherwise works very nicely with mlx-serve.

[weuderr]

I dug through the v0.9.0 source. Here's why you only see effort: auto, and two ways around it.

Why: the /effort picker and Ctrl+T get their levels from two places. First, the model list your provider returns (GET /v1/models, read in internal/providermodeldiscovery). If the model isn't there, Zero falls back to name-based inference (reasoningEffortsForModelName in internal/modelregistry/catalog.go), which only recognizes gpt-5*, codex, o-series, hy3/hunyuan and deepseek. A name like ddalcu/Qwen3.8-27B-MLX-Serve-6bit matches neither, so the list is empty and the TUI says the model "does not expose reasoning effort controls". When no effort is selected, reasoning_effort isn't sent, hence thinking=false on mlx-serve.

Option 1 (TUI): advertise the levels from /v1/models. Zero's discovery parser accepts these fields on each model item:

{
  "id": "ddalcu/Qwen3.8-27B-MLX-Serve-6bit",
  "object": "model",
  "reasoning_efforts": ["low", "medium", "high"],
  "default_reasoning_level": "medium"
}

("supported_reasoning_levels": [{"effort": "medium"}, ...] also works.) If mlx-serve's /v1/models includes that, /effort medium and Ctrl+T will work and Zero will send reasoning_effort on every request. Check what it returns today with curl -s http://127.0.0.1:11234/v1/models. Since mlx-serve already supports the parameter, it would be a small feature request to have it advertised there.

Option 2 (headless, works today): zero exec --reasoning-effort medium "...". In exec, when the model isn't in Zero's registry, forwardedReasoningEffort (internal/cli/exec.go) forwards the requested value as-is, without gating. Your custom model is exactly that case, so the request reaches mlx-serve with "reasoning_effort": "medium".

For a config-level setting (e.g. reasoning_efforts per custom model in the provider config), there's nothing like that in v0.9.0. It's worth a feature request here, since it would cover any local server that can't change its /v1/models output.