Describe the bug
When using geniex serve with streaming enabled ("stream": true), all chunks in the streaming response have "finish_reason": "" (empty string) including the final chunk. According to the OpenAI streaming specification, finish_reason should be null on intermediate chunks and "stop" on the final chunk. This causes OpenAI-compatible clients such as AnythingLLM to never detect that generation has finished, requiring the user to manually stop generation.
Non-streaming responses ("stream": false) correctly return "finish_reason": "stop".
To Reproduce
- Start server:
geniex serve --host 127.0.0.1:18181 --compute npu
- Send a streaming request:
Invoke-WebRequest -Uri "http://127.0.0.1:18181/v1/chat/completions" -Method Post -ContentType "application/json" -UseBasicParsing -Body '{"model": "qualcomm/Qwen3-4B-Instruct-2507", "messages": [{"role": "user", "content": "Say hello"}], "stream": true}' | Select-Object -ExpandProperty Content
- Observe all chunks have
"finish_reason": "" including the final chunk before data: [DONE]
Expected behavior
Per the OpenAI streaming specification:
- Intermediate chunks:
"finish_reason": null
- Final chunk:
"finish_reason": "stop" with empty "delta": {}
GenieX currently sends "finish_reason": "" (empty string) on all chunks including the final one, which is non-compliant and causes clients to never detect completion.
Workaround
Manually stop generation in the UI once the response appears complete. In AnythingLLM, click the stop button after the model finishes generating text — the response is complete but the client keeps waiting due to the missing finish_reason: stop signal.
Desktop:
- OS: Windows 11 ARM64
- Device: Snapdragon X Plus X1P42100
- GenieX CLI Version: v0.3.17
- QAIRT Runtime Version: v2.45.0.260326
- LlamaCPP Runtime Hash: ae9291e
- Model:
ai-hub-models/Qwen3-4B-Instruct-2507 (W4A16, QAIRT runtime)
- AnythingLLM Version: v1.15.0 (used for OpenAI-compatible client testing)
Describe the bug
When using
geniex servewith streaming enabled ("stream": true), all chunks in the streaming response have"finish_reason": ""(empty string) including the final chunk. According to the OpenAI streaming specification,finish_reasonshould benullon intermediate chunks and"stop"on the final chunk. This causes OpenAI-compatible clients such as AnythingLLM to never detect that generation has finished, requiring the user to manually stop generation.Non-streaming responses (
"stream": false) correctly return"finish_reason": "stop".To Reproduce
geniex serve --host 127.0.0.1:18181 --compute npu"finish_reason": ""including the final chunk beforedata: [DONE]Expected behavior
Per the OpenAI streaming specification:
"finish_reason": null"finish_reason": "stop"with empty"delta": {}GenieX currently sends
"finish_reason": ""(empty string) on all chunks including the final one, which is non-compliant and causes clients to never detect completion.Workaround
Manually stop generation in the UI once the response appears complete. In AnythingLLM, click the stop button after the model finishes generating text — the response is complete but the client keeps waiting due to the missing
finish_reason: stopsignal.Desktop:
ai-hub-models/Qwen3-4B-Instruct-2507(W4A16, QAIRT runtime)