Describe the bug
/v1/models endpoint returns model ID with precision suffix (e.g. qualcomm/Qwen3-4B-Instruct-2507:W4A16), but using this exact ID in /v1/chat/completions fails with SDKError(Invalid input parameters or handle), quantization 'W4A16' not found for model 'qualcomm/Qwen3-4B-Instruct-2507'. The request only succeeds when the suffix is omitted. This breaks any OpenAI-compatible client that auto-populates the model ID from /v1/models (confirmed with AnythingLLM).
To Reproduce
- Pull a QAIRT model:
geniex pull ai-hub-models/Qwen3-4B-Instruct-2507
- Start server:
geniex serve --host 127.0.0.1:18181 --compute npu
- Query
/v1/models:
Invoke-RestMethod -Uri "http://127.0.0.1:18181/v1/models" -Method Get
Returns qualcomm/Qwen3-4B-Instruct-2507:W4A16
- Use the returned ID in a chat request:
Invoke-RestMethod -Uri "http://127.0.0.1:18181/v1/chat/completions" -Method Post -ContentType "application/json" -Body '{"model": "qualcomm/Qwen3-4B-Instruct-2507:W4A16", "messages": [{"role": "user", "content": "Say hello"}]}'
- See error:
SDKError(Invalid input parameters or handle), quantization 'W4A16' not found
Expected behavior
/v1/models should return a model ID that works directly with /v1/chat/completions. Either the precision suffix should be stripped from the models list, or the completions endpoint should accept it.
Workaround
Clear the Precision field in geniex.json:
$json = Get-Content "C:\LocalAI\geniex\models\models\qualcomm\Qwen3-4B-Instruct-2507\geniex.json" | ConvertFrom-Json
$json.Precision = ""
$json | ConvertTo-Json -Depth 10 | Set-Content "C:\LocalAI\geniex\models\models\qualcomm\Qwen3-4B-Instruct-2507\geniex.json"
Desktop:
- OS: Windows 11 ARM64
- Device: Snapdragon X Plus X1P42100
- GenieX CLI Version: v0.3.17
- QAIRT Runtime Version: v2.45.0.260326
- LlamaCPP Runtime Hash: ae9291e
- Model:
ai-hub-models/Qwen3-4B-Instruct-2507 (W4A16, QAIRT runtime)
- AnythingLLM Version: v1.15.0 (used for OpenAI-compatible client testing)
Describe the bug
/v1/modelsendpoint returns model ID with precision suffix (e.g.qualcomm/Qwen3-4B-Instruct-2507:W4A16), but using this exact ID in/v1/chat/completionsfails withSDKError(Invalid input parameters or handle), quantization 'W4A16' not found for model 'qualcomm/Qwen3-4B-Instruct-2507'. The request only succeeds when the suffix is omitted. This breaks any OpenAI-compatible client that auto-populates the model ID from/v1/models(confirmed with AnythingLLM).To Reproduce
geniex pull ai-hub-models/Qwen3-4B-Instruct-2507geniex serve --host 127.0.0.1:18181 --compute npu/v1/models:Returns
qualcomm/Qwen3-4B-Instruct-2507:W4A16SDKError(Invalid input parameters or handle), quantization 'W4A16' not foundExpected behavior
/v1/modelsshould return a model ID that works directly with/v1/chat/completions. Either the precision suffix should be stripped from the models list, or the completions endpoint should accept it.Workaround
Clear the
Precisionfield ingeniex.json:Desktop:
ai-hub-models/Qwen3-4B-Instruct-2507(W4A16, QAIRT runtime)