Skip to content

fix(sdk): pin CPU runs to an explicit empty device list - #1245

Closed
MAN$I VERMA (mansiverma897993) wants to merge 1 commit into
qualcomm:mainfrom
mansiverma897993:fix/cpu-run-opencl-crash
Closed

fix(sdk): pin CPU runs to an explicit empty device list#1245
MAN$I VERMA (mansiverma897993) wants to merge 1 commit into
qualcomm:mainfrom
mansiverma897993:fix/cpu-run-opencl-crash

Conversation

@mansiverma897993

@mansiverma897993 MAN$I VERMA (mansiverma897993) commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #1235

Root cause

On Qualcomm devices the crash reported in #1235 is not an OpenCL version requirement — it is llama.cpp's default device selection.

When the cpu alias is used, geniex_resolve_device correctly produces device_id = NULL + n_gpu_layers = 0, but the llama_cpp plugin then leaves llama_model_params::devices null (resolve_devices returns an empty vector for a null id). A null devices list makes llama.cpp claim every registered GPU-type device (llama_prepare_model_devices in src/llama.cpp), and llama_init_from_model creates a backend for each of them regardless of n_gpu_layers.

On Snapdragon 888 / 8 Gen 1 the Adreno 660/730 driver passes ggml-opencl's registration gate (OpenCL C 2.0 + fp16), so the device registers fine — but backend init then hard-aborts: ggml_cl_init uses fatal CL_CHECKs and load_cl_kernels calls build_program_from_source(..., fatal=true), which exit(1)s when the older Adreno compiler rejects any of the ~200 kernels. The whole process dies even though the user asked for CPU.

This also explains why non-Qualcomm chips are unaffected: clGetPlatformIDs finds no platform there, zero OpenCL devices register, and the CPU path never touches the backend.

Fix

For CPU runs with no explicit device selection, pass a device list containing only the null terminator instead of leaving devices null. llama.cpp then resolves zero offload devices and runs pure CPU without ever initializing the OpenCL (or Hexagon) backend.

  • cpu alias → explicit empty list; GPU backends never initialized (the crash in SDK doesn't run on older qualcomm chips #1235)
  • explicit GPUOpenCL / HTP0 selections → unchanged
  • hybrid (empty device_id, ngl ≠ 0) → unchanged, keeps llama.cpp's default HTP+CPU scheduling path
  • third-party/llama.cpp untouched

Applied to both LlamaLlm::create and LlamaVlm::create via the shared resolve_devices helper.

Testing

  • Model-free pytest shard (pytest tests -m api) is covered by CI; the device matrix runs on QDC via pr-check.
  • No public header (geniex.h) or alias-table change, so no FFI/bindings update is required.

A null llama_model_params::devices makes llama.cpp claim every registered
GPU-type device, so a cpu-alias run still initializes the ggml-opencl
backend. On Adreno GPUs with pre-3.0 OpenCL drivers (SD 888 / 8 Gen 1)
that init aborts the process during kernel compilation, killing CPU
inference too. Pass a list containing only the null terminator for CPU
runs so llama.cpp never touches GPU backends the run cannot use.

Fixes qualcomm#1235

Signed-off-by: mansiverma897993 <vmansi756@gmail.com>
@mengshengwu

Copy link
Copy Markdown
Contributor

GenieX currently follows this device selection logic: it uses the NPU for inference by default. Unsupported layers will trigger fallback mechanisms. If the entire NPU is unsupported, an error will be thrown as expected, which is by design. However, users can manually specify the device. Thanks to MAN$I VERMA (@mansiverma897993) for the PR, yet it conflicts with the existing design, so it cannot be merged for the time being.

@mansiverma897993

Copy link
Copy Markdown
Contributor Author

Yes It totally Make sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SDK doesn't run on older qualcomm chips

2 participants