fix(sdk): pin CPU runs to an explicit empty device list - #1245
Closed
MAN$I VERMA (mansiverma897993) wants to merge 1 commit into
Closed
fix(sdk): pin CPU runs to an explicit empty device list#1245MAN$I VERMA (mansiverma897993) wants to merge 1 commit into
MAN$I VERMA (mansiverma897993) wants to merge 1 commit into
Conversation
A null llama_model_params::devices makes llama.cpp claim every registered GPU-type device, so a cpu-alias run still initializes the ggml-opencl backend. On Adreno GPUs with pre-3.0 OpenCL drivers (SD 888 / 8 Gen 1) that init aborts the process during kernel compilation, killing CPU inference too. Pass a list containing only the null terminator for CPU runs so llama.cpp never touches GPU backends the run cannot use. Fixes qualcomm#1235 Signed-off-by: mansiverma897993 <vmansi756@gmail.com>
Contributor
|
GenieX currently follows this device selection logic: it uses the NPU for inference by default. Unsupported layers will trigger fallback mechanisms. If the entire NPU is unsupported, an error will be thrown as expected, which is by design. However, users can manually specify the device. Thanks to MAN$I VERMA (@mansiverma897993) for the PR, yet it conflicts with the existing design, so it cannot be merged for the time being. |
Contributor
Author
|
Yes It totally Make sense |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1235
Root cause
On Qualcomm devices the crash reported in #1235 is not an OpenCL version requirement — it is llama.cpp's default device selection.
When the
cpualias is used,geniex_resolve_devicecorrectly producesdevice_id = NULL+n_gpu_layers = 0, but the llama_cpp plugin then leavesllama_model_params::devicesnull (resolve_devicesreturns an empty vector for a null id). A nulldeviceslist makes llama.cpp claim every registered GPU-type device (llama_prepare_model_devicesinsrc/llama.cpp), andllama_init_from_modelcreates a backend for each of them regardless ofn_gpu_layers.On Snapdragon 888 / 8 Gen 1 the Adreno 660/730 driver passes ggml-opencl's registration gate (OpenCL C 2.0 + fp16), so the device registers fine — but backend init then hard-aborts:
ggml_cl_inituses fatalCL_CHECKs andload_cl_kernelscallsbuild_program_from_source(..., fatal=true), whichexit(1)s when the older Adreno compiler rejects any of the ~200 kernels. The whole process dies even though the user asked for CPU.This also explains why non-Qualcomm chips are unaffected:
clGetPlatformIDsfinds no platform there, zero OpenCL devices register, and the CPU path never touches the backend.Fix
For CPU runs with no explicit device selection, pass a device list containing only the null terminator instead of leaving
devicesnull. llama.cpp then resolves zero offload devices and runs pure CPU without ever initializing the OpenCL (or Hexagon) backend.cpualias → explicit empty list; GPU backends never initialized (the crash in SDK doesn't run on older qualcomm chips #1235)GPUOpenCL/HTP0selections → unchangedhybrid(empty device_id, ngl ≠ 0) → unchanged, keeps llama.cpp's default HTP+CPU scheduling pathApplied to both
LlamaLlm::createandLlamaVlm::createvia the sharedresolve_deviceshelper.Testing
pytest tests -m api) is covered by CI; the device matrix runs on QDC via pr-check.geniex.h) or alias-table change, so no FFI/bindings update is required.