Repository navigation
Description
Environment
- Hardware: Intel Panther Lake NPU5010
- Intel NPU driver: 1.32.1
- OpenVINO: 2026.2.1
- Model:
FluidInference/parakeet-tdt-0.6b-v3-ov - Host: Linux x86_64
Reproduction
EDDY_DEBUG=1 build/examples/cpp/parakeet_cli \
audio/test-single-16k.wav \
--model parakeet-v3 \
--device NPUOn current main, the first clean-install failure is the configured repository
FluidInference/parakeet-tdt-1.1b-v3-ov, which does not exist. The published
OpenVINO export is FluidInference/parakeet-tdt-0.6b-v3-ov. After correcting
that repository ID, model download succeeds and exposes the NPU compilation
failure below.
The encoder and decoder compile on NPU, but the joint component fails and Eddy falls back to CPU:
AlignDimensionsForDPU Pass failed : Got negative index -1 for Dim
Causes
PARAKEET_V3.repo_idpoints to the missing1.1b-v3-ovrepository instead
of the published0.6b-v3-ovOpenVINO export.- The downloaded
parakeet_joint.xmlcontains an opset5LogSoftmaxnamed
logitswithaxis=-1. Its input is static rank 4 with shape
[1, 1, 1, 8198], so OpenVINO semantics makeaxis=-1equivalent to
axis=3.
The Intel NPU compiler's AlignDimensionsForDPU pass rejects the negative axis spelling before graph compilation. Rewriting only this attribute to its canonical non-negative value allows the graph to compile and run on NPU.
Expected behavior
Eddy should download the published Parakeet V3 export and preserve valid OpenVINO semantics while avoiding this device-specific compiler limitation, so its encoder, decoder, and joint components can all run on NPU.
Proposed fix
Correct PARAKEET_V3.repo_id to the published 0.6b-v3-ov repository. Then, for NPU IR compilation only:
- Read the model in memory.
- Visit opset5
LogSoftmaxoperations. - When the axis is negative and the input rank is static, normalize it with
axis += rank. - Revalidate the model and compile the in-memory graph.
Compiled blobs and non-NPU compilation remain unchanged. Invalid or dynamic-rank cases remain untouched and retain the existing CPU fallback behavior.
Validation
On the system above:
- The joint graph compiled on NPU after normalizing
axis=-1toaxis=3. - The full 14.24-second sample produced the same 83-token transcript as CPU and the previous hybrid NPU/CPU path.
- Five speech slices plus a silence sample matched CPU token-for-token; silence remained empty.
- Ten randomized joint inputs selected the same token and duration heads on CPU and NPU.
- Median end-to-end model inference improved from 196 ms with CPU joint fallback to 116 ms with all components on NPU, approximately 41% lower latency.
Activity