Repository navigation
fix: return hybrid cache block counts on the runtime branch - #73
hellolazy1024 wants to merge 2 commits into
Conversation
Co-authored-by: OpenAI Codex
Co-authored-by: OpenAI Codex
|
2026-11-08 maintainer disposition: closing this draft without merging. It intentionally targets the retired runtime snapshot |
A real Qwen3.5-35B-A3B TP=2 Ascend startup reached serving, but its first diagnostic request crashed in KVCacheManager.cache_blocks:
num_cached_blocks > 0received None. On this runtime branch, the base coordinator returns an integer while HybridKVCacheCoordinator discards manager return values.Return the maximum manager count, matching this branch's base coordinator contract, while preserving aligned token limits, EAGLE lookahead clamping, and retention arguments. This targets the deployed runtime branch at 8ac6adf, not main: current main has a different cache/metrics contract.
Validation on the Ascend container using its isolated Python environment:
.venv/bin/python test_hybrid_cache_block_count.py: original source 3 failures; patched source 3 passed.Duplicate check: open PR searches for
cache_blocks return,hybrid cached blocks, andprefix cache crashfound no matching fix; the scheduling admission PR #8 is unrelated.AI assistance was used to diagnose, implement, and test this change. This is a draft for human review; no human line-by-line review or full repository test run is claimed. Private task prompts, patches, credentials, and raw experiment data are not included.