I understand that vLLM provides a metrics interface for cache hit rates. Could llmperf implement the collection of hit rate data during testing?
2026-05-19 12:18:31,301 - INFO - [Completed] Prefix dataset testing completed
2026-05-19 12:18:37,385 - INFO - ----------------------prefix cache metrics: engine 0----------------------
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] Prefix cache queried tokens: 44460
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] Prefix cache hit tokens: 14592
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] Prefix cache hit rate (hit tokens / queried tokens): 32.82%
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] External queried tokens: 29868
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] External hit tokens: 21888
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 0] External hit rate (hit tokens / queried tokens): 73.28%
2026-05-19 12:18:37,385 - INFO - ----------------------prefix cache metrics: engine 1----------------------
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] Prefix cache queried tokens: 14820
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] Prefix cache hit tokens: 0
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] Prefix cache hit rate (hit tokens / queried tokens): 0.00%
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] External queried tokens: 14820
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] External hit tokens: 7296
2026-05-19 12:18:37,385 - INFO - [prefix cache metrics: engine 1] External hit rate (hit tokens / queried tokens): 49.23%
2026-05-19 12:18:37,419 - INFO - Successfully appended aisbench.log content to aisbench_all.log
🚀 The feature, motivation and pitch
I understand that vLLM provides a metrics interface for cache hit rates. Could llmperf implement the collection of hit rate data during testing?
Reference implementation: https://github.com/rayn-zzz/aisbench_auto_tools_prefix/tree/main
Reference logs:
Alternatives
No response
Additional context
No response