Skip to content

feat(codemode): show eval tool-call throughput - #916

Merged
code-yeongyu merged 3 commits into
mainfrom
feat/eval-throughput-badge
Aug 17, 2026
Merged

feat(codemode): show eval tool-call throughput#916
code-yeongyu merged 3 commits into
mainfrom
feat/eval-throughput-badge

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • show exact nested tool-call count and calls/second in completed eval TUI headers
  • use true wall-clock elapsed time for both the visible duration and throughput denominator
  • preserve kernel-reported duration and the existing senpi.eval.execution event contract
  • cover zero-duration, partial/error, multi-cell, legacy, and 40/80/120-column cases
  • add hermetic real-source PTY/xterm/Chrome QA

Test evidence

  • RED: renderer assertions saw ... 1.00 calls/s · 1s ... when wall time was 2s; runtime details lacked toolCallCount
  • GREEN: focused renderer suites 58/58; runtime metadata suites 8/8
  • Full @code-yeongyu/senpi-codemode suite: 565 passed, 6 skipped
  • npm run check: passed
  • npm run build: passed

Manual QA

Command:

node .agents/skills/senpi-qa/scripts/scenarios/eval-throughput-badge-qa.mjs --self-test --evidence eval-throughput-badge

Result: 9/9 passed.

Observed real TUI header:

╭─ eval js done ✓ · 2 calls · 25.32 calls/s · <1s

Evidence includes raw PTY bytes, xterm.js grid JSON/HTML, Chrome PNG, sanitized fake-provider requests, and cleanup/auth receipts under local-ignore/qa-evidence/20260817-eval-throughput-badge/.

Dual visual QA verdicts: PASS / PASS.


Summary by cubic

Shows eval tool-call throughput in the TUI header using wall-clock time. Previously the header showed only kernel duration; now it displays “N calls · X.XX calls/s · elapsed”, making throughput visible while preserving the senpi.eval.execution event contract.

  • Rendering: Badge appears only for final, single-cell, completed evals; hidden for partial/running/error and multi-cell results. Pluralization is correct; zero elapsed shows “0.00 calls/s”; positive calls without positive wall time show “n/a calls/s”. The final metadata line now uses wall time for “took …” and also includes calls and calls/s; timeout/reset badges remain after elapsed.
  • Data model: EvalToolDetails adds wallDurationMs and toolCallCount (additive). Kernel durationMs is unchanged; the renderer prefers wallDurationMs for visible elapsed and throughput.
  • Docs and tests: packages/senpi-codemode/CHANGELOG.md and packages/senpi-codemode/changes.md document behavior and expected conflict zones. Tests updated; added hermetic PTY/xterm/Chrome QA scenario at .agents/skills/senpi-qa/scripts/scenarios/eval-throughput-badge-qa.mjs. No migration required.

Written for commit 3908a8c. Summary will update on new commits.

Review in cubic

@code-yeongyu
code-yeongyu merged commit 4f30118 into main Aug 17, 2026
15 checks passed
@code-yeongyu
code-yeongyu deleted the feat/eval-throughput-badge branch August 17, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant