Repository navigation
Commit 6645f97
perf(agent): cheap compaction — dedicated summarizer model, input cap… (#532)
* perf(agent): cheap compaction — dedicated summarizer model, input caps, free reactive prune, window re-derive
Compaction summarization was the single most expensive recurring event
in a long run: the full elided middle — verbatim tool bodies included —
went to the session's main model at full price (a ~100k-token call near
a 200k window, recursively multiplied on overflow).
Four changes, each independently safe:
- Dedicated summarizer model. Compaction summaries route to a cheap
model resolved as ZERO_COMPACTION_MODEL > preferences.compactionModel
> a curated default on official endpoints (Haiku 4.5 / Gemini
Flash-Lite / gpt-4.1-mini; custom-endpoint catalogs are unknowable so
they get none). Built lazily on the first paid compaction; ANY
failure permanently falls back to the main provider for the run, so a
misconfigured summarizer can never break compaction. "main" at any
level forces the old behavior.
- Summarizer input caps. renderTranscript clamps tool-result bodies to
2,000 chars and tool-call arguments to 500 (head+tail, rune-safe) —
the summary needs what was done and decided, not raw payloads. Cuts
summarizer input 5-10x on tool-heavy sessions.
- Free reactive prune. recover() now runs the same prune-first stage as
the proactive path: a context-limit error is first answered by
pruning stale tool bodies at zero cost, without consuming the
one-shot reactive budget; the paid summarizer only runs when a later
error finds nothing left to prune.
- Window re-derivation on model switch. Options.ContextWindowFor
(wired to the model registry in exec and the TUI) lets the loop
re-derive the compaction threshold after escalate_model, closing the
documented KNOWN LIMITATION where a switch to a smaller-window model
could overflow and a larger one over-compacted.
## Rebase notes (2026-08-25, onto 2749447)
Rebased across 214 commits. main now summarizes through the context planner
(#879) and switches models via turn sessions, so the agent side was reworked
rather than merged textually:
- Summarizer selection lives on compactionState and is resolved lazily per
main model through summarizeWithPlanner (planner + trace preserved); a
failing dedicated summarizer is marked broken and the same slice is
retried once on the main provider.
- Options.Summarizer now receives the main model in force:
`func(ctx, mainModelID string) (Provider, error)`, and may return
(nil, nil) when no dedicated model applies. After a successful
escalate_model switch the loop calls compactor.switchModel, which
re-derives the window and re-resolves the summarizer — a run that started
on the cheap model and escalated regains cheap summaries (review P2).
- recover()'s free prune stage runs before the one-shot reactive guard
(pruning is free and cannot loop) and mirrors maybeCompact's two prune
passes; the free retry is taken only when the pruned history fits the
window the provider enforces (with a 5% estimate margin), otherwise the
same call falls through to the paid summarizer. Gating on the 70%
proactive threshold would have sent histories in the 70-95% band to the
summarizer unnecessarily (review P1 and the follow-up question).
- ContextWindowFor is wired through the turn-session switch path; the
planner's window is updated alongside the compactor's.
Tests: the PR's suite is ported to the new seams, plus insufficient-prune
fall-through, free prune after the budget is spent, window-vs-threshold
gating, and summarizer re-resolution across a model switch.
* fix(agent): branch on a dedicated-summarizer flag instead of comparing providers
Review follow-ups for #532:
- summarizeProvider now reports whether the dedicated summarizer was
selected and summarizeClosure branches on that flag; the previous
`chosen == provider` compared Provider interface values, which panics
when the dynamic type is not comparable. Regression test uses a provider
with a slice field for both the summarizer and the main provider.
- CompactionModelID's curated-default test covers the official OpenAI
profile alongside Anthropic and Google.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>1 parent 2749447 commit 6645f97
12 files changed
Lines changed: 966 additions & 42 deletions
File tree
- internal
- agent
- cli
- config
- providers
- tui
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
| 9 | + | |
8 | 10 | | |
9 | 11 | | |
10 | 12 | | |
| |||
390 | 392 | | |
391 | 393 | | |
392 | 394 | | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
393 | 416 | | |
394 | 417 | | |
395 | 418 | | |
| |||
424 | 447 | | |
425 | 448 | | |
426 | 449 | | |
427 | | - | |
428 | | - | |
429 | | - | |
430 | | - | |
431 | | - | |
432 | | - | |
433 | | - | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
434 | 460 | | |
435 | 461 | | |
436 | 462 | | |
437 | 463 | | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
438 | 532 | | |
439 | 533 | | |
440 | 534 | | |
| |||
528 | 622 | | |
529 | 623 | | |
530 | 624 | | |
531 | | - | |
| 625 | + | |
532 | 626 | | |
533 | 627 | | |
534 | | - | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
535 | 653 | | |
536 | 654 | | |
537 | 655 | | |
| |||
571 | 689 | | |
572 | 690 | | |
573 | 691 | | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
574 | 696 | | |
575 | 697 | | |
576 | 698 | | |
577 | 699 | | |
578 | 700 | | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
579 | 707 | | |
580 | 708 | | |
581 | 709 | | |
| |||
659 | 787 | | |
660 | 788 | | |
661 | 789 | | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
662 | 821 | | |
663 | 822 | | |
| 823 | + | |
| 824 | + | |
664 | 825 | | |
665 | 826 | | |
666 | 827 | | |
| |||
670 | 831 | | |
671 | 832 | | |
672 | 833 | | |
673 | | - | |
| 834 | + | |
674 | 835 | | |
675 | 836 | | |
676 | 837 | | |
677 | 838 | | |
678 | 839 | | |
679 | | - | |
| 840 | + | |
680 | 841 | | |
681 | 842 | | |
682 | 843 | | |
| |||
0 commit comments