Problem
The content-length token heuristic is copy-pasted across the codebase: CHARS_PER_TOKEN is defined independently in src/context-tree.ts, src/context-budget.ts, src/providers/kiro.ts, src/providers/cursor.ts, src/providers/warp.ts, src/providers/codex.ts, and src/providers/cursor-agent.ts, with Math.ceil(len / CHARS_PER_TOKEN) re-derived at each site. If the heuristic is ever refined (different ratio for code vs prose, per-tokenizer), it will silently apply to some providers and not others, skewing exactly the cross-provider comparisons the product exists for.
Evidence
grep -rln 'CHARS_PER_TOKEN = ' src --include='*.ts' lists the 7 files above.- Verified on main at 1a75484.
Proposed fix
One exported estimateTokensFromChars(chars: number): number in a shared leaf module; delete the local constants. Mechanical change, no behavior difference today, and it gives estimated-token provenance a single home (pairs well with #639).
Problem
The content-length token heuristic is copy-pasted across the codebase:
CHARS_PER_TOKENis defined independently in src/context-tree.ts, src/context-budget.ts, src/providers/kiro.ts, src/providers/cursor.ts, src/providers/warp.ts, src/providers/codex.ts, and src/providers/cursor-agent.ts, withMath.ceil(len / CHARS_PER_TOKEN)re-derived at each site. If the heuristic is ever refined (different ratio for code vs prose, per-tokenizer), it will silently apply to some providers and not others, skewing exactly the cross-provider comparisons the product exists for.Evidence
grep -rln 'CHARS_PER_TOKEN = ' src --include='*.ts'lists the 7 files above.Proposed fix
One exported
estimateTokensFromChars(chars: number): numberin a shared leaf module; delete the local constants. Mechanical change, no behavior difference today, and it gives estimated-token provenance a single home (pairs well with #639).