Skip to content

feat(usage): separate auxiliary model calls from a Session's cost and cache hit rate #5691

Description

@liugddx

Problem

A Session's usage figures in Desktop mix the main agent loop with every auxiliary model call made for that Session, and there is no way to separate them.

  • Auxiliary calls pass telemetrySessionId: sessionId to runHostAuxiliaryModelCall. These are Session title, recap, goal evaluation, memory extraction, WorkHub intent/recall, and next-prompt suggestions once feat(desktop): add opt-in next prompt suggestions #5629 lands.
  • The Session Inspector loads usage:summary with { range: 'all', sessionId } (apps/desktop/src/preload/preload.ts:1367). That summary feeds both estimatedSessionCost and usageCacheHitRate (session-inspector-overview-model.ts:194, :245-254, computed as cacheRead / input).
  • UsageQuery (packages/core/src/usage-stats/types.ts:42-50) can filter by session, connection, provider, model, tool and status, but not by callKind. Consumers therefore cannot exclude auxiliary calls, and cannot break a Session's cost down by kind.

Until now this mattered little, because most auxiliary calls run once or rarely per Session. A per-reply auxiliary call such as #5629 changes that:

  • Cost. Every reply adds an extra call that the user sees as Session cost, with no way to tell it apart from the agent's work.
  • Cache hit rate. Auxiliary prompts do not share the main loop's cached prefix. Example: a main turn of 100k input tokens at 95% cache reads, plus one uncached 10k-token suggestion call, moves the displayed rate from 95% to about 86%. The main loop's caching did not change at all.
  • Daily Review. Its request counts are inflated in the same way.

A second, smaller accuracy gap is on the failure path of runHostAuxiliaryModelCall (execution-model-authority.ts:691-701). When a call is aborted or fails, it is recorded with inputTokens: 0, outputTokens: 0, even though the provider may already have processed, and billed, the request. For example, a reasoning call aborted by a short deadline could have been billed for its input plus any reasoning tokens.

Desired outcome

  • Add a callKind (or callKinds / excludeCallKinds) filter to UsageQuery, so consumers can ask for the main loop only, or for a per-kind breakdown.
  • Base the Session Inspector's cache hit rate on main calls only. Optionally show auxiliary spend as its own line next to the Session cost, instead of folding it in silently.
  • Where the provider or SDK reports partial usage on abort or error, record it. If it can't be known, mark the row as usage-unknown rather than zero.

Alternatives or workarounds

  • Auxiliary calls could omit telemetrySessionId, so they only appear in global usage. That hides them from the Session entirely, which is worse for cost transparency.
  • feat(desktop): add opt-in next prompt suggestions #5629 could state in its copy that suggestion spend is counted in the Session's cost. That documents the mixing but does not fix the hit-rate distortion.

Found while reviewing #5629 (review: #5629 (review)).

AI disclosure: Claude Code (Claude Opus) traced the usage path at 276faf1c9 and drafted this issue; liugddx reviewed it and is the human contributor of record.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions