Skip to content

ARM API Reviewer agent: formalize Reviewer / Critic wire protocol as a shared, versioned contract #43646

Closed

Description

@ravimeda

Background

The ARM API Reviewer agent now invokes an ARM API Review Critic sub-agent (#43302, implemented in PR #43494) to independently re-validate every finding before it is presented to a human for posting. The two agents exchange a structured set of inputs, verdicts, and telemetry markers on every iteration. Without a single source of truth for that exchange, each agent file drifts independently: input field names diverge, marker schemas disagree, override semantics fall out of sync, and the gate silently degrades.

Proposal

Maintain the Reviewer / Critic contract in a dedicated .github/agents/protocols/ directory, separate from either agent file. The protocol defines:

  • The ten Critic inputs the Reviewer must pass on every dispatch (PR URL, session SHA, full Step 6 report, file list, previous-version base source, Step 5.5 reconciliation plan, prior FAIL-set summary, considered-and-declined list, graph-production flag, iteration number) plus a verbatim YAML template (# critic-inputs/v1) the Reviewer pastes into every dispatch.
  • The four Critic verdict tracks (Finding accuracy, Graph integrity, Reconciliation accuracy, Coverage quality) and the authorization rule that gates posting.
  • The <!-- review-state: ... --> per-response marker (response-scope: critic-mode, iteration, PR identity).
  • The 6-field per-comment telemetry marker (finding-scope: rule, severity, classification, critic, head-sha, plus optional downstream-rule, override-reason, and the telemetry: degraded fallback form).
  • The Override-reason validator (length + boilerplate denylist + structured-anchor or verbatim-quote requirement) applied identically by both agents.
  • The non-overridable FAIL catalog (reconciliation FAILs, graph fabrication, downstream-ci-conflict, suppression-path-mismatch, unescaped-mention, override-reason-invalid).

References

Acceptance criteria

  • The protocol file is the single source of truth; both agent files reference it and do not duplicate field semantics.
  • Either agent's restatement of a protocol field is byte-faithful or is flagged as a bug.
  • A new contributor can change one of the markers in exactly one place and have both agents pick it up consistently.

Activity

  1. ravimeda commented on Jun 4, 2026

    @ravimeda
    MemberAuthor

    Addressed by PR #43494 (merged). The relevant changes are now in main. Closing as completed.

  2. changed the title [-]ARM API Reviewer agent: formalize Reviewer / Critic wire protocol as a shared, versioned contract[/-] [+]ARM API Reviewer agent formalize Reviewer / Critic wire protocol as a shared, versioned contract[/+] on Aug 25, 2026
  3. changed the title [-]ARM API Reviewer agent formalize Reviewer / Critic wire protocol as a shared, versioned contract[/-] [+]ARM API Reviewer agent: formalize Reviewer / Critic wire protocol as a shared, versioned contract[/+] on Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions