Skip to content

ARM API Reviewer agent: verify dropped readme.md / suppressions.yaml entries against the emitted OpenAPI, not the TypeSpec source #43651

Closed

Description

@ravimeda

Background

When a PR drops a suppression (an entry present in the previous version's readme.md or suppressions.yaml is absent in the new version), the natural inference is "the author fixed the underlying violation." That inference is wrong more often than expected:

  • Suppression removal is a structural diff signal, not evidence of a fix.
  • The lint rule that the suppression silenced runs on emitted swagger, not on the TypeSpec source. A perfectly correct source-level decorator (@visibility(Lifecycle.Read), @readOnly) can still produce a non-compliant emitted shape due to emitter behavior or $ref-interaction patterns.

Several high-frequency rules fire on a specific emitted shape that is independent of the source decorator. For example, ProvisioningStateMustBeReadOnly triggers on { "$ref": "...", "readOnly": true }. Per JSON Schema, siblings of $ref are ignored, so the readOnly is silently dropped at validation time even though the TypeSpec source looks correct.

Proposal

For every dropped suppression, the Reviewer records a violation-absence anchor in the Step 6 internal artifact (same shape as the THANK-AND-RESOLVE proof-of-fix anchor): rule ID + emitted-OpenAPI jsonpath checked + line number at session SHA + one-line description of the shape now present. A dropped suppression without a recorded anchor is a Step 4 omission and blocks the Step 6 report.

Verification is against the emitted OpenAPI, not the TypeSpec source. For high-frequency rules, maintain an anti-pattern lookup table:

  • ProvisioningStateMustBeReadOnly: grep emitted JSON for "provisioningState" and confirm the value is not an object containing both $ref and a sibling readOnly: true.
  • LroPostMustNotUseOriginalUriAsFinalState: grep for x-ms-long-running-operation: true on POST and confirm x-ms-long-running-operation-options.final-state-via is present and is not original-uri.
  • XmsPageableMustHaveCorrespondingResponse: grep for x-ms-pageable and confirm the response schema contains the nextLinkName field.

If the emitted shape still violates the rule, the dropped-suppression case is a Blocking finding (not a Warning). The PR will fail CI and the fix is non-trivial.

References

Acceptance criteria

  • Anti-pattern table grows as new high-frequency rules are identified.
  • Evals cover the $ref-sibling false-positive case (suppression dropped, source looks correct, emitted shape still violates).
  • No suppression-removal is silently accepted as evidence of a fix.

Activity

  1. ravimeda commented on Jun 4, 2026

    @ravimeda
    MemberAuthor

    Addressed by PR #43494 (merged). The relevant changes are now in main. Closing as completed.

  2. changed the title [-]ARM API Reviewer agent: verify dropped readme.md / suppressions.yaml entries against the emitted OpenAPI, not the TypeSpec source[/-] [+]ARM API Reviewer agent verify dropped readme.md / suppressions.yaml entries against the emitted OpenAPI, not the TypeSpec source[/+] on Aug 25, 2026
  3. changed the title [-]ARM API Reviewer agent verify dropped readme.md / suppressions.yaml entries against the emitted OpenAPI, not the TypeSpec source[/-] [+]ARM API Reviewer agent: verify dropped readme.md / suppressions.yaml entries against the emitted OpenAPI, not the TypeSpec source[/+] on Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions