Skip to content

docs: broaden the verify-before-acting rule in AGENTS.md - #3309

Merged
juherr merged 1 commit into
testng-team:masterfrom
juherr:docs/agents-verify-rules
Jul 29, 2026
Merged

docs: broaden the verify-before-acting rule in AGENTS.md#3309
juherr merged 1 commit into
testng-team:masterfrom
juherr:docs/agents-verify-rules

Conversation

@juherr

@juherr juherr commented Jul 29, 2026

Copy link
Copy Markdown
Member

Follow-up to #3302, from the retrospective on writing it.

Why

AGENTS.md shipped with five factual errors, two of them serious — the build JDK was said to live in one place when it lives in six (and test.yml overrides gradle.properties), and a failure mode was described that stopped being true when Gradle 9 landed.

Both came from sources the existing rule did not name. It covered "a statement inherited from a pull request description, an issue or a comment" — but the actual sources were a stale project memory and a conclusion drawn earlier in the same session. Neither felt like hearsay at the time, which is exactly the problem.

Changes

Three lines, at the two places that failed:

  • The verify rule now says including your own, and names memories and earlier-session conclusions alongside PR descriptions.
  • A new line: running a documented command proves it executes, not that it does what the text claims. The CLAUDE.md output filter ran perfectly while matching 2968 of 5293 log lines — it was verified as "executes", never as "filters".
  • The Layout section now says to read docs/ and .github/ before writing documentation. AGENTS.md was itself written as a fifth copy of material already there, before being cut back to links.

Verification

./gradlew clean build green on the merged master — 0 failures. Documentation-only change, so no CHANGES.txt entry, consistent with the other docs: commits in this repository.

Summary by CodeRabbit

  • Documentation
    • Updated contributor guidance to review existing documentation and workflow materials before writing new content.
    • Refined working-style instructions to emphasize verifying claims, distinguishing hearsay, and re-checking against the build.
    • Expanded guidance to clarify that successfully running a documented command confirms execution, not that it matches the described effect.

@juherr
juherr requested a review from krmahadevan as a code owner July 29, 2026 15:46
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 832bce28-1dfd-4335-85db-5a220bb5eca3

📥 Commits

Reviewing files that changed from the base of the PR and between 77f71cb and b5168e6.

📒 Files selected for processing (1)
  • AGENTS.md

📝 Walkthrough

Walkthrough

AGENTS.md updates contributor guidance to consult existing documentation before duplicating content and to verify claims, including hearsay, against the build and documented command effects.

Changes

Contributor guidance updates

Layer / File(s) Summary
Layout guidance
AGENTS.md
The Layout section directs contributors to review existing docs/ and .github/ content before adding duplicated guidance.
Working style verification
AGENTS.md
The Working style guidance expands claim verification to cover hearsay, build re-checks, and measuring documented command effects.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

Suggested reviewers: krmahadevan

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main documentation change in AGENTS.md and accurately highlights the broadened verify-before-acting guidance.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@AGENTS.md`:
- Around line 108-112: Update the “Verify claims before acting on them” guidance
in AGENTS.md so claims are re-checked against the build or the appropriate
authoritative source, preserving the surrounding examples and intent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6b49f52a-4741-4d50-955f-ce7bac0218fa

📥 Commits

Reviewing files that changed from the base of the PR and between 58dfbea and 77f71cb.

📒 Files selected for processing (1)
  • AGENTS.md

Comment thread AGENTS.md Outdated
The rule named only claims inherited from pull requests and issues, so it did
not cover the two sources that actually produced wrong statements while writing
these files: a stale project memory, and a conclusion drawn earlier in the same
session. Both were copied into a committed file before being caught.

Also record that running a documented command only proves it executes -- the
output filter in CLAUDE.md ran fine while matching 2968 of 5293 log lines -- and
that existing docs should be read before new ones are written.
@juherr
juherr force-pushed the docs/agents-verify-rules branch from 77f71cb to b5168e6 Compare July 29, 2026 15:56
@juherr
juherr merged commit 794b9cc into testng-team:master Jul 29, 2026
11 checks passed
@juherr
juherr deleted the docs/agents-verify-rules branch July 29, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant