Skip to content

Add Cursor e2e test generation rules#106

Open
sayak-redhat wants to merge 4 commits into
openshift:mainfrom
sayak-redhat:SPIRE-525
Open

Add Cursor e2e test generation rules#106
sayak-redhat wants to merge 4 commits into
openshift:mainfrom
sayak-redhat:SPIRE-525

Conversation

@sayak-redhat

@sayak-redhat sayak-redhat commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

What it does:

  1. Teaches the agent our test structure, existing helpers, and constants so it reuses them
  2. Enforces a write-scope rule — agent can only touch test/, never production code
  3. Flags gaps in SCC validation, OLM lifecycle, RBAC, and other certification areas
  4. Generates a test-cases.md plan by default; actual code only if you ask for it

What it doesn't do:

  1. No changes to existing code or tests
  2. No new dependencies or tooling
  3. No CI changes

Summary by CodeRabbit

  • Documentation
    • Added a contributor-facing E2E testing rulebook: standardized workflows for creating tests from PRs or tickets, a test-case template with prerequisites and expected outcomes, coverage discovery and deduplication guidance, and a ranked “top 10” missing-scenarios selection.
    • Introduced strict write-scope limits and pre-commit checks to prevent out-of-scope edits, plus mandatory test structure, labeling, async/polling, cleanup, and reviewable style guidelines (including OLM/OpenShift verification).

@openshift-ci

openshift-ci Bot commented Apr 23, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sayak-redhat
Once this PR has been reviewed and has the lgtm label, please assign trilokgeer for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Apr 23, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a contributor-facing E2E test rulebook (.cursor/rules/e2e-rules.md) that prescribes write-scope restrictions, workflows to derive tests from PRs/Jira, coverage deduplication, test-case templates, Ginkgo/Go conventions, pre-commit scope checks, and generation guardrails.

Changes

Cohort / File(s) Summary
E2E Test Contributor Guidelines
​.cursor/rules/e2e-rules.md
New contributor-facing rulebook specifying strict write-scope (only test/ and output/), pre-commit diff checks, workflows to create test plans from GitHub PRs or Jira tickets, discovery and deduplication heuristics, a test-cases.md template, required Ginkgo v2 labeling and Go test patterns, and generation/commit guardrails.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add Cursor e2e test generation rules' directly and clearly describes the main change: introducing a new rulebook for E2E test generation guiding AI agents on test structure, scope, and workflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR adds e2e-rules.md documentation with Ginkgo test naming guidance; no actual test files with code were modified.
Test Structure And Quality ✅ Passed PR adds only markdown documentation file; no Ginkgo test code modifications were made, so the test structure check is not applicable.
Microshift Test Compatibility ✅ Passed This PR does not add any new Ginkgo e2e tests. While the repository contains existing e2e tests, the current PR only adds documentation to .cursor/rules/e2e-rules.md with no modifications to any test code files.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR adds only documentation in .cursor/rules/e2e-rules.md with no executable test code, so Single Node OpenShift compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds only documentation file for E2E test guidance with no changes to deployment manifests, operator code, controllers, or scheduling configurations.
Ote Binary Stdout Contract ✅ Passed PR adds e2e-rules.md configuration file to .cursor/rules directory with no changes to Go source code or test files that could violate OTE Binary Stdout Contract.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR adds only a Markdown documentation file with e2e test guidelines, not actual Ginkgo test implementations, so the IPv6 and disconnected network compatibility check is not applicable.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.cursor/rules/e2e-rules.md:
- Line 299: Several fenced code blocks in the markdown use plain triple
backticks with no language specified; update each instance of a code fence that
currently looks like ``` to include an appropriate language tag (e.g., ```go or
```text) so markdownlint MD040 is satisfied—search for code fences with no
language in the document and replace them with the correct language for those
code snippets.
- Around line 30-43: The doc contains contradictory scope rules: the rule "ONLY
create or modify files inside `test/`" conflicts with later clauses that permit
writing/committing `output/` (e.g. the sentence "The generated `test-cases.md`
is written to a **local** output directory (e.g. `output/<key>/`)"). Update the
first rule to explicitly permit generated output directories by replacing "ONLY
create or modify files inside `test/`" with a scoped rule that allows
writing/committing generated files under `output/<key>/` (or add a clear
exception sentence), and ensure the three occurrences of the output-exemption
text are consistent (the "generated `test-cases.md`" line and the two other
`output/` exemption lines) so the document unambiguously states that only test
source edits are allowed except for generated output written to `output/<key>/`.
- Around line 299-328: The duration constants at the bottom (DefaultInterval,
ShortInterval, DefaultTimeout, ShortTimeout, TestContextTimeout) use shorthand
literals like "10s" and "5m" which are not valid Go duration syntax; update each
to the canonical Go form using the time package (e.g., DefaultInterval = 10 *
time.Second, DefaultTimeout = 5 * time.Minute, etc.), ensure the file imports or
references time where needed and that the constant names (DefaultInterval,
ShortInterval, DefaultTimeout, ShortTimeout, TestContextTimeout) exactly match
the equivalents in test/e2e/utils/constants.go.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 22063276-69e0-4206-8793-a842e38dc095

📥 Commits

Reviewing files that changed from the base of the PR and between e841537 and c1117fb.

📒 Files selected for processing (1)
  • .cursor/rules/e2e-rules.md

Comment thread .cursor/rules/e2e-rules.md Outdated
Comment thread .cursor/rules/e2e-rules.md Outdated
Comment thread .cursor/rules/e2e-rules.md Outdated
…e format

- Refine Step 6 with 10 impact categories (Core Functionality, Config Edge
  Cases, Dynamic Behavior, Integration Gaps, Multi-tenant, Error Handling,
  Upgrade/Compatibility, Performance, Security, Real Customer Scenarios)
- Add prioritization rules (Critical / High / Medium)
- Add structured test case ID format: <TICKET>-TC-NNN
- Enhance Step 7 template with Coverage Gap field and per-step Expected results
- Add "Writing good test steps" guidance for concrete, actionable test plans

Made-with: Cursor

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/e2e/e2e_test.go`:
- Around line 341-368: The test assertions in the e2e test contradict the
production SCC generated by generateSpireAgentSCC; update the expectations in
the It block that reads the scc variable (and references SpireAgentSCCName) to
match production: set Expect(scc.AllowHostNetwork).To(BeTrue()),
Expect(scc.AllowHostPorts).To(BeTrue()),
Expect(scc.AllowPrivilegedContainer).To(BeTrue()), expect
AllowPrivilegeEscalation to be non-nil and true (adjust the pointer assertion
accordingly), and change
Expect(scc.RunAsUser.Type).To(Equal(securityv1.RunAsUserStrategyMustRunAsRange));
alternatively, if you intend to change production behavior instead, leave the
test and open a follow-up to modify generateSpireAgentSCC to align with the
current assertions.
- Around line 370-427: The test's expectations for the spire-agent DaemonSet
(variables ds, podSpec, agentContainer and container SecurityContext sc)
contradict production settings; update the assertions to match
pkg/controller/spire-agent/daemonset.go (HostNetwork should be true, DNSPolicy
should equal corev1.DNSClusterFirstWithHostNet, container sc.Privileged should
be true, sc.AllowPrivilegeEscalation and sc.ReadOnlyRootFilesystem may be
nil/unset, and sc.Capabilities.Drop should not be asserted to contain "ALL"), or
alternatively move this spec check into a separate test that documents the
required privileged configuration for SPIRE Agent's node attestation
(referencing utils.SpireAgentDaemonSetName and the "spire-agent" container name)
and adjust the Expect(...) calls accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 1d1f9edf-4147-4e3c-b29f-0beffc05e2a0

📥 Commits

Reviewing files that changed from the base of the PR and between 1ed6eb9 and f3531a0.

📒 Files selected for processing (5)
  • .cursor/rules/e2e-rules.md
  • output/pr-105/test-cases.md
  • test/e2e/e2e_suite_test.go
  • test/e2e/e2e_test.go
  • test/e2e/utils/constants.go
✅ Files skipped from review due to trivial changes (1)
  • output/pr-105/test-cases.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • .cursor/rules/e2e-rules.md

Comment thread test/e2e/e2e_test.go Outdated
Comment thread test/e2e/e2e_test.go Outdated
Super User added 2 commits April 23, 2026 17:09
Clarify that test/ and output/ are two separate top-level directories,
not output/ being inside test/. Addresses CodeRabbit review comment
about contradictory scope rules.

Made-with: Cursor
@openshift-ci

openshift-ci Bot commented Apr 27, 2026

Copy link
Copy Markdown

@sayak-redhat: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant