✨ Run ruleset tests nightlies #179
Conversation
Signed-off-by: Juan Manuel Leflet Estrada <jleflete@redhat.com>
WalkthroughAdds a new GitHub Actions workflow to run nightly and on-demand ruleset tests (checkout, build, tooling fetch, test run, result parsing, artifacts) and updates .gitignore to ignore IDE/editor swap files. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
.github/workflows/nightly-rulesets.yaml (1)
19-19: Update outdated GitHub Actions versions.Static analysis flags indicate that
actions/setup-go@v4andactions/checkout@v3are outdated for current GitHub Actions runners. Update to latest versions for better compatibility and security.Apply this diff to update the action versions:
- name: Set up Go - uses: actions/setup-go@v4 + uses: actions/setup-go@v5 - name: Checkout analyzer-lsp - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Checkout rulesets - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Checkout kantra - uses: actions/checkout@v3 + uses: actions/checkout@v4Also applies to: 32-32, 39-39, 46-46
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/nightly-rulesets.yaml(1 hunks).gitignore(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.9)
.github/workflows/nightly-rulesets.yaml
19-19: the runner of "actions/setup-go@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
32-32: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
39-39: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
46-46: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Verify api and ui tests run via
global-ci-bundle.ymlwith default input / e2e-ui-integration-tests - GitHub Check: Verify api and ui tests run via
global-ci-bundle.ymlwith default input / e2e-api-integration-tests - GitHub Check: Verify api and ui tests run via
global-ci.ymlwith default inputs / e2e-ui-integration-tests - GitHub Check: Verify api and ui tests run via
global-ci.ymlwith default inputs / e2e-api-integration-tests
🔇 Additional comments (6)
.gitignore (1)
1-3: Standard editor artifacts safely excluded.The additions appropriately ignore common IDE and editor temporary files (.idea/, .swo, .swp) without impacting repository integrity or workflow functionality.
.github/workflows/nightly-rulesets.yaml (5)
8-16: Verify permissions for badge updates.The workflow declares
pull-requests: writeandissues: writepermissions, but the badge update steps at lines 117-135 use the BYOB action withGITHUB_TOKEN. Verify that these permissions are sufficient for the RubbaBoy/BYOB action to update badges. Badge updates may requirecontents: writepermission on the target repository (konveyor/rulesets), which may not be available when running from the CI repository.
52-66: Build step is well-structured.The kantra build correctly replaces the analyzer-lsp dependency with the local checkout to enable testing against unreleased changes. The approach is sound: replacing the module path, tidying, and building.
79-107: Test step uses intentional error suppression for output processing.The
|| trueon line 83 deliberately allows the step to continue even if kantra fails, enabling output processing and result extraction. The step then exits with appropriate codes (lines 103-107) based on parsed results. This is a reasonable approach for capturing test results before determining overall failure.
110-115: Artifact uploads are unconditional, preserving test results regardless of outcome.The step runs regardless of test success, which is appropriate for capturing test outputs for analysis. Compression level 9 is reasonable.
68-77: Podman is available on ubuntu-latest runners.Podman is preinstalled on GitHub Actions ubuntu-latest runners, so the step will not fail due to missing podman. The workflow should proceed without modification for this reason.
Likely an incorrect or invalid review comment.
Signed-off-by: Juan Manuel Leflet Estrada <jleflete@redhat.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
.github/workflows/nightly-rulesets.yaml (1)
19-19: Update deprecated GitHub Actions to their latest versions.The static analysis flagged that
actions/setup-go@v4andactions/checkout@v3are too old to run on GitHub Actions. Update these to the latest versions.Apply this diff to update the action versions:
- name: Set up Go - uses: actions/setup-go@v4 + uses: actions/setup-go@v5 with: go-version: '1.23' - name: Checkout analyzer-lsp - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: repository: konveyor/analyzer-lsp - name: Checkout rulesets - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: repository: konveyor/rulesets - name: Checkout kantra - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: repository: konveyor/kantraAlso applies to: 32-32, 39-39, 46-46
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/nightly-rulesets.yaml(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.9)
.github/workflows/nightly-rulesets.yaml
19-19: the runner of "actions/setup-go@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
32-32: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
39-39: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
46-46: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🔇 Additional comments (3)
.github/workflows/nightly-rulesets.yaml (3)
79-107: Verify error handling for test output parsing.The test parsing logic (lines 85-88) uses several sed/awk transformations on the output. Ensure the output format is consistent and that missing summary lines don't cause silent failures or incorrect exit codes.
Additionally, the command uses
|| trueto continue on failure (line 83), but then attempts to parse output that may be incomplete or malformed. Consider adding validation to confirm the expected summary lines exist before parsing.
1-115: Badge update issue has been resolved.The problematic badge update steps have been removed entirely. The workflow file is exactly 115 lines long with no badge-related code present, eliminating the previous issue where unreachable badge update steps attempted cross-repository access without proper triggers.
68-77: No action needed. Podman is pre-installed by default on GitHub Actions ubuntu-latest runners, as documented in the actions/runner-images repository. The workflow can safely use podman commands without additional setup steps.Likely an incorrect or invalid review comment.
1 participant
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.