Skip to content

test(create-pr): add final validation test file#607

Merged
baleen37 merged 24 commits intomainfrom
test/final-validation
Apr 2, 2026
Merged

test(create-pr): add final validation test file#607
baleen37 merged 24 commits intomainfrom
test/final-validation

Conversation

@baleen37
Copy link
Copy Markdown
Owner

@baleen37 baleen37 commented Apr 2, 2026

Summary

  • Final validation test for the create-pr skill
  • Verifies preflight-check.sh and wait-for-merge.sh are executed as actual bash scripts

Test plan

  • preflight-check.sh executed successfully
  • Commit and push completed
  • PR created and auto-merge enabled
  • wait-for-merge.sh monitors CI and merge status

Summary by CodeRabbit

  • Bug Fixes

    • Improved branch synchronization handling—pre-flight checks now auto-sync branches when behind the base branch.
    • Simplified CI failure recovery workflow.
  • Documentation

    • Streamlined create-PR workflow guidance in SKILL.md.
  • Chores

    • Consolidated PR workflow scripts and removed redundant helpers.

Test User added 24 commits April 2, 2026 20:51
…dundant comments

Result: {"status":"keep","total_bytes":6268,"line_count":183,"word_count":917}
…tighten prose

Result: {"status":"keep","total_bytes":5836,"line_count":169,"word_count":844}
Not referenced by any skill or hook. Was retained for pr-pass per spec
but pr-pass doesn't actually use it.

Result: {"status":"keep","total_bytes":4534,"line_count":141,"word_count":666}
…ync when behind)

Result: {"status":"keep","total_bytes":4019,"line_count":122,"word_count":601}
Only preflight-check.sh used lib.sh. Inlining saves a file and overhead.

Result: {"status":"keep","total_bytes":3558,"line_count":107,"word_count":541}
… inline notes

Result: {"status":"keep","total_bytes":3202,"line_count":85,"word_count":490}
Result: {"status":"keep","total_bytes":3103,"line_count":82,"word_count":471}
Result: {"status":"keep","total_bytes":2884,"line_count":68,"word_count":448}
Scripts don't load into LLM context — only SKILL.md token cost matters.

Result: {"status":"keep","skill_bytes":1081,"skill_lines":27,"skill_words":151}
…edundant sections

Remove stop conditions (obvious), trim CI failure instructions,
shorten description field.

Result: {"status":"keep","skill_bytes":802,"skill_lines":23,"skill_words":110}
Result: {"status":"keep","skill_bytes":732,"skill_lines":19,"skill_words":100}
Result: {"status":"keep","skill_bytes":675,"skill_lines":18,"skill_words":93}
Result: {"status":"keep","skill_bytes":635,"skill_lines":18,"skill_words":82}
Result: {"status":"keep","skill_bytes":605,"skill_lines":17,"skill_words":82}
… preflight

Test found: auto-merge is disabled after force-push, must re-enable.
Also: preflight sync push needs -u for new branches.

Result: {"status":"keep","skill_bytes":665,"skill_lines":17,"skill_words":92}
Test showed agent skipping preflight-check.sh and doing manual logic instead.
Added "Execute each line literally" + "scripts MUST be run" directive.

Result: {"status":"keep","skill_bytes":794,"skill_lines":20,"skill_words":109}
- Remove tests for deleted scripts (lib.sh, verify-pr-status.sh, sync-with-base.sh)
- Update preflight BEHIND test to expect auto-sync behavior
- Fix skill content test to accept code-block-only skills

Result: {"status":"keep","skill_bytes":794}
@baleen37 baleen37 enabled auto-merge (squash) April 2, 2026 12:40
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 2, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 85ad0612-2e9b-49ca-8675-46246db5b207

📥 Commits

Reviewing files that changed from the base of the PR and between 4edb729 and ada6090.

📒 Files selected for processing (15)
  • .autoresearch/autoresearch.jsonl
  • .autoresearch/autoresearch.md
  • .autoresearch/dashboard.md
  • .autoresearch/run.sh
  • .autoresearch/worklog.md
  • plugins/me/skills/create-pr/SKILL.md
  • plugins/me/skills/create-pr/scripts/lib.sh
  • plugins/me/skills/create-pr/scripts/preflight-check.sh
  • plugins/me/skills/create-pr/scripts/sync-with-base.sh
  • plugins/me/skills/create-pr/scripts/verify-pr-status.sh
  • plugins/me/skills/create-pr/scripts/wait-for-merge.sh
  • test-final.md
  • tests/me/me-specific.bats
  • tests/skills/test_create_pr_verify_status.bats
  • tests/skills/test_skill_content.bats

📝 Walkthrough

Walkthrough

This PR optimizes the create-pr skill for reduced token usage by consolidating scripts, inlining shared utilities, and simplifying documentation. It includes autoresearch tracking artifacts, removes three helper/verification scripts, refactors remaining scripts with inline logic, and updates tests accordingly.

Changes

Cohort / File(s) Summary
Autoresearch Tracking
.autoresearch/autoresearch.jsonl, .autoresearch/autoresearch.md, .autoresearch/dashboard.md, .autoresearch/run.sh, .autoresearch/worklog.md
New autoresearch experiment logs and documentation tracking optimization metrics. Dashboard shows ~68% improvement in total_bytes. Run script validates SKILL.md frontmatter and runs shellcheck on all .sh files.
SKILL.md Simplification
plugins/me/skills/create-pr/SKILL.md
Reduced documentation from multi-section guidance to condensed execution guide. Simplified CI failure handling, consolidated git workflow steps, removed detailed pre-flight/checkout/sync instructions.
Script Consolidation & Deletion
plugins/me/skills/create-pr/scripts/lib.sh, plugins/me/skills/create-pr/scripts/sync-with-base.sh, plugins/me/skills/create-pr/scripts/verify-pr-status.sh
Deleted shared library (lib.sh) with require_git_repo and resolve_base_branch functions. Removed standalone sync-with-base.sh and verify-pr-status.sh scripts, consolidating their logic into remaining scripts.
Script Refactoring
plugins/me/skills/create-pr/scripts/preflight-check.sh, plugins/me/skills/create-pr/scripts/wait-for-merge.sh
Inlined environment validation and base branch resolution in preflight-check.sh; changed merge/sync flow to auto-merge when behind. Refactored wait-for-merge.sh state handling to use case statement; simplified output and post-check re-merge logic.
Test Updates
tests/me/me-specific.bats, tests/skills/test_create_pr_verify_status.bats, tests/skills/test_skill_content.bats
Removed test coverage for deleted scripts (verify-pr-status.sh, sync-with-base.sh, lib.sh). Updated preflight-check.sh tests to assert new inline environment checks and auto-sync behavior. Broadened SKILL.md content validation to accept code fences or section headers.
Validation
test-final.md
Placeholder validation file.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Poem

🐰 Hopping code through autoresearch grounds,
Old scripts consolidated, new flows found,
Bytes trimmed to sixty, a leap of great height,
The warren's PR skill now nimble and light!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/final-validation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@baleen37 baleen37 merged commit 4f4395c into main Apr 2, 2026
4 of 5 checks passed
@baleen37 baleen37 deleted the test/final-validation branch April 2, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant