Skip to content

improve(ralph): optimize plugin for simplicity and token efficiency#609

Merged
baleen37 merged 27 commits intomainfrom
autoresearch/ralph-improve-20260403
Apr 3, 2026
Merged

improve(ralph): optimize plugin for simplicity and token efficiency#609
baleen37 merged 27 commits intomainfrom
autoresearch/ralph-improve-20260403

Conversation

@baleen37
Copy link
Copy Markdown
Owner

@baleen37 baleen37 commented Apr 3, 2026

Summary

Ralph 플러그인을 단순화하고 토큰 효율성을 개선했습니다. 7번의 서브에이전트 실제 호출 테스트로 검증.

Changes

SKILL.md (2914 → 1098 bytes, -62.3%)

  • 90줄 → 29줄로 압축
  • --no-prd 플래그 동작 정의 (이전엔 미정의)
  • Slop 정의 명확화, one-at-a-time 명시
  • 불필요한 JSON 예시, 중복 섹션 제거

ralph-cancel SKILL.md (696 → 525 bytes, -24.6%)

  • CWD 컨텍스트 추가
  • Cancel signal과 hook 연동 설명 추가

ralph-persist.ts (3747 → 2390 bytes, -36.2%)

  • TypeScript 간소화 (interfaces 제거, Bun.stdin.text() 사용)
  • Story progress 표시 (N/M stories done)
  • Last failure from progress.txt 표시
  • 변수명/함수 구조 압축

Total: 7357 → 4013 bytes (-45.4%)

Tests

  • BATS 13/13 통과
  • E2E /ralph "Build a calculator" — 성공
  • E2E /ralph --no-prd "Build a calculator" — 성공
  • E2E /ralph-cancel — 성공
  • Agent clarity score: 7/10

Summary by CodeRabbit

  • Chores
    • Updated optimization configuration and metrics targeting a different subsystem with new performance measurement categories
    • Refreshed results dashboard and tracking documentation with updated baseline and final performance metrics
    • Updated validation procedures to include additional component checks and test coverage

Test User added 27 commits April 3, 2026 19:00
Result: {"status":"keep","skill_bytes":2914}
… wording

Result: {"status":"keep","skill_bytes":1364}
Result: {"status":"keep","skill_bytes":1364,"cancel_bytes":431}
… functions

Result: {"status":"keep","skill_bytes":1364,"hook_bytes":2725}
Result: {"status":"keep","skill_bytes":1260}
…vars

Result: {"status":"keep","hook_bytes":2157,"total_bytes":3848}
Result: {"status":"keep","skill_bytes":1237}
Result: {"status":"keep","skill_bytes":1197}
Result: {"status":"keep","skill_bytes":1193}
Result: {"status":"keep","hook_bytes":1962,"total_bytes":3586}
Result: {"status":"keep","skill_bytes":1126}
- progress.txt: clarify "skip on first iteration"
- architect review: add specific review criteria
- deslop: enumerate what to remove
- regression test: add "must pass"
- cancel signal output: "reply" not "output"

Result: {"status":"keep","skill_bytes":1249,"reason":"clarity over bytes"}
…e step

Result: {"status":"keep","skill_bytes":1272}
…json

Agent test showed --no-prd was completely undefined, causing confusion.

Result: {"status":"keep","skill_bytes":1359}
Agent test found "skip PRD" + "create prd.json" contradictory.
Changed to "skip elaboration, auto-generate prd.json".

Result: {"status":"keep","skill_bytes":1396}
Agent test found path base and cancel-signal purpose unclear.

Result: {"status":"keep","cancel_bytes":525}
… update

Agent retry test showed INSIGHT lines valuable for iteration learning.

Result: {"status":"keep","skill_bytes":1438}
…-time + slop

Removed $S variable, [INSIGHT] format, verbose explanations.
Added: "one story at a time", slop definition.
Agent test: 7/10 clarity.

Result: {"status":"keep","skill_bytes":1098,"total_bytes":3909}
Gives agent immediate context about previous failure without reading the file.

Result: {"status":"keep","hook_bytes":2487}
Result: {"status":"keep","hook_bytes":2390}
@baleen37 baleen37 enabled auto-merge (squash) April 3, 2026 10:41
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 3, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2c516019-b0bc-48de-9c9d-8afb9ffb913f

📥 Commits

Reviewing files that changed from the base of the PR and between fe3b97a and 11039d7.

📒 Files selected for processing (8)
  • .autoresearch/autoresearch.jsonl
  • .autoresearch/autoresearch.md
  • .autoresearch/dashboard.md
  • .autoresearch/run.sh
  • .autoresearch/worklog.md
  • plugins/ralph/hooks/ralph-persist.ts
  • plugins/ralph/skills/ralph-cancel/SKILL.md
  • plugins/ralph/skills/ralph/SKILL.md

📝 Walkthrough

Walkthrough

This pull request refactors the autoresearch optimization focus from the create-pr-optimize configuration to a new ralph-optimize configuration targeting the ralph plugin. Changes include updated configuration files, revised documentation, a refactored run script with new metrics (cancel_bytes, hook_bytes, total_bytes), and substantial rewrites to the ralph persistence hook and skill documentation.

Changes

Cohort / File(s) Summary
Autoresearch Configuration
.autoresearch/autoresearch.jsonl, .autoresearch/autoresearch.md, .autoresearch/dashboard.md
Configuration name changed from create-pr-optimize to ralph-optimize; metric target shifted from total_bytes to skill_bytes; baseline and final metrics updated with new byte breakdowns (cancel_bytes, hook_bytes, total_bytes); documentation scope changed to ralph plugin; validation constraints updated to cover persistence loop, cancellation, session recovery, TS compilation, and BATS tests.
Autoresearch Runner
.autoresearch/run.sh
Script root changed from plugins/me/skills/create-pr/ to plugins/ralph/; pre-checks added for skill, cancel-skill, hook, hooks.json, and plugin.json files; metrics computation refactored to read from SKILL.md, ralph-cancel/SKILL.md, and hooks/ralph-persist.ts; validity checks replaced shellcheck with frontmatter validation, jq JSON parsing, bun run TypeScript execution, and targeted BATS tests.
Ralph Worklog
.autoresearch/worklog.md
Session changed from "create-pr token efficiency" to "Ralph Plugin Optimization"; baseline/final metrics updated with new ralph plugin values; prior experiment list replaced with 7 subagent test findings; key insights refocused on clarity, compression limitations, and hook story context.
Ralph Plugin Implementation
plugins/ralph/hooks/ralph-persist.ts
State interfaces and helpers removed; stdin handling switched from accumulation to Bun.stdin.text(); control flow reworked to handle state initialization, cancellation detection via cancel-signal-state.json, and stale-state checks; block output now embeds user story completion status and progress from .ralph/prd.json and .ralph/progress.txt into reason field; error handling simplified with exit code 0.
Ralph Skill Documentation
plugins/ralph/skills/ralph-cancel/SKILL.md, plugins/ralph/skills/ralph/SKILL.md
Cancel skill steps condensed and refocused on CWD-based project targeting; Ralph skill activation simplified, PRD section condensed with auto-generation support, execution loop shortened to priority-based story selection and pass/fail tracking, and completion procedure streamlined; "Important" constraints section removed.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • PR #582: Consolidates autoresearch outputs and refactors autoresearch directory structure, aligning with configuration and documentation updates in this PR.
  • PR #569: Introduced the ralph plugin artifacts (hooks/ralph-persist.ts, skill files, plugin.json) that are being substantially refactored in this PR.
  • PR #573: Implements the same cancel-signal detection and active-state logic in ralph-persist.ts that this PR refactors.

Poem

🐰 Hop along, dear ralph, to optimization's way,
From create-pr's crown to ralph's brighter day,
New metrics dance—cancel, hook, and skill so bright,
Persistence flows cleaner, state and signal in sight,
Simplified loops, no slop left behind,
A plugin refined, with clarity in mind!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch autoresearch/ralph-improve-20260403

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@baleen37 baleen37 merged commit 7ff719e into main Apr 3, 2026
4 of 5 checks passed
@baleen37 baleen37 deleted the autoresearch/ralph-improve-20260403 branch April 3, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant