Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,17 @@ Changes to **how repos are explored or how skills are structured** belong in `cr

Repo-specific findings (build commands, reviewer names, branch prefixes) belong **only** in the generated skill, never back-ported here.

## What belongs here vs. in gardener

[`gardener`](https://github.com/dmccoystephenson/gardener) is a separate open-source project that dispatches generated skills headlessly across a fleet of repos. It reads a generated skill from `~/local-skills/<slug>-dev-loop/<slug>-dev-loop.md` and its `~/.claude/commands/<slug>-dev-loop.md` symlink — the exact paths Steps 3 and 5 write — and invokes `/create-dev-loop` itself to bootstrap a skill for a repo that lacks one.

That makes Steps 3, 5, and 6 a load-bearing interface, not just internal detail: **changing where a skill is written, what it's named, or what Step 6 creates is a breaking change for gardener.** Flag it in the PR description so the corresponding change can be made there.

Conversely, anything about *scheduling, batching, merge authorization, or safety-gating a headless run* belongs in gardener, never here. This skill's output is a skill file; it has no opinion on who runs it or when.

## Promoting a rule into the template

When you add a new rule to `create-dev-loop.md` because the same lesson keeps showing up in multiple skills' self-audits, **also open retrofit PRs against every existing skill that predates the change.** The template only fixes drift forward; existing skills will silently lag until their next self-audit cycle. (The project maintainer tracks this checklist in a private `my-claude-skills/CONVENTIONS.md` reference; external contributors without access should instead note in the PR description which existing skills likely need a retrofit pass, so a maintainer can follow up.)
When you add a new rule to `create-dev-loop.md` because the same lesson keeps showing up in multiple skills' self-audits, **also open retrofit PRs against every existing skill that predates the change.** The template only fixes drift forward; existing skills will silently lag until their next self-audit cycle. Note in the PR description which existing skills likely need a retrofit pass, since only the person running those skills can see which ones exist.

## Grounding work in research

Expand All @@ -45,7 +53,7 @@ CI (`.github/workflows/ci.yml`) runs `scripts/check_docs.py`, which mechanically
4. The `<!-- template-version: <sha> -->` and `<!-- generated-at: <iso8601> -->` HTML comments appear at the top of the generated skill, with the SHA matching the create-dev-loop commit you generated from
5. If the target repo already has a skill, exercise `MODE=update` end-to-end: confirm the abort-on-uncommitted-changes check, the pre-overwrite diff prompt, and that Steps 6–7 are correctly skipped

The maintainer validates against private reference implementations (`dpm-dev-loop`, `herald-dev-loop`); external contributors without access should validate by running `/create-dev-loop` against any real repository they maintain and manually checking the five items above.
Validate by running `/create-dev-loop` against any real repository you maintain and manually checking the five items above. Use a low-stakes repo the first time: the skill creates a GitHub repo in Step 6 and writes to `~/local-skills/` and `~/.claude/commands/`.

## Commit and PR conventions

Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ contributors:
appear correctly
- if you touched update-mode behavior, exercise `MODE=update` end-to-end

The maintainer validates against private reference repos for this step;
if you don't have a suitable repo handy, any real project you maintain
works as a fixture.
Any real project you maintain works as a fixture. Prefer a low-stakes
one the first time — Step 6 creates a GitHub repo, and Steps 3 and 5
write to `~/local-skills/` and `~/.claude/commands/`.
4. Commit using imperative mood, no trailing period (e.g. `Add SKILL_REPO_OWNER placeholder`).
5. Open a PR referencing any related issue with `Closes #N`. Describe what
you tested it against.
Expand Down
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Running `/create-dev-loop` in any repo runs seven Steps (mapped 1:1 to Steps 1
4. **Fill in placeholders** — substitutes every `{{PLACEHOLDER}}` and `{{#if FLAG}}` token from Step 2's findings, using the Step 4 substitution table as the contract
5. **Register** it as a slash command at `~/.claude/commands/<slug>-dev-loop.md`
6. **Create** a private GitHub repo (`<slug>-dev-loop`) to serve as the issue tracker for self-audit findings, and seed it with the five gap-issue labels Phase 9 files against (skipped in update mode)
7. **Record** the skill in `~/my-claude-skills/README.md` for discoverability, if that optional personal catalog repo exists (skipped in update mode, and skipped silently if the catalog isn't present)
7. **Record** the skill in a personal skills catalog for discoverability, if you have one and have pointed `$CLAUDE_SKILLS_CATALOG` at it (skipped in update mode, and skipped silently when that variable is unset — there is no default path)

The generated skill drives a 10-phase loop: triage → work selection → implementation → PR → review → address comments → doc check → merge → self-audit → repeat.

Expand Down Expand Up @@ -68,6 +68,23 @@ Each generated `<slug>-dev-loop` skill encodes:
- **Code patterns** — conventions from `CLAUDE.md` encoded directly into Phase 3
- **Do-not-auto-merge paths** — files that require human review before merge (e.g. `plugin.yml`, `pom.xml`), in addition to universal entries like `.github/workflows/*`

## Running it across many repos

A generated skill is invoked interactively, one repo at a time, as
`/<slug>-dev-loop`. To run them unattended across a whole fleet of repos —
on a schedule, with safety gating and a merge allow-list — see
[`gardener`](https://github.com/dmccoystephenson/gardener), this project's
open-source companion. `gardener tend` dispatches a repo's generated skill
headlessly and bootstraps one via `/create-dev-loop` if the repo doesn't
have one yet; `gardener overnight` does that across an opt-in list of repos
within a time budget.

The split is deliberate: create-dev-loop decides *what a skill knows about
its repo*, gardener decides *when and how safely one gets to run*. Neither
requires the other — skills generated here work standalone as slash
commands, and gardener is useful for any repo that has a dev-loop skill
however it was authored.

## Research basis

Design decisions for this project are grounded in empirical research on autonomous coding agents — PR-size effects on agent success, self-critique failure modes, context rot in long-horizon runs, and related findings. See [`RESEARCH.md`](RESEARCH.md) for citations and the implications for each phase.
Expand Down
6 changes: 3 additions & 3 deletions RESEARCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Last reviewed: 2026-07-27.
- PR #34 (Structure Phase 9 self-audit prompts and label gap issues): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Extends the rubric-over-free-form pattern from Phase 4 (self-review) to Phase 9 (self-audit), with a 6-item structured checklist and a 5-label taxonomy for filed gap issues.
- PR #54 (Apply consolidated template-rule backlog, #44/#46/#47): shipped 2026-06-14. Observed effect: pending — needs N cycles of data. Sharpens the external-anchor definition (#44: UNVERIFIED handling when the anchor can't run; #46: green CI is not verification when CI's scope excludes the changed files) and grounds the Tests-fix rubric item empirically (#47: stash-and-run FAIL→PASS, not scored from reasoning alone) — directly targeting the self-preference failure mode this finding's evidence documents for judgment-based rubric scoring.
- PR #57 (Fix documentation drift across the three sources of truth): shipped 2026-07-25. Observed effect: pending — needs N cycles of data. Restores `{{EXTERNAL_SIGNAL_LABEL}}` in the Phase 1 carried-over-PR bullet, where PR #54 had reintroduced a hardcoded "CI must be green first" after PR #28 generalized the anchor. An anchor named `CI` in a repo that has no CI is an anchor the loop cannot check, which degrades the self-review to exactly the unanchored critique this finding shows is neutral-to-harmful.
- PR #67 (Add Tests-fix CI-revert fallback ladder): shipped 2026-07-26. Observed effect: pending — needs N cycles of data. Closes a gap left by PR #54: the Tests-fix rubric item required a local `git stash` experiment with no substitute when the local anchor can't run, even though the same PR's UNVERIFIED handling fully anticipates that case. Without a defined fallback, Tests-fix silently degraded back to unanchored judgment on exactly the repos this finding is about (confirmed twice on `medieval-factions-dev-loop`, PRs #1976 and #1980).
- PR #67 (Add Tests-fix CI-revert fallback ladder): shipped 2026-07-26. Observed effect: pending — needs N cycles of data. Closes a gap left by PR #54: the Tests-fix rubric item required a local `git stash` experiment with no substitute when the local anchor can't run, even though the same PR's UNVERIFIED handling fully anticipates that case. Without a defined fallback, Tests-fix silently degraded back to unanchored judgment on exactly the repos this finding is about confirmed twice, in two separate PRs opened by a generated child skill running against a repo whose local test anchor could not be run.

---

Expand Down Expand Up @@ -194,10 +194,10 @@ Last reviewed: 2026-07-27.
**Implication for create-dev-loop.**
- The existing retrofit checklist in `CLAUDE.md` (promote rule to template *and* retrofit existing child skills) is empirically justified — forward-only template fixes leave existing skills silently lagging.
- Consider adding a "skill version + tested-against-model" header to generated skills, so a future self-audit can distinguish drift from a fresh bug.
- Consider periodic regression checks: dry-run the generated skill against a known-good fixture repo and compare output to a recorded baseline. The maintainer's private `dpm-dev-loop` / `herald-dev-loop` repos serve as fixtures internally; external contributors can substitute any repo they maintain.
- Consider periodic regression checks: dry-run the generated skill against a known-good fixture repo and compare output to a recorded baseline. Any repo you maintain works as the fixture — the value is in the recorded baseline and the diff against it, not in which repo produced it.

**Implementations.**
- PR #33 (Template-version header for drift detection): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds `<!-- template-version: SHA -->` and `<!-- generated-at: ISO-timestamp -->` HTML comments to the top of every newly generated skill, plus `{{TEMPLATE_VERSION}}` and `{{GENERATED_AT}}` rows to the Step 4 substitution table. Implements the second bullet of this finding's implications; the periodic regression-check (third bullet) is a separate future change. The drift-detection step in `cdl-dev-loop` Phase 9 is also deferred — it belongs in the cdl-dev-loop repo, not here.
- PR #33 (Template-version header for drift detection): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds `<!-- template-version: SHA -->` and `<!-- generated-at: ISO-timestamp -->` HTML comments to the top of every newly generated skill, plus `{{TEMPLATE_VERSION}}` and `{{GENERATED_AT}}` rows to the Step 4 substitution table. Implements the second bullet of this finding's implications; the periodic regression-check (third bullet) is a separate future change. Acting on a detected drift is also deferred — that step belongs in this repo's own generated dev-loop skill (its Phase 9 self-audit), not in the template here.
- PR #35 (Update mode — re-derive existing skills from current template): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Adds an `update` mode to `/create-dev-loop` so the retrofit work that CLAUDE.md's promotion checklist currently mandates can be driven by the skill itself rather than by hand. Complements PR #33's drift detection — #33 detects drift; PR #35 is the remediation channel.

---
Expand Down
12 changes: 7 additions & 5 deletions create-dev-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -667,15 +667,15 @@ done

**Skip this step if `MODE=update`** — the entry already exists from the initial generation, if one was made. (Run only when `MODE=fresh` or `MODE=overwrite`.)

`~/my-claude-skills` is an optional personal catalog convention, not something every user has. Check first:
Some users keep a personal catalog repo listing every skill they have and where it lives. This is entirely opt-in: it is configured by pointing `$CLAUDE_SKILLS_CATALOG` at that repo's working directory, and there is no default path — a hardcoded one would mean writing into a directory the user never asked this skill to touch. Check first:

```bash
[ -d ~/my-claude-skills/.git ] && echo present || echo absent
[ -n "$CLAUDE_SKILLS_CATALOG" ] && [ -d "$CLAUDE_SKILLS_CATALOG/.git" ] && echo present || echo absent
```

**If absent**, skip this step silently — it is not required for the generated skill to work (Step 5 already registered the slash command). Do not create the directory or repo on the user's behalf.
**If absent** (unset, or set to something that isn't a git repo), skip this step silently — it is not required for the generated skill to work (Step 5 already registered the slash command). Do not create the directory or repo on the user's behalf, and do not guess at a catalog location.

**If present**, open `~/my-claude-skills/README.md` and append a new row to the skills table using the GitHub repo created in Step 6 (`OWNER=$(gh api user -q .login)`, same as Step 6):
**If present**, open `$CLAUDE_SKILLS_CATALOG/README.md` and append a new row to its skills table using the GitHub repo created in Step 6 (`OWNER=$(gh api user -q .login)`, same as Step 6). Match the existing table's column order rather than assuming this one:

```
| <slug>-dev-loop | `/<slug>-dev-loop` | [$OWNER/<slug>-dev-loop](https://github.com/$OWNER/<slug>-dev-loop) | Autonomous dev loop for {{PROJECT_NAME}} |
Expand All @@ -684,8 +684,10 @@ done
Then commit and push:

```bash
cd ~/my-claude-skills
cd "$CLAUDE_SKILLS_CATALOG"
git add README.md
git commit -m "Add <slug>-dev-loop skill"
git push
```

If the catalog has no README table to append to, skip rather than inventing one.
Loading