Skip to content

feat(typesafe-triage): add dry-run label triage with fixture replay - #22

Open
glifocat wants to merge 3 commits into
mainfrom
feat/typesafe-triage-dry-run
Open

glifocat wants to merge 3 commits into
mainfrom
feat/typesafe-triage-dry-run

Conversation

@glifocat

Copy link
Copy Markdown
Collaborator

What changes?

Adds a typesafe-triage skill: an offline-testable, dry-run label triage for nanocoai/nanoclaw issues and pull requests using the TypeSafe System One decision API (POST /v1/systemone, model jev-latest).

  • Fetches the 30 most recently updated open issues and 20 open PRs read-only via gh api (title, body, author association, labels; PR changed files and nanoclaw-pr-template:v2 section status). The issues endpoint is paged until enough real issues arrive, since it interleaves PRs.
  • One fan-out request per item: area (choice, 16 area/* labels with rubrics from CLAUDE.md, docs and .github/labeler.yml), kind (choice, 7 kind/*), priority (score, 4 levels), plus needs_repro (issues) or pr_ready (PRs). Existing labels are never sent to the model.
  • Confidence gate (area/kind 0.6, priority 0.8, yes/no 0.7; all flags) proposes triage/unresolved below the gate instead of guessing. Prints a per-proposal table with confidence, existing labels and AGREE / DISAGREE / NEW, then a summary with agreement rate per question, gated-item count, token usage and wall time.
  • Raw answers saved under gitignored skills/typesafe-triage/output/. --fixture replays recorded items and responses offline (shipped fixture: 5 real items fetched read-only on 2026-09-16 with hand-written answers); --record captures a live run in the same format.
  • TYPESAFE_API_KEY is read from the environment only, rejected if it contains whitespace, and redacted from error text. No SDK dependency (standard library only). Nothing is ever written to GitHub.
  • Plugin version 0.12.0; README, skills catalog, CHANGELOG and a doc page (docs/typesafe-triage.md) updated. No GitHub Action added.

Validation

Offline only; no live TypeSafe request has been made from this repository.

  • python -m unittest discover -s tests in a venv with the hash-pinned wizard requirements -> 278 tests OK (239 existing + 39 new in tests/test_typesafe_triage.py: question/request shape, template detection, gate and verdict logic, paginated gh parsing with a mocked runner, HTTP transport with a mocked opener (bearer header, 429/529 retry, no 422 retry, key redaction), partial-failure handling, fixture replay end to end, key never in stdout/stderr/saved JSON).
  • python3 skills/typesafe-triage/scripts/typesafe-triage.py --fixture skills/typesafe-triage/fixtures/nanoclaw-sample.json -> table for 5 items, area agreement 100% (3 compared), 5 items with a gated question, 10,960 input tokens recorded.
  • git diff --check, bash -n on all skill scripts, plugin manifests parse as JSON.
  • Codex adversarial review (uncommitted diff): 7 findings, all fixed (key could reach a traceback, PR-dominated issue page, lost results on partial failure, late-bound opener for the offline guard, nested headings in template sections, file-count cap, output-name collision).
  • Docs: links in docs/typesafe-triage.md and skills/typesafe-triage/SKILL.md point at existing paths; the gh label list taxonomy was re-read on 2026-09-16.
  • Live dry run with TYPESAFE_API_KEY set is still to do by the operator.

Add a typesafe-triage skill that fetches recently updated open NanoClaw
issues and PRs read-only through gh, sends one TypeSafe System One fan-out
request per item (area, kind, priority, needs_repro or pr_ready), gates on
confidence with triage/unresolved below the gate, and prints proposals next
to existing labels with AGREE/DISAGREE/NEW verdicts. Raw answers are saved
under a gitignored directory; --fixture replays recorded responses offline
and --record captures a live run in the same format.

The key is read from TYPESAFE_API_KEY only, rejected if it contains
whitespace, and redacted from error text. Nothing is written to GitHub.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant