diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e26670f..d6ddfcb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,6 +17,7 @@ Anything. Code in any language, documentation, proposals, architecture decisions 1. Fork this repository 2. Pick a concrete starting point: - extend [`tools/receipt-log/`](tools/receipt-log/) + - use [`tools/collaboration-compass/`](tools/collaboration-compass/) to sharpen a human/AI collaboration feature idea before building - pick up [issue #9](https://github.com/fielding/slop-farm/issues/9) - pick up [issue #10](https://github.com/fielding/slop-farm/issues/10) - or open a small PR that leaves behind inspectable residue @@ -62,6 +63,9 @@ Reusable utilities live in `tools/`. Each tool gets its own subdirectory with a Current tools: - [`tools/memory-health/`](tools/memory-health/README.md) — CLI auditor for agent memory directories: detects stale files, bloat, contradictions, and orphaned notes. +- [`tools/receipt-log/`](tools/receipt-log/README.md) — append-only receipts for agent work, provenance, and review notes. +- [`tools/proposal-pile/`](tools/proposal-pile/README.md) — append-only proposal capture for future work. +- [`tools/collaboration-compass/`](tools/collaboration-compass/README.md) — local scorecard for evaluating human/AI collaboration feature ideas before implementation. Each tool README should explain purpose, usage, assumptions, and failure modes. If your tool has a risky mode (`--fix`, network access, installation, browser automation, file mutation, etc.), document the safe/default path first. diff --git a/README.md b/README.md index 9893853..fd07a73 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ Slop Farm now has two tiny in-repo collaboration artifacts: - [`tools/receipt-log/`](tools/receipt-log/) — append-only receipts for what agents actually did - [`tools/proposal-pile/`](tools/proposal-pile/) — append-only proposals for what agents think should happen next +- [`tools/collaboration-compass/`](tools/collaboration-compass/) — a local scorecard for making human/AI collaboration feature ideas reviewable before anyone builds them If this repo is going to mean anything, it needs more residue than slogans. These are small, but they are real. @@ -28,6 +29,7 @@ If you want to contribute right now, pick one of these paths: - read [`AGENTS.md`](AGENTS.md) if you are an AI agent looking for repository-specific working guidance - extend [`tools/receipt-log/`](tools/receipt-log/) with signed receipts, richer provenance, or a tiny viewer - extend [`tools/proposal-pile/`](tools/proposal-pile/) with better proposal review/decision flows +- use [`tools/collaboration-compass/`](tools/collaboration-compass/) to turn a human/AI collaboration feature idea into a reviewable scorecard before building it - pick up [issue #9](https://github.com/fielding/slop-farm/issues/9) if you want to grow the collaboration trail around the receipt log - pick up [issue #10](https://github.com/fielding/slop-farm/issues/10) if you want to make the first artifact more trustworthy or more legible without bloating it - open a PR with a small artifact that another agent can inspect or build on diff --git a/tools/collaboration-compass/README.md b/tools/collaboration-compass/README.md new file mode 100644 index 0000000..ea9442b --- /dev/null +++ b/tools/collaboration-compass/README.md @@ -0,0 +1,57 @@ +# Collaboration Compass + +A tiny, local-only rubric for turning “this might improve human/AI collaboration” into an inspectable proposal scorecard. + +It is intentionally not an optimizer and not a judge. It asks for a feature idea in JSON, scores a few reviewable dimensions, and prints the weak spots another contributor should inspect before building. + +## Input + +Create a JSON file with these fields: + +```json +{ + "title": "Receipt-review handoff queue", + "problem": "Agents leave receipts, but humans need a quick way to see what changed and what needs review.", + "users": ["human maintainer", "agent contributor"], + "artifact": "A local report that groups receipts by changed file and risk note.", + "human_benefit": "Humans can review residue without reading every JSONL line.", + "agent_benefit": "Agents get a clearer target for useful follow-up work.", + "review_plan": "Run the report against fixture receipts and compare it to the raw log.", + "risks": ["Could over-summarize important context"], + "rollback": "Delete the generated report; no source data is changed." +} +``` + +## Usage + +Safe/default mode reads one local JSON file and prints a Markdown scorecard: + +```bash +python3 tools/collaboration-compass/compass.py examples/idea.json +``` + +There is no network access, no dependency install, and no file mutation. + +## Scoring + +The compass awards one point for each inspectable collaboration property: + +- named human-side benefit +- named agent-side benefit +- concrete artifact/residue +- review plan +- explicit risks +- rollback path +- at least two user roles/stakeholders + +Scores are prompts, not authority: + +- `6-7`: buildable enough for a small PR +- `4-5`: needs one sharper review hook +- `0-3`: still mostly vibes + +## Failure modes + +- The rubric can reward well-written bad ideas; reviewers still need judgment. +- Sparse input produces sparse output instead of inventing confidence. +- This tool does not validate safety beyond checking whether risks and rollback were named. diff --git a/tools/collaboration-compass/compass.py b/tools/collaboration-compass/compass.py new file mode 100644 index 0000000..fff6eba --- /dev/null +++ b/tools/collaboration-compass/compass.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 +"""Score a human/AI collaboration feature idea as an inspectable Markdown card.""" + +from __future__ import annotations + +import argparse +import json +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +REQUIRED_TEXT_FIELDS = ( + "title", + "problem", + "artifact", + "human_benefit", + "agent_benefit", + "review_plan", + "rollback", +) + + +@dataclass(frozen=True) +class Check: + key: str + label: str + passed: bool + note: str + + +def _clean_text(value: Any) -> str: + return value.strip() if isinstance(value, str) else "" + + +def _clean_list(value: Any) -> list[str]: + if not isinstance(value, list): + return [] + return [item.strip() for item in value if isinstance(item, str) and item.strip()] + + +def load_idea(path: Path) -> dict[str, Any]: + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise SystemExit(f"Invalid JSON in {path}: {exc}") from exc + if not isinstance(data, dict): + raise SystemExit(f"Expected a JSON object in {path}") + return data + + +def evaluate(idea: dict[str, Any]) -> list[Check]: + users = _clean_list(idea.get("users")) + risks = _clean_list(idea.get("risks")) + return [ + Check( + "human_benefit", + "Human benefit is named", + bool(_clean_text(idea.get("human_benefit"))), + _clean_text(idea.get("human_benefit")) or "missing", + ), + Check( + "agent_benefit", + "Agent benefit is named", + bool(_clean_text(idea.get("agent_benefit"))), + _clean_text(idea.get("agent_benefit")) or "missing", + ), + Check( + "artifact", + "Concrete residue/artifact exists", + bool(_clean_text(idea.get("artifact"))), + _clean_text(idea.get("artifact")) or "missing", + ), + Check( + "review_plan", + "Review plan is explicit", + bool(_clean_text(idea.get("review_plan"))), + _clean_text(idea.get("review_plan")) or "missing", + ), + Check( + "risks", + "Risks are named", + bool(risks), + "; ".join(risks) if risks else "missing", + ), + Check( + "rollback", + "Rollback path is named", + bool(_clean_text(idea.get("rollback"))), + _clean_text(idea.get("rollback")) or "missing", + ), + Check( + "users", + "At least two stakeholders are named", + len(users) >= 2, + ", ".join(users) if users else "missing", + ), + ] + + +def recommendation(score: int) -> str: + if score >= 6: + return "Buildable enough for a small, reviewable PR." + if score >= 4: + return "Promising, but sharpen one or two review hooks before building." + return "Still mostly vibes; name the artifact, review plan, and rollback before building." + + +def render_markdown(idea: dict[str, Any], checks: list[Check]) -> str: + title = _clean_text(idea.get("title")) or "Untitled collaboration idea" + problem = _clean_text(idea.get("problem")) or "No problem statement provided." + score = sum(1 for check in checks if check.passed) + lines = [ + f"# Collaboration Compass: {title}", + "", + f"**Score:** {score}/{len(checks)} — {recommendation(score)}", + "", + "## Problem", + problem, + "", + "## Checks", + ] + for check in checks: + marker = "✅" if check.passed else "⚠️" + lines.append(f"- {marker} **{check.label}:** {check.note}") + missing = [field for field in REQUIRED_TEXT_FIELDS if not _clean_text(idea.get(field))] + if missing: + lines.extend(["", "## Missing text fields", ", ".join(missing)]) + return "\n".join(lines) + "\n" + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("idea_json", type=Path, help="Path to a local collaboration idea JSON file") + args = parser.parse_args(argv) + idea = load_idea(args.idea_json) + sys.stdout.write(render_markdown(idea, evaluate(idea))) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/collaboration-compass/test_compass.py b/tools/collaboration-compass/test_compass.py new file mode 100644 index 0000000..b9d3a26 --- /dev/null +++ b/tools/collaboration-compass/test_compass.py @@ -0,0 +1,58 @@ +import json +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +TOOL = Path(__file__).with_name("compass.py") + + +class CollaborationCompassTests(unittest.TestCase): + def run_tool(self, idea): + with tempfile.TemporaryDirectory() as tmpdir: + path = Path(tmpdir) / "idea.json" + path.write_text(json.dumps(idea), encoding="utf-8") + return subprocess.run( + [sys.executable, str(TOOL), str(path)], + text=True, + capture_output=True, + check=True, + ).stdout + + def test_complete_idea_scores_full_marks(self): + output = self.run_tool( + { + "title": "Receipt review queue", + "problem": "Humans need a fast way to review agent residue.", + "users": ["human maintainer", "agent contributor"], + "artifact": "Markdown review queue generated from receipts.", + "human_benefit": "Maintainers see risk notes before opening files.", + "agent_benefit": "Agents can target follow-up work at reviewed gaps.", + "review_plan": "Compare output against fixture receipt logs.", + "risks": ["Could hide context if summaries are too terse"], + "rollback": "Delete the generated report; source receipts are unchanged.", + } + ) + self.assertIn("**Score:** 7/7", output) + self.assertIn("Buildable enough", output) + self.assertIn("✅ **At least two stakeholders are named:** human maintainer, agent contributor", output) + + def test_sparse_idea_names_missing_fields(self): + output = self.run_tool( + { + "title": "Vibes board", + "problem": "Ideas get vague.", + "artifact": "A board.", + "users": ["agent"], + } + ) + self.assertIn("**Score:** 1/7", output) + self.assertIn("Still mostly vibes", output) + self.assertIn("human_benefit", output) + self.assertIn("rollback", output) + self.assertIn("⚠️ **At least two stakeholders are named:** agent", output) + + +if __name__ == "__main__": + unittest.main()