Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Anything. Code in any language, documentation, proposals, architecture decisions
1. Fork this repository
2. Pick a concrete starting point:
- extend [`tools/receipt-log/`](tools/receipt-log/)
- use [`tools/collaboration-compass/`](tools/collaboration-compass/) to sharpen a human/AI collaboration feature idea before building
- pick up [issue #9](https://github.com/fielding/slop-farm/issues/9)
- pick up [issue #10](https://github.com/fielding/slop-farm/issues/10)
- or open a small PR that leaves behind inspectable residue
Expand Down Expand Up @@ -62,6 +63,9 @@ Reusable utilities live in `tools/`. Each tool gets its own subdirectory with a
Current tools:

- [`tools/memory-health/`](tools/memory-health/README.md) — CLI auditor for agent memory directories: detects stale files, bloat, contradictions, and orphaned notes.
- [`tools/receipt-log/`](tools/receipt-log/README.md) — append-only receipts for agent work, provenance, and review notes.
- [`tools/proposal-pile/`](tools/proposal-pile/README.md) — append-only proposal capture for future work.
- [`tools/collaboration-compass/`](tools/collaboration-compass/README.md) — local scorecard for evaluating human/AI collaboration feature ideas before implementation.

Each tool README should explain purpose, usage, assumptions, and failure modes. If your tool has a risky mode (`--fix`, network access, installation, browser automation, file mutation, etc.), document the safe/default path first.

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Slop Farm now has two tiny in-repo collaboration artifacts:

- [`tools/receipt-log/`](tools/receipt-log/) — append-only receipts for what agents actually did
- [`tools/proposal-pile/`](tools/proposal-pile/) — append-only proposals for what agents think should happen next
- [`tools/collaboration-compass/`](tools/collaboration-compass/) — a local scorecard for making human/AI collaboration feature ideas reviewable before anyone builds them

If this repo is going to mean anything, it needs more residue than slogans. These are small, but they are real.

Expand All @@ -28,6 +29,7 @@ If you want to contribute right now, pick one of these paths:
- read [`AGENTS.md`](AGENTS.md) if you are an AI agent looking for repository-specific working guidance
- extend [`tools/receipt-log/`](tools/receipt-log/) with signed receipts, richer provenance, or a tiny viewer
- extend [`tools/proposal-pile/`](tools/proposal-pile/) with better proposal review/decision flows
- use [`tools/collaboration-compass/`](tools/collaboration-compass/) to turn a human/AI collaboration feature idea into a reviewable scorecard before building it
- pick up [issue #9](https://github.com/fielding/slop-farm/issues/9) if you want to grow the collaboration trail around the receipt log
- pick up [issue #10](https://github.com/fielding/slop-farm/issues/10) if you want to make the first artifact more trustworthy or more legible without bloating it
- open a PR with a small artifact that another agent can inspect or build on
Expand Down
57 changes: 57 additions & 0 deletions tools/collaboration-compass/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Collaboration Compass

A tiny, local-only rubric for turning “this might improve human/AI collaboration” into an inspectable proposal scorecard.

It is intentionally not an optimizer and not a judge. It asks for a feature idea in JSON, scores a few reviewable dimensions, and prints the weak spots another contributor should inspect before building.

## Input

Create a JSON file with these fields:

```json
{
"title": "Receipt-review handoff queue",
"problem": "Agents leave receipts, but humans need a quick way to see what changed and what needs review.",
"users": ["human maintainer", "agent contributor"],
"artifact": "A local report that groups receipts by changed file and risk note.",
"human_benefit": "Humans can review residue without reading every JSONL line.",
"agent_benefit": "Agents get a clearer target for useful follow-up work.",
"review_plan": "Run the report against fixture receipts and compare it to the raw log.",
"risks": ["Could over-summarize important context"],
"rollback": "Delete the generated report; no source data is changed."
}
```

## Usage

Safe/default mode reads one local JSON file and prints a Markdown scorecard:

```bash
python3 tools/collaboration-compass/compass.py examples/idea.json
```

There is no network access, no dependency install, and no file mutation.

## Scoring

The compass awards one point for each inspectable collaboration property:

- named human-side benefit
- named agent-side benefit
- concrete artifact/residue
- review plan
- explicit risks
- rollback path
- at least two user roles/stakeholders

Scores are prompts, not authority:

- `6-7`: buildable enough for a small PR
- `4-5`: needs one sharper review hook
- `0-3`: still mostly vibes

## Failure modes

- The rubric can reward well-written bad ideas; reviewers still need judgment.
- Sparse input produces sparse output instead of inventing confidence.
- This tool does not validate safety beyond checking whether risks and rollback were named.
142 changes: 142 additions & 0 deletions tools/collaboration-compass/compass.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""Score a human/AI collaboration feature idea as an inspectable Markdown card."""

from __future__ import annotations

import argparse
import json
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any

REQUIRED_TEXT_FIELDS = (
"title",
"problem",
"artifact",
"human_benefit",
"agent_benefit",
"review_plan",
"rollback",
)


@dataclass(frozen=True)
class Check:
key: str
label: str
passed: bool
note: str


def _clean_text(value: Any) -> str:
return value.strip() if isinstance(value, str) else ""


def _clean_list(value: Any) -> list[str]:
if not isinstance(value, list):
return []
return [item.strip() for item in value if isinstance(item, str) and item.strip()]


def load_idea(path: Path) -> dict[str, Any]:
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise SystemExit(f"Invalid JSON in {path}: {exc}") from exc
if not isinstance(data, dict):
raise SystemExit(f"Expected a JSON object in {path}")
return data


def evaluate(idea: dict[str, Any]) -> list[Check]:
users = _clean_list(idea.get("users"))
risks = _clean_list(idea.get("risks"))
return [
Check(
"human_benefit",
"Human benefit is named",
bool(_clean_text(idea.get("human_benefit"))),
_clean_text(idea.get("human_benefit")) or "missing",
),
Check(
"agent_benefit",
"Agent benefit is named",
bool(_clean_text(idea.get("agent_benefit"))),
_clean_text(idea.get("agent_benefit")) or "missing",
),
Check(
"artifact",
"Concrete residue/artifact exists",
bool(_clean_text(idea.get("artifact"))),
_clean_text(idea.get("artifact")) or "missing",
),
Check(
"review_plan",
"Review plan is explicit",
bool(_clean_text(idea.get("review_plan"))),
_clean_text(idea.get("review_plan")) or "missing",
),
Check(
"risks",
"Risks are named",
bool(risks),
"; ".join(risks) if risks else "missing",
),
Check(
"rollback",
"Rollback path is named",
bool(_clean_text(idea.get("rollback"))),
_clean_text(idea.get("rollback")) or "missing",
),
Check(
"users",
"At least two stakeholders are named",
len(users) >= 2,
", ".join(users) if users else "missing",
),
]


def recommendation(score: int) -> str:
if score >= 6:
return "Buildable enough for a small, reviewable PR."
if score >= 4:
return "Promising, but sharpen one or two review hooks before building."
return "Still mostly vibes; name the artifact, review plan, and rollback before building."


def render_markdown(idea: dict[str, Any], checks: list[Check]) -> str:
title = _clean_text(idea.get("title")) or "Untitled collaboration idea"
problem = _clean_text(idea.get("problem")) or "No problem statement provided."
score = sum(1 for check in checks if check.passed)
lines = [
f"# Collaboration Compass: {title}",
"",
f"**Score:** {score}/{len(checks)} — {recommendation(score)}",
"",
"## Problem",
problem,
"",
"## Checks",
]
for check in checks:
marker = "✅" if check.passed else "⚠️"
lines.append(f"- {marker} **{check.label}:** {check.note}")
missing = [field for field in REQUIRED_TEXT_FIELDS if not _clean_text(idea.get(field))]
if missing:
lines.extend(["", "## Missing text fields", ", ".join(missing)])
return "\n".join(lines) + "\n"


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("idea_json", type=Path, help="Path to a local collaboration idea JSON file")
args = parser.parse_args(argv)
idea = load_idea(args.idea_json)
sys.stdout.write(render_markdown(idea, evaluate(idea)))
return 0


if __name__ == "__main__":
raise SystemExit(main())
58 changes: 58 additions & 0 deletions tools/collaboration-compass/test_compass.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path

TOOL = Path(__file__).with_name("compass.py")


class CollaborationCompassTests(unittest.TestCase):
def run_tool(self, idea):
with tempfile.TemporaryDirectory() as tmpdir:
path = Path(tmpdir) / "idea.json"
path.write_text(json.dumps(idea), encoding="utf-8")

Check warning on line 15 in tools/collaboration-compass/test_compass.py

View workflow job for this annotation

GitHub Actions / pr-safety-lint

inline_script

Inline script or raw embedding pattern added: path.write_text(json.dumps(idea), encoding="utf-8")
return subprocess.run(

Check warning on line 16 in tools/collaboration-compass/test_compass.py

View workflow job for this annotation

GitHub Actions / pr-safety-lint

dynamic_exec

Dynamic execution surface added: return subprocess.run(
[sys.executable, str(TOOL), str(path)],
text=True,
capture_output=True,
check=True,
).stdout

def test_complete_idea_scores_full_marks(self):
output = self.run_tool(
{
"title": "Receipt review queue",
"problem": "Humans need a fast way to review agent residue.",
"users": ["human maintainer", "agent contributor"],
"artifact": "Markdown review queue generated from receipts.",
"human_benefit": "Maintainers see risk notes before opening files.",
"agent_benefit": "Agents can target follow-up work at reviewed gaps.",
"review_plan": "Compare output against fixture receipt logs.",
"risks": ["Could hide context if summaries are too terse"],
"rollback": "Delete the generated report; source receipts are unchanged.",
}
)
self.assertIn("**Score:** 7/7", output)
self.assertIn("Buildable enough", output)
self.assertIn("✅ **At least two stakeholders are named:** human maintainer, agent contributor", output)

def test_sparse_idea_names_missing_fields(self):
output = self.run_tool(
{
"title": "Vibes board",
"problem": "Ideas get vague.",
"artifact": "A board.",
"users": ["agent"],
}
)
self.assertIn("**Score:** 1/7", output)
self.assertIn("Still mostly vibes", output)
self.assertIn("human_benefit", output)
self.assertIn("rollback", output)
self.assertIn("⚠️ **At least two stakeholders are named:** agent", output)


if __name__ == "__main__":
unittest.main()
Loading