Skip to content

feat(capture+review): scrub personal/proprietary identifiers from screenshots - #2

Draft
nicolasiscoding wants to merge 1 commit into
mainfrom
feature/scrub-identifiers-before-capture
Draft

feat(capture+review): scrub personal/proprietary identifiers from screenshots#2
nicolasiscoding wants to merge 1 commit into
mainfrom
feature/scrub-identifiers-before-capture

Conversation

@nicolasiscoding

Copy link
Copy Markdown
Member

What

Makes DOM-scrubbing of personal/proprietary identifiers a built-in part of the capture flow, plus a pre-publish review check. Closes #1.

guidewright-capture

  • New Principle 4 — never show a real account, ID, or org name.
  • Scrub step added to the red-box per-step loop (run last, right before the shot, so a framework re-render can't restore the real text) and to the workflow checklist.
  • New references/scrubbing.md: a parameterized, product-agnostic evaluate_script snippet (login email, org/workspace name, UUID/tenant id -> neutral placeholders like you@example.com / ACME Corp / masked all-zeros), with verify guidance. Placeholders/patterns are configurable per product, not hardcoded.

guidewright-review

  • Pre-publish identifier-leak check in the walk step, the checklist, and lens 8 of the rubric: flag any screenshot/alt-text/page text exposing a real email, raw UUID, or non-generic org/customer name (Blocker/Major; checkable read-only).

Evals

  • capture-scrubs-identifiers — a described flow seeded with a real-shaped email/UUID/workspace name; the authored page must use placeholders and plan the scrub.
  • review-catches-leaked-identifiers — new evals/fixtures/leaky-howto.md with planted leaks; review must flag all three.
  • Bundle now 5 evals; JSON validated. plugin 0.2.0.

Testing note

guidewright's evals are behavioral (one agent-run per eval, LLM-judged). They were authored in the existing evals.json format and dry-run-validated: the edited skills satisfy every new assertion, and the pre-change skills would fail them (so the evals actually guard the new behavior). Full harness run (spawns per-eval agents) should be run by a non-fork session or CI.

All planted identifiers in fixtures are fictional; no real accounts/orgs/ids appear anywhere in the diff.

…eenshots

guidewright-capture: add a scrub-before-screenshot step to the red-box loop +
a 4th principle (never show real accounts/IDs/org names) + references/scrubbing.md
with a parameterized, product-agnostic scrub snippet (email/org name/UUID ->
placeholders), run last so a re-render can't restore the real text.

guidewright-review: pre-publish check that flags any screenshot/alt-text/page
text still carrying a real email, raw UUID, or non-generic org/customer name
(walk step + checklist + lenses rubric).

evals: add capture-scrubs-identifiers and review-catches-leaked-identifiers
(+ leaky-howto fixture). Bump plugin 0.1.0 -> 0.2.0. Closes #1.
@nicolasiscoding

Copy link
Copy Markdown
Member Author

Eval run — all green ✅ (16/16 assertions)

Ran the full behavioral suite for real (per eval: a performer agent follows the edited skill → a strict LLM judge scores each assertion). Not a dry-run.

# Eval Skill Result
0 howto-stays-recipe capture ✅ 3/3
1 conceptual-becomes-explanation capture ✅ 2/2
2 review-catches-planted-issues review ✅ 4/4
3 capture-scrubs-identifiers (new) capture ✅ 3/3
4 review-catches-leaked-identifiers (new) review ✅ 4/4

New behavior confirmed:

  • capture-scrubs-identifiers — a capture following the edited skill replaced the real login email / workspace name / raw UUID with neutral placeholders (you@example.com, ACME Corp, masked all-zeros id) and planned the DOM scrub as the last step before each shot — while still producing a clean action-only how-to.
  • review-catches-leaked-identifiers — the review flagged the planted email, tenant UUID, and internal workspace name as Blocker/Major in a prioritized report.

No regression: the 3 pre-existing evals (recipe discipline, Diátaxis type-fit, planted-issue review) still pass with the edits in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scrub personal/proprietary identifiers (emails, org/tenant IDs) from screenshots before capture

1 participant