[Feature] gitagent diff -- semantic diff between agent versions

[shreyas-lyzr]

Summary

Add a gitagent diff command that shows semantic differences between agent versions.

Problem

When an agent evolves across git commits, git diff shows raw text changes. A semantic diff would highlight meaningful changes: new skills added, compliance rules changed, model updated, personality shifts in SOUL.md, etc.

Proposed Solution

# Compare current agent with a previous commit
gitagent diff HEAD~1

# Compare two branches
gitagent diff main..feature/new-skill

# Compare two agent directories
gitagent diff ./agent-v1 ./agent-v2

Output categories:

• Schema changes: fields added/removed/modified in agent.yaml
• Capability changes: new/removed skills, tools, workflows
• Behavioral changes: SOUL.md, RULES.md modifications
• Compliance changes: risk tier, framework, supervision changes

Acceptance Criteria

• gitagent diff command implemented
• Structured output (not just raw text diff)
• JSON output option for programmatic use
• Documentation added

[shreyas-lyzr]

Still a gap. Apologies for the delay — and refreshing the framing here, since the project has matured since this was filed.

The semantic-diff value proposition is now clearer because of how agent.yaml, SOUL.md, RULES.md, DUTIES.md, skills/, and compliance/ decompose. A useful gapman diff should report at this granularity:

gapman diff v1.0.0..v1.1.0

Identity (SOUL.md):       unchanged
Rules (RULES.md):         +2 added (no spending without approval; require PII redaction), 0 removed
Duties (DUTIES.md):       conflict pair added: [analyst, reviewer]
Skills:                   +1 added (compliance-checker), 0 removed, 1 modified (code-review v1.2 → v1.3)
Tools:                    +1 added (mcp-policy-search), 0 removed
Compliance.risk_tier:     medium → high             ⚠ tier escalation
Compliance.rules:         +1 added (FINRA 4511)
Hooks:                    +1 added (pre_tool_use: spending-cap, fail_open=false)  ⚠ enforcement added
Memory:                   MEMORY.md changed (cross-session memory edit)            ⚠ review needed

The ⚠ markers identify changes that compliance reviewers should look at. This is the value-add over git diff — a compliance reviewer or a release manager can read the output in one screen.

Implementation sketch:

• New skill: src/commands/diff.ts
• Compares two refs (commits, tags, dirs) by loading each as a GAPAgent tuple via loadAgentManifest + sibling files
• Diffs each component (identity, rules, duties, skills, tools, compliance, hooks, memory)
• Prefixes with ⚠ markers for changes that escalate risk_tier, add enforcement hooks, or modify memory

Anyone want to take this? good first issue for someone with TS familiarity. Real apologies for the long wait.

[shreyas-lyzr]

Useful command. A few thoughts on the semantic layer:

What counts as semantic — a raw git diff on agent.yaml is already readable YAML. The value of gitagent diff is in surfacing meaning: "2 skills added", "compliance.supervision changed from human_approval to automated", "max_turns increased from 10 to 50", "SOUL.md content changed substantially". The output should be a structured summary, not a re-wrapped YAML diff.

SOUL.md diff — personality/identity changes in SOUL.md are hard to summarize programmatically. Consider a simple line-count and a first-changed-line summary rather than trying to do NLP over the diff.

Breaking change detection — if a tool is removed or renamed between versions, that is a breaking change for anything that depends on the agent. gitagent diff should flag tool removals explicitly (similar to how API diff tools flag removed endpoints).

Output format — --output json would allow downstream tooling (dashboards, audit logs, PR bots) to consume the diff programmatically. Worth building in from the start rather than retrofitting.

This composes well with issue #59 (agent performance tracking) — a diff between versions is exactly what you need to correlate behavior changes with performance changes.

[shreyas-lyzr]

A semantic diff command would be genuinely useful for agent lifecycle management. The structured nature of agent.yaml makes this tractable — unlike diffing arbitrary code, the schema fields have defined semantics.

Suggested output categories for the diff:

• Identity changes: name, version, description in agent.yaml
• Capability changes: skills added/removed, tools added/removed
• Compliance changes: any field under compliance: in agent.yaml (these warrant special attention)
• Model changes: preferred model, temperature, max_tokens
• Personality changes: SOUL.md diff (could be rendered as a word-level diff)
• Delegation changes: added/removed sub-agents, changed delegation mode

For the SOUL.md diff specifically, a word-level diff (similar to git diff --word-diff) would be more readable than a line diff since SOUL.md tends to be flowing prose.

Assigning @shreyas-lyzr to prioritize against the roadmap.