I built this because I got tired of AI saying "done" when it wasn't.
You know the drill: you ask an AI coding assistant to build something, it says "all tests pass," and then you find out it never actually ran them. Or it ran them once, failed, and "fixed" the test to make it green.
plan-auditor forces the AI to prove its work. Not with words-with actual command output, file hashes, and deterministic checks that can't be faked.
- You describe what you want built
- plan-auditor turns that into explicit, machine-checkable requirements
- The AI implements each step and must provide real evidence (command output, file contents, test results)
- A separate auditor process verifies everything independently
- Only when every check passes does it say "done"
No more "trust me bro" from language models.
- Verify AI coding agents before accepting a claimed completion.
- Require deterministic evidence for tests, builds, lint, type checks, generated files, and CI gates.
- Audit multi-step implementation plans with sealed requirements and tamper-evident evidence.
- Use STRIPS/PDDL-style planning checks for machine-verifiable task reachability.
- Add an independent software quality assurance layer when the same agent should not grade its own work.
pip install plan-auditor
# or
npx plan-auditor Then just tell your AI: "Use plan-auditor for this task." It handles the rest.
I was working on a scientific computing project and an AI assistant kept insisting it had fixed a bug. It hadn't. Three times. I wasted hours before I realized the "fix" was just the AI editing its own test output.
So I built a system where the AI can't lie. Where "done" means "here's the terminal output proving it." Where the plan is sealed cryptographically and any tampering breaks the verification.
It's not perfect. But it's a hell of a lot better than taking an LLM's word for anything.
- Architecture - how the pieces fit together
- Formal Planning - the STRIPS/PDDL verification stuff
- Threat Model - what it can and can't protect against
MIT. Do whatever you want with it.