|
| 1 | +# Pull request review agent |
| 2 | + |
| 3 | +**Status:** Draft |
| 4 | +**Date:** November 2026 |
| 5 | +**Author:** Nic Cope |
| 6 | + |
| 7 | +## Background |
| 8 | + |
| 9 | +Agents write all of Modelplane's code, and most of its review. Each maintainer |
| 10 | +points their own agent at a PR, with their own harness and agent skills. We |
| 11 | +don't have skills of our own for finding correctness bugs, so one shared agent |
| 12 | +could look for them on every PR as well as ours do. For style and taste, each |
| 13 | +agent applies its owner's conventions. Skills kept in the repo would let us |
| 14 | +agree on ours, and give the same feedback whoever reviews. |
| 15 | + |
| 16 | +This doesn't replace maintainer review. It frees maintainers to spend review on |
| 17 | +the insight only they have. |
| 18 | + |
| 19 | +## Proposal |
| 20 | + |
| 21 | +An agent reviews a PR when an org member opens it or marks it ready. To review |
| 22 | +anyone else's PR, or to re-review after new pushes, a maintainer comments |
| 23 | +`/review`. A re-review covers only what `git range-diff` shows changed since |
| 24 | +the last one. |
| 25 | + |
| 26 | +The review agent runs four reviewers with fresh context: design, prose, craft |
| 27 | +and correctness. It checks their findings against the bar in |
| 28 | +`.agents/skills/reviewing-pull-requests` and posts the ones that pass as one |
| 29 | +comment review from a GitHub App. It never approves or requests changes. The |
| 30 | +agents live in `.opencode/agents`, and the skills they load in `.agents/skills`, |
| 31 | +where any maintainer can tune them or run the review locally: |
| 32 | + |
| 33 | +```bash |
| 34 | +opencode run --agent review "Review PR #556 in modelplaneai/modelplane." |
| 35 | +``` |
| 36 | + |
| 37 | +## Security |
| 38 | + |
| 39 | +Every PR comes from a fork, and the workflow runs on `pull_request_target`, the |
| 40 | +trigger that gives a fork's PR access to secrets. The agent then runs the PR's |
| 41 | +code with a shell and network, because it verifies findings by running tests and |
| 42 | +reading upstream source. We assume the PR is malicious. Its code must never get |
| 43 | +the Anthropic key or any credential that can write to the repo. |
| 44 | + |
| 45 | +**The agent runs in a Firecracker microVM.** GitHub's x86 runners expose KVM. |
| 46 | +Escaping the VM takes an exploit in KVM or Firecracker, which is the boundary |
| 47 | +AWS Lambda relies on, rather than in the shared kernel a container would give |
| 48 | +us. |
| 49 | + |
| 50 | +**The only credential in the VM can't do harm.** |
| 51 | + |
| 52 | +- The Anthropic key stays on the runner. A proxy there adds it to the agent's |
| 53 | + model requests, and forwards nothing but `POST /v1/messages`. The key belongs |
| 54 | + to a workspace of its own, with a spend limit. |
| 55 | +- The VM gets the job's GitHub token, so the agent can read issues and PRs with |
| 56 | + `gh`. It can't write anything, everything it can read is public, and it |
| 57 | + expires with the job. |
| 58 | +- The App's key exists only in a second job with no model in it. That job |
| 59 | + validates the review and posts it. |
| 60 | + |
| 61 | +**The VM can only connect to the proxy and the internet.** A firewall on the |
| 62 | +runner drops its connections to the runner's other ports and private network, |
| 63 | +and to the cloud metadata service. It drops all IPv6. It doesn't share files |
| 64 | +with the runner. Its job arrives as a tar on a read-only disk, and it writes the |
| 65 | +review as a tar that the post job reads in memory, never extracting it. |
| 66 | + |
| 67 | +**A PR can't change the reviewer's setup.** The VM image, the workflow, the |
| 68 | +agents, the skills and every other file opencode reads come from `main`. The |
| 69 | +post job escapes any marker the agent writes, so the agent can't set what the |
| 70 | +next re-review covers. |
| 71 | + |
| 72 | +What's left: a malicious PR can spend API budget, and post a misleading comment |
| 73 | +review. The spend limit bounds the first, a maintainer reads the second, and |
| 74 | +non-members get a review only when a maintainer asks for one. PR code running |
| 75 | +in the VM can rewrite the agent's instructions mid-review, so a PR can steer its |
| 76 | +own review. It's still only a comment, so we accept that. |
| 77 | + |
| 78 | +GitHub doesn't officially support KVM on its runners. About one runner in ten |
| 79 | +during testing had none. A small workflow re-runs a review that failed for that |
| 80 | +reason, up to twice. |
| 81 | + |
| 82 | +## Alternatives considered |
| 83 | + |
| 84 | +**opencode's GitHub Action.** The agent posts directly, so it holds a token that |
| 85 | +can write to the repo. |
| 86 | + |
| 87 | +**A container instead of a VM.** A kernel exploit in the container would give it |
| 88 | +the runner, and the key on it. Docker can't run inside an unprivileged container |
| 89 | +either, so the agent couldn't create a kind cluster. |
| 90 | + |
| 91 | +**No shell for the agent.** It wouldn't need a sandbox, but it couldn't verify |
| 92 | +findings. Unverified findings are most of the noise maintainers decline. |
0 commit comments