Skip to content

Add code-security-fixer: patch real CVEs with a test- and Bandit-gated ADK 2.0 workflow - #2815

Merged
tommywagz merged 14 commits into
google:mainfrom
debanshd:codemender-security-fixer
Nov 11, 2026
Merged

tommywagz merged 14 commits into
google:mainfrom
debanshd:codemender-security-fixer

Conversation

@debanshd

@debanshd debanshd commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2811

What

contrib/python/code-security-fixer is an ADK 2.0 workflow that detects, patches and validates security vulnerabilities. Gemini proposes a fix. A deterministic validator accepts it only if no previously passing test fails, Bandit reports no new finding, and at least one finding or exploit test is resolved. The agent never commits; its output is a unified diff for human review.

Real-world anchor

  • Target: Django's django/utils/archive.py, vendored at the commit before the fix for CVE-2021-3281 (archive path traversal). It's reproduced with Django's own regression cases, plus CVE-2025-59682, the later bypass of that fix. Vendored code keeps Django's BSD-3-Clause licence.
  • Tests: Django's functional extract() tests, plus exploit tests for both CVEs. The reference fixes in unit tests are Django's actual patches.
  • Live run (gemini-3.7-flash): exploit tests 0/3 → 3/3 in one accepted patch, with functional tests still passing. The fix matches Django's 2025 approach (os.path.commonpath).
  • TARGET_DIR and TEST_COMMAND point the agent at your own repo. It's copied to a private working copy and never modified.

How it works

  • Workflow: scan → (has_findings → prepare_patch → patch_agent → validate → continue / done | clean) → report. patch_agent is the only model call.
  • Validator: runs the project's tests and Bandit in a resource-limited subprocess with a minimal environment and no credentials. It rejects edits to tests, real # nosec directives, syntax errors and paths outside the target. Malformed tool output becomes a rejection rather than a crash.
  • Serving: A2A JSON-RPC (0.3 + 1.0) and HTTP+JSON, an Agent Engine adapter, and a Dockerfile.

Checks run locally

  • validate, ruff, mypy (CI command), check_env_vars, check_model_literals, check_recipe_pyproject and uv lock --check: all pass.
  • pytest: 40 passed, with and without a local .env. test_server_e2e.py also passes.
  • osv-scanner.toml ignores GHSA-xcq7-p3pf-p88h with a reason and an expiry. Its fix (opentelemetry-api 1.45.1) is blocked by google-adk 2.13's <=1.44 pin. All other flagged packages are upgraded.

Note

app/django_archive/ intentionally contains vulnerable upstream code as the demo target. Its exploit tests are excluded from the recipe's own pytest run.

A CodeMender-style detect -> patch -> validate agent for AppSec finding
backlogs. Gemini proposes file-level fixes; a deterministic validator runs
Bandit and the target's own exploit/regression tests in a resource-limited
subprocess on a scratch copy and keeps only patches that fix a finding
without regressions. Ships an intentionally vulnerable orders API as the
sample target, TARGET_DIR support, and model-free unit tests.

Refs google#2811
@debanshd
debanshd requested a review from tommywagz as a code owner October 17, 2026 09:48

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated House Rules review — 2 finding(s).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Hygiene review — 4 finding(s).

⚠️ This PR's diff was too large to review in full, so 5 file(s) were not looked at by this lane:

  • contrib/python/codemender-security-fixer/README.md
  • contrib/python/codemender-security-fixer/app/sample_target/tests/test_security.py
  • contrib/python/codemender-security-fixer/tests/eval/evalsets/README.md
  • contrib/python/codemender-security-fixer/tests/integration/test_server_e2e.py
  • contrib/python/codemender-security-fixer/tests/unit/test_tools.py

Splitting the PR up would get them reviewed.


Round 1 · 0 of this PR's 25 automated comments used · this round is capped at 20 across all reviewers.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Maintainability review — 2 finding(s).

⚠️ This PR's diff was too large to review in full, so 4 file(s) were not looked at by this lane:

  • contrib/python/codemender-security-fixer/README.md
  • contrib/python/codemender-security-fixer/app/sample_target/tests/test_security.py
  • contrib/python/codemender-security-fixer/tests/integration/test_server_e2e.py
  • contrib/python/codemender-security-fixer/tests/unit/test_tools.py

Splitting the PR up would get them reviewed.


Round 1 · 0 of this PR's 25 automated comments used · this round is capped at 20 across all reviewers.

@debanshd

Copy link
Copy Markdown
Contributor Author

@pierpaolo28 this implements the recipe proposal you and @kgu-here discussed with me. It's backed by a real-world scenario, per your feedback. I'd appreciate your review when you have a chance.

@github-actions

Copy link
Copy Markdown
Contributor

There is a problem with the Antigravity CLI Correctness PR review. Please check the action logs for details.

- Read config through app/config.py with no in-code defaults; values live
  in .env.example, and tests load it via tests/conftest.py.
- Drop the duplicate load_dotenv() in fast_api_app.py, use the standard
  ALLOW_ORIGINS parsing and port 8080; A2A reads APP_URL/AGENT_VERSION
  from the environment.
- Dockerfile runs as non-root appuser and ships .env.example defaults.
- License header on app_utils/__init__.py, ownership.team "Cloud AI",
  _MAX_DETAIL_CHARS instead of 300, and an assertion in test_cookie_is_ascii.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Security review — 1 finding(s).


Round 2 · 6 of this PR's 25 automated comments used · this round is capped at 3 across all reviewers · after this round only Security and Correctness run.

Call sites read each variable with a direct os.getenv() (no default) and
pass the value to app/config.py, which only validates it. check_env_vars.py
again detects all 16 variables.
Avoid baking a developer's local .env into image layers, and replace the
literal 8080 with SERVER_PORT.

Refs google#2811
The agent is now a google.adk.Workflow: scan -> prepare_patch ->
patch_agent (single_turn Agent with PatchRequest/PatchProposal schemas) ->
validate, looping on routed edges while findings and budget remain, then
report. Deterministic steps are function nodes over ctx.state; unit tests
run the real graph with a scripted patcher. Adds an A2A integration test
(agent card 0.3 shape + message/send) and drops empty optional serving env
vars so the Agent Engine adapter starts.

Refs google#2811
@github-actions github-actions Bot added the status/not-ready-for-review Review requests held until checks pass and bot comments are resolved label Oct 18, 2026
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Ready for review. All checks pass.

Re-requested review from tommywagz.

Narrow optional values before indexing so the Mypy Type Check passes.

Refs google#2811
@github-actions github-actions Bot removed the status/not-ready-for-review Review requests held until checks pass and bot comments are resolved label Oct 18, 2026
- Rename contrib/python/codemender-security-fixer to code-security-fixer
  and drop the CodeMender naming.
- Replace the home-made sample app with Django's django/utils/archive.py at
  6822aa5c (parent of the CVE-2021-3281 fix), vendored byte-identical with
  its BSD-3 licence, Django's functional tests and exploit tests for
  CVE-2021-3281 and its incomplete-fix follow-up CVE-2025-59682.
- Unit tests apply Django's exact historical patches (05413afa, 924a0c09):
  the 2021 patch is accepted as partial, the 2025 patch closes the rest.
- Serve the A2A HTTP+JSON binding (/v1/card, /v1/message:send) next to
  JSON-RPC for the Gemini agent / Agent Registry, and test it.

Refs google#2811
With the HTTP+JSON binding added, the 0.3 card prefers HTTP+JSON and lists
JSON-RPC in additionalInterfaces at the same URL. Assert both are advertised.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated House Rules review — 1 finding(s).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Security review — 1 finding(s).

⚠️ This PR's diff was too large to review in full, so 2 file(s) were not looked at by this lane:

  • contrib/python/code-security-fixer/README.md
  • contrib/python/code-security-fixer/tests/unit/test_tools.py

Splitting the PR up would get them reviewed.


Round 3 · 7 of this PR's 25 automated comments used · this round is capped at 1 across all reviewers · only Security and Correctness still run.

@github-actions github-actions Bot added the status/not-ready-for-review Review requests held until checks pass and bot comments are resolved label Oct 28, 2026
Move to google-adk 2.13.0 with matching google-cloud-aiplatform /
google-cloud-agentplatform 2.6.0 (as main did in google#2827). This removes the
package collision that broke test_web_server_runnability in a fresh
install, and imports AdkApp from vertexai.agent_engines. Bumps certifi to
2026.10.23 (PYSEC-2026-4320) and opentelemetry to 1.44.0, the highest
google-adk 2.13 allows.

Also writes the review patch to a private per-run temp directory and adds
licence headers: Apache for the package markers we wrote, Django's BSD-3
notice for the two vendored upstream files.

Refs google#2811
@debanshd

Copy link
Copy Markdown
Contributor Author

Thanks for updating the branch, @tommywagz. Pushed a fix for the failing checks:

  • Recipe pytest / runnability: moved to google-adk 2.13.0 with matching google-cloud-aiplatform / google-cloud-agentplatform 2.6.0, as main did in fix(brand-search-optimization): upgrade to google-adk 2.13.0 and remove Computer Use alias workarounds #2827. The older pair both installed an agentplatform package, so a fresh install could fail at import.
  • Lockfile scan: certifi is now 2026.10.23, which clears PYSEC-2026-4320. opentelemetry-api stays flagged for GHSA-xcq7-p3pf-p88h (fixed in 1.45.1), because google-adk 2.13 pins opentelemetry-api>=1.39,<=1.44. The lock is on 1.44.0, the highest it allows. Recipes already on main carry the same or older versions (e.g. brand-search-optimization is on 1.41.1); the scan only fails here because these are new lockfiles.
  • pytest (tools + skills + CI scripts): this fails in .github/scripts/tests/test_process_issue_response.py::test_routing_rules_sync_with_codeowners_and_workflow, which this PR doesn't touch. It looks like a CODEOWNERS/routing mismatch on main after chore(codeowners): add genmedia-for-commerce owners #2826.

Locally: validate, ruff, mypy, check_env_vars, check_model_literals and check_recipe_pyproject all pass, and pytest passes with and without a local .env.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated House Rules review — 1 finding(s).

@github-actions github-actions Bot removed the status/not-ready-for-review Review requests held until checks pass and bot comments are resolved label Oct 28, 2026
# Model Configuration
MODEL_NAME=gemini-3.7-flash

# Code to fix. Empty = the bundled, intentionally vulnerable app/sample_target.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this reference app/django_archive instead of app/sample_target?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. Fixed in 390e0a4: the comment now points at app/django_archive (Django's archive.extract() before the CVE-2021-3281 / CVE-2025-59682 fixes).

# Run them locally with: uv run pytest tests/integration
addopts = "--ignore=tests/integration"
# Only collect the recipe's own tests. The bundled sample target under
# app/sample_target/ ships exploit tests that FAIL on purpose until fixed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment still refers to app/sample_target/ rather than app/django_archive

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed in 390e0a4: the testpaths comment now refers to app/django_archive/.

start_time = time.time()
while time.time() - start_time < timeout:
try:
response = requests.get("http://127.0.0.1:8000/docs", timeout=10)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this use f"{BASE_URL}/docs" instead of the hardcoded URL?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, fixed in 390e0a4: wait_for_server now polls f"{BASE_URL}/docs". While running it I also removed a stale /feedback test (that route does not exist in this app) and gave the SSE request a read timeout that fits a full run; test_server_e2e.py passes locally (2 passed).

"app.fast_api_app:app",
"--host",
"0.0.0.0",
"--port",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

port 8000 here differs from SERVER_PORT 8080 in fast_api_app.py and Dockerfile

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed it was inconsistent. Fixed in 390e0a4: the test no longer hard-codes a port at all; it starts uvicorn on an ephemeral port (same as test_a2a.py) and derives BASE_URL and APP_URL from it, so it can't clash with 8080 (SERVER_PORT) or anything else running locally.

"tests": [],
"error": f"test run crashed: {tail[0][:_MAX_DETAIL_CHARS]}",
}
payload = json.loads(run["stdout"][marker + len("@@RESULT@@") :])

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json.loads here is unguarded against JSONDecodeError if output is truncated or malformed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Fixed in 390e0a4: _parse_unittest now catches JSONDecodeError/KeyError/TypeError (and a non-list tests) and returns error: "test harness output could not be parsed", which the validator treats as a rejection instead of crashing. Covered by test_malformed_harness_output_is_an_error_not_a_crash (truncated JSON, wrong shape, missing key, non-list).

return f"new content exceeds {workspace.MAX_FILE_CHARS} characters"
if new_content == old:
return "new content is identical to the current file"
if new_content.count("nosec") > old.count("nosec"):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't count("nosec") also catch legitimate identifiers or comments mentioning "nosec"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it would. Fixed in 390e0a4: the precheck now counts only real Bandit directives, i.e. #\s*nosec inside comment tokens (via tokenize, with a line-based fallback for code that does not tokenize). Identifiers like nosec_count, string literals and prose comments such as # see the nosec policy no longer count. Covered by test_nosec_directives_counts_only_real_markers and test_mentioning_nosec_is_not_a_suppression. (Bandit also runs with --ignore-nosec, so a directive could never hide a finding; the precheck just rejects such patches early with a clear reason.)

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

BASE_URL = "http://127.0.0.1:8000"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this use an ephemeral port like test_a2a.py does to prevent local port conflicts?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, done in 390e0a4: _free_port() binds port 0 like test_a2a.py, and the server, BASE_URL and APP_URL all use it.

def _child_env(cwd: str) -> dict[str, str]:
"""A minimal environment: no credentials or API keys leak to the target."""
return {
"PATH": "/usr/local/bin:/usr/bin:/bin",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if TEST_COMMAND invokes pytest or a venv binary directly, restricted PATH could cause FileNotFoundError?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, it could. Fixed in 390e0a4: the child PATH now starts with the directory of the recipe's interpreter (the venv bin/ under uv run and in the container) before /usr/local/bin:/usr/bin:/bin, so TEST_COMMAND="pytest -rA -q" or another console script installed in the venv resolves. If a command still can't start, run_limited returns a clear cannot run '<cmd>' ... (child PATH: ...) error instead of raising. Covered by test_test_command_resolves_venv_console_scripts and test_missing_test_command_is_reported.

@debanshd debanshd changed the title Add codemender-security-fixer: detect, patch and validate security findings Add code-security-fixer: patch real CVEs with a test- and Bandit-gated ADK 2.0 workflow Nov 10, 2026
- .env.example / pyproject.toml: point the comments at app/django_archive.
- test_server_e2e.py: ephemeral port (as in test_a2a.py) and f"{BASE_URL}/docs";
  drop the stale /feedback test (no such route) and allow for long runs.
- scanner: malformed or truncated harness output is reported as an error
  instead of raising JSONDecodeError; commands that cannot start are
  reported too.
- validator: only real Bandit `# nosec` comment directives are refused, not
  identifiers, strings or prose comments.
- sandbox: child PATH starts with the interpreter's bin directory, so a
  TEST_COMMAND like `pytest -rA` or another venv console script resolves.
- Bump joserfc 1.8.0, pyasn1 0.6.5, oauthlib 4.0.1 (OSV advisories).

Refs google#2811

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated House Rules review — 1 finding(s).

@@ -0,0 +1,237 @@
# ruff: noqa

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] no licence header on this file; 2 .py file(s) have none while 25 carry the full block (contrib/python/code-security-fixer/app/django_archive/django/utils/archive.py; contrib/python/code-security-fixer/app/django_archive/django/core/exceptions.py)

GHSA-xcq7-p3pf-p88h is fixed in opentelemetry-api 1.45.1, but google-adk
2.13 pins opentelemetry-api <= 1.44. Record it in osv-scanner.toml with a
reason and an expiry, as other contrib recipes do.

Refs google#2811
@debanshd

Copy link
Copy Markdown
Contributor Author

@tommywagz thanks for the review. All 8 comments are addressed in 390e0a4, each with a test, and I replied on each thread. Since then:

  • 534e18f adds an osv-scanner.toml that ignores GHSA-xcq7-p3pf-p88h. Its fix (opentelemetry-api 1.45.1) is blocked by google-adk 2.13's <=1.44 pin. joserfc, pyasn1 and oauthlib are upgraded.
  • The PR description is updated to match the current recipe.

The remaining pytest (tools + skills + CI scripts) failure comes from main and is fixed there. Could you click Update branch? My token can't push the workflow-file changes from main. Ready for another look whenever you have time.

@tommywagz tommywagz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No more critiques from me. LTGM!

@tommywagz
tommywagz merged commit b4fa327 into google:main Nov 11, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RECIPE PROPOSAL] CodeMender-style security vulnerability fixer agent

2 participants