Repository navigation
Add code-security-fixer: patch real CVEs with a test- and Bandit-gated ADK 2.0 workflow - #2815
Conversation
A CodeMender-style detect -> patch -> validate agent for AppSec finding backlogs. Gemini proposes file-level fixes; a deterministic validator runs Bandit and the target's own exploit/regression tests in a resource-limited subprocess on a scratch copy and keeps only patches that fix a finding without regressions. Ships an intentionally vulnerable orders API as the sample target, TARGET_DIR support, and model-free unit tests. Refs google#2811
There was a problem hiding this comment.
Automated Hygiene review — 4 finding(s).
contrib/python/codemender-security-fixer/README.mdcontrib/python/codemender-security-fixer/app/sample_target/tests/test_security.pycontrib/python/codemender-security-fixer/tests/eval/evalsets/README.mdcontrib/python/codemender-security-fixer/tests/integration/test_server_e2e.pycontrib/python/codemender-security-fixer/tests/unit/test_tools.py
Splitting the PR up would get them reviewed.
Round 1 · 0 of this PR's 25 automated comments used · this round is capped at 20 across all reviewers.
There was a problem hiding this comment.
Automated Maintainability review — 2 finding(s).
contrib/python/codemender-security-fixer/README.mdcontrib/python/codemender-security-fixer/app/sample_target/tests/test_security.pycontrib/python/codemender-security-fixer/tests/integration/test_server_e2e.pycontrib/python/codemender-security-fixer/tests/unit/test_tools.py
Splitting the PR up would get them reviewed.
Round 1 · 0 of this PR's 25 automated comments used · this round is capped at 20 across all reviewers.
|
@pierpaolo28 this implements the recipe proposal you and @kgu-here discussed with me. It's backed by a real-world scenario, per your feedback. I'd appreciate your review when you have a chance. |
|
There is a problem with the Antigravity CLI Correctness PR review. Please check the action logs for details. |
- Read config through app/config.py with no in-code defaults; values live in .env.example, and tests load it via tests/conftest.py. - Drop the duplicate load_dotenv() in fast_api_app.py, use the standard ALLOW_ORIGINS parsing and port 8080; A2A reads APP_URL/AGENT_VERSION from the environment. - Dockerfile runs as non-root appuser and ships .env.example defaults. - License header on app_utils/__init__.py, ownership.team "Cloud AI", _MAX_DETAIL_CHARS instead of 300, and an assertion in test_cookie_is_ascii.
Call sites read each variable with a direct os.getenv() (no default) and pass the value to app/config.py, which only validates it. check_env_vars.py again detects all 16 variables.
Avoid baking a developer's local .env into image layers, and replace the literal 8080 with SERVER_PORT. Refs google#2811
The agent is now a google.adk.Workflow: scan -> prepare_patch -> patch_agent (single_turn Agent with PatchRequest/PatchProposal schemas) -> validate, looping on routed edges while findings and budget remain, then report. Deterministic steps are function nodes over ctx.state; unit tests run the real graph with a scripted patcher. Adds an A2A integration test (agent card 0.3 shape + message/send) and drops empty optional serving env vars so the Agent Engine adapter starts. Refs google#2811
|
Ready for review. All checks pass. Re-requested review from |
Narrow optional values before indexing so the Mypy Type Check passes. Refs google#2811
- Rename contrib/python/codemender-security-fixer to code-security-fixer and drop the CodeMender naming. - Replace the home-made sample app with Django's django/utils/archive.py at 6822aa5c (parent of the CVE-2021-3281 fix), vendored byte-identical with its BSD-3 licence, Django's functional tests and exploit tests for CVE-2021-3281 and its incomplete-fix follow-up CVE-2025-59682. - Unit tests apply Django's exact historical patches (05413afa, 924a0c09): the 2021 patch is accepted as partial, the 2025 patch closes the rest. - Serve the A2A HTTP+JSON binding (/v1/card, /v1/message:send) next to JSON-RPC for the Gemini agent / Agent Registry, and test it. Refs google#2811
With the HTTP+JSON binding added, the 0.3 card prefers HTTP+JSON and lists JSON-RPC in additionalInterfaces at the same URL. Assert both are advertised.
There was a problem hiding this comment.
Automated Security review — 1 finding(s).
contrib/python/code-security-fixer/README.mdcontrib/python/code-security-fixer/tests/unit/test_tools.py
Splitting the PR up would get them reviewed.
Round 3 · 7 of this PR's 25 automated comments used · this round is capped at 1 across all reviewers · only Security and Correctness still run.
Move to google-adk 2.13.0 with matching google-cloud-aiplatform / google-cloud-agentplatform 2.6.0 (as main did in google#2827). This removes the package collision that broke test_web_server_runnability in a fresh install, and imports AdkApp from vertexai.agent_engines. Bumps certifi to 2026.10.23 (PYSEC-2026-4320) and opentelemetry to 1.44.0, the highest google-adk 2.13 allows. Also writes the review patch to a private per-run temp directory and adds licence headers: Apache for the package markers we wrote, Django's BSD-3 notice for the two vendored upstream files. Refs google#2811
|
Thanks for updating the branch, @tommywagz. Pushed a fix for the failing checks:
Locally: validate, ruff, mypy, check_env_vars, check_model_literals and check_recipe_pyproject all pass, and pytest passes with and without a local |
| # Model Configuration | ||
| MODEL_NAME=gemini-3.7-flash | ||
|
|
||
| # Code to fix. Empty = the bundled, intentionally vulnerable app/sample_target. |
There was a problem hiding this comment.
should this reference app/django_archive instead of app/sample_target?
There was a problem hiding this comment.
Good catch, thanks. Fixed in 390e0a4: the comment now points at app/django_archive (Django's archive.extract() before the CVE-2021-3281 / CVE-2025-59682 fixes).
| # Run them locally with: uv run pytest tests/integration | ||
| addopts = "--ignore=tests/integration" | ||
| # Only collect the recipe's own tests. The bundled sample target under | ||
| # app/sample_target/ ships exploit tests that FAIL on purpose until fixed. |
There was a problem hiding this comment.
comment still refers to app/sample_target/ rather than app/django_archive
There was a problem hiding this comment.
Good catch. Fixed in 390e0a4: the testpaths comment now refers to app/django_archive/.
| start_time = time.time() | ||
| while time.time() - start_time < timeout: | ||
| try: | ||
| response = requests.get("http://127.0.0.1:8000/docs", timeout=10) |
There was a problem hiding this comment.
should this use f"{BASE_URL}/docs" instead of the hardcoded URL?
There was a problem hiding this comment.
Yes, fixed in 390e0a4: wait_for_server now polls f"{BASE_URL}/docs". While running it I also removed a stale /feedback test (that route does not exist in this app) and gave the SSE request a read timeout that fits a full run; test_server_e2e.py passes locally (2 passed).
| "app.fast_api_app:app", | ||
| "--host", | ||
| "0.0.0.0", | ||
| "--port", |
There was a problem hiding this comment.
port 8000 here differs from SERVER_PORT 8080 in fast_api_app.py and Dockerfile
There was a problem hiding this comment.
Agreed it was inconsistent. Fixed in 390e0a4: the test no longer hard-codes a port at all; it starts uvicorn on an ephemeral port (same as test_a2a.py) and derives BASE_URL and APP_URL from it, so it can't clash with 8080 (SERVER_PORT) or anything else running locally.
| "tests": [], | ||
| "error": f"test run crashed: {tail[0][:_MAX_DETAIL_CHARS]}", | ||
| } | ||
| payload = json.loads(run["stdout"][marker + len("@@RESULT@@") :]) |
There was a problem hiding this comment.
json.loads here is unguarded against JSONDecodeError if output is truncated or malformed
There was a problem hiding this comment.
Good point. Fixed in 390e0a4: _parse_unittest now catches JSONDecodeError/KeyError/TypeError (and a non-list tests) and returns error: "test harness output could not be parsed", which the validator treats as a rejection instead of crashing. Covered by test_malformed_harness_output_is_an_error_not_a_crash (truncated JSON, wrong shape, missing key, non-list).
| return f"new content exceeds {workspace.MAX_FILE_CHARS} characters" | ||
| if new_content == old: | ||
| return "new content is identical to the current file" | ||
| if new_content.count("nosec") > old.count("nosec"): |
There was a problem hiding this comment.
won't count("nosec") also catch legitimate identifiers or comments mentioning "nosec"?
There was a problem hiding this comment.
Yes, it would. Fixed in 390e0a4: the precheck now counts only real Bandit directives, i.e. #\s*nosec inside comment tokens (via tokenize, with a line-based fallback for code that does not tokenize). Identifiers like nosec_count, string literals and prose comments such as # see the nosec policy no longer count. Covered by test_nosec_directives_counts_only_real_markers and test_mentioning_nosec_is_not_a_suppression. (Bandit also runs with --ignore-nosec, so a directive could never hide a finding; the precheck just rejects such patches early with a clear reason.)
| logging.basicConfig(level=logging.INFO) | ||
| logger = logging.getLogger(__name__) | ||
|
|
||
| BASE_URL = "http://127.0.0.1:8000" |
There was a problem hiding this comment.
could this use an ephemeral port like test_a2a.py does to prevent local port conflicts?
There was a problem hiding this comment.
Yes, done in 390e0a4: _free_port() binds port 0 like test_a2a.py, and the server, BASE_URL and APP_URL all use it.
| def _child_env(cwd: str) -> dict[str, str]: | ||
| """A minimal environment: no credentials or API keys leak to the target.""" | ||
| return { | ||
| "PATH": "/usr/local/bin:/usr/bin:/bin", |
There was a problem hiding this comment.
if TEST_COMMAND invokes pytest or a venv binary directly, restricted PATH could cause FileNotFoundError?
There was a problem hiding this comment.
Good catch, it could. Fixed in 390e0a4: the child PATH now starts with the directory of the recipe's interpreter (the venv bin/ under uv run and in the container) before /usr/local/bin:/usr/bin:/bin, so TEST_COMMAND="pytest -rA -q" or another console script installed in the venv resolves. If a command still can't start, run_limited returns a clear cannot run '<cmd>' ... (child PATH: ...) error instead of raising. Covered by test_test_command_resolves_venv_console_scripts and test_missing_test_command_is_reported.
- .env.example / pyproject.toml: point the comments at app/django_archive.
- test_server_e2e.py: ephemeral port (as in test_a2a.py) and f"{BASE_URL}/docs";
drop the stale /feedback test (no such route) and allow for long runs.
- scanner: malformed or truncated harness output is reported as an error
instead of raising JSONDecodeError; commands that cannot start are
reported too.
- validator: only real Bandit `# nosec` comment directives are refused, not
identifiers, strings or prose comments.
- sandbox: child PATH starts with the interpreter's bin directory, so a
TEST_COMMAND like `pytest -rA` or another venv console script resolves.
- Bump joserfc 1.8.0, pyasn1 0.6.5, oauthlib 4.0.1 (OSV advisories).
Refs google#2811
| @@ -0,0 +1,237 @@ | |||
| # ruff: noqa | |||
There was a problem hiding this comment.
[MINOR] no licence header on this file; 2 .py file(s) have none while 25 carry the full block (contrib/python/code-security-fixer/app/django_archive/django/utils/archive.py; contrib/python/code-security-fixer/app/django_archive/django/core/exceptions.py)
GHSA-xcq7-p3pf-p88h is fixed in opentelemetry-api 1.45.1, but google-adk 2.13 pins opentelemetry-api <= 1.44. Record it in osv-scanner.toml with a reason and an expiry, as other contrib recipes do. Refs google#2811
|
@tommywagz thanks for the review. All 8 comments are addressed in 390e0a4, each with a test, and I replied on each thread. Since then:
The remaining |
tommywagz
left a comment
There was a problem hiding this comment.
No more critiques from me. LTGM!
Closes #2811
What
contrib/python/code-security-fixeris an ADK 2.0 workflow that detects, patches and validates security vulnerabilities. Gemini proposes a fix. A deterministic validator accepts it only if no previously passing test fails, Bandit reports no new finding, and at least one finding or exploit test is resolved. The agent never commits; its output is a unified diff for human review.Real-world anchor
django/utils/archive.py, vendored at the commit before the fix for CVE-2021-3281 (archive path traversal). It's reproduced with Django's own regression cases, plus CVE-2025-59682, the later bypass of that fix. Vendored code keeps Django's BSD-3-Clause licence.extract()tests, plus exploit tests for both CVEs. The reference fixes in unit tests are Django's actual patches.gemini-3.7-flash): exploit tests 0/3 → 3/3 in one accepted patch, with functional tests still passing. The fix matches Django's 2025 approach (os.path.commonpath).TARGET_DIRandTEST_COMMANDpoint the agent at your own repo. It's copied to a private working copy and never modified.How it works
scan→ (has_findings→prepare_patch→patch_agent→validate→continue/done|clean) →report.patch_agentis the only model call.# nosecdirectives, syntax errors and paths outside the target. Malformed tool output becomes a rejection rather than a crash.Checks run locally
validate,ruff,mypy(CI command),check_env_vars,check_model_literals,check_recipe_pyprojectanduv lock --check: all pass..env.test_server_e2e.pyalso passes.osv-scanner.tomlignores GHSA-xcq7-p3pf-p88h with a reason and an expiry. Its fix (opentelemetry-api1.45.1) is blocked bygoogle-adk2.13's<=1.44pin. All other flagged packages are upgraded.Note
app/django_archive/intentionally contains vulnerable upstream code as the demo target. Its exploit tests are excluded from the recipe's own pytest run.