Skip to content

test: migrate ACP approval fixtures to tool-authoritative replies - #769

Merged
AlexanderZ-Band merged 8 commits into
mainfrom
feat/testpython-migrate-acp-approval-and-hosted-fixture-INT-1757
Oct 12, 2026
Merged

AlexanderZ-Band merged 8 commits into
mainfrom
feat/testpython-migrate-acp-approval-and-hosted-fixture-INT-1757

Conversation

@AlexanderZ-Band

@AlexanderZ-Band AlexanderZ-Band commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Hosted Copilot and Cursor approval fixtures disabled Band MCP tools while requiring room replies. Native ACP text is thought telemetry, so these turns could not deliver the required reply. Keep completion tools injected and preserve the native-command approval assertions.

Cursor approval scenarios use a test-only resolver that selects an offered one-shot permission only for a registered, schema-valid send-message call in the active room/session. Every other permission delegates to the existing manual resolver. The pinned Cursor CLI permission formatter omits rawInput and sends a decorated MCP title plus fenced JSON arguments. Adapt only its exact completion-reply frame before the inherited handler; do not interpret other display content as tool input. Production approval policy and turn judgment remain unchanged.

Evidence and validation:

  • Real ACP JSON-RPC reproduction: native text produced thought/task events and no room reply.
  • Executed the installed pinned Cursor permission formatter and captured its actual emitted frame as an independent regression fixture. The original plan had mistaken the richer tool-update shape for the permission shape.
  • 24 fixture regressions passed over real ACP and local MCP. They cover the pinned frame, malformed/foreign/lookalike inputs, missing one-shot options, unregistered tools, room/session mismatches, and native-command file effects before/after completion.
  • 161 focused adapter/conformance/emitter tests passed. Ruff, formatting, Pyrefly and diff checks passed.
  • Full non-live suite on the final repair revision with reruns disabled: 7,875 passed, 170 reported skips. All 24 new regressions ran.
  • Initial live attempt, revision 8338029, remains RED: Copilot 3/3 passed on each OS; Cursor 0/6 passed on each OS. Five Cursor follow-up permissions per OS exposed the formatter mismatch; the session-grant timeout's precise pending action was not logged.
  • Fresh repair validation, revision fce0d6d, runs exactly nine required cells once on each OS, with pytest reruns disabled and SDK delivery limited to one attempt. PASS: 9/9 required cells on Ubuntu and 9/9 on Windows, with the exact same required-cell inventory and no required skips or missing rows. Both backend jobs and the merged scorecard gate passed. This is a new-revision result; the failed earlier attempt remains recorded above.

Final-revision CI passed, including Ubuntu and Windows Python 3.11-3.13, lint/type checks, packaging, conformance/collection, and executable documentation. Python core coverage passed as well.

Implements INT-1757. Focused validation does not certify the full nightly baseline; parent acceptance remains INT-1563.

Review follow-up (2bdefc5d): fix the scripted ACP peer to record approval from the selected offered option rather than a legacy ID substring. Four real ACP/MCP regressions cover custom one-shot/permanent grants, rejection, and cancellation; three fail against the previous helper and all four pass after the repair. Reuse the shared server constant and ACP content builders, and remove the remaining obsolete hosted-tools explanation. The permission fixture policy is unchanged. Local validation on this commit: 28 targeted tests and 7,879 non-live tests passed (170 reported skips), reruns disabled; Ruff, formatting, and Pyrefly passed. The review-follow-up CI remains RED: Windows Python 3.11 timed out running the existing native PowerShell write probe after 10 seconds. That shell subprocess test and command generation are unchanged by the cleanup; its underlying timeout cause is unresolved. The earlier live acceptance evidence remains tied to its recorded revision.

Plumbing cleanup (0fe714e0): consolidate the Cursor test factory into one construction path and declare the exact pinned frame with structural matching. Parse its reply through SendMessageInput and pass canonical title/direct arguments to the inherited handler, removing the synthetic MCP envelope. Invalid display frames remain manual; an added nested-invocation case guards that boundary. Six-angle review and independent logical check found no new defects. Local gates pass: Ruff/format/Pyrefly, 128 focused tests and 7,880 nonlive tests (170 reported skips), reruns disabled. Fresh first-attempt Ubuntu/Windows acceptance passed on this exact revision: 9/9 required cells on each OS, exact inventory compared with the previous accepted scorecards, no missing/skipped required cells, and both backend jobs plus merged scorecard gate successful. Cleanup CI passed on this exact revision, including Ubuntu/Windows Python 3.11–3.13, all extra lanes, lint/type checks, packaging and executable documentation. Coverage and all required PR checks also passed. The weekly-report job is schedule-only and skipped on this PR run. No previous run was retried; the earlier failures remain recorded.

Auto-merge is enabled. GitHub requires one independent approving review; the authenticated author cannot approve this PR, so it remains open pending that approval before merge-queue admission.

@linear-code

linear-code Bot commented Oct 11, 2026

Copy link
Copy Markdown

INT-1757

@AlexanderZ-Band
AlexanderZ-Band marked this pull request as ready for review October 11, 2026 07:21

@AlexanderZ-Band AlexanderZ-Band left a comment •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: one confirmed Logical Bugs issue and three nits (A:2, B:1; C/D/E:0). All four fixed in 2bdefc5. The outside-hunk finding at tests/e2e/baseline/smoke/adapters/test_copilot_acp.py:403 is fixed by removing the obsolete Band-tools-off explanation. The three inline threads have replies and are resolved. No findings declined. Focused post-fix Logical Bugs pass found no new defects. Full local gates passed: Ruff, format, Pyrefly, 28 targeted tests and 7879 nonlive tests (170 reported skips), reruns disabled. One review cycle.

Comment thread tests/toolkit/test_cursor_approval_fixture.py Outdated
Comment thread tests/toolkit/test_cursor_approval_fixture.py Outdated
Comment thread tests/integrations/acp/acp_toolkit/agent.py Outdated

@AlexanderZ-Band AlexanderZ-Band left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second full my-code-review pass: zero new findings across A-F; prior resolved findings remain fixed. Scope matches INT-1757: reachable tool-authoritative replies in hosted Copilot and Cursor approval fixtures, preserving native approval/lifecycle/delivery contracts. Independent logical skeptic found no defects; original approval repro and 28 ACP/MCP regressions pass. Edge-case reviewer passed 78 focused tests with reruns disabled. No changes required; one cycle.

@AlexanderZ-Band AlexanderZ-Band left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plumbing cleanup reviewed across A-F: zero new findings. One Cursor fixture construction path replaces duplicate factory forwarding. Structural matching and SendMessageInput JSON validation normalize only the pinned completion frame directly, without a synthetic MCP envelope. Native/manual permission handling, active room/session, registration, offered one-shot selection, cancellation and inherited logging remain intact. Independent logical check found no defects. Local gates: Ruff/format/Pyrefly, 128 focused tests, full nonlive 7880 passed with 170 reported skips, reruns disabled. New-revision CI and first-attempt Ubuntu/Windows live acceptance are pending.

@AlexanderZ-Band AlexanderZ-Band left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fresh my-code-review pass on this head: zero findings across Style & Hygiene, Design & Reuse, Runtime Safety & Observability, Test Quality, Edge-Case Flow Analysis, and independently verified Logical Bugs. The three prior inline findings remain fixed and resolved. Scope matches INT-1757 and its plan, with the documented correction for the pinned Cursor display-only permission frame. Real ACP/MCP regressions passed 29/29 with no skips or reruns; exact-head required CI and first-attempt Ubuntu/Windows live acceptance are green. No additional changes required.

…d-fixture-INT-1757

Main's CursorACPAdapter now pre-approves registered Band tools during an
active turn, which supersedes this branch's test-only
CursorApprovalAdapter. Drop it, cursor_approval_adapter and their
fixture tests, and let the Cursor approval cells build the product
adapter as main does. ALLOW_ONCE_KIND now comes from main's
permissions module, and FakeACPAgent keeps the general offered-options
check, typed through AllowedOutcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9yodkF7egXLBeXULEb7PM
@AlexanderZ-Band
AlexanderZ-Band requested a review from a team October 12, 2026 10:27

@amit-gazal-band amit-gazal-band left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. After the main merge (#750 pre-approves Band tools in production), the net diff is just the Copilot hosted-fixture cleanup and the scripted ACP peer's selected-option approval fix, plus regressions. ACP + Cursor adapter tests (615 passed) and ruff pass locally on 3575914; CI green. Non-blocking: keep an eye on nightly for the previously documented hosted Copilot band_send_message loop now that inject_band_tools=False is gone; will_ask_permission(options=...) silently ignores allow_option_id.

@AlexanderZ-Band
AlexanderZ-Band added this pull request to the merge queue Oct 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 12, 2026
AlexanderZ-Band and others added 2 commits October 12, 2026 17:45
will_ask_permission took both allow_option_id and options, and silently
dropped allow_option_id whenever options was given. Keep options as the
only argument and build the default allow/reject menu with
allow_or_reject().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FyJ4C6gG4U83aLNhFzQ3i8
@AlexanderZ-Band

AlexanderZ-Band commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Replying to the two non-blocking notes in @amit-gazal-band's approval:

will_ask_permission(options=...) silently ignores allow_option_id.

Confirmed and fixed in 9280a9e. options is now the only parameter. The default allow/reject menu is built by allow_or_reject(allow_option_id) in acp_toolkit/agent.py, so the two can no longer be passed together. The six Cursor callers now pass options=allow_or_reject("allow-once"). Ruff, format, Pyrefly, and 8,110 non-live tests pass (reruns disabled).

keep an eye on nightly for the previously documented hosted Copilot band_send_message loop now that inject_band_tools=False is gone

Agreed. It's worth watching, because the main merge brought about 1.4k lines of ACP changes after this PR's recorded live acceptance. I re-ran the hosted cells live against the merged head (test_copilot_hosted_auth_replies ×2 and test_copilot_switches_model_and_effort_in_a_live_room, reruns disabled). All 3 passed in 95s, and none of the turns looped. Nightly remains the broader check.

@AlexanderZ-Band
AlexanderZ-Band added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 9b8c5b8 Oct 12, 2026
28 checks passed
@AlexanderZ-Band
AlexanderZ-Band deleted the feat/testpython-migrate-acp-approval-and-hosted-fixture-INT-1757 branch October 12, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants