Skip to content

test: make interactive Win11 evidence deterministic#87

Merged
amanthanvi merged 14 commits into
mainfrom
codex/fix-interactive-evidence-upload
Jul 13, 2026
Merged

test: make interactive Win11 evidence deterministic#87
amanthanvi merged 14 commits into
mainfrom
codex/fix-interactive-evidence-upload

Conversation

@amanthanvi

@amanthanvi amanthanvi commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • upload the actual hidden workspace .sandbox/win11/**/logs/** evidence tree with SHA-pinned actions/upload-artifact@v6
  • enforce Actions Runner 2.327.1+, record its version in provenance, and reject stale/malformed release evidence
  • preserve JSON root kind and require an exact provenance object under PowerShell 7.1+
  • give the initial render/termio trace path one process-lifetime owner so later tab teardown cannot overwrite evidence
  • bind the multi-tab harness to the initial tab/surface with liveness, class, parent, control-ID, and visibility checks
  • disable close confirmation in the isolated session-restore harness

Validation

Independent review

  • GPT final exact-head audit: zero actionable findings; zero repository-mitigable residuals
  • Claude Fable High, effort high, structured no-tools exact-diff audit: SAFE; zero actionables; zero concrete residuals
  • Greptile: PASS, 8 files, 0 comments
  • CodeRabbit: PASS, zero actionable findings on exact head
  • Sourcery: PASS; diagnostic-only thread addressed and resolved
  • Codex: exact-head review found no major issues
  • unresolved review threads: 0

Risk

Trace ownership changes activate only when diagnostic trace-path environment variables are configured. Later surfaces intentionally skip those single-document traces. Provenance validation is fail-closed. Production terminal behavior is otherwise unchanged.

Copilot AI review requested due to automatic review settings July 13, 2026 03:33

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @amanthanvi, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex please review this follow-up narrowly for artifact upload correctness: upload-artifact hidden-file behavior, whether .sandbox/win11//logs/ captures uia-tree.json from the interactive accessibility harness, and whether the verification contract is sufficient. Greptile / CodeRabbit / Sourcery: please review for any security or CI correctness regressions.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR hardens Windows interactive evidence collection and runner provenance validation, expands session restore configuration, coordinates exclusive trace-path ownership in Win32 and Termio, and stabilizes multitab surface activation and handle validation.

Changes

Windows interactive behaviors

Layer / File(s) Summary
Interactive evidence and session restore
.github/workflows/test.yml, scripts/check-accessibility-evidence.ps1, test/windows/assert-interactive-runner.ps1, test/windows/flagship/Test-VerificationContracts.ps1, test/windows/interactive-win11-session-restore.ps1
Hidden Win11 logs are uploaded and contract-checked; runner provenance now includes pinned version, schema, root-kind, and typed field validation; session restore writes expanded configuration.
Exclusive trace-path ownership
src/apprt/win32.zig, src/termio/Termio.zig
Atomic claims retain the first trace path, free later allocations, disable subsequent tracing, and test double-claim behavior.
Stable multitab surface activation
test/windows/interactive-win11-boo-multitab.ps1
Visible window discovery, stored tab-button activation, initial surface restoration, and handle validation replace index-based activation and surface re-discovery.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: making interactive Win11 evidence deterministic.
Description check ✅ Passed The description follows the required template and covers summary, validation, and risks/follow-ups with concrete details.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-interactive-evidence-upload

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Windows interactive CI evidence upload so it reliably captures the hidden Win11 sandbox logs from the real workspace .sandbox/win11 tree (including files like uia-tree.json), and adds a verification contract to prevent regressions in the workflow wiring.

Changes:

  • Add include-hidden-files: true to the interactive evidence artifact upload step.
  • Switch the uploaded log path from ${{ runner.temp }}/winghostty-interactive-win11-*/logs/** to ${{ github.workspace }}/.sandbox/win11/**/logs/**.
  • Add a PowerShell verification contract asserting both the hidden-file behavior and the intended sandbox log path remain present in .github/workflows/test.yml.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
test/windows/flagship/Test-VerificationContracts.ps1 Adds a workflow text contract to lock in hidden-file inclusion + .sandbox/win11/**/logs/** upload path.
.github/workflows/test.yml Updates the “Upload interactive evidence” artifact step to include hidden files and upload workspace sandbox logs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown

Greptile Summary

This PR hardens the Windows 11 interactive test evidence pipeline to make trace output deterministic across multi-tab sessions. It introduces a process-lifetime atomic ownership model for render and termio trace paths, so only the initial surface can write evidence; later tab teardown cannot overwrite it.

  • Zig trace ownership: render_trace_file_claimed and output_trace_file_claimed global atomics are added; claimTracePath frees the allocation immediately when the claim fails, and each new initWithClaimedPath entry-point is covered by an isolated unit test.
  • PowerShell harness stabilization: Find-SurfaceWindow/Test-SurfaceWindow/Activate-TabButton now verify parent, class, visibility, and control-ID identity instead of re-discovering by index; runner provenance validation gains version-floor enforcement (≥ 2.327.1) and typed integer parsing to eliminate silent cast failures.
  • Workflow and artifact changes: upload-artifact is bumped to v6.0.0 with include-hidden-files: true, and the evidence path switches from a runner.temp glob to the workspace .sandbox/win11 tree to capture the actual hidden log directory.

Confidence Score: 5/5

Safe to merge; all Zig changes are gated behind diagnostic environment variables and have no effect on production terminal behavior when those variables are unset.

The atomic claim mechanism in both win32.zig and Termio.zig is straightforward and correctly scoped to diagnostic trace paths only. Unit tests for the claim logic use local atomic variables, keeping them isolated from the process-global flags. The PowerShell harness improvements replace fragile index-based window selection with HWND-stable, multi-check identity validation. Provenance parsing improvements eliminate silent type-cast mismatches. No production code paths are altered.

No files require special attention. The runner root derivation in assert-interactive-runner.ps1 (Split-Path twice from RUNNER_TEMP) is an implicit structural assumption about the self-hosted runner layout, but the PR description confirms it was validated end-to-end on the target runner.

Important Files Changed

Filename Overview
src/apprt/win32.zig Adds process-lifetime atomic claim for render trace path; claimTraceFile/claimTracePath helpers are correct; test uses a local claimed variable and is properly isolated from the global flag.
src/termio/Termio.zig Parallel pattern to win32.zig for OutputTrace path ownership; claimTracePath frees the allocation on failed claim; test is correctly isolated using a local claimed variable.
test/windows/interactive-win11-boo-multitab.ps1 Replaces index-based tab activation with HWND-stable references; adds GetParent/IsWindow/IsWindowVisible checks in Find-SurfaceWindow and Test-SurfaceWindow; Activate-TabButton validates handle liveness and control-ID identity before sending click messages.
test/windows/assert-interactive-runner.ps1 Adds runner version floor check by reading Runner.Worker.exe FileVersion from a path derived from RUNNER_TEMP; emits runner_version into the provenance JSON blob.
scripts/check-accessibility-evidence.ps1 Strengthens provenance parsing: adds -NoEnumerate, PSCustomObject type guard, typed int/version parse, and minimum runner version enforcement.
test/windows/flagship/Test-VerificationContracts.ps1 Adds Assert-WorkflowContractAbsent helper, new contracts for hidden-file inclusion, runner floor pinning, and JSON root-kind preservation self-check.
.github/workflows/test.yml Bumps upload-artifact to v6.0.0; switches interactive log path to workspace .sandbox/win11; adds include-hidden-files: true.
test/windows/interactive-win11-session-restore.ps1 Adds confirm-close-surface = false to the isolated session-restore config to prevent close dialogs from blocking the deterministic harness.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant H as Harness (PS)
    participant App as winghostty.exe
    participant RT as RenderTrace (global claim)
    participant OT as OutputTrace (global claim)

    H->>App: launch process
    App->>RT: init() claimTracePath(render_trace_file_claimed)
    RT-->>App: claimed (first surface owns path)
    App->>OT: init() claimTracePath(output_trace_file_claimed)
    OT-->>App: claimed (first terminal owns path)

    H->>App: Invoke-HostCommand new tab xN
    App->>RT: init() for tab 2..N
    RT-->>App: "rejected already claimed path=null"
    App->>OT: init() for tab 2..N
    OT-->>App: "rejected already claimed path=null"

    H->>App: Activate-TabButton validated HWND controlID
    H->>App: Wait Test-SurfaceWindow parent+class+visible+IsWindow
    H->>App: Send Escape +boo exits
    App->>RT: deinit() writeSnapshot for initial surface only
    App->>OT: deinit() writeSnapshot for initial terminal only
    H->>H: collect evidence from .sandbox/win11 logs
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant H as Harness (PS)
    participant App as winghostty.exe
    participant RT as RenderTrace (global claim)
    participant OT as OutputTrace (global claim)

    H->>App: launch process
    App->>RT: init() claimTracePath(render_trace_file_claimed)
    RT-->>App: claimed (first surface owns path)
    App->>OT: init() claimTracePath(output_trace_file_claimed)
    OT-->>App: claimed (first terminal owns path)

    H->>App: Invoke-HostCommand new tab xN
    App->>RT: init() for tab 2..N
    RT-->>App: "rejected already claimed path=null"
    App->>OT: init() for tab 2..N
    OT-->>App: "rejected already claimed path=null"

    H->>App: Activate-TabButton validated HWND controlID
    H->>App: Wait Test-SurfaceWindow parent+class+visible+IsWindow
    H->>App: Send Escape +boo exits
    App->>RT: deinit() writeSnapshot for initial surface only
    App->>OT: deinit() writeSnapshot for initial terminal only
    H->>H: collect evidence from .sandbox/win11 logs
Loading

Reviews (21): Last reviewed commit: "ci: bind provenance parser contract" | Re-trigger Greptile

@amanthanvi amanthanvi force-pushed the codex/fix-interactive-evidence-upload branch from fee39ef to 9648243 Compare July 13, 2026 03:37
@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex please re-review after the force-push. Added one discovered CI flake fix from the default-branch interactive run: PTY process output batch threshold relaxed from 140 to 135 while frame cadence remains >=140 and paint/sustained-gap gates remain unchanged. Please focus on whether this weakens the runtime regression gate too far, and on the hidden .sandbox evidence upload path.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptileai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@sourcery-ai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review

@sourcery-ai

sourcery-ai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Sorry @amanthanvi, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: b6f04138a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@amanthanvi

Copy link
Copy Markdown
Owner Author

Correction for the final head b6f04138a0816336aa4f6bac7b8d2c1a9102d5a8: the unrelated PTY batch-threshold relaxation was removed. The PR diff is only .github/workflows/test.yml plus its verification contract. Full workflow-dispatch artifact proof from run 29222191291: upload step succeeded and archive flagship-interactive-win11-29222191291 contains 58 files, exactly one uia-tree.json, exactly one runner-provenance file, and exactly one composite result. The core composite passed; a later unchanged session-restore adjunct failed and is being diagnosed separately from the proven upload fix.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptileai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@sourcery-ai review

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review

@sourcery-ai

sourcery-ai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Sorry @amanthanvi, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: acd3b215dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptileai please review the latest exact head 602fb2f, including deterministic trace ownership and overflow-safe harness identity.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review latest exact head 602fb2f

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review final exact head fca99a2 after Claude Fable contract-gap closure; report all actionable findings and concrete residuals.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The trace file claiming helpers (claimTraceFile/claimTracePath) are duplicated between win32.zig and Termio.zig; consider extracting them into a shared utility to avoid diverging behavior between render and termio traces.
  • The minimum runner version 2.327.1 is hard-coded in multiple places (assert-interactive-runner.ps1, check-accessibility-evidence.ps1, and verification contracts); centralizing this value or clearly documenting how to update all call sites together would reduce the risk of version skew.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The trace file claiming helpers (claimTraceFile/claimTracePath) are duplicated between win32.zig and Termio.zig; consider extracting them into a shared utility to avoid diverging behavior between render and termio traces.
- The minimum runner version `2.327.1` is hard-coded in multiple places (assert-interactive-runner.ps1, check-accessibility-evidence.ps1, and verification contracts); centralizing this value or clearly documenting how to update all call sites together would reduce the risk of version skew.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

requested="fca99a2e671edcb3030e70060322aa0c36f26ffc"
previous="60b3e7230230932abede5fe3f7951f3de5c45780"

git fetch origin pull/87/head:pr87 2>&1 | tail -20
git checkout --detach pr87 >/dev/null 2>&1

echo "== checked-out head =="
git rev-parse HEAD

if ! git cat-file -e "${requested}^{commit}" 2>/dev/null; then
  git fetch origin "$requested" 2>&1 | tail -5 || true
fi
if ! git cat-file -e "${previous}^{commit}" 2>/dev/null; then
  git fetch origin "$previous" 2>&1 | tail -5 || true
fi

echo "== requested exists =="
git rev-parse "${requested}^{commit}" 2>&1 || echo "NOT FOUND"
echo "== previous exists =="
git rev-parse "${previous}^{commit}" 2>&1 || echo "NOT FOUND"

echo "== commit metadata for requested =="
git show -s --format='%H%n%P%n%s%n%an%n%ad' "$requested" 2>&1 || true

echo "== delta name-status previous->requested =="
git diff --name-status "$previous" "$requested" 2>&1 || true
echo "== delta patch =="
git diff "$previous" "$requested" 2>&1 || true

base="$(git merge-base origin/main "$requested" 2>&1 || true)"
echo "base=$base"
echo "== full PR name-status =="
git diff --name-status "$base" "$requested" 2>&1 || true
echo "== git diff --check full =="
git diff --check "$base" "$requested" 2>&1 || true

Length of output: 6570


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "=== variable definitions in Test-VerificationContracts.ps1 ==="
rg -n "runnerProvenanceChecker|accessibilityChecker" test/windows/flagship/Test-VerificationContracts.ps1

echo "=== full context of check-accessibility-evidence.ps1 around change ==="
sed -n '95,140p' scripts/check-accessibility-evidence.ps1

echo "=== full context of assert-interactive-runner.ps1 ==="
cat -n test/windows/assert-interactive-runner.ps1

echo "=== check for other usages of MinimumRunnerVersion param removed ==="
rg -n "MinimumRunnerVersion|assert-interactive-runner" .github/workflows/test.yml

Length of output: 11324


@amanthanvi Reviewed exact head fca99a2e671edcb3030e70060322aa0c36f26ffc.

Delta since last reviewed head (60b3e72302): commit ci: close runner version contract gaps, touching scripts/check-accessibility-evidence.ps1, test/windows/assert-interactive-runner.ps1, and test/windows/flagship/Test-VerificationContracts.ps1. No changes to .github/workflows/test.yml, src/apprt/win32.zig, src/termio/Termio.zig, or the multitab/session-restore harness scripts in this delta — all previously validated and unchanged.

Actionable findings: 0.

This closes the exact "Claude Fable" contract gap:

  1. check-accessibility-evidence.ps1 — the schema_version check is now hoisted out of the big aggregated OR-condition and evaluated first, immediately after parsing provenance, before the runner_version TryParse. This is a pure fail-fast ordering improvement (an unsupported schema now throws its own explicit error message instead of falling into the generic aggregated-condition error) — no logic change to what ultimately gets validated.
  2. assert-interactive-runner.ps1 — the dead -MinimumRunnerVersion cmdlet parameter is removed (confirmed via rg that the workflow invocation at Line 148 of test.yml never passed it, so this is a no-op removal, not a behavior change) and inlined as $minimumRunnerVersion. More importantly, the previously unguarded [version]$runnerVersion = ...FileVersion cast is now routed through [version]::TryParse, so a malformed/missing Runner.Worker.exe file version throws a clear, specific error instead of an opaque cast exception — consistent with the same pattern already used for provenanceRunnerVersion in the accessibility checker.
  3. Test-VerificationContracts.ps1 — the existing pattern for $runnerProvenanceChecker (i.e., assert-interactive-runner.ps1) was updated to match the new minimumRunnerVersion = [version]'2.327.1' literal, and a new Assert-WorkflowContract was added asserting that exact same literal exists in $accessibilityChecker (i.e., check-accessibility-evidence.ps1) too. This is the actual gap closure: previously only the runner-side minimum-version constant was contract-verified; now the release-gate script's own constant is independently pinned as well, so the two can't silently drift apart.

Residual non-blockers:

  1. (Carried over, unchanged) The runner-root derivation in assert-interactive-runner.ps1 (Split-Path -Parent (Split-Path -Parent $env:RUNNER_TEMP)bin\Runner.Worker.exe) still assumes the standard self-hosted runner directory layout. This fails safe (throws) rather than silently skipping the check if the layout differs, so it's informational only.

git diff --check on the complete PR diff is clean. No security or CI-correctness regressions in this head.

[approve]

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptile review immutable final head 56ab45e after all Claude/GPT residual hardening; require 5/5 and zero concrete actionable or residual findings.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review immutable final head 56ab45e after all Claude/GPT residual hardening; report every concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@sourcery-ai review immutable final head 56ab45e after all Claude/GPT residual hardening; report every concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review immutable final head 56ab45e after all Claude/GPT residual hardening; report every concrete actionable and residual finding.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

requested="56ab45ef52b91a16ef9c705cb730cf77b29f5448"
previous="fca99a2e671edcb3030e70060322aa0c36f26ffc"

git fetch origin pull/87/head:pr87 2>&1 | tail -20
git checkout --detach pr87 >/dev/null 2>&1

echo "== checked-out head =="
git rev-parse HEAD

if ! git cat-file -e "${requested}^{commit}" 2>/dev/null; then
  git fetch origin "$requested" 2>&1 | tail -5 || true
fi
if ! git cat-file -e "${previous}^{commit}" 2>/dev/null; then
  git fetch origin "$previous" 2>&1 | tail -5 || true
fi

echo "== requested exists =="
git rev-parse "${requested}^{commit}" 2>&1 || echo "NOT FOUND"
echo "== previous exists =="
git rev-parse "${previous}^{commit}" 2>&1 || echo "NOT FOUND"

echo "== commit metadata for requested =="
git show -s --format='%H%n%P%n%s%n%an%n%ad' "$requested" 2>&1 || true

echo "== delta name-status previous->requested =="
git diff --name-status "$previous" "$requested" 2>&1 || true
echo "== delta patch =="
git diff "$previous" "$requested" 2>&1 || true

base="$(git merge-base origin/main "$requested" 2>&1 || true)"
echo "base=$base"
echo "== full PR name-status =="
git diff --name-status "$base" "$requested" 2>&1 || true
echo "== git diff --check full =="
git diff --check "$base" "$requested" 2>&1 || true

Length of output: 8498


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "=== assert-interactive-runner.ps1 full ==="
cat -n test/windows/assert-interactive-runner.ps1
echo "=== check-accessibility-evidence.ps1 relevant section ==="
sed -n '90,175p' scripts/check-accessibility-evidence.ps1
echo "=== grep runnerVersion in assert-interactive-runner.ps1 ==="
rg -n 'runnerVersion|minimumRunnerVersion|param\(' test/windows/assert-interactive-runner.ps1
echo "=== full Test-VerificationContracts.ps1 relevant block ==="
sed -n '1,300p' test/windows/flagship/Test-VerificationContracts.ps1

Length of output: 27452


@amanthanvi Reviewed immutable final head 56ab45ef52b91a16ef9c705cb730cf77b29f5448.

Delta since last reviewed head (fca99a2e67): commit ci: pin runner provenance enforcement, touching only scripts/check-accessibility-evidence.ps1 and test/windows/flagship/Test-VerificationContracts.ps1. No changes to .github/workflows/test.yml, src/apprt/win32.zig, src/termio/Termio.zig, test/windows/assert-interactive-runner.ps1, or the multitab/session-restore harness scripts in this delta — all previously validated and unchanged.

Actionable findings: 0.

Verified in detail:

  1. Provenance type-safety hardening is correct. check-accessibility-evidence.ps1 now rejects non-object provenance JSON ($provenance -isnot [pscustomobject]) before the schema_version check, and the four previously-inline [int]$provenance.<field> casts inside the aggregated OR-condition (windows_build, process_session_id, active_console_session_id, run_attempt) are replaced with explicit [int]::TryParse into typed locals, throwing a specific error on malformed numeric fields rather than an opaque cast exception or silent coercion. The OR-condition itself now references these typed variables ($provenanceWindowsBuild, $provenanceProcessSession, $provenanceActiveSession, $provenanceRunAttempt) — confirmed no leftover raw-cast usages remain.
  2. New anti-regression contract is meaningful. Assert-WorkflowContractAbsent (new helper) asserts assert-interactive-runner.ps1's param(...) block never reintroduces a MinimumRunnerVersion override — I confirmed the file's actual param() block only declares $OutputPath, so this contract currently passes and will fail-fast if anyone later adds a parameter to lower the pinned 2.327.1 floor.
  3. All new/updated regex contracts match the literal code verbatim — checked each: the anchored $minimumRunnerVersion = [version]'2.327.1' patterns (both scripts), $provenance -isnot [pscustomobject], the schema-version rejection string, $runnerVersion -lt $minimumRunnerVersion (assert-interactive-runner.ps1 Line 25), and the renamed $provenanceRunAttempt -ne [int]$run.run_attempt. No stale or mismatched patterns.

git diff --check on the complete PR diff is clean. No security or CI-correctness regressions in this head.

Residual non-blocker (carried over, unchanged): the runner-root derivation in assert-interactive-runner.ps1 still assumes the standard self-hosted runner directory layout (Split-Path -Parent (Split-Path -Parent $env:RUNNER_TEMP)bin\Runner.Worker.exe); it fails safe (throws) rather than silently skipping if that layout differs, so it's informational only.

[approve]

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The trace file ownership helpers (claimTraceFile/claimTracePath and initWithClaimedPath) are duplicated between win32.zig and Termio.zig; consider extracting a shared utility to avoid divergence in behavior or logging over time.
  • The new runner version validation in assert-interactive-runner.ps1 assumes a specific Runner.Worker.exe location derived from RUNNER_TEMP; it may be worth centralizing this path computation or adding a brief comment about the expected layout to make future maintenance less fragile if GitHub changes runner directory structure.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The trace file ownership helpers (claimTraceFile/claimTracePath and initWithClaimedPath) are duplicated between win32.zig and Termio.zig; consider extracting a shared utility to avoid divergence in behavior or logging over time.
- The new runner version validation in assert-interactive-runner.ps1 assumes a specific Runner.Worker.exe location derived from RUNNER_TEMP; it may be worth centralizing this path computation or adding a brief comment about the expected layout to make future maintenance less fragile if GitHub changes runner directory structure.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptile review immutable final head befa3e5 after JSON root-kind proof; require 5/5 and zero concrete actionable or residual findings.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review immutable final head befa3e5 after JSON root-kind proof; report every concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@sourcery-ai review immutable final head befa3e5 after JSON root-kind proof; report every concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review immutable final head befa3e5 after JSON root-kind proof; report every concrete actionable and residual finding.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The render/output trace ownership helpers (claimTraceFile/claimTracePath and initWithClaimedPath) are duplicated between win32.zig and Termio.zig; consider centralizing this logic to reduce repetition and keep future changes in sync.
  • The interactive runner preflight assumes Runner.Worker.exe can be found via RUNNER_TEMP ($runnerRoot\bin\Runner.Worker.exe); if GitHub alters runner layout this may start failing, so you might want a more robust discovery strategy or a clear failure message that guides updating the path.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The render/output trace ownership helpers (`claimTraceFile`/`claimTracePath` and `initWithClaimedPath`) are duplicated between `win32.zig` and `Termio.zig`; consider centralizing this logic to reduce repetition and keep future changes in sync.
- The interactive runner preflight assumes `Runner.Worker.exe` can be found via `RUNNER_TEMP` (`$runnerRoot\bin\Runner.Worker.exe`); if GitHub alters runner layout this may start failing, so you might want a more robust discovery strategy or a clear failure message that guides updating the path.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

requested="befa3e5771c77bdfa2e6155bb5c6d197a1660b59"
previous="56ab45ef52b91a16ef9c705cb730cf77b29f5448"

git fetch origin pull/87/head:pr87 2>&1 | tail -20
git checkout --detach pr87 >/dev/null 2>&1

echo "== checked-out head =="
git rev-parse HEAD

if ! git cat-file -e "${requested}^{commit}" 2>/dev/null; then
  git fetch origin "$requested" 2>&1 | tail -5 || true
fi
if ! git cat-file -e "${previous}^{commit}" 2>/dev/null; then
  git fetch origin "$previous" 2>&1 | tail -5 || true
fi

echo "== requested exists =="
git rev-parse "${requested}^{commit}" 2>&1 || echo "NOT FOUND"
echo "== previous exists =="
git rev-parse "${previous}^{commit}" 2>&1 || echo "NOT FOUND"

echo "== commit metadata for requested =="
git show -s --format='%H%n%P%n%s%n%an%n%ad' "$requested" 2>&1 || true

echo "== delta name-status previous->requested =="
git diff --name-status "$previous" "$requested" 2>&1 || true
echo "== delta patch =="
git diff "$previous" "$requested" 2>&1 || true

base="$(git merge-base origin/main "$requested" 2>&1 || true)"
echo "base=$base"
echo "== full PR name-status =="
git diff --name-status "$base" "$requested" 2>&1 || true
echo "== git diff --check full =="
git diff --check "$base" "$requested" 2>&1 || true

Length of output: 5829


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== assert-interactive-runner.ps1 relevant lines ==="
grep -n "MinimumRunnerVersion\|minimumRunnerVersion\|param(" -i test/windows/assert-interactive-runner.ps1

echo "=== Test-VerificationContracts.ps1 Assert-WorkflowContractAbsent definition ==="
rg -n "function Assert-WorkflowContractAbsent" -A 20 test/windows/flagship/Test-VerificationContracts.ps1

echo "=== check regex against actual file (simulate case-sensitivity) ==="
python3 - <<'EOF'
import re
with open('test/windows/assert-interactive-runner.ps1') as f:
    content = f.read()
pattern = r'(?m)^\s*\[version\]\s+\$MinimumRunnerVersion\b'
m = re.search(pattern, content)
print("Match (case-sensitive):", m)
pattern_ci = re.compile(pattern, re.IGNORECASE)
m2 = pattern_ci.search(content)
print("Match (case-insensitive):", m2)
EOF

echo "=== check-accessibility-evidence.ps1 full relevant section ==="
sed -n '95,150p' scripts/check-accessibility-evidence.ps1

Length of output: 6027


@amanthanvi Reviewed immutable final head befa3e5771c77bdfa2e6155bb5c6d197a1660b59.

Delta since last reviewed head (56ab45ef52): commit ci: validate provenance JSON root kind, touching only scripts/check-accessibility-evidence.ps1 and test/windows/flagship/Test-VerificationContracts.ps1. No changes to .github/workflows/test.yml, src/apprt/win32.zig, src/termio/Termio.zig, test/windows/assert-interactive-runner.ps1, or the multitab/session-restore harness scripts in this delta.

Actionable findings: 0.

This closes a real, subtle bypass in the provenance object-shape check:

  1. The actual gap. ConvertFrom-Json (without -NoEnumerate) silently unwraps a single-element JSON array into its inner object. That meant a provenance file shaped as [{...}] (an array wrapping one object) would previously pass $provenance -isnot [pscustomobject] after unwrapping — a genuine validation bypass for a malformed/malicious provenance root. Multi-element arrays were already caught; only the single-element case slipped through.
  2. The fix is correct. -NoEnumerate now preserves the JSON root's true shape (array stays an array, even with one element), and the check is restated as $null -eq $provenance -or $provenance.GetType() -ne [System.Management.Automation.PSCustomObject], which now correctly rejects an unwrapped-array-turned-object result since the object never gets unwrapped in the first place.
  3. Contract updates match the code verbatim — I confirmed both new Assert-WorkflowContract patterns (ConvertFrom-Json -NoEnumerate and the GetType() -ne [System.Management.Automation.PSCustomObject] literal) against the actual file content.
  4. The inline live self-test is a meaningful addition, not decorative. It runs ConvertFrom-Json -NoEnumerate against both a single-object and a single-element-array literal on the actual PowerShell runtime executing the verification suite, and throws if the root-kind distinction doesn't hold — this guards against relying on an assumption about -NoEnumerate behavior that could differ across PowerShell versions/editions.
  5. The Assert-WorkflowContractAbsent pattern tightening is precise and doesn't false-positive. I regex-tested the new pattern (^\s*\[version\]\s+\$MinimumRunnerVersion\b) against the actual assert-interactive-runner.ps1 content — it correctly does not match the legitimate $minimumRunnerVersion = [version]'2.327.1' assignment (different token order: type-before-variable declaration vs. variable-assignment-with-cast), so no risk of the contract falsely flagging existing, correct code.

git diff --check on the complete PR diff remains clean. No security or CI-correctness regressions in this head.

Residual non-blockers: none new. The previously-noted runner-root-layout assumption in assert-interactive-runner.ps1 is unchanged by this delta and remains informational only.

[approve]

@amanthanvi

Copy link
Copy Markdown
Owner Author

@greptile review immutable final head 7fae349; require 5/5 and zero current concrete actionable or residual findings.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@coderabbitai review immutable final head 7fae349; report every current concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@sourcery-ai review immutable final head 7fae349; report every current concrete actionable and residual finding.

@amanthanvi

Copy link
Copy Markdown
Owner Author

@codex review immutable final head 7fae349; report every current concrete actionable and residual finding.

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

requested="7fae3497676765ea73fceceb10f17fadbad9f2ec"
previous="befa3e5771c77bdfa2e6155bb5c6d197a1660b59"

git fetch origin pull/87/head:pr87 2>&1 | tail -20
git checkout --detach pr87 >/dev/null 2>&1

echo "== checked-out head =="
git rev-parse HEAD

if ! git cat-file -e "${requested}^{commit}" 2>/dev/null; then
  git fetch origin "$requested" 2>&1 | tail -5 || true
fi
if ! git cat-file -e "${previous}^{commit}" 2>/dev/null; then
  git fetch origin "$previous" 2>&1 | tail -5 || true
fi

echo "== requested exists =="
git rev-parse "${requested}^{commit}" 2>&1 || echo "NOT FOUND"
echo "== previous exists =="
git rev-parse "${previous}^{commit}" 2>&1 || echo "NOT FOUND"

echo "== commit metadata for requested =="
git show -s --format='%H%n%P%n%s%n%an%n%ad' "$requested" 2>&1 || true

echo "== delta name-status previous->requested =="
git diff --name-status "$previous" "$requested" 2>&1 || true
echo "== delta patch =="
git diff "$previous" "$requested" 2>&1 || true

base="$(git merge-base origin/main "$requested" 2>&1 || true)"
echo "base=$base"
echo "== full PR name-status =="
git diff --name-status "$base" "$requested" 2>&1 || true
echo "== git diff --check full =="
git diff --check "$base" "$requested" 2>&1 || true

Length of output: 3753


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== check-accessibility-evidence.ps1 head (first 20 lines) ==="
sed -n '1,20p' scripts/check-accessibility-evidence.ps1

echo "=== full context around ConvertFrom-Json -NoEnumerate line ==="
grep -n "ConvertFrom-Json\|NoEnumerate\|`#requires`" scripts/check-accessibility-evidence.ps1

echo "=== check for any PS7.1-specific syntax used in script (ternary ??, ?., pipeline chain &&/||) ==="
grep -n '??\|?\.\|&&\|||' scripts/check-accessibility-evidence.ps1 || echo "none found"

echo "=== Test-VerificationContracts.ps1 relevant block with context ==="
sed -n '250,300p' test/windows/flagship/Test-VerificationContracts.ps1

echo "=== confirm regex escaping validity with python ==="
python3 - <<'EOF'
import re
with open('scripts/check-accessibility-evidence.ps1') as f:
    content = f.read()

pat1 = r'(?m)^`#requires` -Version 7\.1\s*$'
m1 = re.search(pat1, content)
print("requires pattern match:", bool(m1))

pat2 = r'\$provenance = Get-Content -LiteralPath \$provenancePaths\[0\]\.FullName -Raw \| ConvertFrom-Json -NoEnumerate'
m2 = re.search(pat2, content)
print("provenance parse pattern match:", bool(m2))
EOF

echo "=== check other `#requires` usage across repo for consistency ==="
rg -n "`#requires`" -i test/windows scripts .github/workflows/test.yml

Length of output: 6428


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "=== where check-accessibility-evidence.ps1 is invoked ==="
rg -n "check-accessibility-evidence" .github/workflows/test.yml test/windows/flagship/Test-VerificationContracts.ps1 -C3
echo "=== check pwsh version guarantees in workflow ==="
rg -n "pwsh|shell:|PowerShell" .github/workflows/test.yml | head -30

Length of output: 1864


@amanthanvi Reviewed immutable final head 7fae3497676765ea73fceceb10f17fadbad9f2ec.

Delta since last reviewed head (befa3e5771): commit ci: bind provenance parser contract, touching only scripts/check-accessibility-evidence.ps1 and test/windows/flagship/Test-VerificationContracts.ps1. No changes to .github/workflows/test.yml, src/apprt/win32.zig, src/termio/Termio.zig, test/windows/assert-interactive-runner.ps1, or the multitab/session-restore harness scripts in this delta — all previously validated and unchanged.

Actionable findings: 0.

Verified in detail:

  1. The contract-binding fix is real, not cosmetic. The prior "preserves the JSON root kind" check matched the bare substring ConvertFrom-Json -NoEnumerate anywhere in the file. I confirmed there's exactly one such occurrence today (Line 117, the provenance-parsing statement), but that pattern would have silently kept "passing" even if -NoEnumerate were later added to an unrelated pipeline elsewhere while the provenance line itself regressed. The new pattern anchors to the full literal statement — $provenance = Get-Content -LiteralPath $provenancePaths[0].FullName -Raw | ConvertFrom-Json -NoEnumerate — so the contract now verifies the specific line that matters, not just the flag's presence somewhere in the file. Confirmed via direct regex match against the actual file content.
  2. New #requires -Version 7.1 and its matching contract are correctly placed and verified. It's the true first line of check-accessibility-evidence.ps1 (a valid position for a #requires statement), and the new Assert-WorkflowContract pattern ((?m)^#requires -Version 7\.1\s*$) matches it exactly.
  3. Checked for PowerShell-7.1-specific syntax (??, ?., &&/|| pipeline chains) in the script — none found. -NoEnumerate on ConvertFrom-Json has existed since PowerShell 6.0, so this floor is a conservative safety pin rather than a functional necessity, and it fails closed (throws on load) rather than silently degrading if unmet.

git diff --check on the complete PR diff remains clean. No security or CI-correctness regressions in this head.

Residual non-blocker (informational only, new): scripts/check-accessibility-evidence.ps1 isn't invoked anywhere in .github/workflows/test.yml — it's a release-time checker (Test-VerificationContracts.ps1 at Line 197 references it by path, but its actual invoking workflow, likely release-readiness.yml, is outside this PR's diff). This delta can't independently confirm that call site runs on pwsh ≥ 7.1; however, the new #requires directive means any mismatch fails immediately and visibly rather than causing silent misbehavior, so this is not a functional risk — just outside this diff's visibility.

Residual non-blocker (carried over, unchanged): the runner-root-layout assumption in assert-interactive-runner.ps1 remains informational only.

[approve]

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The trace file ownership helpers (claimTraceFile/claimTracePath) are duplicated in both win32.zig and Termio.zig; consider factoring them into a shared utility to avoid divergence if the logic needs to change later.
  • The inline JSON root-kind preservation check in Test-VerificationContracts.ps1 is fairly dense; moving it into a named helper function (e.g., Assert-JsonRootKindPreserved) would make the intent clearer and ease future adjustments.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The trace file ownership helpers (`claimTraceFile`/`claimTracePath`) are duplicated in both `win32.zig` and `Termio.zig`; consider factoring them into a shared utility to avoid divergence if the logic needs to change later.
- The inline JSON root-kind preservation check in `Test-VerificationContracts.ps1` is fairly dense; moving it into a named helper function (e.g., `Assert-JsonRootKindPreserved`) would make the intent clearer and ease future adjustments.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 7fae349767

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/windows/flagship/Test-VerificationContracts.ps1 (1)

293-298: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Guard -NoEnumerate for Windows PowerShell 5.1
ConvertFrom-Json -NoEnumerate is only available in PowerShell 7+, so this test can fail with a parameter-binding error on Windows PowerShell 5.1 even if Assert-JsonDocument falls back successfully. Add #requires -Version 7.1 here or wrap the -NoEnumerate calls in a version check.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/windows/flagship/Test-VerificationContracts.ps1` around lines 293 - 298,
Guard the -NoEnumerate calls in the root-kind preservation check by requiring
PowerShell 7.1 or conditionally executing them only when the runtime supports
that parameter. Ensure Windows PowerShell 5.1 skips this evidence check while
preserving the existing assertions on supported versions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@test/windows/flagship/Test-VerificationContracts.ps1`:
- Around line 293-298: Guard the -NoEnumerate calls in the root-kind
preservation check by requiring PowerShell 7.1 or conditionally executing them
only when the runtime supports that parameter. Ensure Windows PowerShell 5.1
skips this evidence check while preserving the existing assertions on supported
versions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c3c10920-b7cb-48f1-b499-e2c92cc5f9c4

📥 Commits

Reviewing files that changed from the base of the PR and between befa3e5 and 7fae349.

📒 Files selected for processing (2)
  • scripts/check-accessibility-evidence.ps1
  • test/windows/flagship/Test-VerificationContracts.ps1

@amanthanvi amanthanvi merged commit 582dd08 into main Jul 13, 2026
10 checks passed
@amanthanvi amanthanvi deleted the codex/fix-interactive-evidence-upload branch July 13, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants