Skip to content

fix(tests): align full-route fixtures with canonical receipt contracts - #3353

Merged
Daniel Meppiel (danielmeppiel) merged 3 commits into
mainfrom
danielmeppiel-ci-release-fixture-contracts
Oct 22, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 3 commits into
mainfrom
danielmeppiel-ci-release-fixture-contracts

Conversation

@danielmeppiel

@danielmeppiel Daniel Meppiel (danielmeppiel) commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

fix(tests): align full-route fixtures with canonical receipt contracts

Description

TL;DR

Correct the fixtures behind ten failures in the trusted-main nonpublishing run: two mutation anchors, six generated-lifecycle cases, and two manually rebound receipt cases. Preserve production authorization and receipt guards, while adding seven scope-boundary cases and two trusted-receipt controls. The final head passes 298 focused cases and its hosted CI; the separate macOS ARM timeout remains unresolved.

Important

This is a four-file test/documentation repair, not a declaration that the release pipeline is green. No workflow, timeout, production code, policy, publication condition, or release version changes.

Problem (WHY)

  • Two uninstall mutation fixtures no longer matched the canonical sink's indentation, so they failed before proving mutant detection.
  • Generated lifecycle snapshots rejected legitimate installation-identity writes. Project snapshots also needed the exact user-state receipt, including after prune.
  • Manually rewriting claim targets invalidated their receipt binding. The untracked-content fixture then expected retirement despite the real no trustworthy local deployment baseline refusal.
  • [!] The same full run exhausted macOS ARM's 60-minute integration limit while workers were still progressing. That separate scheduling problem is deliberately not folded into this repair.

The actual failure evidence is Linux ARM job 119851234998, Intel job 119843775989, and ARM job 119843776399, all on abd363ff.

The diagnosis follows "Read agent execution traces, not just final outputs."; qualification follows "Only proceed when validation passes". Failed candidates and failed evidence guards remain failed, rather than being rewritten as successes.

Approach (WHAT)

  • Retarget only the two mutation anchors; keep their AST parsing and exact-rule diagnostic assertions.
  • Admit three identity paths at the correct root, plus one project receipt filename derived read-only through the canonical owner under the fixture HOME.
  • Use the same scoped snapshot oracle for install and prune; explicitly reject identity siblings, wrong-root identities, and another project receipt.
  • Keep legacy-rebound preservation, warning, and failing-audit assertions; add untouched-receipt controls that retain the original retirement and personal-file safety assertions.

Implementation (HOW)

File Intent
test_architecture_install_compound_mutations.py Match the current canonical uninstall call without weakening mutant detection.
test_generated_lifecycle_state_machine.py Centralize the fixture's exact receipt allowance and use it after prune; add seven real-CLI boundary cases.
test_obsolete_dependency_claim_lifecycle.py Distinguish trusted receipts from fixture-invalidated legacy bindings at project and user scope.
integration-testing.md Document narrow snapshot allowances and receipt-binding controls.

Diagrams

Dashed nodes identify changed fixture/oracle behavior; the reinstall outcomes remain existing production behavior.

flowchart LR
    subgraph Fixtures["Receipt fixtures"]
        I["Real install writes local receipt"]
        T["Trusted control: keep claim binding"]
        L["Legacy control: rewrite claim target"]
        I --> T
        I --> L
    end
    subgraph Production["Existing reinstall behavior"]
        R["Retire absent owned claim"]
        K["Retain claim and warn"]
        P["Preserve personal bytes without claiming them"]
        T --> R
        L --> K
        R --> P
        K --> P
    end
    subgraph Oracle["Scoped snapshot oracle"]
        A["Three identity paths and exact project receipt"]
        B["Install and prune use fixture write set"]
        C["Reject siblings and wrong-root writes"]
        A --> B --> C
    end
    classDef new stroke-dasharray: 5 5;
    class T,A,B,C new;
Loading

Trade-offs

  • Exact allowances intentionally expose future write-set changes; allowing whole identity or project-receipt subtrees would hide unrelated writes.
  • The fixture invokes the existing receipt owner read-only under its isolated HOME instead of duplicating the filename-key algorithm. Existing global .apm permissions are unchanged.
  • Trusted and untrusted receipt scenarios are separate: forcing both to retire claims would weaken the production baseline contract.
  • The ARM timeout needs separate scheduling work. This PR neither increases budgets nor removes tests, and does not authorize another full dispatch by itself.

Benefits

  1. Both mutation cases reach their exact-rule violation assertions again.
  2. All six original project/global model cases execute successfully, including prune.
  3. Seven boundary cases reject unrelated writes; four project/user receipt scenarios distinguish trustworthy from invalidated evidence.

Issue and approved scope

Issue: None. The maintainer explicitly authorized this bounded CI-recovery work without a new issue/approval round. This is a task-specific exception, not an accepted-issue claim or a general governance change.

Human scope-approval comment: Direct task instruction, not a fabricated GitHub approval comment. Scope is the demonstrated full-route fixture corrections starting from actual public 33f738e7, with focused qualification, independent review, and one carrier.

Does this PR complete the issue, or what remains? No issue is closed. It repairs the demonstrated fixture defects; actual changed-main CI/full nonpublishing validation and the separate ARM timeout remain outstanding.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

Validation

Qualified head: 3c798e1305fcfe9ecee33851314e427936addf74, based on actual public 33f738e7bdb17e948d5551ae5177a9d01bcc272e. Fresh REST and GraphQL branch refs agreed; the worktree stayed clean through execution. Canonical changed-owner detection returned zero.

Evidence Actual result
Three changed modules plus tests/quality 298 unique passed; 0 failures, errors, or skips
Widened identity/receipt permission control 7 intended failures at the missing-rejection assertion
Disabled exact target-rule control 2 intended failures because that exact rule's diagnostic disappeared
Forced legacy rebinding of trusted controls 2 intended failures at absent-claim retirement assertions
Independent coordinator source/delta reviews Final static finding closed; bounded review, not formal panel approval
Own hosted CI SUCCESS; all 16 declared jobs matched by workflow/matrix identity and passed
Merge Gate / CodeQL SUCCESS; both real CodeQL analyses and the same-attempt workflow aggregate completed
Copilot review at this head COMMENTED with no blocking findings; no inline comments
Historical full run 39784442659 Failed; Linux x86 cancelled; required Release Validation skipped; ARM integration timed out

The separate neutral CodeQL check reports a missing imposed-native API upload / <default> configuration. It remains explicitly excepted, not passed. Docs deployment is correctly skipped for this pull-request event. CodeQL's earlier aggregate delay is preserved; attempt 1 later completed without cancellation, rerun, or waiver.

The negative target-rule controls still returned other diagnostics; they failed the exact-rule assertion, not an assumed overall exit-code change. Earlier d336 and cf88 qualification failures, and a stricter intake guard's wrong exit-code expectation, remain preserved.

Executed focused command and output

Absolute local workspace/evidence prefixes are represented below by $ROOT and $EVIDENCE; executable, arguments, prerequisite flags, and outcomes are unchanged.

APM_BINARY_PATH="$ROOT/.venv/bin/apm" APM_E2E_TESTS=1 \
  "$ROOT/.venv/bin/python3" -m pytest \
  tests/integration/test_architecture_install_compound_mutations.py \
  tests/integration/test_generated_lifecycle_state_machine.py \
  tests/integration/test_obsolete_dependency_claim_lifecycle.py tests/quality \
  -p no:randomly -p no:cacheprovider -q --color=no \
  --junitxml="$EVIDENCE/fullroute-v3-restored.xml" \
  --basetemp="$EVIDENCE/fullroute-v3-restored-tmp"
298 passed in 785.18s (0:13:05)

Populations: 93 compound-mutation, 14 generated-model, 30 obsolete-claim, and 161 quality cases. The six original model nodes, seven new boundaries, and four trusted/untrusted receipt nodes are included, not added to that total.

Current-head canonical lint commands: every command exited 0
Exact command Result
uv run --frozen --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py exit 0
uv run --frozen --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py exit 0
uv run --frozen --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ scripts/windows_native_symlink_probe_entry.py exit 0
bash scripts/lint-auth-signals.sh exit 0
bash scripts/lint-architecture-boundaries.sh exit 0
uv run --frozen python scripts/check_test_assertions.py exit 0
uv run --frozen python scripts/check_exact_test_duplicates.py exit 0

The current CI YAML-I/O, 2100-line, and portable-relative-path guards also passed. Diagram validation: mmdc 12.0.0 rendered successfully and the image was inspected. No conformance regeneration or product changelog entry is needed for these test-only corrections.

Scenario Evidence

# Scenario (user promise) Principle(s) Test(s) proving it Type
1 Install, update, prune, and remove packages without unrelated writes Secure by default, DevX (pragmatic as npm) tests/integration/test_generated_lifecycle_state_machine.py::test_generated_lifecycle_mandatory_replay and test_generated_lifecycle_sequences_preserve_reference_model (regression traps for this CI failure) e2e
2 A project operation cannot silently write another identity or receipt Secure by default tests/integration/test_generated_lifecycle_state_machine.py::test_generated_identity_write_scope_is_exact e2e
3 Preserve personal files while retiring only claims supported by trustworthy local evidence Secure by default, Governed by policy tests/integration/test_obsolete_dependency_claim_lifecycle.py::test_historical_absence_respects_live_preservation_boundaries e2e

How to test

  • Use the project's installed .venv/bin/apm entry point; set APM_BINARY_PATH to it and APM_E2E_TESTS=1, then run the focused command above with fresh output paths. Expect 298 passes and no skips.
  • Run -k test_generated_identity_write_scope_is_exact on the generated-model module. Expect seven passes covering identity siblings, wrong roots, and a project receipt sibling.
  • Run the obsolete-claim module with -k untracked-content. Expect four passes: trusted claims retire; rebound claims remain with the exact warning; personal bytes remain unclaimed in both.
  • Run the seven lint commands and static CI guards above. Expect no diagnostics.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Keep canonical materialization guards unchanged while correcting mutation anchors, exact identity write allowances, and trusted versus rebound receipt controls.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 067b35fc-52fb-4db4-b316-c606d23b4b34
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 067b35fc-52fb-4db4-b316-c606d23b4b34
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 067b35fc-52fb-4db4-b316-c606d23b4b34
@danielmeppiel

Daniel Meppiel (danielmeppiel) commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

Docs sync advisory

Verdict: no_change * Pages affected: 0 * LLM calls: 0/15 * Took: 1.06s

The canonical L0 path gate matches all four changed paths under tests/** or docs/**, with no user-surface path match. The contributor fixture guidance is already included; no additional corpus update is indicated.

This was a direct deterministic classification, not a persona panel, formal approval, or merge recommendation. No panel agents were spawned.


Generated by docs-sync. This comment is AI-generated and may contain errors.

Merge Master assessment

ship_now for 3c798e1305fcfe9ecee33851314e427936addf74 onto current main 33f738e7bdb17e948d5551ae5177a9d01bcc272e, limited to this four-file fixture/documentation repair.

The separate-actor bounded reviews and this released-packet qualification leave no outstanding in-scope finding. All 104 artifact seals, the complete completion schema and strict terminal owner verification pass (zero production owners touched). Original execution events support 298 unique passing cases, 11 intended negative assertions with every exact counterpart restored, seven canonical lint commands and three static guards. No tests were replayed for this assessment.

Current-head CI and Merge Gate pass, including all 16 declared CI jobs. CodeQL's actual workflow and both analyses succeeded naturally on the original attempt; no cancellation or rerun occurred. Direct workflow/job/check endpoints reconcile the metadata replacement and incomplete discovery index. Both source guard failures and the MM auxiliary-inventory correction remain recorded as failures, not promoted results. The bot review is COMMENTED, not formal approval; this is not a persona panel.

Current admin/bypass capability, exact source/main compatibility, released ownership, final body and all 14 protected open issues are verified. Landing uses the maintainer's standing bounded CI-repair authorization with no closing issue. The imposed native CodeQL comparison remains neutral/missing under the existing explicit exception, not a passed analysis.

Still open: the separate parallel macOS ARM scheduling repair and a successful full actual-main nonpublishing qualification. This PR does not establish overall release recovery, authorize a dispatch, change production guards or increase time budgets.


Generated by autopilot-pr-merge-worker. This comment is AI-generated and may contain errors.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bounded fixture and documentation repairs preserve production safeguards, with no unresolved blocking findings.

No open issues 🎉

What changed in this PR

This PR repairs lifecycle test fixtures to match existing installation-identity and receipt contracts without changing production behavior or release workflows. The separate macOS ARM timeout remains outside scope.

Changes:

  • Correct two uninstall mutation anchors while preserving exact-rule assertions.
  • Add narrowly scoped identity and receipt snapshot allowances, including boundary rejection tests.
  • Separate trusted-receipt recovery from invalidated-receipt preservation and document the fixture requirements.
File Description
tests/​integration/​test_obsolete_dependency_claim_lifecycle.py Separates trusted and rebound receipt cases while retaining personal-file safety checks.
tests/​integration/​test_generated_lifecycle_state_machine.py Shares scoped snapshot allowances across install and prune and adds seven boundary cases.
tests/​integration/​test_architecture_install_compound_mutations.py Aligns two mutation anchors with current uninstall indentation.
docs/​src/​content/​docs/​contributing/​integration-testing.md Documents exact identity allowances and receipt-binding controls.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants