Skip to content

fix(audit): replay the project's own content the way install deploys it - #3209

Open
Aditya Nikam (adityaanikam) wants to merge 1 commit into
microsoft:mainfrom
adityaanikam:fix/audit-project-self-replay-3151
Open

Aditya Nikam (adityaanikam) wants to merge 1 commit into
microsoft:mainfrom
adityaanikam:fix/audit-project-self-replay-3151

Conversation

@adityaanikam

Copy link
Copy Markdown

Description

apm audit reported a root skills/ folder as unintegrated drift after a clean apm install, although install never deploys it.

Install integrates the project's own content through a synthetic _local package that is always an APM_PACKAGE, so only .apm/ content is deployed. The audit replay built the package for the project's own lock entry from apm.yml and detected its type from the project root. For a project with .apm/ content and a sibling skills/<name>/SKILL.md that detection gives SKILL_BUNDLE, so the replay promoted skills/<name> and the diff showed it as missing.

Both paths now build that package in one place, build_project_local_package_info in the new install/project_package.py. integrate_local_content (install) and the project's own entry in run_replay (audit) call it. Dependencies in the replay still use _build_package_info as before, so their behavior does not change.

Issue and approved scope

Issue: #3151

Human scope-approval comment: #3151 (comment) (acceptance clarification: #3151 (comment))

Does this PR complete the issue, or what remains? Fixes #3151.

How it follows the acceptance clarification:

  • Install and the project's own replay entry use the same package builder. I did not add a second classification. A boundary guard requires both call sites: the existing install-deployment-audit-replay rule now fails if either one stops using build_project_local_package_info.
  • The lifecycle test tests/integration/test_project_self_audit_lifecycle.py runs install, audit, reinstall and audit for explicit and auto includes, each with one target and with two targets. It asserts the deployed bytes, that nothing is deployed for the sibling skills/hello, that local_deployed_files lists only the instruction paths, and that the audit leaves the project state unchanged. It then deletes a managed file and, after a reinstall, edits it: the audit fails with deployed-files-present for the deletion and with content-integrity and drift for the edit, and a reinstall repairs both.
  • Nothing is exempted from the audit. A genuinely missing or changed managed file still fails it, as the test shows.
  • Added tests/fixtures/lifecycle_bug_ledger.json row for [BUG] apm audit reports a root skills/ folder as unintegrated drift, although install never deploys it #3151, a CHANGELOG entry, and a short paragraph in drift-and-secure-by-default.md.
  • drift.py is now 1025 lines, so I raised its entry in KNOWN_LARGE_MODULES from 1020 to 1025. services.py got smaller.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

On Windows with Python 3.12, using uv run, with APM_E2E_TESTS=1 and APM_BINARY_PATH pointing at the apm.exe of the checkout:

  • test_project_self_audit_lifecycle.py: 4 passed. With the drift.py change reverted, all 4 fail at the first audit after a clean install (exit 1).
  • tests/unit selected with -k "drift or replay or audit or local_content or project_package": 903 passed, 9 skipped.
  • tests/quality: 64 passed.
  • tests/unit/install, scripts, integration, commands, core and deps: 8926 passed, 91 failed, 4 errors. The same 91 failures and 4 errors occur on main on my machine (the ones I looked at create symlinks and fail with WinError 1314, no symlink privilege), and the sets of failing tests are identical with and without this change.
  • test_architecture_owner_rule_mutations.py, test_deps_models_validation.py, test_drift_detection.py, test_services.py and test_architecture_invariants.py pass, apart from the symlink test in test_services.py that fails the same way on main.
  • scripts/lint_architecture_boundaries.py and ruff check / ruff format --check on the changed files are clean.

I did not run the whole test suite, so that box is unchecked.

Spec conformance (OpenAPM v0.1)

  • N/A -- this PR does not change OpenAPM-observable behaviour.

apm install integrates the project's own content through a synthetic
_local package that is always an APM_PACKAGE, so a sibling
skills/<name>/SKILL.md next to .apm/ is never deployed. Audit replay
rebuilt the package for the project's own lock entry and detected its
type from the project root, which is SKILL_BUNDLE for that layout. The
replay promoted skills/<name>, so apm audit --ci failed a clean install
with an unintegrated drift finding.

Build the synthetic package in one place, build_project_local_package_info,
and use it for both install and the project's own entry in the replay.
The install-deployment-audit-replay guard now requires both call sites.

Add a lifecycle test that installs, audits, reinstalls and audits again
for explicit and automatic includes with one and two targets, and checks
that a missing or changed managed file still fails the audit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] apm audit reports a root skills/ folder as unintegrated drift, although install never deploys it

2 participants