Skip to content

feat(ci): automate asset docs governance with completeness and region-sync validation#2480

Open
jkim323 wants to merge 14 commits into
mainfrom
feat/2338-automate-governance
Open

feat(ci): automate asset docs governance with completeness and region-sync validation#2480
jkim323 wants to merge 14 commits into
mainfrom
feat/2338-automate-governance

Conversation

@jkim323

@jkim323 jkim323 commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Automates governance of the AI asset documentation catalog so that every agent, prompt, instruction, and skill stays documented and in sync with its generated reference page.

Key changes:

  • Asset-doc validation — extends scripts/linting/Validate-AssetDocs.ps1 with completeness and region-sync checks, backed by helper additions in scripts/linting/Modules/LintingHelpers.psm1 and new Pester coverage.
  • CI enforcement — adds the asset-docs-validation.yml workflow and gates pull requests on it through pr-validation.yml, so incomplete or drifted asset docs fail the PR.
  • Workflow hardening — prevents input expression injection in the asset-docs validation workflow.
  • Catalog regeneration — regenerates the docs/reference/** asset reference catalog: adds subagent reference pages (for example hve-artifact-*, accessibility-surface-inventory, rpi-walkthrough), renames prompt-updater to hve-artifact-explorer and related pages, and refreshes README counts.
  • Build flag — minor package.json script/flag update supporting the validation run.

Related Issue(s)

Closes #2338
Closes #2336

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update (N/A — package.json change is a build flag, not a dependency bump)

AI Artifacts:

  • [] Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

Automated validation relevant to this change:

  • Added Pester coverage: scripts/tests/linting/Validate-AssetDocs.Tests.ps1 (+112) and scripts/tests/linting/LintingHelpers.Tests.ps1 (+37) exercise the new completeness and region-sync logic.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Eval spec schema and coverage (if AI artifacts changed): npm run eval:lint:schema
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

@jkim323
jkim323 requested a review from a team as a code owner July 19, 2026 23:52
@jkim323 jkim323 changed the title Feat/2338 automate governance feat(ci): automate asset docs governance with completeness and region-sync validation Jul 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
actions/actions/checkout 9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 🟢 7
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Maintained🟢 1020 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 10no binaries found in the repo
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Packaging⚠️ -1packaging workflow not detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
SAST🟢 10SAST tool is run on all commits
Branch-Protection🟢 6branch protection is not maximal on development and all release branches
actions/actions/upload-artifact 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a 🟢 5.2
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1packaging workflow not detected
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 1dependency not pinned by hash detected -- score normalized to 1
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
SAST🟢 10SAST tool is run on all commits
Branch-Protection⚠️ 0branch protection not enabled on development/release branches

Scanned Files

  • .github/workflows/asset-docs-validation.yml

@github-actions github-actions Bot mentioned this pull request Jul 19, 2026
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Passed

  • Artifacts evaluated: 0
  • Specs run: 0
  • Assertions passed: 0
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 0
  • Failed specs (merge-blocking): 0

No changed AI artifacts required evaluation.

@katriendg katriendg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The line-ending normalization in Validate-AssetDocs.ps1 is a clean, well-motivated fix — it normalizes both sides of the generated-region comparison so Windows/autocrlf checkouts no longer report false CRLF drift, and the adjacent tamper test confirms real drift detection is unaffected. New CRLF Pester test passes (41/41 locally), and the asset-docs.md contributor guide is thorough with all command and cross-link references resolving. Two optional non-blocking nits left inline (a lone-CR test case and an ms.date freshness note) — neither blocks merge.

Comment thread scripts/tests/linting/Validate-AssetDocs.Tests.ps1
Comment thread docs/contributing/ai-artifacts-common.md Outdated
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.74%. Comparing base (53ddf1a) to head (77e1c7a).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2480      +/-   ##
==========================================
- Coverage   82.76%   82.74%   -0.02%     
==========================================
  Files         154      154              
  Lines       20853    20838      -15     
  Branches       13       13              
==========================================
- Hits        17259    17243      -16     
- Misses       3592     3593       +1     
  Partials        2        2              
Flag Coverage Δ
docusaurus 94.44% <ø> (ø)
pester 86.26% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/linting/Validate-AssetDocs.ps1 94.79% <100.00%> (+0.06%) ⬆️
scripts/security/Update-ActionSHAPinning.ps1 72.43% <ø> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jkim323

jkim323 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

@katriendg / @rezatnoMsirhC another set of eyes would be appreciated ty!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: [6/7] automation governance to prevent docs drift feat: [7/7] full validation of the asset-docs system

5 participants