feat(skills): add release-readiness-gate skill and release-readiness collection#2422
feat(skills): add release-readiness-gate skill and release-readiness collection#2422rekhako wants to merge 6 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2422 +/- ##
==========================================
- Coverage 82.75% 82.75% -0.01%
==========================================
Files 154 154
Lines 20844 20841 -3
Branches 13 13
==========================================
- Hits 17250 17246 -4
- Misses 3592 3593 +1
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Adds a new release-readiness collection and a release-readiness-gate skill intended to consolidate multiple planner artifacts into a single Go / Conditional-Go / No-Go release decision and scorecard, and wires it into the existing marketplace + hve-core-all bundle.
Changes:
- Introduces the documentation-driven
release-readiness-gateskill with default pillars, verdict rules, and a scorecard output format. - Adds a new
release-readinesscollection (YAML + docs) and registers it in the marketplace andhve-core-all. - Extends shared governance instructions (
disclaimer-language,untrusted-content-boundary) to cover.copilot-tracking/release-readiness/**.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| plugins/release-readiness/skills/release-readiness/release-readiness-gate | Adds the plugin’s release-readiness-gate entry (currently as a 1-line pointer file). |
| plugins/release-readiness/scripts/lib | Adds plugin scripts/lib entry (currently as a 1-line pointer file). |
| plugins/release-readiness/README.md | Adds plugin README describing the release-readiness collection contents. |
| plugins/release-readiness/instructions/shared/untrusted-content-boundary.instructions.md | Adds plugin instruction entry (currently as a 1-line pointer file). |
| plugins/release-readiness/instructions/shared/hve-core-location.instructions.md | Adds plugin instruction entry (currently as a 1-line pointer file). |
| plugins/release-readiness/instructions/shared/disclaimer-language.instructions.md | Adds plugin instruction entry (currently as a 1-line pointer file). |
| plugins/release-readiness/docs/templates | Adds plugin docs/templates entry (currently as a 1-line pointer file). |
| plugins/release-readiness/.github/plugin/plugin.json | Defines the new release-readiness plugin metadata and skills root. |
| plugins/hve-core-all/skills/release-readiness/release-readiness-gate | Adds the release-readiness gate entry to the hve-core-all plugin (currently as a 1-line pointer file). |
| plugins/hve-core-all/README.md | Regenerates the hve-core-all plugin README to include the new skill. |
| plugins/hve-core-all/.github/plugin/plugin.json | Registers skills/release-readiness/ in the hve-core-all plugin. |
| collections/release-readiness.collection.yml | Adds the new release-readiness collection manifest. |
| collections/release-readiness.collection.md | Adds the new release-readiness collection documentation. |
| collections/hve-core-all.collection.yml | Registers the new gate skill in the hve-core-all collection manifest. |
| collections/hve-core-all.collection.md | Regenerates the hve-core-all collection documentation to include the new skill. |
| .github/skills/release-readiness/release-readiness-gate/SKILL.md | Adds the release-readiness-gate skill playbook (inputs, pillars, verdict rules, output format, handoff). |
| .github/plugin/marketplace.json | Registers the release-readiness plugin in the marketplace list. |
| .github/instructions/shared/untrusted-content-boundary.instructions.md | Adds .copilot-tracking/release-readiness/** to the untrusted-content boundary applyTo globs. |
| .github/instructions/shared/disclaimer-language.instructions.md | Adds .copilot-tracking/release-readiness/** to the disclaimer-language applyTo globs. |
|
|
||
| | Pillar | Reads as ready when... | Evidence source | | ||
| |--------|------------------------|-----------------| | ||
| | Reliability & Performance | SLOs defined and load behavior characterized | performance-slo-planner (`performance-plans/`) | |
There was a problem hiding this comment.
Update: after weighing this against the merge-order risk raised in the companion thread, we went with the decouple option in dc6ce9a2 — the Reliability & Performance row now reads "No dedicated planner; codebase signals (CI, load-test results, performance-plans/ when available)", matching the existing Observability/Operational Readiness rows. This keeps #2422 self-contained rather than gating it on #2420. Superseding my earlier "keeping the reference" note above.
|
|
||
| This skill produces a decision, not a fix. After writing the scorecard: | ||
|
|
||
| - **No-Go / Red blockers** — route each blocking gap back to the pillar's owning planner (for example a Security Red → Security Planner, a Privacy Red → Privacy Planner, a Reliability Red → performance-slo-planner). Do not remediate here. |
There was a problem hiding this comment.
Update: after weighing this against the merge-order risk raised in the companion thread, we went with the decouple option in dc6ce9a2 — the Reliability & Performance row now reads "No dedicated planner; codebase signals (CI, load-test results, performance-plans/ when available)", matching the existing Observability/Operational Readiness rows. This keeps #2422 self-contained rather than gating it on #2420. Superseding my earlier "keeping the reference" note above.
katriendg
left a comment
katriendg
left a comment
There was a problem hiding this comment.
Thanks for this contribution — the skill logic and verdict rules are solid and the handoff contract is well-thought-out. There are a few things to address before merging, most are added inline where possible:
High — collection placement: The new release-readiness collection contains only one skill, and all its upstream planner dependencies (rai-planner, security-planning, privacy-standards, etc.) already live in project-planning. Please drop the standalone collection and add the skill to project-planning.collection.yml instead. We are trying to keep collection count minimal and related items together, and potentially diminish the amount of collections in future.
Medium — skill description verbosity: The description frontmatter is 259 words with USE FOR / DO NOT USE FOR blocks. This hurts Tier 1 semantic invocation precision. Please replace it with a single focused sentence — the content is still available in the skill body.
Medium — playbook section order: The skill body is missing canonical ## Success criteria, ## Constraints, and ## Stop rules sections, and ## Output Format appears before ## Handoff. The content is all there — it just needs to be reorganized into the named sections the prompt-builder template requires.
Low — progressive disclosure: The Default Pillars table and scorecard template are good candidates to extract into references/ and templates/ to keep the skill body compact.
You will also see there are still some issues already logged by Copilot around the plugin generation and symlinks.
| 2. **Specialist plan artifacts** — any existing planner outputs under `.copilot-tracking/` (for example `rai-plans/`, `security-plans/`, `sssc-plans/`, `performance-plans/`, `privacy-plans/`, `accessibility/`). | ||
| 3. **Codebase signals** — CI config, test coverage, observability wiring, error handling, and any open backlog. | ||
|
|
||
| ## Default Pillars |
There was a problem hiding this comment.
F4 [Standards] — Progressive disclosure opportunity (Low)
The Default Pillars table is detailed reference material that could live in a references/pillars.md file, loaded on demand rather than inline in SKILL.md. Similarly, the full scorecard template (line 65) is a good candidate for templates/scorecard.md. The prompt-builder guidance recommends keeping SKILL.md compact and using references/ and templates/ for detailed material.
There was a problem hiding this comment.
Good idea. Deferring to a follow-up (#2430) to keep this PR focused on the placement + structure changes; it moves Default Pillars to references/pillars.md and the scorecard to templates/scorecard.md. Please let me know if you would prefer it to land in this PR instead.
…collection Go/No-Go release gate consolidating specialist planner outputs into an evidence-grounded ship decision (RAG scorecard, blocking gaps, sign-off, Handoff). Adds a new release-readiness collection bundling the gate skill and shared governance instructions, brings the release-readiness output folder under the disclaimer and untrusted-content globs, registers the gate in hve-core-all, and adds the release-readiness plugin to marketplace.json.
27b7566 to
dafddeb
Compare
- move collection registration into project-planning; delete standalone collection - shorten description; restructure into Goal/Success criteria/Constraints/Stop rules - add Accessibility to consolidated planners; remove em dashes - regenerate plugins and marketplace 🚀 - Generated by Copilot
|
Thanks all for the thorough review. Summary of the latest changes:
Deferred: F4 (progressive disclosure), tracked as follow-up #2430. |
|
|
||
| | Pillar | Reads as ready when... | Evidence source | | ||
| |---------------------------|----------------------------------------------------|---------------------------------------------------------------------------| | ||
| | Reliability & Performance | SLOs defined and load behavior characterized | performance-slo-planner (`performance-plans/`) | |
There was a problem hiding this comment.
Both the Default Pillars table (here) and the Handoff section (line 85) reference performance-slo-planner as the owning planner for Reliability & Performance, but that skill is in companion PR #2420 and has not yet merged. If this PR lands first, the routing guidance for that pillar is non-actionable.
Two options to consider:
Option A — enforce merge order. Merge #2420 before or alongside this PR. The reference is accurate from day one and requires no cleanup.
Option D — mirror the no-dedicated-planner pattern. The Observability and Operational Readiness rows already use "No dedicated planner; codebase signals (...)." Applying the same pattern to Reliability & Performance until #2420 lands is self-explanatory and requires no annotation cleanup afterward:
| Reliability & Performance | SLOs defined and load behavior characterized | No dedicated planner until #2420 lands; codebase signals (CI, load test results, `performance-plans/` when available) |Once #2420 merges, the Evidence column for this row can be updated in that PR to point to the new planner. Option D has no time-bomb annotation to remove later.
There was a problem hiding this comment.
Went with Option D in dc6ce9a2. The Reliability & Performance evidence source now reads No dedicated planner; codebase signals (CI, load-test results, performance-plans/ when available), matching the existing Observability and Operational Readiness rows. This keeps #2422 self-contained (no merge-order dependency on #2420) and, per your point, leaves no time-bomb annotation to remove later — I deliberately dropped the #2420 from the wording so it reads as a steady-state pattern rather than a dated placeholder. When #2420 lands it can update this row to point at the new planner as part of its own diff. I also fixed the Handoff example, which referenced the same non-existent planner, to route a Supply Chain Red -> SSSC Planner and added a line covering how no-dedicated-planner pillars are handed off. Thanks!
| - **Go:** hand the signed-off scorecard to the launch owner as the go/no-go record. | ||
| - Re-run the gate whenever a routed gap closes or a specialist artifact changes, so the verdict reflects current evidence. | ||
|
|
||
| ## Output Format |
There was a problem hiding this comment.
The ## Constraints section correctly mandates carrying the professional-review disclaimer, but the output template here does not include a placeholder for it. This PR extends disclaimer-language.instructions.md to apply to .copilot-tracking/release-readiness/**, so the obligation is active. A user following the template verbatim will produce a scorecard missing the required disclaimer.
Suggested fix: add a blockquote disclaimer near the top of the template:
# Release Readiness Scorecard: <app> (<scope>)
> **AI-assisted assessment:** This scorecard was produced with AI assistance and requires review and validation by a qualified human reviewer before use in a launch decision. It does not constitute professional advice.
**Verdict:** Go | Conditional-Go | No-Go
**Date:** <date> · **Rubric:** <trust bar source or "default pillars">There was a problem hiding this comment.
Good catch — added the disclaimer to the scorecard template in dc6ce9a2, using the blockquote you suggested:
AI-assisted assessment: This scorecard was produced with AI assistance and requires review and validation by a qualified human reviewer before use in a launch decision. It does not constitute professional advice.
It now sits directly under the # Release Readiness Scorecard heading, so anyone following the template verbatim produces output that satisfies the disclaimer-language.instructions.md obligation this PR activates for .copilot-tracking/release-readiness/**. Thanks!
|
One observation I noticed: The verdict rules currently leave a reachable state with no computable result. Step 3 defines Red as either a launch-blocking gap or no evidence. However, Step 4 independently allows that gap to be tagged
This conflicts with the success criterion that exactly one verdict is computed and with the “evidence or it didn’t happen” constraint. Suggestion: Could we make the verdict partition exhaustive by treating Red as intrinsically launch-blocking? I this preserves scope awareness while ensuring missing evidence cannot accidentally become non-blocking. A small truth-table example covering this case would also help prevent regression.
|
Any Red pillar (including Red for missing evidence) is now launch-blocking and yields No-Go, closing a reachable state that matched no verdict. Scope/tolerability is decided at RAG scoring time, so a Red gap can no longer be tagged non-blocking. Adds a worked truth-table example. Addresses review feedback from @jkim323 on microsoft#2422.
|
@jkim323 great catch, thank you. You're right that a Red-for-missing-evidence pillar tagged |
1 similar comment
|
@jkim323 great catch, thank you. You're right that a Red-for-missing-evidence pillar tagged |
|
Hi @katriendg @rezatnoMsirhC — gentle follow-up on this one. I pushed an update on Jul 13 ( ix: make release-readiness-gate verdict rules exhaustive) that addresses the earlier feedback, and it's ready for another look. Could you please take a moment to re-review when you can? Thank you! |
…lanner ref L101: add AI-assisted professional-review disclaimer blockquote to the release-readiness scorecard template so output satisfies the disclaimer-language obligation this skill activates for .copilot-tracking/release-readiness/**. L33: replace the not-yet-merged performance-slo-planner reference with the no-dedicated-planner codebase-signals pattern (matching Observability/Operational Readiness), decoupling this PR from microsoft#2420; update the Handoff example to a real planner (SSSC).
…s-gate # Conflicts: # collections/hve-core-all.collection.md # plugins/hve-core-all/README.md
There was a problem hiding this comment.
Before this merges, could you run the hve-builder skill in review mode against this SKILL.md and share or link the findings? New prompt-engineering artifacts in this repo are expected to go through a systematic rubric pass. A clean or addressed report would give the review a complete paper trail.
Summary
Adds a
release-readiness-gateskill and a newrelease-readinesscollection. The gate consolidates specialist planner outputs (RAI, Security, Supply Chain, Performance, Privacy, Accessibility) into a single, evidence-grounded Go / Conditional-Go / No-Go decision with a RAG scorecard per pillar, a blocking-gap list, and a sign-off checklist. It decides; it does not remediate.Changes
.github/skills/release-readiness/release-readiness-gate/SKILL.md— documentation-driven playbook (Inputs, Default Pillars with per-pillar Evidence-source mapping, Procedure, Verdict Rules, Handoff, Output Format). No scripts.release-readinesscollection (.yml+.md) bundling the gate skill plus shared governance instructions (disclaimer-language,untrusted-content-boundary,hve-core-location) for isolated-install portability.**/.copilot-tracking/release-readiness/**to thedisclaimer-languageanduntrusted-content-boundaryapplyToglobs (matching every upstream planner it consolidates).hve-core-allbundle; added therelease-readinessplugin tomarketplace.json(15 total).release-readinessandhve-core-allplugin/collection docs only.Notes
.copilot-tracking/release-readiness/<date>-<scope>-readiness.md.Validation
npm run lint:collections-metadata— 0 errorsnpm run validate:skills— 0 errors / 0 warningsnpm run lint:marketplace— OK (15 plugins)npm run lint:frontmatter— edited/new files pass (0 errors)Closes #2421