Conversation
|
Thanks, @a-moskvin. This matches the pilot shape from #101: optional, one file, cited, and every value DRAFT. Verification of this branch (
Nits, non-blocking:
On the many-to-many follow-up: please open it as an issue once this pilot settles, so it's discussed separately from this PR. Merge-order note. This PR, #185 and #187 all regenerate shared files. I simulated CI hasn't run here. Workflows on fork PRs wait for a maintainer to approve them, so every result above is from a local run, not a GitHub check. |
|
Following up after a closer look at where this fits long term. My first comment only checked correctness. This one is about direction, and it changes my earlier "merge-ready" read. The maintainer has decided to hold this PR in its current shape. The idea isn't being turned down: a "how do I check it" layer is a real gap. SCF ships Assessment Objectives and an Evidence Request List, and CSA's AI Controls Matrix has auditing guidelines. We want this to be done in a shape that lasts. Three things stand in the way: 1. AIUC-1 already publishes its own verification layer. The standard has official evidence for each requirement (
(While checking this we found that the repo's AIUC-1 registry names the wrong publisher, licence and URL: #189. That one's ours to fix, not yours.) 2. The method mostly belongs to the control, not the risk–control pair. B001's text repeats across five entries with small variations. Free text on each row will duplicate and drift. The review state (
3. The rows underneath aren't reviewed yet. Proposed next step: open the many-to-many follow-up as an issue now, with the data model above (or your counter-proposal), and settle it there first. Once it's agreed, a reworked PR can land against the agreed shape. Most of your work carries over: the parser and the test, and the 36 method phrases that trace verbatim to NPW controls all port directly. One more thing, for transparency: since NPW is your catalogue, please say so in the file header and the PR when the reworked version lands. That's normal open-source practice and not a concern in itself; we just apply it to everything sourced from a single author. Leaving this open so the history stays in one place. Thanks for the careful work and for raising #101. |
|
Hi @emmanuelgjr , Thank you for the feedback, the direction makes sense. Agreed on all three points:
|
What this PR changes
Refs #101. Adds an optional
verification_methodfield to the schema v2 row format and pilots it in one file.scripts/generate.js: parses a "Verification method" column as a v2 metadata column (stored asverification_method, excluded fromnotes)data/schema.json,src/index.ts: optional string fielddocs/SCHEMA_V2_MIGRATION.md: field documentedscripts/generate.test.mjs: regression test — fails if method text lands innotesagentic-top10/Agentic_AIUC1.md: column added to all 10 tables (ASI01–ASI10); 22 of 26 requirement-level rows filled, allDRAFTdata/entries/ASI*.json(9),docs/data.jsType of change
Source / evidence
Methods adapted from the NPW Agentic AI Control Catalogue v2.4.0 (CC BY-SA 4.0): https://www.newpacificway.com/ai-controls.
Checklist
Content
tags, detailed per-entry mappings, references, changelog)
shared/SEVERITY.mdAgentic_X.md, that file mentions this one backLLM01–LLM10,ASI01–ASI10,DSGAI01–DSGAI21)CC BY-SA 4.0Links & data
.mdlinks resolve to real fileslychee)data/schema.jsoncompatible (if adding a new entry type)Project hygiene
YYYY-MM-DDformat)CHANGELOG.mdupdated if this is a new mapping file (include in correct version section)README.mdcounts updated if file count changed (badge + summary table + section heading + repo tree)node scripts/validate.js --file <path>locally and it passesFor new mapping files only
README.mdmapping table with "Standout content" descriptionCROSSREF.mdprimary frameworks column where relevantREADME.mdupdated (✅for the new cell)SourceList_Framework.md(e.g.,Agentic_SAMM.md)Notes for reviewers
Non-breaking: with no file carrying the column, the schema commit regenerates nothing.
DRAFT status: every value awaits SME review, like relationship/confidence.
Intentional gaps: 12 domain-level rows (A, C, D, E, F) are out of scope as too generic. 4 rows (ASI04/B003, ASI05/B005, ASI09/B009, and ASI10/B006) have no defensible method. Contributions welcome.
Attribution: text after the
(NPW …)citation is contributor-authored.One source: all methods come from one catalogue. Methods from other sources, each with its own citation, are welcome.
OLIR: the field is not exported; OLIR has no corresponding field.
Open design question: I suggest a many-to-many model of mapping controls to verification methods. Happy to propose it as a follow-up issue if this pilot is accepted.