Skip to content

data_validation: point riskassessment central schema at the dataset-local one - #79

Merged
emmanuelgjr merged 1 commit into
GenAI-Security-Project:mainfrom
Prasad-desh:patch-1
Oct 1, 2026
Merged

emmanuelgjr merged 1 commit into
GenAI-Security-Project:mainfrom
Prasad-desh:patch-1

Conversation

@Prasad-desh

Copy link
Copy Markdown
Contributor

Summary

Refs #76. Follows the pattern from #73. Supports the Data Security Initiative's data_validation workstream.

Converts data_validation/schemas/riskassessment.schema.json into a $ref pointer to datasets/riskassessment_dataset/schema.json, which is authoritative per the policy ratified in #62.

Reference check: grep -rn "riskassessment.schema" returns no matches. Nothing validates against the central file; only READMEs link to the dataset folder.

Evidence:

  • datasets/riskassessment_dataset/validate.py: all 23 entries pass the local schema
  • Validating the same 23 entries against the central draft-07 schema: 23/23 fail (missing deployment_scenario, components_assessed, dsgai_risks_identified; assessment_id does not match ^RA-\d{4}$)

Divergences recorded in $comment:

  • Draft version: draft-07 to 2020-12, with additionalProperties: false
  • assessment_id: RA-NNNN to DSGAI-RA- slug
  • components_assessed (string list) to typed components with role enum and sensitivity tier
  • dsgai_risks_identified with single risk_rating to risks_identified with inherent and residual ratings plus required rationale
  • existing_controls, control_gaps, framework_alignment from free text to structured arrays
  • industry (free text) to industry_sector enum
  • New required fields: scenario_name, entry_type, deployment_pattern, description, date_added

Analysis drafted with AI assistance; I reviewed the diff and verified results by running the validators locally.

DSGAI mapping

None. Schema housekeeping only; no dataset entries are added or changed.

Checklist

  • Data is anonymized (no PII, credentials, or org names): N/A, no data in this change
  • Entries reference at least one DSGAI ID where applicable: N/A, no entries changed
  • Validation scripts pass (python run_all_checks.py): run_all_checks.py is currently a stub ("Validators not yet implemented"). Ran datasets/riskassessment_dataset/validate.py instead: 23/23 entries pass.
  • One entry per file for dataset contributions: N/A, not a dataset contribution

…oject#76)

Updated schema version and added comment regarding schema authority.

Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 29, 2026 09:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Prasad-desh

Copy link
Copy Markdown
Contributor Author

@emmanuelgjr — opened this per the plan above. grep -rn "riskassessment.schema" confirmed nothing references the central file, and the concrete divergences are recorded in the $comment.

One note beyond this PR: the checklist's python run_all_checks.py is currently a stub ("Validators not yet implemented"), so I validated with datasets/riskassessment_dataset/validate.py instead — 23/23 entries pass. Happy to open a separate issue for that if it's useful.

I'll take vulnerability.schema.json next if no one else has claimed it.

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified locally: all 23 entries pass the dataset-local schema and the new $ref pointer, and all 23 fail the old central file. Nothing references the central file. Same pattern as #73. Thanks!

@emmanuelgjr
emmanuelgjr merged commit 733f3c8 into GenAI-Security-Project:main Oct 1, 2026
emmanuelgjr added a commit that referenced this pull request Oct 1, 2026
Same pattern as #73, #79 and #80. Nothing references the central file;
datasets/exploit_dataset/validate.py validates all 59 entries against
the dataset-local schema, and all 59 fail the central draft-07 file
(missing name and source; exploit_id, prerequisites, impact and
detection_indicators of the wrong shape). The divergences are recorded
in $comment.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@Prasad-desh
Prasad-desh deleted the patch-1 branch October 8, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants