Repository navigation
data_validation: OWASP 2026 lists, CVSS v4.0, format checks, 2020-12 schemas - #93
Merged
Merged
Conversation
…schemas Brings the validation onto current standards: - OWASP Top 10 for LLM Applications 2026 (renumbered) and the Agentic Top 10 (ASI01-ASI10:2026). Schemas accept LLMnn:2025 and LLMnn:2026 and a new owasp_agentic_top10_mapping; reference_data/owasp_top10.csv lists all three with a 2026-to-2025 crosswalk, used to add the 2026 equivalent to every existing LLM mapping (111 files). crossref_validator rejects unknown OWASP IDs and warns on IDs without an edition year. - CVSS v4.0: cvss_v4_score/cvss_v4_vector in the vulnerability schema; the qualitative "Informational" becomes the FIRST rating "None". New severity_check.py recomputes every score from its vector and checks the qualitative rating against the FIRST band. Fixes from it, against NVD: CVE-2024-37032 8.8 is High, not Critical; CVE-2025-0845's vector gives 6.4, not 6.5; vectors added for CVE-2024-34359, CVE-2025-0845 and CVE-2025-32711. - The exploit, vulnerability and risk-assessment validators now enforce "format" (dates, URIs); jsonschema[format-nongpl] provides the URI checker. All current data passes. - incident, rag, promptinj_testcase and crossframework schemas move from draft-07 to 2020-12, matching the rest. - Python 3.11+ (3.10 reaches end of life this month); CI runs the tests on 3.11, 3.12 and 3.13. - reference-data-refresh workflow: monthly ATLAS/CWE regeneration; on a change it pushes a branch and opens an issue (workflows here may not open PRs). - test_standards.py guards the above: schemas are 2020-12, OWASP enums match the reference table, dataset validators enforce format. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dependency ReviewThe following issues were found:
License Issuesdata_validation/requirements.txt
OpenSSF Scorecard
Scanned Files
|
| # Contributor-maintained schema for one sub-collection; its own validator owns it. | ||
| CONTRIBUTOR_SCHEMAS = {"contrastive_testcase.schema.json"} | ||
|
|
||
| OWASP = list(csv.DictReader(open(REFERENCE_DIR / "owasp_top10.csv", newline="", encoding="utf-8"))) |
Dependency Review rejected cvss (LGPL-3.0); the repository allows only permissive licences. cvss_score.py computes base scores itself: v3.0/v3.1 from the FIRST specification formulas, v4.0 as a port of cvss40.js, the FIRST reference calculator (BSD-2-Clause, licence and lookup tables included). The port matches the reference on 500 vectors with base, threat and environmental metrics; the v3 scorer matches the NVD/CNA scores of the dataset's CVEs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Dependency Review flagged the
The tooling is back to one runtime dependency ( |
This was referenced Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements items 1–6 and 14 of the data-validation review: the places where the validation lagged current standards or silently skipped a check.
LLMnn:2025andLLMnn:2026, always with the year.reference_data/owasp_top10.csvholds both editions plus a 2026→2025 crosswalk, which I used to add the 2026 equivalent to every existing mapping (111 files, 163 IDs; nothing removed).owasp_agentic_top10_mappingfield (ASI01:2026…ASI10:2026) in the exploit, vulnerability and agent-trace schemas. OWASP writes these IDs without a year; the suffix keeps a future edition from colliding with this one, as just happened with the LLM list.cvss_v4_scoreandcvss_v4_vector. The qualitative rating "Informational" becomes FIRST's "None" (no entry used it). A newseverity_check.pyrecomputes every score from its vector (v3.0/3.1/4.0) and checks that the qualitative rating matches the score's FIRST band. Data fixes found by the check, against NVD:•
CVE-2024-37032: 8.8 is High, not Critical.•
CVE-2025-0845: the published vector gives 6.4, not 6.5.• Vectors added for
CVE-2024-34359,CVE-2025-0845andCVE-2025-32711(the CNA vector, which matches the stored 9.3)."format": "date"and"uri"were never checked. They now check both, andjsonschema[format-nongpl]supplies the URI checker. All current data passes.reference-data-refreshworkflow. Runs on the 2nd of each month (and on demand) and regenerates the ATLAS and CWE tables. If anything changed, it pushesautomation/reference-data-YYYY-MMand opens an issue with the compare link: this repo doesn't let workflows open PRs (can_approve_pull_request_reviews: false), and I didn't loosen that setting.The new
tests/test_standards.pyfails if a schema drifts from 2020-12, if the OWASP enums stop matching the reference table, or if a dataset validator drops its format checker.Verification
run_all_checks.py: 7 validators pass. The shared checks find 0 errors across 591 files; the only 2 warnings are the intentional retired-ID notes from exploit_dataset: move retired ATLAS IDs to AML.T0115 sub-techniques #90.actionlintis clean on both workflows.🤖 Generated with Claude Code