Severity Ranker: Prioritize Findings Using CVSS & Scanner Metadata

[arpit2006]

Description

Currently, findings generated by Semgrep, OSV-Scanner, and Gitleaks are displayed in the order returned by each scanner. This order does not reflect the actual security risk, forcing users to manually inspect severity values across different scanner formats before deciding which issues to fix first.

This issue introduces the first component of the Machine Learning Tier 1 – Intelligent Triage pipeline by assigning a normalized severity score (0–10) to every finding. Although the initial implementation is deterministic, it establishes the labeled dataset required for future ML-based prioritization models.

Once implemented, every finding will contain a standardized numerical severity score regardless of its originating scanner, enabling consistent sorting, filtering, and downstream ML training.

---

Problem Statement

Each scanner represents severity differently:

• OSV-Scanner

  • Uses CVSS scores embedded inside vulnerability metadata.
  • Example:

    {
      "severity": [
        {
          "type": "CVSS_V3",
          "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H"
        }
      ]
    }

• Semgrep

  • Uses categorical severity labels.

    {
      "extra": {
        "severity": "ERROR"
      }
    }

• Gitleaks

  • Uses confidence values instead of CVSS.

    {
      "RuleID": "...",
      "Confidence": "HIGH"
    }

Because these formats are inconsistent:

• High-risk vulnerabilities may appear below informational findings.
• Users cannot quickly identify the most critical issues.
• Later ML ranking models have no normalized training labels.
• Dashboard sorting becomes unreliable.

---

Proposed Solution

Create a new ranking module:

backend/
└── app/
    └── ml/
        └── severity_ranker.py

Expose a public API:

def rank(findings: list[dict]) -> list[dict]:
    ...

The function should:

1. Inspect each finding.
2. Detect which scanner produced it.
3. Extract the highest-confidence severity metadata.
4. Convert it into a normalized float between 0–10.
5. Add

severity_score

to every finding.

Example:

{
    "scanner": "semgrep",
    "rule_id": "python.lang.security.audit.eval",
    "severity_score": 9.0
}

---

Severity Derivation Priority

The score should be derived using the following priority:

1. OSV Scanner (Highest Priority)

Extract CVSS score from

finding["severity"][0]["score"]

Example:

"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H"

Use the cvss Python package to parse the vector and obtain the numeric base score.

Example:

9.8

---

2. Semgrep

Read

finding["extra"]["severity"]

Map values:

Semgrep Severity	Score
ERROR	9
WARNING	6
INFO	3

Unknown values should default to 5.

---

3. Gitleaks

Read

finding["Confidence"]

Mapping:

Confidence	Score
HIGH	8
MEDIUM	5
LOW	2

---

4. Fallback

If none of the above metadata exists:

severity_score = 5.0

---

Database Changes

Extend the findings table:

ALTER TABLE findings
ADD COLUMN severity_score REAL DEFAULT 5.0;

Every persisted finding should include:

severity_score

---

API Changes

The following endpoints should return findings ordered by severity:

• /scan
• /scan-url

Sorting:

ORDER BY severity_score DESC

or

sorted(findings,
       key=lambda x: x["severity_score"],
       reverse=True)

---

Unit Tests

Create

backend/tests/test_severity_ranker.py

Tests should verify:

✅ OSV CVSS Parsing

Input:

{
    "severity":[
        {
            "score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H"
        }
    ]
}

Expected:

severity_score ≈ 9.8

---

✅ Semgrep Mapping

ERROR → 9
WARNING → 6
INFO → 3

---

✅ Gitleaks Mapping

HIGH → 8
MEDIUM → 5
LOW → 2

---

✅ Fallback

Missing metadata should produce

5.0

---

✅ Sorting

Provide multiple findings and ensure:

9.8
8
6
5
3
2

is returned in descending order.

---

Dependencies

Add:

cvss

to

backend/requirements-ml.txt

or

backend/requirements.txt

depending on project structure.

---

Acceptance Criteria

• Create backend/app/ml/severity_ranker.py
• Implement rank(findings: list[dict])
• Parse OSV CVSS vectors using the cvss package
• Map Semgrep severities correctly
• Map Gitleaks confidence values correctly
• Apply fallback score of 5.0
• Add severity_score to every finding
• Persist severity_score in SQLite
• Return findings sorted by descending severity
• Add comprehensive unit tests covering every derivation path
• Update any API schemas or documentation to include severity_score

---

Expected Impact

• 📊 Prioritizes critical findings automatically.
• ⚡ Enables users to focus on high-risk vulnerabilities first.
• 🧠 Provides normalized labels for future ML ranking models.
• 📈 Improves dashboard sorting and filtering.
• 🔄 Establishes the foundation for Tier 2 intelligent remediation and recommendation models.

---

Difficulty

Medium

Estimated Time: 4–6 hours

Skills Required:

• Python
• FastAPI
• SQLite
• Unit Testing (pytest)
• Basic understanding of CVSS and security scanner outputs

[github-actions]

👋 Welcome @arpit2006!

Thank you for opening an issue and contributing to Patchpilot. 🚀

Next Steps

• Please ensure you've provided all the required details.
• A maintainer will review your issue shortly.
• Feel free to ask questions if anything is unclear.

⚡ Want faster assignment and quicker responses?

Join our Discord community:

🔗 https://discord.gg/FcXuyw2Rs

We actively discuss issues, features, roadmap updates, and contributor support there.

Happy Contributing! 🎉