We integrated SkillSpector as an attributed independent evidence provider in
AI SAFE². Thank you for exposing coverage and degraded-analysis status explicitly;
those fields are important to our downstream review process.
Environment and scope
- SkillSpector v2.11.1, pinned source revision
704bc9544260c2f41222dc0f92982521709496ab.
- Windows; Python 3.13.14;
scan TARGET --format json --no-llm.
- Scan date: 2026-09-09. No target skill/test code executed by us.
- Model credentials were not forwarded. Static mode is not a claim of zero
network access: supply-chain lookups can contact OSV.
- Public target: AI SAFE²'s tracked
skills/ tree at
6455eec76caec259ba3f9b902a4ab24f27b34263.
- This is a version-pinned observation, not a claim that every subsequent
upstream revision is affected.
Observed result
The tracked-source snapshot produced 50 issues and a risk assessment of
100 / CRITICAL / DO_NOT_INSTALL. A separate development checkout also produced
50 issues. Our clean and inert-hostile controls produced zero and seven issues,
respectively. These counts are not a calibrated precision or recall benchmark.
Several explanations appear to infer executable behavior from inert content:
- Prompt-injection strings in sanitizer regression tests are test inputs, with
assertions about sanitization. A lexical match should not itself establish
that the package follows those instructions.
- The
self_modification risk-taxonomy key describes a risk being assessed;
it is not an operation that modifies the program or its policy.
- The similarity between the package names uvicorn and gunicorn is insufficient
on its own to establish typosquatting.
- An environment-derived health-check URL is not itself a credential. Any
exfiltration explanation should identify the actual sensitive source and sink.
Important caveat: some smoke tests send bearer tokens to a configured URL.
Their transport/destination validation merits separate review. We are not asking
for all test-path findings to be suppressed or claiming every finding is false.
Coverage warnings, degraded OSV checks, and unresolved references are genuine
limitations and should remain visible.
Reproduction
Recommend using a clean, disposable directory and isolated environment. Install the pinned scanner from its official source, not an unpinned dependency set. The source pin
does not itself pin all transitive dependencies.
git clone https://github.com/CyberStrategyInstitute/ai-safe2-framework.git safe2-repro
git -C safe2-repro archive --format=zip --output=../safe2-skills.zip 6455eec76caec259ba3f9b902a4ab24f27b34263 skills
Extract that public-source archive into a new specimen directory using your
platform's archive utility. Scan only; do not install or execute specimen code:
skillspector --version
skillspector scan specimen/skills --format json --no-llm --output report.json
The historical count is environment-dependent, particularly for network checks;
the useful reproduction target is the particular finding explanation and source
context, not an invariant total of 50. A fresh minimal specimen and current-main
comparison have not yet been run for this upstream submission.
Requested improvement / questions
- Distinguish lexical pattern matches, data-flow evidence, and confirmed
executable behavior in the explanation and confidence semantics.
- Consider syntax/data-flow context for inert strings and taxonomy definitions.
- Preserve raw findings and coverage independently from reviewed dispositions.
- Add paired regression cases: inert test input versus active instruction/code,
including malicious content placed under a test-like path.
- Do not exempt
tests/, private IPs, or loopback wholesale: those are not trust
boundaries. Baselines should be reviewer-controlled and narrowly scoped.
- Is this already covered by an existing issue, recommended baseline workflow,
or a newer analyzer revision? We can help supply focused specimens.
Our integration preserves SkillSpector's original score and recommendation. We
do not rewrite those into approval or claim NVIDIA endorsement or conformance.
Public supporting analysis
AI SAFE² second-pass findings and limitations
Related reports
Related: #37 (text-content false positives), #314 (security-test patterns flagged), and #103 (earlier static documentation findings). This report supplies a pinned v2.11.1 AI SAFE2 specimen and distinguishes specific explanation/precision concerns from genuine coverage limitations. Please consolidate if an existing issue is the preferred tracking location.
We integrated SkillSpector as an attributed independent evidence provider in
AI SAFE². Thank you for exposing coverage and degraded-analysis status explicitly;
those fields are important to our downstream review process.
Environment and scope
704bc9544260c2f41222dc0f92982521709496ab.scan TARGET --format json --no-llm.network access: supply-chain lookups can contact OSV.
skills/tree at6455eec76caec259ba3f9b902a4ab24f27b34263.upstream revision is affected.
Observed result
The tracked-source snapshot produced 50 issues and a risk assessment of
100 / CRITICAL / DO_NOT_INSTALL. A separate development checkout also produced
50 issues. Our clean and inert-hostile controls produced zero and seven issues,
respectively. These counts are not a calibrated precision or recall benchmark.
Several explanations appear to infer executable behavior from inert content:
assertions about sanitization. A lexical match should not itself establish
that the package follows those instructions.
self_modificationrisk-taxonomy key describes a risk being assessed;it is not an operation that modifies the program or its policy.
on its own to establish typosquatting.
exfiltration explanation should identify the actual sensitive source and sink.
Important caveat: some smoke tests send bearer tokens to a configured URL.
Their transport/destination validation merits separate review. We are not asking
for all test-path findings to be suppressed or claiming every finding is false.
Coverage warnings, degraded OSV checks, and unresolved references are genuine
limitations and should remain visible.
Reproduction
Recommend using a clean, disposable directory and isolated environment. Install the pinned scanner from its official source, not an unpinned dependency set. The source pin
does not itself pin all transitive dependencies.
Extract that public-source archive into a new
specimendirectory using yourplatform's archive utility. Scan only; do not install or execute specimen code:
The historical count is environment-dependent, particularly for network checks;
the useful reproduction target is the particular finding explanation and source
context, not an invariant total of 50. A fresh minimal specimen and current-main
comparison have not yet been run for this upstream submission.
Requested improvement / questions
executable behavior in the explanation and confidence semantics.
including malicious content placed under a test-like path.
tests/, private IPs, or loopback wholesale: those are not trustboundaries. Baselines should be reviewer-controlled and narrowly scoped.
or a newer analyzer revision? We can help supply focused specimens.
Our integration preserves SkillSpector's original score and recommendation. We
do not rewrite those into approval or claim NVIDIA endorsement or conformance.
Public supporting analysis
AI SAFE² second-pass findings and limitations
Related reports
Related: #37 (text-content false positives), #314 (security-test patterns flagged), and #103 (earlier static documentation findings). This report supplies a pinned v2.11.1 AI SAFE2 specimen and distinguishes specific explanation/precision concerns from genuine coverage limitations. Please consolidate if an existing issue is the preferred tracking location.