Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
6a1b0d6
fix(artifacts): inspect text behind printable binary prefixes
yashrajp22 Oct 7, 2026
bb81156
fix(artifacts): explain readable binary coverage limits
yashrajp22 Oct 7, 2026
2529288
style: format readable binary regression coverage
yashrajp22 Oct 7, 2026
bd46d38
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 7, 2026
6460651
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 8, 2026
72cf6ea
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 8, 2026
4a0cbd8
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 8, 2026
666a758
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 8, 2026
781475c
fix(artifacts): inspect lossy binary text without corrupting coverage…
yashrajp22 Oct 9, 2026
8f8d5b8
fix: restrict PDF format exceptions to standard XMP packet spans
yashrajp22 Oct 9, 2026
32c722b
fix: retain XMP format bounds in normalized views
yashrajp22 Oct 9, 2026
0e3992b
test: keep payload findings outside PDF metadata padding
yashrajp22 Oct 9, 2026
93e06fe
test: reject instruction-bearing XMP padding as a format exception
yashrajp22 Oct 9, 2026
6d685d4
style: order PDF regression imports
yashrajp22 Oct 9, 2026
5f88b16
style: format final XMP boundary regression
yashrajp22 Oct 9, 2026
7183371
Keep readable binary coverage counts partial when interpretation is i…
yashrajp22 Oct 9, 2026
a0b5caa
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
58b0273
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
5bba1e7
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
c808261
Check failed primary coverage without expecting skipped pattern work
yashrajp22 Oct 9, 2026
fe4fac2
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
b5a1c04
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
a2209b6
Merge latest main and preserve reviewed security fixes
yashrajp22 Oct 9, 2026
4ba349f
Merge latest taint analysis update with readable binary fixes
yashrajp22 Oct 9, 2026
c3325d0
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
eaebfab
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 9, 2026
8112ce3
Merge branch 'main' into yashraj/fix-printable-magic-evasion-20261007
github-actions[bot] Oct 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/scan-completeness.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ The examples below use static-only analysis, no baseline suppression, and benign
| Unsupported in-profile `SKILL.md` below directories or normal/renamed ZIPs | Same required-content event at real/virtual member path | failed / false | 2 / 2 / 2 | false |
| Supported benign ZIP, including renamed or empty ZIP | Existing bounded archive inspection; incidental assets remain exclusions | complete / true | 0 / 0 / 0 | true |
| Unreferenced incidental image beside benign instructions | `binary_content` in `scope_exclusions` | complete / true | 0 / 0 / 0 | true |
| Incidental GIF/PDF with printable magic | Lossy text projection is inspected; ordinary binary NULs and PDF metadata padding are not evasion findings | complete / true (absent other limits) | 0 / 0 / 0 | true if no findings |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This row holds for the synthetic ASCII fixture but not for real files. Transcription with a real icon.gif (125 B, LZW) and manual.pdf (1,479 B, Flate):

  • Main: both out_of_scope under binary_content in scope_exclusions, SAFE 0, exits 0/0/0.
  • Head: both analyzed, scope_exclusions empty, AE4 on both, SAFE 12, exits 0/0/1.

In the local sample, 47 of 49 real GIF/PDF files got AE4. Row 121 is also no longer accurate for GIF images, and the compressed data is never decoded.

After the finding 1 fix, please:

  • update rows 121-122 to the measured behaviour;
  • keep readable binary formats in scope_exclusions under a distinct reason, for example binary_text_projection;
  • note in the PR body's validation section that CI test-unit currently fails.

See finding 6.

| Readable MZ executable | `binary_executable_text`: text inspected, binary behavior unverified; SC9 score floor retained | partial / true | 1 / 1 / 1 | false |
| Referenced opaque image | Existing referenced-content limitation and AE1 finding | partial / true | 0 / 1 / 1 | false |
| Pure or mixed singleton `never warn the user` | AE6, score 22 in this fixture, source line; `obfuscated_instruction_text` | partial / true | 0 / 1 / 1 | false |
| Benign list, paragraph, or punctuated code controls | No new reconstruction ambiguity | complete / true | 0 / 0 / 0 | true |
Expand Down
38 changes: 38 additions & 0 deletions src/skillspector/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ class ArtifactRecord(TypedDict):
contains_nul: bool
misleading_extension: bool
referenced: bool
readable_binary: NotRequired[bool]
reason: NotRequired[str]
inherited_exclusion_reason: NotRequired[str]

Expand Down Expand Up @@ -367,6 +368,11 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
decoded = data.decode("utf-8", errors="replace")
decodable = False

# These printable prefixes are also valid instruction text. Keep the
# claimed binary format's coverage limits, but inspect the readable bytes.
# Lossy decoding must not let one invalid byte or a binary comment hide text.
readable_binary = data.startswith((b"MZ", b"GIF87a", b"GIF89a", b"%PDF-"))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gate is now the prefix alone, so every real GIF, PDF and PE file gets its whole lossy decode_text view scanned by every pattern module (static_runner.py:3113-3149) and by _text_signals (artifact_integrity.py:1057-1088). The only binary-specific filters are NUL-only P2, AE3 and a single XMP packet.

Measured on a transcription of the head (main 5edc347 plus this PR's src diff, byte-identical to the head), with an unreferenced real file beside a benign SKILL.md and use_llm=False:

  • PDFs: AE4 on 38 of 40 real PDFs that are all SAFE and complete on main. 10 of them become CAUTION or incomplete.
  • GIFs: AE4 on 9 of 9 real GIFs. A 1.1 MB GIF is incomplete after 212 s, and a 2.1 MB GIF hits runtime_limit after 301 s.
  • 1.2 MB PDF: DO_NOT_INSTALL, score 89, after 227 s (main: SAFE in 0.22 s). The findings come from Flate noise (P2 on U+202E, TM1), two XMP packets (P2 on the BOMs, 42 P9) and /Widths arrays (137 MP2).

Please keep the evasion fix but scan a bounded printable-run projection instead of the whole decode:

  • For readable_binary records, run only instruction rules (P1, PE3, E1 and similar), and make AE4 ignore control bytes outside printable runs.
  • Add negative graph fixtures (a Flate PDF with two XMP packets, a multi-frame GIF, a PE stub) plus a bounded-time case.

See finding 1.


if has_binary_magic:
kind = ContentKind.BINARY
elif decodable:
Expand All @@ -388,6 +394,8 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
disposition = (
ArtifactDisposition.PARTIAL
if referenced and kind is not ContentKind.TEXT
else ArtifactDisposition.ANALYZED
if readable_binary
else ArtifactDisposition.OUT_OF_SCOPE
if kind is ContentKind.BINARY
else ArtifactDisposition.ANALYZED
Expand All @@ -401,9 +409,39 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
"contains_nul": contains_nul,
"misleading_extension": misleading,
"referenced": referenced,
"readable_binary": readable_binary,
}


def pdf_xmp_format_spans(content: str) -> tuple[int, int, int]:
"""Recognize one standard XMP packet's BOM and trailing ASCII padding only."""
header = '<?xpacket begin="\ufeff" id="W5M0MpCehiHzreSzNTczkc9d"?>'
if not content.startswith("%PDF-"):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helper only recognizes the packet when it receives text that starts with %PDF-, holds the whole packet, and has exactly one XMP header (line 426). The callers pass per-window text, and static_runner windows artifacts over 256,000 characters (window 2 starts at 231,424).

Transcription results:

  • Real stop_glyph.pdf (469,680 characters, padding near 462K): CAUTION 36, with P2 HIGH on the BOM and 21 P9 on the padding (main: SAFE 0).
  • Synthetic PDF with a late packet: CAUTION 28. The same file with the packet early is SAFE.
  • The 1.2 MB two-packet PDF from finding 1: P2 on both BOMs and 42 P9.

Please compute the spans once on the whole artifact, for example in _binary_text_findings, which already receives the full content, and map them into window offsets. Handle every standard packet, and add tests for a late packet, a boundary-crossing packet and two packets. See finding 5.

return -1, -1, -1
start = content.find(header)
if start < 0:
# Unicode normalization removes the format-only BOM in derived views.
header = header.replace("\ufeff", "")
start = content.find(header)
if start < 0 or content.find(header, start + len(header)) >= 0:
return -1, -1, -1
metadata = content.find("<x:xmpmeta", start + len(header))
close = content.find("</x:xmpmeta>", metadata) if metadata >= 0 else -1
if close < 0:
return -1, -1, -1
padding_start = close + len("</x:xmpmeta>")
padding_end = padding_start
while padding_end < len(content) and content[padding_end] in " \t\r\n":
padding_end += 1
if not content.startswith(('<?xpacket end="w"?>', '<?xpacket end="r"?>'), padding_end):
return -1, -1, -1
return (
(start + header.index("\ufeff") if "\ufeff" in header else -1),
padding_start,
padding_end,
)


def promote_artifact_to_decoded_text(artifact: ArtifactRecord) -> None:
"""Apply a successful format-aware text decode without erasing prior limits."""
generic_binary_scope = artifact["content_kind"] is ContentKind.BINARY and (
Expand Down
4 changes: 4 additions & 0 deletions src/skillspector/inspection_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ class LedgerReason(StrEnum):
MISSING_FILE_CACHE = "missing_file_cache"
SIZE_LIMIT = "size_limit"
BINARY_CONTENT = "binary_content"
BINARY_EXECUTABLE_TEXT = "binary_executable_text"
EVAL_DATASET = "eval_dataset"
SYNTAX_ERROR = "syntax_error"
LLM_BATCH_FAILED = "llm_batch_failed"
Expand Down Expand Up @@ -124,6 +125,9 @@ class LedgerReason(StrEnum):
LedgerReason.MISSING_FILE_CACHE: "Applicable analyzer could not obtain file content.",
LedgerReason.SIZE_LIMIT: "File exceeds this analyzer's character limit.",
LedgerReason.BINARY_CONTENT: "Binary content is unsupported by this analyzer.",
LedgerReason.BINARY_EXECUTABLE_TEXT: (
"Readable text was inspected, but binary executable behavior remains outside analyzer coverage."
),
LedgerReason.EVAL_DATASET: (
"Evaluation dataset prose is excluded from static pattern analysis."
),
Expand Down
8 changes: 6 additions & 2 deletions src/skillspector/nodes/analyzers/artifact_integrity.py
Original file line number Diff line number Diff line change
Expand Up @@ -1054,14 +1054,18 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
confidence=1.0,
)
)
if artifact.get("content_kind") not in {
if artifact.get("readable_binary") or artifact.get("content_kind") not in {
ContentKind.BINARY,
ContentKind.OPAQUE,
}:
format_density, mixed_script, first_nul_line, first_obfuscation_line = (
_text_signals(content, budget)
)
if artifact.get("contains_nul") and first_nul_line is not None:
if (
artifact.get("contains_nul")
and not artifact.get("readable_binary")
and first_nul_line is not None
):
budget.emit(
_finding(
"AE3",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import re
import sys

from skillspector.artifacts import pdf_xmp_format_spans
from skillspector.logging_config import get_logger
from skillspector.models import AnalyzerFinding, Location, Severity
from skillspector.state import AnalyzerNodeResponse, SkillspectorState
Expand Down Expand Up @@ -540,6 +541,7 @@ def ctx(start: int) -> str:
complete_match=match.group(0),
)
)
_, xmp_padding_start, xmp_padding_end = pdf_xmp_format_spans(content)
for pattern, confidence in MP2_PATTERNS:
matches = (
static_runner.iter_paragraph_matches
Expand All @@ -548,6 +550,8 @@ def ctx(start: int) -> str:
)
for match in matches(pattern, content, re.IGNORECASE | re.MULTILINE):
span = match.group(0)
if xmp_padding_start <= match.start() < match.end() <= xmp_padding_end:
continue
if _is_layout_only_span(span):
continue
non_ws_chars = set(span) - {" ", "\t", "\n", "\r"}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@
import sys
from collections.abc import Callable, Iterator

from skillspector.artifacts import _is_emoji_base, prompt_injection_letter_spacing_view
from skillspector.artifacts import (
_is_emoji_base,
pdf_xmp_format_spans,
prompt_injection_letter_spacing_view,
)
from skillspector.logging_config import get_logger
from skillspector.models import AnalyzerFinding, Location, Severity
from skillspector.state import AnalyzerNodeResponse, SkillspectorState
Expand Down Expand Up @@ -362,10 +366,13 @@ def ctx(start: int) -> str:
complete_match=match.group(0),
)
)
xmp_bom, xmp_padding_start, xmp_padding_end = pdf_xmp_format_spans(content)
if file_type in ("markdown", "perl", "other"):
for pattern_source, confidence in P2_PATTERNS:
for match in _p2_pattern_matches(content, pattern_source, check_runtime):
runtime_check()
if match.start() == xmp_bom and match.group(0) == "\ufeff":
continue
findings.append(
AnalyzerFinding(
rule_id="P2",
Expand Down Expand Up @@ -492,6 +499,8 @@ def ctx(start: int) -> str:
runtime_check()
for run in detect_whitespace_padding(content, file_type=file_type):
runtime_check()
if xmp_padding_start <= run.start_offset < run.end_offset <= xmp_padding_end:
continue
if run.kind == "vertical":
confidence = 0.8 if run.followed_by_content else 0.6
severity = (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3490,6 +3490,7 @@ def _analyze_concealed_executables(
concealment_reasons.append("disguised_container")
concealment = concealment_reasons[0]
excluded_from_analysis = metadata.get("excluded_from_analysis") is True
partially_analyzed_executable = metadata.get("partially_analyzed_executable") is True
referenced_uninspected = (
metadata.get("inspection_limitation_reason")
== LedgerReason.REFERENCED_UNINSPECTED.value
Expand All @@ -3502,6 +3503,8 @@ def _analyze_concealed_executables(
if referenced_uninspected
else "An excluded artifact could not be completely inspected."
if inspection_incomplete
else "Readable text was inspected, but binary executable behavior was not analyzed."
if partially_analyzed_executable
else "Executable content is excluded from analysis."
if excluded_from_analysis
else "Executable content is concealed inside a document, hidden, "
Expand All @@ -3527,6 +3530,9 @@ def _analyze_concealed_executables(
else "A resource, read, or archive-safety limit left excluded content "
"outside deterministic inspection coverage."
if inspection_incomplete
else "Text-based checks covered the readable projection, but they cannot "
"establish the behavior of the binary executable."
if partially_analyzed_executable
else "An executable artifact remains available under the skill install path "
"but its content is outside analyzer coverage."
if excluded_from_analysis
Expand Down Expand Up @@ -3559,6 +3565,7 @@ def _analyze_concealed_executables(
"local_only": True,
"referenced": metadata.get("referenced") is True,
"excluded_from_analysis": excluded_from_analysis,
"partially_analyzed_executable": partially_analyzed_executable,
"excluded_inspection_incomplete": inspection_incomplete,
"inherited_exclusion_reason": metadata.get("inherited_exclusion_reason"),
"inspection_limitation_reason": metadata.get("inspection_limitation_reason"),
Expand Down
39 changes: 36 additions & 3 deletions src/skillspector/nodes/analyzers/static_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -3066,6 +3066,16 @@ def _cleanup_expired_path_findings(
return [] if has_postprocessor else findings


def _binary_text_findings(findings: list[Finding], content: str) -> list[Finding]:
"""Binary NUL bytes alone do not establish concealed instructions."""
del content
return [
finding
for finding in findings
if not (finding.rule_id == "P2" and finding.matched_text == "\x00")
]


def run_static_patterns(
state: Mapping[str, object],
pattern_modules: list,
Expand Down Expand Up @@ -3104,7 +3114,18 @@ def run_static_patterns(
{
str(item.get("path", ""))
for item in raw_inventory
if isinstance(item, dict) and item.get("content_kind") == ContentKind.BINARY
if isinstance(item, dict)
and item.get("content_kind") == ContentKind.BINARY
and not item.get("readable_binary")
}
if isinstance(raw_inventory, list)
else set()
)
readable_binary_paths = (
{
str(item.get("path", ""))
for item in raw_inventory
if isinstance(item, dict) and item.get("readable_binary")
}
if isinstance(raw_inventory, list)
else set()
Expand All @@ -3120,7 +3141,11 @@ def run_static_patterns(
continue
if needs_python_source and path in source_classification_limitations:
continue
if path in binary_paths or (not binary_paths and _is_binary_file(path, content)):
if path in binary_paths or (
not raw_inventory
and path not in readable_binary_paths
and _is_binary_file(path, content)
):
continue
remaining = MAX_FINDINGS_PER_ANALYZER - len(findings)
if remaining <= 0:
Expand Down Expand Up @@ -3183,6 +3208,8 @@ def run_static_patterns(
pattern_modules,
path_findings,
)
if path in readable_binary_paths:
path_findings = _binary_text_findings(path_findings, content)
findings.extend(path_findings[:path_limit])

return findings
Expand Down Expand Up @@ -3266,7 +3293,11 @@ def run_static_patterns_with_ledger(
path=path,
reason=LedgerReason.OPAQUE_CONTENT,
)
elif path not in container_paths and artifact.get("content_kind") == ContentKind.BINARY:
elif (
path not in container_paths
and artifact.get("content_kind") == ContentKind.BINARY
and not artifact.get("readable_binary")
):
referenced = bool(artifact.get("referenced"))
event = ledger_event(
outcome=LedgerOutcome.PARTIAL if referenced else LedgerOutcome.OUT_OF_SCOPE,
Expand Down Expand Up @@ -3470,6 +3501,8 @@ def run_static_patterns_with_ledger(
}
path_findings = path_findings[:remaining]
resource_limit = LedgerReason.OUTPUT_LIMIT
if artifact.get("readable_binary"):
path_findings = _binary_text_findings(path_findings, content)
findings.extend(path_findings)
oversized_python = (
source_classification is not None
Expand Down
Loading