-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat(analyzer): add OpenCode-native SkillSpector invocation skill and tool #537
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
8657399
8dde131
da6cbbd
fc1773a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| description: Scan an AI agent skill for security risks with SkillSpector (static analysis by default); use when vetting a skill before install or auditing one in use. | ||
| --- | ||
| Scan the skill at $ARGUMENTS with the skillspector_scan tool (static analysis only, no LLM calls). | ||
| To opt into LLM semantic analysis instead, call skillspector_scan with noLlm false and SKILLSPECTOR_PROVIDER/SKILLSPECTOR_MODEL set. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| import { tool } from "@opencode-ai/plugin" | ||
| import { execFile } from "node:child_process" | ||
| import fs from "node:fs" | ||
| import path from "node:path" | ||
| import { promisify } from "node:util" | ||
| import { | ||
| TIMEOUT_MS, | ||
| buildCliArgs, | ||
| formatExecError, | ||
| formatSuccess, | ||
| isUrlOrAbsolute, | ||
| resolveBinary, | ||
| } from "./skillspector_scan_lib.ts" | ||
|
|
||
| const runFile = promisify(execFile) | ||
|
|
||
| export default tool({ | ||
| description: "Scan an AI agent skill for security risks with SkillSpector. Static analysis only by default; opt into LLM analysis explicitly.", | ||
| args: { | ||
| target: tool.schema.string().describe("Skill to scan: local path, .md/.zip file, or Git/file URL"), | ||
| format: tool.schema.enum(["terminal", "json", "markdown", "sarif"]).default("json").describe("Report format"), | ||
| noLlm: tool.schema.boolean().default(true).describe("Skip LLM analysis (static checks only). Set false to opt into LLM semantic analysis via SKILLSPECTOR_PROVIDER/SKILLSPECTOR_MODEL"), | ||
| output: tool.schema.string().optional().describe("Write the report to this file instead of returning it (resolved against the session directory if relative)"), | ||
| }, | ||
| async execute(args, context) { | ||
| const baseDir = context.directory ?? context.worktree ?? process.cwd() | ||
| const target = isUrlOrAbsolute(args.target) ? args.target : path.resolve(baseDir, args.target) | ||
| const output = args.output | ||
| ? (isUrlOrAbsolute(args.output) ? args.output : path.resolve(baseDir, args.output)) | ||
| : undefined | ||
| const bin = resolveBinary(context.worktree ?? baseDir, { | ||
| existsSync: fs.existsSync, | ||
| }) | ||
| const cliArgs = buildCliArgs({ target, format: args.format, noLlm: args.noLlm, output }) | ||
|
|
||
| let result: { stdout: string; stderr: string } | ||
| try { | ||
| result = (await runFile(bin, cliArgs, { | ||
| timeout: TIMEOUT_MS, | ||
| maxBuffer: 32 * 1024 * 1024, | ||
| cwd: baseDir, | ||
| })) as { stdout: string; stderr: string } | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P2] Propagate the OpenCode cancellation signal
|
||
| } catch (err: unknown) { | ||
| return formatExecError(bin, err) | ||
| } | ||
|
|
||
| return formatSuccess(output, result.stdout, result.stderr) | ||
| }, | ||
| }) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| // Pure helpers for the skillspector_scan tool. Dependency-free (no | ||
| // @opencode-ai/plugin import) so this module runs under plain node --test. | ||
|
|
||
| import path from "node:path" | ||
|
|
||
| export const TIMEOUT_MS = 120_000 | ||
| export const MAX_STDOUT = 12_000 | ||
| export const MAX_STDERR = 6_000 | ||
| export const INSTALL_HINT = | ||
| "uv tool install git+https://github.com/NVIDIA/skillspector.git" | ||
|
|
||
| export function truncate(text: string, max: number): string { | ||
| if (text.length <= max) return text | ||
| return text.slice(0, max) + `\n...[truncated ${text.length - max} chars]` | ||
| } | ||
|
|
||
| export function redact(text: string): string { | ||
| return text | ||
| .replace(/sk-ant-[A-Za-z0-9_-]+/g, "[REDACTED]") | ||
| .replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[REDACTED]") | ||
| .replace( | ||
| /\b([A-Z][A-Z0-9_]*_(?:API_KEY|TOKEN))(\s*[:=]\s*["']?)[^"'\s,}]+/g, | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P1] Redact every supported credential name The generic pattern only covers names ending in |
||
| "$1$2[REDACTED]", | ||
| ) | ||
| } | ||
|
|
||
| export function isUrlOrAbsolute(target: string): boolean { | ||
| return ( | ||
| /^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//.test(target) || path.isAbsolute(target) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P1] Recognize supported SCP-style Git targets The SkillSpector CLI explicitly accepts |
||
| ) | ||
| } | ||
|
|
||
| export interface ResolveBinaryOpts { | ||
| env?: Record<string, string | undefined> | ||
| platform?: string | ||
| existsSync?: (p: string) => boolean | ||
| } | ||
|
|
||
| export function resolveBinary( | ||
| worktree: string, | ||
| opts: ResolveBinaryOpts = {}, | ||
| ): string { | ||
| const env = opts.env ?? process.env | ||
| const platform = opts.platform ?? process.platform | ||
| const existsSync = opts.existsSync | ||
| const fromEnv = env.SKILLSPECTOR_BIN?.trim() | ||
| if (fromEnv) return fromEnv | ||
| // Repo-checkout fallback: <worktree>/.venv (Scripts/skillspector.exe on Windows, bin/skillspector elsewhere). | ||
| const binDir = platform === "win32" ? "Scripts" : "bin" | ||
| const exe = platform === "win32" ? "skillspector.exe" : "skillspector" | ||
| const venvBin = path.join(worktree, ".venv", binDir, exe) | ||
| if (existsSync ? existsSync(venvBin) : false) return venvBin | ||
| return "skillspector" | ||
| } | ||
|
|
||
| export interface ScanArgs { | ||
| target: string | ||
| format?: string | ||
| noLlm?: boolean | ||
| output?: string | ||
| } | ||
|
|
||
| export function buildCliArgs(args: ScanArgs): string[] { | ||
| // The host may omit declared defaults, so re-apply them here. noLlm | ||
| // defaults to true: LLM analysis must stay strictly opt-in. | ||
| const format = args.format ?? "json" | ||
| const noLlm = args.noLlm ?? true | ||
| const cliArgs = ["scan", args.target, "--format", format] | ||
| if (noLlm) cliArgs.push("--no-llm") | ||
| if (args.output) cliArgs.push("--output", args.output) | ||
| return cliArgs | ||
| } | ||
|
|
||
| export interface ExecFailure { | ||
| code?: unknown | ||
| killed?: boolean | ||
| stdout?: unknown | ||
| stderr?: unknown | ||
| message?: string | ||
| } | ||
|
|
||
| export function formatExecError(bin: string, err: unknown): string { | ||
| const e = err as ExecFailure | ||
| const partialOut = typeof e.stdout === "string" ? e.stdout : "" | ||
| const partialErr = typeof e.stderr === "string" ? e.stderr : "" | ||
| if (e.code === "ENOENT") { | ||
| return `SkillSpector CLI not found (tried "${bin}"). Install it with \`${INSTALL_HINT}\`, or point SKILLSPECTOR_BIN at the binary.` | ||
| } | ||
| if (e.killed) { | ||
| return redact( | ||
| `SkillSpector scan timed out after ${TIMEOUT_MS / 1000}s (killed; partial output below):\n` + | ||
| truncate(partialOut, MAX_STDOUT) + | ||
| (partialErr ? `\nstderr:\n${truncate(partialErr, MAX_STDERR)}` : ""), | ||
| ) | ||
| } | ||
| if (e.code === 1 && partialOut) { | ||
| // Findings above the risk threshold: the JSON report is the answer, not a crash. | ||
| return redact(truncate(partialOut, MAX_STDOUT)) | ||
| } | ||
| if (e.code === 2) { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P1] Preserve reports produced before exit 2 Exit 2 is not exclusively a usage error: the scan command writes a valid report first and then exits 2 when |
||
| return redact( | ||
| `SkillSpector usage error (exit 2):\n${truncate(partialErr || e.message || "", MAX_STDERR)}`, | ||
| ) | ||
| } | ||
| return redact( | ||
| `SkillSpector scan failed: ${truncate(partialErr || e.message || String(err), MAX_STDERR)}`, | ||
| ) | ||
| } | ||
|
|
||
| export function formatSuccess( | ||
| output: string | undefined, | ||
| stdout: string, | ||
| stderr: string, | ||
| ): string { | ||
| if (output && !stdout) return `Report saved to: ${output}` | ||
| return redact(truncate(stdout || stderr, MAX_STDOUT)) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P1] Do not discard successful-run warnings Whenever stdout is nonempty, this drops all stderr. SkillSpector intentionally emits security-relevant warnings there—for example incomplete discovery and detection of an author-shipped baseline—while still returning a JSON report on stdout. Return independently bounded/redacted stdout and stderr so the caller sees both, and test a successful report accompanied by a warning. |
||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # SkillSpector OpenCode Extension | ||
|
|
||
| SkillSpector can be installed into OpenCode as a local extension. The extension registers a `skillspector_scan` tool and a `/skillspector` slash command that run the existing SkillSpector CLI. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - OpenCode installed. | ||
| - Python `>=3.12,<3.15`. | ||
| - `uv` recommended. | ||
| - This repo checked out locally. | ||
| - Node 22+ to run the extension unit tests (type stripping, no extra dependencies). | ||
|
|
||
| ## Install | ||
|
|
||
| Copy this repo's `.opencode/` directory into your project (or `~/.config/opencode/` for global use): | ||
|
|
||
| ```bash | ||
| cp -r /path/to/SkillSpector/.opencode /path/to/my-project/ | ||
| ``` | ||
|
|
||
| Make sure `skillspector` is on PATH, or point `SKILLSPECTOR_BIN` at the binary: | ||
|
|
||
| ```bash | ||
| export SKILLSPECTOR_BIN=/path/to/SkillSpector/.venv/bin/skillspector | ||
| ``` | ||
|
|
||
| Then reload OpenCode or start a new session; `/skillspector` is auto-discovered. | ||
|
|
||
| ## Basic scan | ||
|
|
||
| In OpenCode: | ||
|
|
||
| ```text | ||
| /skillspector ./my-skill | ||
| ``` | ||
|
|
||
| Equivalent CLI (static analysis only): | ||
|
|
||
| ```bash | ||
| skillspector scan ./my-skill --no-llm | ||
| ``` | ||
|
|
||
| ## Tool parameters | ||
|
|
||
| - `target`: path, URL, zip, Git repo, or `SKILL.md` to scan. | ||
| - `format`: `terminal`, `json`, `markdown`, or `sarif`. Default: `json`. | ||
| - `output`: optional report path. | ||
| - `noLlm`: default `true`. | ||
|
|
||
| Unlike the [Pi extension](PI_EXTENSION.md), this tool has no `provider`, `model`, `yaraRulesDir`, or `verbose` parameters: LLM-backed analysis is configured through the environment instead (see below). | ||
|
|
||
| ## LLM-backed analysis | ||
|
|
||
| Static scan is default. To use semantic LLM analysis, configure provider credentials in your shell before launching OpenCode, then call the tool with `noLlm` false: | ||
|
|
||
| ```text | ||
| Use skillspector_scan on ./my-skill with noLlm=false. | ||
| ``` | ||
|
|
||
| ```bash | ||
| export SKILLSPECTOR_PROVIDER=opencode_cli | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P1] Use a provider that SkillSpector actually registers
|
||
| export SKILLSPECTOR_MODEL=opencode/nemotron-3-ultra-free | ||
| ``` | ||
|
|
||
| The extension never reads API keys itself and redacts secret-looking output. | ||
|
|
||
| ## Unit tests | ||
|
|
||
| Pure tool helpers live in dependency-free `.opencode/tools/skillspector_scan_lib.ts`, covered by `tests/opencode/skillspector_scan_lib.test.ts` via stdlib `node --test` (zero new dependencies): | ||
|
|
||
| ```bash | ||
| node --test tests/opencode/skillspector_scan_lib.test.ts | ||
| ``` | ||
|
|
||
| ## Remove | ||
|
|
||
| Delete the copied `.opencode/tools/skillspector_scan.*` and `.opencode/commands/skillspector.md` files. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Request permission for host access and LLM egress
Custom tools do not inherit OpenCode's built-in read/edit/external-directory permission prompts. This model-callable tool accepts arbitrary absolute input and output paths and can opt into sending skill content to an external LLM, then executes both without
context.ask. Require an explicit, narrowly scoped permission request (including external-directory reads/writes and network/LLM egress), or restrict targets to the session tree; add denial-path tests before launching the process.