feat: security analysis guidelines - #44
Conversation
…aire, threat model, enforcement mapping) Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-104-106.watson.ibm.com>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
…tion after each step) or Autonomous (all four steps back-to-back, one final review at the end Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
… threat vectors Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
…t existing Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
…d list against the target's existing Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
…y analysis steps Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
…' into sc/security-analysis
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@dhcp-9-31-107-214.watson.ibm.com>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
araujof
left a comment
There was a problem hiding this comment.
Nice work!
Can you please make the following changes before merging?
-
opa_policy/guidelines-security-analysis/steps/enforcement_mapping.md:42: Smith usesinput.args.*, notinput.arguments.*. Using the wrong path here could produce rules for fields that do not exist. Please update the workflow for consistency. -
opa_policy/guidelines-security-analysis/steps/enforcement_mapping.md:319: Step 8 assumes guidance contains numbered rules, but most bundled examples use headings and bullets. Since Step E overwritesguidance.txt, this could potentially discard user-written context. What if we preserved the original file and appended only missing rules? -
opa_policy/guidelines-security-analysis/guidelines-security-analysis.md:149: This says to include non-OPA gaps inguidance_updated.txt, while Step 8 explicitly forbids it. Please choose one behavior; keeping those gaps only inowasp_policy_guidelines.mdseems safest.
…erwrite) guidance.txt, drop gap register from guidance_updated.txt Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
|
Updated based on the comments. |
Thanks! I just checked, and it looks like some of the issues have some residuals:
|
…KILL/READMEs, teach STEP 8 to handle prose-with-headers guidance.txt Signed-off-by: Sai Sree Laya Chukkappalli <saisree@Sais-MacBook-Pro.local>
Summary
Adds the guidelines-security-analysis skill: a 4-step pipeline (architecture analysis -> policy guidance questionnaire -> threat model -> enforcement mapping) that turns a target MCP server's source and OWASP Top 10 for Agentic AI Security (ASI01–ASI10) catalog into a scoped set of security guidelines for OPA policy.
Closes: #issue-number
Changes
opa_policy/guidelines-security-analysis/guidelines-security-analysis.md— new orchestrator doc that sequences the 4 stepsopa_policy/guidelines-security-analysis/owasp/architecture_analysis.md— maps an MCP server's layers, trust boundaries, and OPA enforcement points intoarchitecture.mdopa_policy/guidelines-security-analysis/owasp/policy_guidance_questionnaire.md— derives a 22-question policy-intent questionnaire fromarchitecture.md+guidance.txt/system_vars.json/tool_definitions.jsonwhere presentopa_policy/guidelines-security-analysis/owasp/threat_model.md— evaluates all 10 ASI categories against the tool using a three-question applicability test, sourced live from the catalog rather than hardcodedopa_policy/guidelines-security-analysis/owasp/enforcement_mapping.md— maps each threat instance to an enforcement layer (OPA vs. agent/tool/infra) and produces plain-English policy-rule requirements grounded in the catalog'smitigationssrc/smith/data/owasp_10_ai_catalog.json— new repo-relative data file: the OWASP Top 10 for Agentic AI Security catalog (ASI01–ASI10), single source of truth for category names, descriptions, impact, mitigations, and attack scenariosAll four step docs share a consistent "Authoritative Paths" contract: inputs are read from ONLY the named files under
<TARGET_AGENT_PATH>/smith/guidelines-security-analysis/(or the repo-relative catalog path), with an explicit stop-and-ask instruction if a required file is missing.Checks
make cipasses (lint, Rego lint, license headers, build smoke)make testpasses (policy scorecard — needed if policy behavior changed) — not applicable, noassets/policy.regochanges in this PRCHANGELOG.mdupdated under## [Unreleased](if user-facing)git commit -s) — verified viagit log -1 --format='%B',Signed-off-bypresentNotes (optional)
Base branch should be
hl/promptfoo_config_auto, notmain— this skill was built on top of unreleased work already on that branch. Confirmed clean:git diff --stat hl/promptfoo_config_auto..feat/guidelines-security-analysisshows exactly these 6 files (1392 insertions, 0 deletions), nopolicy_build/CLI/example files leaking in.