feat: support auto-generation of configs and test generation improvements

[dhl123]

Summary

Closes: #36 #40 #41

Changes

• Auto-generate Promptfoo config: new smith --flag generate_promptfoo_config stage that creates/updates a Promptfoo redteam configuration from guidance and system variables, using a customizable template (PROMPTFOO_CONFIG_TEMPLATE).
• Policy Explorer UI bridge: added explorer_server.py to serve an interactive HTML view of the policy alongside IR data for visual inspection. It also supports reseting smith configuration based on selected guidance.
• LLM-based tool classification for promptfoo cases: during test generation, promptfoo cases are now classified to a target tool name via a single LLM call against the MCP tool definitions, removing the hardcoded "Promptfoo" placeholder. This steps aims to make test translation apply the same tool-name mismatch check to all cases uniformly. Also, when LLM calls several tools in one turn to make the decision, it avoids the promptfoo cases are assigned with the first tool call name blindly, causing false positives. For example, employee agent usually calls list_employee before every add_employee prompt. So a malicious promptfoo case will be assigned a wrong label list_employee
• Include target tool parameters in the LLM prompt so generated test cases include concrete parameter values: src/smith/test_generation/case_generation.py. For detailed reasons please see [BUG]: Test case generation produces vague prompts missing required tool parameters #41
• src/smith/test_generation/generate_promptfoo_config.py — Append tool parameter info to testGenerationInstructions in generated configs, same for [BUG]: Test case generation produces vague prompts missing required tool parameters #41
• Refactored get_tool_definitions() helper to deduplicate MCP tool extraction across CLI flags.
• Session config for IR mode: When working with IR-generated specs via the Policy Explorer UI, selected tool names are written to session_config.json; during test generation, translate_case filters out cases targeting tools not in that selection, keeping the test suite scoped to the tools under review.

Additional UI to read from raw guidance file.

• New Guidance Classifier feature, src/smith/tools/classify_guidance_lines.py uses LLM classifier mapping each guidance line → the MCP tool call(s) it governs, referencing tool_definitions.json.
• src/smith/tools/guidance_classifier_server.py, guidance_classifier.html serves the UI, independent from the IR explore html page and server.
• save a copy of produced files: src/smith/tools/save_snapshot.py , src/smith/cli.py
• scripts/clean_generated.sh now also empties assets/policy.rego.
• test_generation/test_generation.md adds promptfoo config reminder
• Uploads smith outputs grouped by tools.

Cpex agent demo

• Updated the agent.py file to include three additional endpoints: /chat, /extract_tool_call, /extract_tool_definition (specifically for this agent example, since its tool definitions are inside of the agent, not server.py).
• Added supports in smith to allow extract tool details from /extract_tool_definition endpoint.
• Bypass cpex proxy as for now when extracting tool call values, since the proxy is not deployed for now.
• Updated guidance.txt based on case understanding in https://github.com/praxis-proxy/demos/tree/main/demos/cpex.

Checks

• make ci passes (lint, Rego lint, license headers, build smoke)
• make test passes (policy scorecard — needed if policy behavior changed)
• CHANGELOG.md updated under ## [Unreleased] (if user-facing)
• Commits are signed off for the DCO (git commit -s)

[araujof]

Approving it with the following issues to followup as separate PRs.

• Incomplete arguments → args migration. Translation now writes args in src/smith/test_generation/extract_tool_args.py:73, but policy cross-validation still directs policies to read input.arguments in opa_policy/policy_cross_validation/policy_cross_validation.md:38. This can produce policies that ignore tool parameters.

• Unit suite fails. make unit has three failures: load_tool_list’s input contract disagrees with its test, and _resolve_tool_definitions_path is missing. The green CI does not run the new unit target (Makefile:177). Need to fix that in the CI too.

• Promptfoo updates are not idempotent. Every config update appends the same tool-parameter instructions again in src/smith/test_generation/generate_promptfoo_config.py:386, causing duplicated, growing prompts.