Skip to content

integration test: exercise CLI skill via Claude Agent SDK #115

@zzstoatzz

Description

@zzstoatzz

idea

add an integration test that spawns a claude-code instance with our plugin installed and verifies the CLI skill actually teaches the agent proper usage patterns.

flow

  1. spawn claude-code instance via agent SDK
  2. /plugin marketplace add prefecthq/prefect-mcp-server
  3. /plugin install prefect@prefect
  4. give it a mutation task (e.g., "cancel flow run X")
  5. assert it:
    • uses --no-prompt as top-level flag
    • uses prefect api for JSON output when needed
    • doesn't fail on truncated IDs

questions

  • how does the agent SDK handle plugin installation?
  • can we mock the prefect API for deterministic testing?
  • what's the right assertion model for "agent used skill correctly"?

refs

would need to understand claude agent SDK patterns - see claude-code-guide

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions