Skip to content

Phase F: gateway CLI content, Stage 04 - Iterate scenario, and tester fixes#8

Merged
leggetter merged 5 commits intomainfrom
feat/phase-f-gateway-cli
Feb 25, 2026
Merged

Phase F: gateway CLI content, Stage 04 - Iterate scenario, and tester fixes#8
leggetter merged 5 commits intomainfrom
feat/phase-f-gateway-cli

Conversation

@leggetter
Copy link
Copy Markdown
Collaborator

Summary

Adds Phase F gateway CLI content to the event-gateway skill and extends the receive-webhooks agent scenario with a Stage 04 - Iterate (documentation-only): the agent is asked to document how to inspect and retry (list request → event → attempt, retry event) using the CLI. Also adds a convention for request → event → attempt ordering and fixes the scenario tester result-directory timestamp.

Changes

1. Convention and glossary (AGENTS.md)

  • Request → event → attempt ordering: When listing or describing these three together, always use the order request, then event, then attempt (data flow).
  • Checklist: skill authoring must use that order when mentioning requests, events, and attempts together.

2. Phase F skill content (event-gateway references)

  • 01-setup.md: hookdeck gateway connection create, hookdeck listen <port>, --path /webhooks, browser auth via hookdeck login.
  • 04-iterate.md: CLI inspect/retry workflow: hookdeck gateway request listevent listattempt list --event-id evt_xxx, and event retry evt_xxx.
  • cli-workflows.md: Gateway resource commands (connection, source, destination, transformation, request, event, attempt); request→event→attempt ordering; listen with port/path/source/connection.
  • monitoring-debugging.md: CLI inspection commands (request list/get/retry, event list/get/retry, attempt list/get).
  • referencing-docs.md: CLI doc URLs for gateway.
  • connection-architecture.md, connection-rules.md, authentication.md, api-patterns.md: Gateway/connection wording and consistency.

3. Agent scenario: Stage 04 - Iterate

  • scenarios.yaml (receive-webhooks):
    • New stage: iterate.
    • Prompt: assume user has received at least one webhook; ask agent to add a README/docs section explaining how to list requests, then events, then attempts, and how to retry a failed event with the CLI; cite 04-iterate.md if used.
    • Evaluation: Stage 04 - Iterate (2 pts): references 04-iterate/cli-workflows/monitoring-debugging; documents hookdeck gateway request/event/attempt list and retry; request→event→attempt order in documentation.
  • assess.ts: passesCheck() for Stage 04 (doc-based checks on combined log + README).

4. Testing docs and tester fix

  • TESTING.md: receive-webhooks total 17 pts; Stage 04 - Iterate rubric (documentation-only, no live traffic); note on preflight timeout (“Claude CLI did not respond within 15s” in restricted environments).
  • index.ts: Result directory timestamp no longer has a trailing dot (ISO string had . before ms; now strip . and use 14-char suffix).

Testing

Automated

  • Agent scenario run: ./scripts/test-agent-scenario.sh run receive-webhooks express --timeout 600
    • Result: Passed 17/17 (all stages including Stage 04 - Iterate).
    • Agent produced Express app with README containing inspect/retry workflow and cited verification-code.md and cli-workflows.md.
    • Stage 04 checks: reference to iterate docs ✓, gateway list + retry documented ✓, request→event→attempt order ✓.
  • Assessor: Stage 04 - Iterate checks evaluated correctly on run output (report.md generated with 2/2 for Stage 04).

Manual

  • CLI command verification: Confirmed that all CLI examples used in the skills exist and work with hookdeck (from PATH):
    • hookdeck gateway, hookdeck gateway request list, hookdeck gateway event list, hookdeck gateway event list --status FAILED --limit 20, hookdeck gateway event retry <event-id>, hookdeck gateway attempt list --event-id <event-id>, hookdeck listen, hookdeck listen 3000 --path /webhooks, hookdeck login.
  • Preflight: Running the scenario in a sandbox (network only) failed with “Claude CLI did not respond within 15s”; running with full permissions succeeded. TESTING.md updated to document this.

Follow-up

leggetter and others added 5 commits February 24, 2026 20:50
- Add convention to terminology: always list request, then event, then attempt
- Add checklist item for skill authoring

Co-authored-by: Cursor <cursoragent@cursor.com>
- 01-setup: hookdeck gateway connection create, listen with port/path
- 04-iterate: CLI inspect/retry (request list → event list → attempt list, event retry)
- cli-workflows: gateway resource commands, request→event→attempt ordering
- monitoring-debugging: CLI inspection commands
- referencing-docs: CLI doc URLs
- connection-*, authentication, api-patterns: gateway/connection wording

Co-authored-by: Cursor <cursoragent@cursor.com>
- scenarios.yaml: add iterate stage, extend prompt (document inspect/retry assuming traffic exists), add evaluation Stage 04 - Iterate (2 pts)
- assess.ts: passesCheck for Stage 04 (ref 04-iterate/cli-workflows/monitoring-debugging, gateway list+retry, request→event→attempt order)

Co-authored-by: Cursor <cursoragent@cursor.com>
…imestamp

- TESTING.md: receive-webhooks 17 pts, Stage 04 - Iterate rubric, preflight timeout note
- index.ts: timestamp without trailing dot (replace . in ISO string, slice 14 chars)

Co-authored-by: Cursor <cursoragent@cursor.com>
- Fix hookdeck listen argument order in 01-setup.md (port first, per CLI help)
- Add source prompting context for localhost-only use case
- Fix .md fetch URLs → standard URLs in monitoring-debugging.md footer
- Clarify "route" → "endpoint path" to avoid terminology confusion
- Remove redundant regex i flag in assess.ts (docNorm already lowercased)
- Add TOC to connection-architecture.md (over 100 lines, per AGENTS.md)
- Add live docs link to hookdeck router SKILL.md (checklist compliance)
- Add 04-iterate.md to SKILL.md Reference Material table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@leggetter leggetter merged commit 2b41b0b into main Feb 25, 2026
7 checks passed
@leggetter leggetter deleted the feat/phase-f-gateway-cli branch February 25, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant