Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions cmd/rune-mcp/main_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package main

import (
"context"
"errors"
"fmt"
"io"
"testing"
)

func TestIsNormalShutdown(t *testing.T) {
cases := []struct {
name string
err error
want bool
}{
{"nil — stdin EOF surfaces here (SDK filters io.EOF to nil)", nil, true},
{"context.Canceled — SIGINT/SIGTERM path", context.Canceled, true},
{"wrapped context.Canceled", fmt.Errorf("wrap: %w", context.Canceled), true},
{"raw io.EOF should NOT match — SDK already filters it", io.EOF, false},
{"context.DeadlineExceeded is a real error in shutdown", context.DeadlineExceeded, false},
{"arbitrary error", errors.New("transport broken"), false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := isNormalShutdown(tc.err); got != tc.want {
t.Errorf("isNormalShutdown(%v) = %v, want %v", tc.err, got, tc.want)
}
})
}
}
2 changes: 1 addition & 1 deletion docs/v04/progress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ README의 7-Phase 로드맵(Phase 1 외부 deps → Phase 7 검증)이 **horizon
| 마일스톤 | 상태 | 문서 | 관련 커밋 |
|---|---|---|---|
| Phase A — MCP boot (handshake + tools/list) | ✅ 합격 | [phase-a-mcp-boot.md](phase-a-mcp-boot.md) | `19b7bf6` (브랜치 `yg/first-mcp-boot`) |
| Phase A.5 — smoke test 추가 (CI 회귀 방지) | ⏳ 예정 | — | — |
| Phase A.5 — smoke test 추가 (CI 회귀 방지) | ✅ 합격 | [phase-a5-smoke-test.md](phase-a5-smoke-test.md) | `c353926` (브랜치 `yg/phase-a5-smoke-test`, PR #87) |
| Phase B — `rune_diagnostics` environment 섹션 진짜 응답 (stdlib only) | ⏳ 예정 | — | — |
| Phase 1 — `go.mod` 외부 deps 본격 추가 (gRPC · envector SDK · embedder proto) | ⏳ 예정 | — | — |
| Phase 2 — `internal/domain` + `internal/policy` 순수 로직 (TM scope) | ⏳ 예정 | — | — |
Expand Down
72 changes: 72 additions & 0 deletions docs/v04/progress/phase-a5-smoke-test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Phase A.5 — smoke tests

> ✅ 통과 (2026-04-27) · 커밋 `c353926` · 브랜치 `yg/phase-a5-smoke-test` · PR #87
> 핵심 파일 2개: `internal/mcp/register_test.go` (137줄) · `cmd/rune-mcp/main_test.go` (33줄)

## 1. 한 줄 요약

Phase A 의 bash/jq cookbook (`phase-a-mcp-boot.md` §4.2 / §4.3) 을 Go test 로 정착. `go test ./...` 한 번이 handshake + tools/list + tools/call stub + shutdown 정상성 회귀를 모두 가드. 외부 의존 0 (SDK 의 `NewInMemoryTransports` 사용 — 네트워크/stdio 없음, ~1.5s).

## 2. 동작하는 것

| 테스트 | 위치 | 잡아주는 회귀 |
|---|---|---|
| `TestRegister_All8ToolsListed` | `internal/mcp/register_test.go` | tool 누락 / 오타 / 이름 변경 (예: `rune_caputre`), SDK 알파벳 정렬 회귀 |
| `TestRegister_SchemasInferred` | 동 | domain/service struct 가 InputSchema · OutputSchema 추론 불가 형태로 변경 |
| `TestRegister_StubReturnsIsError` (8 subtest) | 동 | stub 응답 포맷 변경, IsError 누락, "not yet implemented" 문구 깨짐, 잘못된 핸들러 매핑 |
| `TestIsNormalShutdown` (6 subtest) | `cmd/rune-mcp/main_test.go` | EOF 매칭 잘못 추가, ctx.Canceled 누락, 진짜 에러를 정상 종료로 오판 |

총 4 함수 / 16 케이스, ~1.5s.

## 3. 검증 — 30초 컷

### 3.1. 전체 (가장 흔한 사용)

```bash
go test ./...
```

기대: 두 패키지 `ok`, 나머지 `[no test files]`.

### 3.2. Phase A.5 만

```bash
go test ./internal/mcp/ ./cmd/rune-mcp/
```

### 3.3. verbose / 단일 테스트

```bash
go test -v ./internal/mcp/ # 모든 subtest 출력
go test -v -run TestRegister_All8ToolsListed ./internal/mcp/ # 한 함수
go test -v -run 'TestRegister_StubReturnsIsError/rune_diagnostics' ./internal/mcp/ # 한 subtest
```

### 3.4. 캐시 무시 / 커버리지

```bash
go test -count=1 ./... # 캐시 우회
go test -cover ./internal/mcp/ ./cmd/rune-mcp/ # 커버리지 한 줄
go test -coverprofile=coverage.out ./internal/mcp/ && go tool cover -html=coverage.out # HTML
```

## 4. 새 tool 추가 시 절차

1. `internal/mcp/tools.go` `Register` 에 `sdkmcp.AddTool(...)` 한 줄 추가
2. `internal/mcp/register_test.go` 의 `expectedTools` 슬라이스에 새 이름 추가 (알파벳순 유지)
3. `go test ./internal/mcp/` 통과 확인 — 안 통과하면 보통 알파벳순 깨졌거나 이름 오타

## 5. 흔한 실패 → 원인

| 메시지 | 원인 |
|---|---|
| `tool count: got 7, want 8` | `tools.go` 에서 `AddTool` 빠뜨림 |
| `tool[N]: got "X", want "Y"` | SDK 알파벳 정렬 회귀, `expectedTools` 순서 미준수, 또는 이름 변경 |
| `InputSchema is nil` / `OutputSchema is nil` | args/result struct 가 schema inference 안 되는 타입 |
| `Content[0].Text does not contain stub marker` | `stubResult` 메시지 변경 — Phase 5 진입 전엔 실패해야 정상 |
| `isNormalShutdown(...) = true, want false` | `isNormalShutdown` 에 의도치 않은 매칭 추가 |

## 6. 다음 마일스톤

- **Phase B** — `rune_diagnostics` environment 섹션 stdlib 응답 (`runtime.GOOS` · `runtime.Version` · `os.Getwd`). 첫 진짜 응답 흐름. Phase A.5 의 `TestRegister_StubReturnsIsError/rune_diagnostics` 는 의도적으로 깨짐 — 새 메시지 expectations 로 갱신 필요
- **Phase 4 이후** — Vault/envector/embedder adapter 가 들어오면 in-memory mock 패턴 같이 도입 (Phase A.5 의 `NewInMemoryTransports` 패턴이 reference)
158 changes: 158 additions & 0 deletions internal/mcp/register_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
// Phase A.5 smoke tests — in-memory MCP server/client to assert that the
// 8-tool catalog and stub responses survive future refactors.
//
// These mirror the bash/jq cookbook in docs/v04/progress/phase-a-mcp-boot.md
// §4.2 (tools/list) and §4.3 (tools/call). Replacing the cookbook with Go
// tests turns the verification into a CI gate.

package mcp_test

import (
"strings"
"testing"

sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp"

"github.com/envector/rune-go/internal/mcp"
)

// expectedTools — alphabetical order matches what the SDK advertises in
// tools/list (Python rune v0.3.x bit-identical names).
var expectedTools = []string{
"rune_batch_capture",
"rune_capture",
"rune_capture_history",
"rune_delete_capture",
"rune_diagnostics",
"rune_recall",
"rune_reload_pipelines",
"rune_vault_status",
}

// newSession spins up an in-memory MCP server with all 8 tools registered
// and returns a connected client session ready for tools/list and tools/call.
func newSession(t *testing.T) *sdkmcp.ClientSession {
t.Helper()
ctx := t.Context()

srv := sdkmcp.NewServer(&sdkmcp.Implementation{
Name: "rune-mcp-test",
Version: "0.0.0-test",
}, nil)
if err := mcp.Register(srv, &mcp.Deps{}); err != nil {
t.Fatalf("Register: %v", err)
}

st, ct := sdkmcp.NewInMemoryTransports()
ss, err := srv.Connect(ctx, st, nil)
if err != nil {
t.Fatalf("server.Connect: %v", err)
}
t.Cleanup(func() { _ = ss.Close() })

client := sdkmcp.NewClient(&sdkmcp.Implementation{
Name: "rune-mcp-test-client",
Version: "0.0.0-test",
}, nil)
cs, err := client.Connect(ctx, ct, nil)
if err != nil {
t.Fatalf("client.Connect: %v", err)
}
t.Cleanup(func() { _ = cs.Close() })
return cs
}

func TestRegister_All8ToolsListed(t *testing.T) {
cs := newSession(t)

res, err := cs.ListTools(t.Context(), &sdkmcp.ListToolsParams{})
if err != nil {
t.Fatalf("ListTools: %v", err)
}

got := make([]string, len(res.Tools))
for i, tool := range res.Tools {
got[i] = tool.Name
}

if len(got) != len(expectedTools) {
t.Fatalf("tool count: got %d, want %d (got=%v)", len(got), len(expectedTools), got)
}
// SDK contract: tools/list returns alphabetical order (go-sdk sorts on
// emit). Compare position-by-position so a regression in the SDK or
// in registration ordering surfaces here.
for i, name := range expectedTools {
if got[i] != name {
t.Errorf("tool[%d]: got %q, want %q (full list: %v)", i, got[i], name, got)
}
}
}

func TestRegister_SchemasInferred(t *testing.T) {
cs := newSession(t)

res, err := cs.ListTools(t.Context(), &sdkmcp.ListToolsParams{})
if err != nil {
t.Fatalf("ListTools: %v", err)
}

// tools.go promises both input AND output schema are preserved
// (see Register / stubHandler comments). A nil on either side means
// schema inference broke for that tool.
for _, tool := range res.Tools {
if tool.InputSchema == nil {
t.Errorf("%s: InputSchema is nil — input schema inference regressed", tool.Name)
}
if tool.OutputSchema == nil {
t.Errorf("%s: OutputSchema is nil — output schema inference regressed", tool.Name)
}
}
Comment thread
esifea marked this conversation as resolved.
}

func TestRegister_StubReturnsIsError(t *testing.T) {
cs := newSession(t)

// All 8 tools — each carries minimal schema-valid args so SDK input
// validation passes and the stubHandler fires. Order matches expectedTools.
cases := []struct {
name string
args map[string]any
}{
{"rune_batch_capture", map[string]any{"items": "[]"}},
{"rune_capture", map[string]any{"text": "hi", "source": "test", "extracted": map[string]any{}}},
{"rune_capture_history", nil},
{"rune_delete_capture", map[string]any{"record_id": "test-id"}},
{"rune_diagnostics", nil},
{"rune_recall", map[string]any{"query": "hello"}},
{"rune_reload_pipelines", nil},
{"rune_vault_status", nil},
}

for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
res, err := cs.CallTool(t.Context(), &sdkmcp.CallToolParams{
Name: tc.name,
Arguments: tc.args,
})
if err != nil {
t.Fatalf("CallTool transport error: %v", err)
}
if !res.IsError {
t.Errorf("IsError: got false, want true (Phase A stub)")
}
if len(res.Content) == 0 {
t.Fatalf("Content: empty")
}
tc0, ok := res.Content[0].(*sdkmcp.TextContent)
if !ok {
t.Fatalf("Content[0]: got %T, want *TextContent", res.Content[0])
}
if !strings.Contains(tc0.Text, "not yet implemented") {
t.Errorf("Content[0].Text: %q does not contain stub marker", tc0.Text)
}
if !strings.Contains(tc0.Text, tc.name) {
t.Errorf("Content[0].Text: %q does not name the tool", tc0.Text)
}
})
}
}
Loading