Skip to content

Commit 30ab924

Browse files
committed
feat: complete code generation architecture for assert/require packages
Inverts the generation model: internal/assertions is now the single source of truth, with assert/ and require/ packages fully generated including all variants (format, forward, tests, examples). Architecture: - Scanner: AST/types analysis extracts functions, signatures, and test examples - Generator: Template-based generation of 76 functions × 8 variants = 608 functions - Model: Structured representation bridging scanner and generator Key improvements: - Example-driven test generation from godoc "Examples:" sections - Achieves ~100% coverage in generated packages (99.5% - 0.5% gap from helpers) - Modern Go 1.23 iterator-based table-driven tests throughout - Domain-organized source (boolean, collection, compare, equal, error, etc.) - Function type detection for proper generation of type/var function signatures - Comprehensive documentation (MAINTAINERS.md, CLAUDE.md) - Codegen smoke tests (24.4% coverage) Generated packages: - assert/require: assertions, format variants, forward methods - All with corresponding tests and runnable examples - Helper types and interfaces This replaces the previous semi-hand-written/semi-generated approach where adding a single assertion required manually updating 6+ files that already had thousands of lines. Now: write once in internal/assertions/ in small focused source files, run go generate, done. Trade-off: the existing code generator has been rewritten entirely. The new one is more complex, but also more readable. This added (mostly stable) complexity should be outweighted by the simplification of the assertion development workflow. Signed-off-by: Frederic BIDON <[email protected]> ci: enforced Windows TMP to reside on the same drive Signed-off-by: Frederic BIDON <[email protected]>
1 parent b5da3fc commit 30ab924

File tree

162 files changed

+38111
-16066
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

162 files changed

+38111
-16066
lines changed

.claude/CLAUDE.md

Lines changed: 249 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,72 @@ This is the go-openapi fork of the testify testing package. The main goal is to
88

99
## Key Architecture
1010

11-
### Core Packages
11+
### Single Source of Truth: `internal/assertions/`
12+
13+
All assertion implementations live in `internal/assertions/`, organized by domain:
14+
- **boolean.go** - True, False
15+
- **collection.go** - Contains, Empty, Len, ElementsMatch, Subset, etc.
16+
- **compare.go** - Greater, Less, comparison assertions
17+
- **equal.go** - Equal, EqualValues, NotEqual, Same, etc.
18+
- **error.go** - Error, NoError, ErrorIs, ErrorAs, etc.
19+
- **file.go** - FileExists, DirExists, FileEmpty, FileNotEmpty
20+
- **http.go** - HTTPSuccess, HTTPError, HTTPStatusCode, etc.
21+
- **json.go** - JSONEq
22+
- **number.go** - InDelta, InEpsilon, Positive, Negative
23+
- **panic.go** - Panics, NotPanics, PanicsWithValue
24+
- **string.go** - Regexp, NotRegexp
25+
- **time.go** - WithinDuration
26+
- **type.go** - IsType, Zero, NotZero, Implements
27+
- **yaml.go** - YAMLEq
28+
29+
**Key principle:** Write assertions once in `internal/assertions/` with comprehensive tests. Everything else is generated.
30+
31+
### Core Packages (Generated)
1232
- **assert**: Provides non-fatal test assertions (tests continue after failures)
33+
- Generated from `internal/assertions/` by `codegen/`
34+
- Returns `bool` to indicate success/failure
1335
- **require**: Provides fatal test assertions (tests stop immediately on failure via `FailNow()`)
14-
- Both packages share similar APIs, but `require` wraps `assert` functions to make them fatal
36+
- Generated from `internal/assertions/` by `codegen/`
37+
- Void functions that call `FailNow()` on failure
1538

16-
### Code Generation
17-
- The codebase uses code generation extensively via `_codegen/main.go`
18-
- Generated files include:
19-
- `assert/assertion_format.go` - Format string variants of assertions
20-
- `assert/assertion_forward.go` - Forwarded assertion methods
21-
- `require/require.go` - Require variants of all assert functions
22-
- `require/require_forward.go` - Forwarded require methods
39+
Both packages are 100% generated and maintain API consistency mechanically.
40+
41+
### Code Generation Architecture
42+
43+
The codebase uses sophisticated code generation via the `codegen/` directory:
44+
45+
**Structure:**
46+
```
47+
codegen/
48+
├── internal/
49+
│ ├── scanner/ # Parses internal/assertions using go/packages and go/types
50+
│ ├── generator/ # Template-based code generation engine
51+
│ ├── model/ # Data model for assertions
52+
├── main.go # CLI orchestration
53+
└── (generated outputs in assert/ and require/)
54+
```
55+
56+
**Generated files include:**
57+
- **assert/assertion_assertions.go** - Package-level assertion functions
58+
- **assert/assertion_format.go** - Format string variants (Equalf, Truef, etc.)
59+
- **assert/assertion_forward.go** - Forwarded assertion methods for chaining
60+
- **assert/assertion_*_test.go** - Generated tests for all assert variants
61+
- **require/requirement_assertions.go** - Fatal assertion functions
62+
- **require/requirement_format.go** - Fatal format variants
63+
- **require/requirement_forward.go** - Fatal forwarded methods
64+
- **require/requirement_*_test.go** - Generated tests for all require variants
65+
66+
**Each assertion function generates 8 variants:**
67+
1. `assert.Equal(t, ...)` - package-level function
68+
2. `assert.Equalf(t, ..., "msg")` - format variant
69+
3. `a.Equal(...)` - forward method (where `a := assert.New(t)`)
70+
4. `a.Equalf(..., "msg")` - forward format variant
71+
5. `require.Equal(t, ...)` - fatal package-level
72+
6. `require.Equalf(t, ..., "msg")` - fatal format variant
73+
7. `r.Equal(...)` - fatal forward method
74+
8. `r.Equalf(..., "msg")` - fatal forward format variant
75+
76+
With 76 assertion functions, this generates 608 functions automatically.
2377

2478
### Dependency Isolation Strategy
2579
- **internal/spew**: Internalized copy of go-spew for pretty-printing values
@@ -36,42 +90,129 @@ The "enable" pattern allows YAML functionality to be opt-in: import `_ "github.c
3690
# Run all tests
3791
go test ./...
3892

39-
# Run tests in a specific package
40-
go test ./assert
41-
go test ./require
93+
# Run tests in specific packages
94+
go test ./internal/assertions # Source of truth with exhaustive tests
95+
go test ./assert # Generated package tests
96+
go test ./require # Generated package tests
4297

4398
# Run a single test
44-
go test ./assert -run TestEqual
99+
go test ./internal/assertions -run TestEqual
100+
101+
# Run with coverage
102+
go test -cover ./internal/assertions # Should be 90%+
103+
go test -cover ./assert # Should be ~100%
104+
go test -cover ./require # Should be ~100%
45105

46106
# Run tests with verbose output
47107
go test -v ./...
48108
```
49109

110+
### Adding a New Assertion
111+
112+
**The entire workflow:**
113+
1. Add function to appropriate file in `internal/assertions/`
114+
2. Add "Examples:" section to doc comment
115+
3. Add tests to corresponding `*_test.go` file
116+
4. Run `go generate ./...`
117+
5. Done - all 8 variants generated with tests
118+
119+
**Example - Adding a new assertion:**
120+
```go
121+
// In internal/assertions/string.go
122+
123+
// StartsWith asserts that the string starts with the given prefix.
124+
//
125+
// Examples:
126+
//
127+
// success: "hello world", "hello"
128+
// failure: "hello world", "bye"
129+
func StartsWith(t T, str, prefix string, msgAndArgs ...any) bool {
130+
if h, ok := t.(H); ok {
131+
h.Helper()
132+
}
133+
if !strings.HasPrefix(str, prefix) {
134+
return Fail(t, fmt.Sprintf("Expected %q to start with %q", str, prefix), msgAndArgs...)
135+
}
136+
return true
137+
}
138+
```
139+
140+
Then add tests in `internal/assertions/string_test.go` and run `go generate ./...`.
141+
142+
This generates:
143+
- `assert.StartsWith(t, str, prefix)`
144+
- `assert.StartsWithf(t, str, prefix, "msg")`
145+
- `a.StartsWith(str, prefix)` (forward method)
146+
- `a.StartsWithf(str, prefix, "msg")`
147+
- `require.StartsWith(t, str, prefix)`
148+
- `require.StartsWithf(t, str, prefix, "msg")`
149+
- `r.StartsWith(str, prefix)` (forward method)
150+
- `r.StartsWithf(str, prefix, "msg")`
151+
- Tests for all 8 variants
152+
50153
### Code Generation
51-
When modifying assertion functions in `assert/assertions.go`, regenerate derived code:
52154
```bash
53-
# Generate all code
155+
# Generate all code from internal/assertions
54156
go generate ./...
55157

56-
# This runs the codegen tool which:
57-
# 1. Parses assert/assertions.go for TestingT functions
58-
# 2. Generates format variants (e.g., Equalf from Equal)
59-
# 3. Generates require variants (fatal versions)
60-
# 4. Generates forwarded assertion methods
158+
# Or run the generator directly
159+
cd codegen && go run . -target assert
160+
cd codegen && go run . -target require
161+
162+
# The generator:
163+
# 1. Scans internal/assertions/ for exported functions
164+
# 2. Extracts "Examples:" from doc comments
165+
# 3. Generates assert/ package with all variants + tests
166+
# 4. Generates require/ package with all variants + tests
167+
# 5. Ensures 100% test coverage via example-driven tests
168+
```
169+
170+
### Example-Driven Test Generation
171+
172+
The generator reads "Examples:" sections from doc comments:
173+
174+
```go
175+
// Equal asserts that two objects are equal.
176+
//
177+
// Examples:
178+
//
179+
// success: 123, 123
180+
// failure: 123, 456
181+
func Equal(t T, expected, actual any, msgAndArgs ...any) bool {
182+
// implementation
183+
}
61184
```
62185

63-
The code generator looks for functions with signature `func(TestingT, ...) bool` in the assert package and creates corresponding variants.
186+
From this, it generates tests that verify:
187+
- Success case works correctly
188+
- Failure case works correctly and calls appropriate failure methods
189+
- Format variants work with message parameter
190+
- Forward methods work with chaining
191+
192+
**Test case types:**
193+
- `success: <args>` - Test should pass
194+
- `failure: <args>` - Test should fail
195+
- `panic: <args>` - Test should panic (followed by assertion message on next line)
196+
`<expected panic message>`
64197

65198
### Build and Verify
66199
```bash
67200
# Tidy dependencies
68201
go mod tidy
69202

70203
# Build code generator
71-
cd _codegen && go build
204+
cd codegen && go build
72205

73206
# Format code
74207
go fmt ./...
208+
209+
# Run all tests
210+
go test ./...
211+
212+
# Check coverage
213+
go test -cover ./internal/assertions
214+
go test -cover ./assert
215+
go test -cover ./require
75216
```
76217

77218
## Important Constraints
@@ -106,3 +247,89 @@ When using YAML assertions (YAMLEq, YAMLEqf):
106247
## Testing Philosophy
107248

108249
Keep tests simple and focused. The assert package provides detailed failure messages automatically, so test code should be minimal and readable. Use `require` when a test cannot continue meaningfully after a failure, and `assert` when subsequent checks might provide additional context.
250+
251+
### Testing Strategy: Layered Coverage
252+
253+
**Layer 1: Exhaustive Tests in `internal/assertions/`** (94% coverage)
254+
- Comprehensive table-driven tests using Go 1.23 `iter.Seq` patterns
255+
- Error message content and format validation
256+
- Edge cases, nil handling, type coercion scenarios
257+
- Domain-organized test files mirroring implementation
258+
- Source of truth for assertion correctness
259+
260+
**Layer 2: Generated Smoke Tests in `assert/` and `require/`** (~100% coverage)
261+
- Minimal mechanical tests proving functions exist and work
262+
- Success case: verify correct return value / no FailNow
263+
- Failure case: verify correct return value / FailNow called
264+
- Generated from "Examples:" in doc comments
265+
- No error message testing (already covered in Layer 1)
266+
267+
**Layer 3: Meta Tests for Generator** (future)
268+
- Test that code generation produces correct output
269+
- Verify function signatures, imports, structure
270+
- Optional golden file testing
271+
272+
This layered approach ensures:
273+
- Deep testing where it matters (source implementation)
274+
- Complete coverage of generated forwarding code
275+
- Simple, maintainable test generation
276+
- No duplication of complex test logic
277+
278+
## Architecture Benefits
279+
280+
### Why This Design Wins
281+
282+
**For Contributors:**
283+
- Add assertion in focused, domain-organized file
284+
- Write tests once in single location
285+
- Run `go generate` and get all variants for free
286+
- Clear separation: source vs generated code
287+
288+
**For Maintainers:**
289+
- Mechanical consistency across 608 generated functions
290+
- Template changes affect all functions uniformly
291+
- Easy to add new variants (e.g., generics)
292+
- Single source of truth prevents drift
293+
294+
**For Users:**
295+
- Comprehensive API with 76 assertions
296+
- All expected variants (package, format, forward, require)
297+
- Zero external dependencies
298+
- Drop-in replacement for stretchr/testify
299+
300+
**The Math:**
301+
- 76 assertion functions × 8 variants = 608 functions
302+
- Old model: Manually maintain 608 functions across multiple packages
303+
- New model: Write 76 functions once, generate the rest
304+
- Result: 87% reduction in manual code maintenance
305+
306+
### Technical Innovations
307+
308+
**Go AST/Types Integration:**
309+
- Scanner uses `go/packages` and `go/types` for semantic analysis
310+
- Position-based lookup bridges AST and type information
311+
- Import alias resolution for accurate code generation
312+
- Handles complex Go constructs (generics, interfaces, variadic args)
313+
314+
**Example-Driven Testing:**
315+
- "Examples:" sections in doc comments drive test generation
316+
- success/failure/panic cases extracted automatically
317+
- Tests generated for all 8 variants per function
318+
- Achieves 100% coverage with minimal test complexity
319+
320+
**Template Architecture:**
321+
- Separate templates for assert vs require packages
322+
- Conditional logic handles return values vs void functions
323+
- Mock selection based on FailNow requirements
324+
- Consistent formatting and structure across all output
325+
326+
## Example Coverage Status
327+
328+
Most assertion functions now have "Examples:" sections in their doc comments. The generator extracts these to create both tests and testable examples.
329+
330+
**Coverage notes:**
331+
- Basic assertions (Equal, Error, Contains, Len, True, False) have complete examples
332+
- Some complex assertions use TODO placeholders for pointer/struct values
333+
- All new assertions should include Examples before merging
334+
335+
For the complete guide on adding examples, see `docs/MAINTAINERS.md` section "Maintaining Generated Code".

.codecov.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
ignore:
2-
- _codegen
2+
- codegen

0 commit comments

Comments
 (0)