Skip to content

Improve JSON validation throughput by 11% - #122

Open
trevorprater wants to merge 1 commit into
valyala:masterfrom
trevorprater:codex/optimize-validate-strings
Open

Improve JSON validation throughput by 11%#122
trevorprater wants to merge 1 commit into
valyala:masterfrom
trevorprater:codex/optimize-validate-strings

Conversation

@trevorprater

@trevorprater trevorprater commented Jul 3, 2026

Copy link
Copy Markdown

What changed

This replaces the validation string/key path with a single-pass scanner that validates control characters and escape sequences while scanning, instead of first locating strings and then doing follow-up scans/parsing.

It also avoids the strconv/strings imports on this path and keeps Validate/ValidateBytes behavior unchanged.

Why

The existing Validate benchmark fixtures spend a meaningful amount of time in string/key validation. The new scan keeps the same JSON validity checks but removes extra passes over already-scanned strings.

This patch was found while experimenting with Sleepy - https://sleepy.run/mcp, then manually reviewed and validated locally before opening this PR.

Validation

Commands run on Apple M4 / darwin arm64:

go test -count=1 ./...

Result:

ok  	github.com/valyala/fastjson	0.480s
ok  	github.com/valyala/fastjson/fastfloat	0.319s

I also ran an additional local differential test in a temporary worktree comparing both Validate and ValidateBytes against encoding/json.Valid over focused edge cases plus 20,000 deterministic randomized ASCII inputs. That check passed.

Benchmark command:

go test -run '^$' -bench '^BenchmarkValidate/(small|medium|large|canada|citm|twitter)/fastjson$' -benchmem -count=20 .

benchstat summary:

Validate/small/fastjson-10     21.86n +/- 1%  -> 18.84n +/- 1%  -13.79%
Validate/medium/fastjson-10    224.3n +/- 1%  -> 192.3n +/- 1%  -14.29%
Validate/large/fastjson-10     3.149u +/- 1%  -> 2.635u +/- 1%  -16.32%
Validate/canada/fastjson-10    222.0u +/- 0%  -> 214.9u +/- 1%   -3.19%
Validate/citm/fastjson-10      116.1u +/-11%  -> 104.5u +/- 1%  -10.01%
Validate/twitter/fastjson-10   55.94u +/- 1%  -> 49.86u +/- 2%  -10.88%
geomean                         5.304u        -> 4.693u       -11.51%

Allocations stayed at 0 B/op and 0 allocs/op.

@trevorprater trevorprater changed the title [codex] optimize JSON validation string scanning Improve JSON validation throughput by 11% Jul 3, 2026
@trevorprater
trevorprater marked this pull request as ready for review July 3, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant