fix(npz): int64 overflow computing shape sub-products - #90
Merged
Conversation
A .npy header can declare an EMPTY array — any dimension zero — while its
other dimensions are astronomically large. The declared-size check computes
the product of every dimension, so a single zero makes it zero: the size
check passes for free, and because the loop's overflow guard is
`if (dd != 0 && elems > UINT64_MAX / dd)`, every guard after the zero is
vacuous too.
The readers, though, multiply *sub-ranges* of the shape to compute strides
and slice sizes. The 3-D+ path collapses the trailing dimensions into a
single column count, and
shape = (1, 1, 392361265078550784, 29, 0)
made that product overflow int64:
main.cpp:10365: runtime error: signed integer overflow:
392361265078550784 * 29 cannot be represented in type 'long int'
Reachable just by opening a crafted .npz — this is the real reader path, not
only the fuzz harness (which hits the same class one line-range away, where
it collapses dims 1.. instead of 2..).
Found by the .npy fuzz harness from #85, on the 24.04 fuzz smoke step of an
unrelated PR. That step is time-bounded, so it explores a different corpus
each run; this is the third pre-existing bug it has surfaced that way.
The validator now also tracks the product of the NON-ZERO dimensions and
rejects when it passes INT64_MAX. Every sub-product of the shape divides
that product, so bounding it bounds all of them — one check instead of
hardening each multiplication site separately. Legitimate empty arrays
((0, 3), the tiny.zerorow.npz fixture) and ordinary 3-D arrays are
unaffected, since their non-zero product is tiny.
Fixture tiny.shapeovf.npz (hand-built — numpy will not write such a header)
plus assertions that it is rejected with a clear message and a non-zero exit,
so the ASan/UBSan CI job is the regression test rather than the fuzzer's
luck. 474 tests pass, and the same suite reruns green under ASan+UBSan.
fuzz_npy is clean over 1.24 M iterations seeded with the repro.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by the
.npyfuzz harness from #85, on the 24.04 fuzz smoke step of #89. That step is time-bounded, so it explores a different corpus each run — this is the third pre-existing bug it has surfaced that way (after #87 and the two in #85).This one is reachable just by opening a crafted
.npz, not only through the harness.The bug
A
.npyheader can declare an empty array — any dimension zero — while its other dimensions are astronomically large. The declared-size check multiplies every dimension, so a single zero makes the total zero: the size check passes for free. Worse, the loop's guard isso once
elemsis zero, every later guard is vacuous too.The readers, though, multiply sub-ranges of the shape to compute strides and slice sizes. The 3-D+ path collapses the trailing dimensions into a single column count, and
makes that product overflow
int64:That line is
NpzSource's real 3-D+ reader, reached byvv crafted.npz --tab a. The fuzz harness hits the same class a few lines away, where it collapses dims1..instead of2...The fix
The validator now also tracks the product of the non-zero dimensions and rejects when it passes
INT64_MAX. Every sub-product of the shape divides that product, so bounding it bounds all of them — one check, rather than hardening each multiplication site separately and hoping none was missed.Legitimate empty arrays (
(0, 3)— thetiny.zerorow.npzfixture from #87) and ordinary 3-D arrays are unaffected, since their non-zero product is tiny.Verification
tiny.shapeovf.npz, hand-built (numpy will not write such a header), with assertions that it is rejected with a clear message and a non-zero exit — so the ASan/UBSan CI job is the regression test rather than the fuzzer's luck..npythroughfuzz_npy, and the.npzthrough the real reader under sanitizers.fuzz_npyclean over 1.24 M iterations seeded with the repro.(0, 3)in both memory orders, andtiny.npz's(2, 3, 4)cube with slice stepping.🤖 Generated with Claude Code