Skip to content

fix(npz): int64 overflow computing shape sub-products - #90

Merged
balwierz merged 1 commit into
mainfrom
fix/npy-shape-subproduct-overflow
Jul 25, 2026
Merged

fix(npz): int64 overflow computing shape sub-products#90
balwierz merged 1 commit into
mainfrom
fix/npy-shape-subproduct-overflow

Conversation

@balwierz

Copy link
Copy Markdown
Owner

Found by the .npy fuzz harness from #85, on the 24.04 fuzz smoke step of #89. That step is time-bounded, so it explores a different corpus each run — this is the third pre-existing bug it has surfaced that way (after #87 and the two in #85).

This one is reachable just by opening a crafted .npz, not only through the harness.

The bug

A .npy header can declare an empty array — any dimension zero — while its other dimensions are astronomically large. The declared-size check multiplies every dimension, so a single zero makes the total zero: the size check passes for free. Worse, the loop's guard is

if (dd != 0 && elems > UINT64_MAX / dd) return "shape too large";

so once elems is zero, every later guard is vacuous too.

The readers, though, multiply sub-ranges of the shape to compute strides and slice sizes. The 3-D+ path collapses the trailing dimensions into a single column count, and

shape = (1, 1, 392361265078550784, 29, 0)

makes that product overflow int64:

main.cpp:10365: runtime error: signed integer overflow:
392361265078550784 * 29 cannot be represented in type 'long int'

That line is NpzSource's real 3-D+ reader, reached by vv crafted.npz --tab a. The fuzz harness hits the same class a few lines away, where it collapses dims 1.. instead of 2...

The fix

The validator now also tracks the product of the non-zero dimensions and rejects when it passes INT64_MAX. Every sub-product of the shape divides that product, so bounding it bounds all of them — one check, rather than hardening each multiplication site separately and hoping none was missed.

Legitimate empty arrays ((0, 3) — the tiny.zerorow.npz fixture from #87) and ordinary 3-D arrays are unaffected, since their non-zero product is tiny.

Verification

  • 474 tests pass, and the same suite reruns green under ASan + UBSan.
  • New fixture tiny.shapeovf.npz, hand-built (numpy will not write such a header), with assertions that it is rejected with a clear message and a non-zero exit — so the ASan/UBSan CI job is the regression test rather than the fuzzer's luck.
  • Both repros confirmed fixed: the standalone .npy through fuzz_npy, and the .npz through the real reader under sanitizers.
  • fuzz_npy clean over 1.24 M iterations seeded with the repro.
  • Verified the legitimate cases still work: (0, 3) in both memory orders, and tiny.npz's (2, 3, 4) cube with slice stepping.

🤖 Generated with Claude Code

A .npy header can declare an EMPTY array — any dimension zero — while its
other dimensions are astronomically large. The declared-size check computes
the product of every dimension, so a single zero makes it zero: the size
check passes for free, and because the loop's overflow guard is
`if (dd != 0 && elems > UINT64_MAX / dd)`, every guard after the zero is
vacuous too.

The readers, though, multiply *sub-ranges* of the shape to compute strides
and slice sizes. The 3-D+ path collapses the trailing dimensions into a
single column count, and

    shape = (1, 1, 392361265078550784, 29, 0)

made that product overflow int64:

    main.cpp:10365: runtime error: signed integer overflow:
    392361265078550784 * 29 cannot be represented in type 'long int'

Reachable just by opening a crafted .npz — this is the real reader path, not
only the fuzz harness (which hits the same class one line-range away, where
it collapses dims 1.. instead of 2..).

Found by the .npy fuzz harness from #85, on the 24.04 fuzz smoke step of an
unrelated PR. That step is time-bounded, so it explores a different corpus
each run; this is the third pre-existing bug it has surfaced that way.

The validator now also tracks the product of the NON-ZERO dimensions and
rejects when it passes INT64_MAX. Every sub-product of the shape divides
that product, so bounding it bounds all of them — one check instead of
hardening each multiplication site separately. Legitimate empty arrays
((0, 3), the tiny.zerorow.npz fixture) and ordinary 3-D arrays are
unaffected, since their non-zero product is tiny.

Fixture tiny.shapeovf.npz (hand-built — numpy will not write such a header)
plus assertions that it is rejected with a clear message and a non-zero exit,
so the ASan/UBSan CI job is the regression test rather than the fuzzer's
luck. 474 tests pass, and the same suite reruns green under ASan+UBSan.
fuzz_npy is clean over 1.24 M iterations seeded with the repro.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@balwierz
balwierz merged commit a0281bd into main Jul 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant