Skip to content

Use 64-bit arithmetic for deep sample-count table offset - #2843

Open
cary-ilm wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
cary-ilm:fix-deep-samplecount-overflow
Open

cary-ilm wants to merge 2 commits into
AcademySoftwareFoundation:mainfrom
cary-ilm:fix-deep-samplecount-overflow

Conversation

@cary-ilm

Copy link
Copy Markdown
Member

The deep scanline and tiled readers computed the per-line offset into the sample-count table with 32-bit int arithmetic:

const int32_t* counts = decoder.sample_count_table + y * cinfo.width;

For a sufficiently large chunk/tile, y * cinfo.width can overflow a signed 32-bit int and wrap negative, producing a pointer before the allocated table and an out-of-bounds read in the following memcpy.

Promote the multiplication to int64_t, matching the already-safe sibling code paths (ImfDeepScanLineInputFile.cpp:942, ImfDeepTiledInputFile.cpp:1147).

In practice the deep scanline path is not reachable: deep data only permits compressions with one line per chunk (NONE/RLE/ZIPS/ZSTD), enforced on read, so cinfo.height is 1 and y is always 0. The deep tiled path is reachable only for tiles whose width*height exceeds INT_MAX, which also requires the >8 GB sample-count allocation to succeed first. This change is defense-in-depth and restores consistency with the 64-bit-safe fill paths.

Addresses https://github.com/AcademySoftwareFoundation/openexr/security/advisories/GHSA-54mf-6gmg-pr4v

Assisted-by: Copilot / Claude Opus 4.8

…gmg-pr4v)

The deep scanline and tiled readers computed the per-line offset into
the sample-count table with 32-bit int arithmetic:

    const int32_t* counts = decoder.sample_count_table + y * cinfo.width;

For a sufficiently large chunk/tile, y * cinfo.width can overflow a
signed 32-bit int and wrap negative, producing a pointer before the
allocated table and an out-of-bounds read in the following memcpy.

Promote the multiplication to int64_t, matching the already-safe
sibling code paths (ImfDeepScanLineInputFile.cpp:942,
ImfDeepTiledInputFile.cpp:1147).

In practice the deep scanline path is not reachable: deep data only
permits compressions with one line per chunk (NONE/RLE/ZIPS/ZSTD),
enforced on read, so cinfo.height is 1 and y is always 0. The deep
tiled path is reachable only for tiles whose width*height exceeds
INT_MAX, which also requires the >8 GB sample-count allocation to
succeed first. This change is defense-in-depth and restores
consistency with the 64-bit-safe fill paths.

Addresses https://github.com/AcademySoftwareFoundation/openexr/security/advisories/GHSA-54mf-6gmg-pr4v

Assisted-by: Copilot / Claude Opus 4.8

Signed-off-by: Cary Phillips <seabeepea@gmail.com>

@peterhillman peterhillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but do we not prefer static_cast<uint64_t>(y) instead of (uint64_t) y ?

Signed-off-by: Cary Phillips <seabeepea@gmail.com>
@cary-ilm

Copy link
Copy Markdown
Member Author

Yes, updated, and simplified several neighboring casts in the process.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants