Skip to content

Preserve integer-weight precision in Int64 and AtomicInt64 fills - #1207

Open
dhairyajangir wants to merge 4 commits into
scikit-hep:developfrom
dhairyajangir:fix/exact-integer-weights-20261112
Open

dhairyajangir wants to merge 4 commits into
scikit-hep:developfrom
dhairyajangir:fix/exact-integer-weights-20261112

Conversation

@dhairyajangir

@dhairyajangir dhairyajangir commented Nov 12, 2026 •

Copy link
Copy Markdown

Int64 and AtomicInt64 currently convert fill weights to double, so weight=2**53 + 1 stores 9007199254740992 instead of 9007199254740993, even though direct bin assignment preserves the value.

Keep Python integers, NumPy integer scalars/0-D arrays, and integer-dtype weight buffers exact through the native binding when all weights fit signed 64-bit storage. Select that converter for the two integer storages, check unsigned values before narrowing, and preserve the existing scalar protocol for objects that also export arrays. Conversion stays under the GIL and the typed buffers remain alive during filling.

Related to #289. This addresses ingestion precision; the existing fractional-weight conversion and accumulator-overflow behavior remain. Sequences already promoted by NumPy to floating/object dtype retain their existing route.

Validation on Linux, CPython 3.13.16 and NumPy 2.5.4, using a fresh C++14 build:

  • The same 100 focused cases go from 70 failures / 30 passes on unchanged develop to 100 passes. They cover signed endpoints, in-range unsigned inputs, non-contiguous buffers, broadcasting, threaded filling and scalar/array protocol controls.
  • pytest -n auto --benchmark-disable: 1,400 passed, four existing xfails.
  • Configured all-files prek hooks pass, including formatting and Mypy; an explicit changed-files run also covers the new test module.

Codex assisted implementation and validation.

The upstream Tests workflow for 26adaa8d is held for maintainer authorization; please approve that run. Pre-commit.ci and Read the Docs have passed.

dhairyajangir and others added 4 commits November 12, 2026 08:34
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
- Object-dtype integer weights keep exact int64 values
- Out-of-range integer weights raise OverflowError on integer storages
- Select the exact path with a storage trait; Unlimited uses it too
- Avoid repeated conversions and the numpy import on each fill

Assisted-by: ClaudeCode:claude-opus-5-5
@henryiii

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

I pushed 3dc13d8 with fixes from a review of this branch:

  • Object-dtype weights: object arrays of Python or NumPy ints (for example a list that mixes int64 values with larger ints) now keep exact int64 values. Before, they went through double.
  • Overflow: on Int64 and AtomicInt64, an integer weight outside the int64 range now raises OverflowError. Before, it went to double and then to an undefined double-to-int64 cast. On arm64, weight=2**64 stored INT64_MAX.
  • Storage trait: a trait on the storage type now selects the exact path, not a fixed list of two value types. Unlimited also uses it, so integer weights no longer change all cells to double. Values outside int64 still use double there.
  • Performance: the fallback reuses the converted array, the numpy.integer lookup is done once, and uint64 weights are copied once.

Regression tests for each change are in tests/test_integer_weights.py.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants