Skip to content

fix(hdf5): read AnnData >= 0.13 nullable-string-array columns#79

Merged
balwierz merged 1 commit into
mainfrom
fix/anndata-compat
Jul 9, 2026
Merged

fix(hdf5): read AnnData >= 0.13 nullable-string-array columns#79
balwierz merged 1 commit into
mainfrom
fix/anndata-compat

Conversation

@balwierz

@balwierz balwierz commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

CI on Ubuntu 24.04 went red (9 AnnData/HDF5 tests) because the runner picked up anndata 0.13, which changed the on-disk encoding of string obs/var columns and the DataFrame _index from a plain string-array dataset to a nullable-string-array group (a values string dataset + a boolean mask). vv's reader only knew the dataset form and silently skipped the group — so obs/var string columns, the row index (cell/gene identifiers), and the labelled X preview all came up empty on files written by current anndata.

vv now decodes the new form:

  • read_anndata_dataframe's column walker and read_anndata_index_labels handle a nullable-string-array group via a shared read_nullable_string_array() helper, applying the mask as Arrow nulls.
  • HDF5 bools decode to their enum member name ("TRUE"/"FALSE"), so the mask check accepts the string form as well as a genuine Arrow boolean.
  • The legacy string-array dataset path is unchanged — both old and new files read.

Verification

  • Reproduced by installing the latest anndata in a fresh venv (0.13.1) and regenerating the fixtures — 9 failures, matching CI. After the fix: full suite 374 against anndata 0.13 and 0.12 (backward + forward compatible).
  • New tiny.nullstr.h5ad — hand-written with raw h5py, so the new path (including the mask → NA case, which the anndata-generated fixtures don't exercise: label='beta' is masked to null) is covered regardless of the installed anndata version.
  • Committed anndata fixtures are left in the local form; CI regenerates them (now with 0.13) and the reader handles them.

This turns 24.04 green again (it was breaking every PR, not just this branch).

🤖 Generated with Claude Code

anndata 0.13 changed string obs/var columns and the DataFrame `_index` from a
plain `string-array` dataset to a `nullable-string-array` group (a `values`
string dataset + a boolean `mask`). vv's reader only handled the dataset form and
silently skipped the group, so on files written by current anndata the obs/var
string columns, the row index (cell/gene identifiers), and the labelled X preview
all came up empty — the CI 24.04 runner picked up anndata 0.13 and went red.

Decode the new form: read_anndata_dataframe's column walker and
read_anndata_index_labels now handle a `nullable-string-array` group via a shared
read_nullable_string_array() helper, applying the mask as nulls. HDF5 bools
decode to their enum member name ("TRUE"/"FALSE"), so the mask check accepts the
string form too. The legacy `string-array` dataset path is unchanged, so both
old and new fixtures read (verified against anndata 0.12 and 0.13; suite 374).

Adds tiny.nullstr.h5ad — hand-written with raw h5py so the new path (including the
mask -> NA case, which the anndata-generated fixtures don't exercise) is covered
regardless of the installed anndata version.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@balwierz
balwierz merged commit 2228b15 into main Jul 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant