fix(hdf5): read AnnData >= 0.13 nullable-string-array columns#79
Merged
Conversation
anndata 0.13 changed string obs/var columns and the DataFrame `_index` from a
plain `string-array` dataset to a `nullable-string-array` group (a `values`
string dataset + a boolean `mask`). vv's reader only handled the dataset form and
silently skipped the group, so on files written by current anndata the obs/var
string columns, the row index (cell/gene identifiers), and the labelled X preview
all came up empty — the CI 24.04 runner picked up anndata 0.13 and went red.
Decode the new form: read_anndata_dataframe's column walker and
read_anndata_index_labels now handle a `nullable-string-array` group via a shared
read_nullable_string_array() helper, applying the mask as nulls. HDF5 bools
decode to their enum member name ("TRUE"/"FALSE"), so the mask check accepts the
string form too. The legacy `string-array` dataset path is unchanged, so both
old and new fixtures read (verified against anndata 0.12 and 0.13; suite 374).
Adds tiny.nullstr.h5ad — hand-written with raw h5py so the new path (including the
mask -> NA case, which the anndata-generated fixtures don't exercise) is covered
regardless of the installed anndata version.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CI on Ubuntu 24.04 went red (9 AnnData/HDF5 tests) because the runner picked up anndata 0.13, which changed the on-disk encoding of string
obs/varcolumns and the DataFrame_indexfrom a plainstring-arraydataset to anullable-string-arraygroup (avaluesstring dataset + a booleanmask). vv's reader only knew the dataset form and silently skipped the group — so obs/var string columns, the row index (cell/gene identifiers), and the labelled X preview all came up empty on files written by current anndata.vv now decodes the new form:
read_anndata_dataframe's column walker andread_anndata_index_labelshandle anullable-string-arraygroup via a sharedread_nullable_string_array()helper, applying themaskas Arrow nulls."TRUE"/"FALSE"), so the mask check accepts the string form as well as a genuine Arrow boolean.string-arraydataset path is unchanged — both old and new files read.Verification
tiny.nullstr.h5ad— hand-written with raw h5py, so the new path (including the mask → NA case, which the anndata-generated fixtures don't exercise:label='beta'is masked to null) is covered regardless of the installed anndata version.This turns 24.04 green again (it was breaking every PR, not just this branch).
🤖 Generated with Claude Code