Repository navigation
Audio array remains NumPy after Torch formatting #8755
Description
Describe the bug
Since the migration to TorchCodec in Datasets 4.0.0, calling set_format(type="torch") or with_format("torch") on a dataset no longer makes the "array" field of an audio column a PyTorch tensor. It is a NumPy array instead.
However, the current Use with PyTorch documentation explicitly shows the "audio" field being a proper PyTorch tensor after with_format("torch").
Steps to reproduce the bug
from datasets import load_dataset
ds = load_dataset(
"hf-internal-testing/librispeech_asr_demo",
"clean",
split="validation",
)
ds.set_format(type="torch")
audio = ds[0]["audio"]["array"]
print(type(audio)) # numpy.ndarrayExpected behavior
Following the documentation and the behavior before 4.0.0, the "array" field of an audio column should return a PyTorch Tensor when Torch formatting is enabled.
Environment info
datasetsversion: 5.0.1- Platform: Linux-7.0.14-linuxkit-aarch64-with-glibc2.41
- Python version: 3.13.9
huggingface_hubversion: 1.33.0- PyArrow version: 25.0.1
- Pandas version: 3.0.6
fsspecversion: 2026.6.0
Activity
ssivitskii commentedon Oct 8, 2026
Hi, I'd like to help with this issue. I see #8757 is already updating the audio examples in the formatting docs. I'd be interested in taking on any remaining investigation or test coverage around
set_format(type="torch")/with_format("torch")and the audio decoding behavior.Could you confirm the intended behavior after the TorchCodec migration, and whether there is a useful follow-up I could work on alongside the docs PR?
krao19 commentedon Oct 26, 2026
Hi, I looked into this and can reproduce it on current
mainwith a local wav file (no Hub access needed):It comes from two places.
TorchFormatterpasses the torchcodecAudioDecoderthrough untouched, and theAudioDecoderwrapper infeatures/_torchcodec.pyalways does.cpu().numpy()when you ask for["array"], even though torchcodec already returns a tensor. So you get numpy whatever the format is.#8757 updates the docs to match this behaviour. If you'd rather have the code match the old docs instead, I have a small change ready: the torch formatter flags the decoder so
["array"]stays a tensor (channel averaging done in torch), and the numpy path is unchanged. It comes with a regression test that fails onmainand passes with the change, and stereo files give the same values as the numpy path.Which way would you prefer? Happy to open a PR if the code fix is wanted.