← All notes

Summary

with_format("numpy") makes supported values appear as NumPy arrays when read. Without a requested format, Datasets returns ordinary Python objects. [S1]

Details

Fixed-shape nested data can form a single multidimensional array. Varying shapes may instead produce object arrays containing separate arrays. The NumPy guide recommends an explicit Array feature with a declared shape to avoid repeated shape checks and copies. Formatting can target selected columns, and it can be reset with None. [S1, S2]

Claims

• id: C1 claim: "Unformatted reads return regular Python values." source: S1 evidence: "By default, datasets return regular Python objects"
• id: C2 claim: "An explicit Array feature can declare the tensor shape." source: S1 evidence: "specify the shape of your tensors"
• id: C3 claim: "Formatting can be reset to its original behavior with None." source: S2 evidence: "set the format to None"

Sources

• [S1] Use with NumPy. Hugging Face contributors. https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/use_with_numpy.mdx. Apache-2.0; license: https://raw.githubusercontent.com/huggingface/datasets/main/LICENSE. Accessed 2026-11-05.
• [S2] Process. Hugging Face contributors. https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/process.mdx. Apache-2.0; license: https://raw.githubusercontent.com/huggingface/datasets/main/LICENSE. Accessed 2026-11-05.

Attribution and notices

This note is CC BY 4.0. Source material has been condensed and restated; short evidence excerpts retain their source license.

Hugging Face Datasets documentation is Apache-2.0. This note restates the documentation and includes only short evidence excerpts. See the accompanying source license.

@misc{iil_note_0hz5a0wz,
  author = {{Librarian}},
  title = {{NumPy formatting controls dataset reads, not the stored table}},
  howpublished = {iinventedlight Library},
  url = {https://iinventedlight.com/library/notes/hf-datasets-numpy-format/},
  note = {Permanent ID iil:note:0hz5a0wz. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-datasets-numpy-format.md.