← All notes

Hub repository structure declares splits and configurations

Summary

A Hub dataset repository holds data files and a README dataset card. Split names can be inferred from conventional file names or declared in README YAML. [S1]

Details

The configs field supports explicit file-to-split mappings and multiple named configurations. Each configuration needs a config_name, even when there is only one. Callers can pass a configuration name as the second load_dataset argument. Separately, revision selects the repository version; a configuration is not a branch or commit. [S1, S2]

Claims

Sources

Attribution and notices

This note is CC BY 4.0. Source material has been condensed and restated; short evidence excerpts retain their source license.

Hugging Face Datasets documentation is Apache-2.0. This note restates the documentation and includes only short evidence excerpts. See the accompanying source license.

License: CC-BY-4.0 · Original synthesis by iinventedlight, drafted by Librarian with AI assistance; based on the attributed sources

Cite this

Permanent ID: iil:note:bjksyetn

Librarian. n.d.. "Hub repository structure declares splits and configurations." iinventedlight Library, iil:note:bjksyetn. https://iinventedlight.com/library/notes/hf-datasets-repository-structure/. License: CC-BY-4.0.

@misc{iil_note_bjksyetn,
  author = {{Librarian}},
  title = {{Hub repository structure declares splits and configurations}},
  howpublished = {iinventedlight Library},
  url = {https://iinventedlight.com/library/notes/hf-datasets-repository-structure/},
  note = {Permanent ID iil:note:bjksyetn. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-datasets-repository-structure.md.