← All notes

Summary

load_dataset accepts a Hub repository identifier. Its revision argument selects a tag, branch, or commit, while file and split arguments select data within that version. [S1]

Details

data_files can map chosen files to named splits. data_dir narrows loading to a directory. A repository can also declare split assignments in the YAML block of its README. These controls address different questions: which repository version, which files, and which portion of the dataset. A simple repository with train.csv and test.csv yields the corresponding two splits. [S1, S2]

Claims

• id: C1 claim: "revision selects a version of a Hub dataset." source: S1 evidence: "Use the revision parameter to specify the dataset version"
• id: C2 claim: "data_files can assign files to named splits." source: S1 evidence: "Use the data_files parameter to map data files to splits"
• id: C3 claim: "The simple train.csv and test.csv layout produces two splits." source: S2 evidence: "you'll get a dataset with two splits"

Sources

• [S1] Load. Hugging Face contributors. https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/loading.mdx. Apache-2.0; license: https://raw.githubusercontent.com/huggingface/datasets/main/LICENSE. Accessed 2026-11-05.
• [S2] Structure your repository. Hugging Face contributors. https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/repository_structure.mdx. Apache-2.0; license: https://raw.githubusercontent.com/huggingface/datasets/main/LICENSE. Accessed 2026-11-05.

Attribution and notices

This note is CC BY 4.0. Source material has been condensed and restated; short evidence excerpts retain their source license.

Hugging Face Datasets documentation is Apache-2.0. This note restates the documentation and includes only short evidence excerpts. See the accompanying source license.

@misc{iil_note_q0fp855z,
  author = {{Librarian}},
  title = {{Loading Hub data separates revision, files, and splits}},
  howpublished = {iinventedlight Library},
  url = {https://iinventedlight.com/library/notes/hf-datasets-load-from-hub/},
  note = {Permanent ID iil:note:q0fp855z. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-datasets-load-from-hub.md.