← All notes

Short answer

Build datasets from local folders (CSV/JSON/Parquet/text), from generators, or with ImageFolder/AudioFolder builders. [S1][S2]

Points

• Folder builders are low-code ways to create image or audio sets. [S1]
• Dataset.from_generator supports data larger than memory. [S2]
• You can also load from the Hub, local files, or in-memory objects. [S2]

Claims

• id: C1 claim: Folder builders include ImageFolder and AudioFolder. source: S1 evidence: "ImageFolder and AudioFolder"
• id: C2 claim: Common formats include csv, json/jsonl, parquet, txt. source: S1 evidence: "formats such as csv , json/jsonl , parquet , txt"
• id: C3 claim: from_generator supports data larger than memory. source: S2 evidence: "This approach supports loading data larger than available memory"
• id: C4 claim: Datasets can load from the Hub or local files. source: S2 evidence: "load a dataset from: The Hugging Face Hub Local files"

Sources

• [S1] Create a dataset · Hugging Face. Hugging Face (Datasets docs). https://huggingface.co/docs/datasets/create_dataset . License: Apache-2.0 (huggingface/datasets docs). Accessed 2026-11-05.
• [S2] Load. Hugging Face. https://huggingface.co/docs/datasets/loading . License: Apache-2.0 (huggingface/datasets docs). Accessed 2026-11-05.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

Source notice: Hugging Face Datasets documentation, Apache License 2.0 (https://github.com/huggingface/datasets/blob/main/LICENSE).

@misc{iil_note_q1yqmrj1,
  author = {{iinventedlight}},
  title = {{Create a Hugging Face dataset from your own files}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/hf-datasets-create-dataset/},
  note = {Permanent ID iil:note:q1yqmrj1. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-datasets-create-dataset.md.