← All notes

Answer: At the heart of PyTorch data loading utility is the torch.utils.data.DataLoader class. It represents a Python iterable over a dataset, with support for map-style and iterable-style datasets, customizing data loading order, automatic batching, single- and multi-process data loading, automatic memory pinning. (S1)

Dataset styles

• The most important argument of DataLoader constructor is dataset, which indicates a dataset object to load data from. (S1)
• A map-style dataset, when accessed with dataset[idx], could read the idx-th image and its corresponding label from a folder on the disk. (S1)
• An iterable-style dataset is an instance of a subclass of IterableDataset that implements the __iter__() protocol, and represents an iterable over data samples. (S1)
• This type of datasets is particularly suitable for cases where random reads are expensive or even improbable, and where the batch size depends on the fetched data. (S1)

Multiple workers

• The same dataset object is replicated on each worker process, and thus the replicas must be configured differently to avoid duplicated data. (S1)
• Even if you have a pool of processes sending data to a single one, make it send the buffers back - this is nearly free and will let you avoid a copy when sending next batch. (S2)

Claims

• id: C1 claim: DataLoader is at the heart of data loading. source: S1 evidence: "At the heart of PyTorch data loading utility is the torch.utils.data.DataLoader"
• id: C2 claim: Iterable datasets implement __iter__. source: S1 evidence: "that implements the __iter__() protocol"
• id: C3 claim: Replicas on workers must be configured differently. source: S1 evidence: "replicas must be configured differently to avoid duplicated data."
• id: C4 claim: Send buffers back to avoid a copy. source: S2 evidence: "make it send the buffers back - this"

Sources

• [S1] torch.utils.data — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/data.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.
• [S2] Multiprocessing best practices — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/notes/multiprocessing.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_61q8dpd1,
  author = {{iinventedlight}},
  title = {{Datasets and DataLoader in PyTorch}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/pytorch-data-loading/},
  note = {Permanent ID iil:note:61q8dpd1. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/pytorch-data-loading.md.