---
title: "How the Hugging Face Datasets cache works and where it lives"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://iinventedlight.com/library/notes/hf-datasets-cache.md
source_license: "Apache-2.0 (huggingface/datasets docs); restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: Hugging Face (Datasets docs), The cache · Hugging Face, Apache-2.0 (huggingface/datasets docs); Hugging Face (Datasets docs), Cache management · Hugging Face, Apache-2.0 (huggingface/datasets docs)."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No code was run; only the fetched Hugging Face documentation pages were read."
category: huggingface-datasets
topic_area: "datasets: caching"
tags: huggingface, datasets, cache
sources:
  - id: S1
    title: "The cache · Hugging Face"
    publisher: Hugging Face (Datasets docs)
    url: https://huggingface.co/docs/datasets/about_cache
    accessed: 2026-11-04
    license: "Apache-2.0 (huggingface/datasets docs)"
  - id: S2
    title: "Cache management · Hugging Face"
    publisher: Hugging Face (Datasets docs)
    url: https://huggingface.co/docs/datasets/cache
    accessed: 2026-11-04
    license: "Apache-2.0 (huggingface/datasets docs)"
---
# How the Hugging Face Datasets cache works and where it lives

## Short answer
Downloads go to the Hub cache (`~/.cache/huggingface/hub`); Arrow versions and processed results go to `~/.cache/huggingface/datasets`. Move both by setting `HF_HOME`. Re-running the same `map` reuses cached results because each transform has a fingerprint. [S1][S2]

## Gotchas
- A function that cannot be pickled gets a random fingerprint, so it is recomputed every time. [S1]
- With caching off, results go to a temp folder; call `save_to_disk()` to keep them. [S1]

## Claims
- id: C1
  claim: The cache reloads previously processed datasets.
  source: S1
  evidence: "It stores previously downloaded and processed datasets"
- id: C2
  claim: Non-hashable transforms get a random fingerprint.
  source: S1
  evidence: "uses a random fingerprint instead and raises a warning"
- id: C3
  claim: Use save_to_disk when caching is disabled.
  source: S1
  evidence: "When caching is disabled, use Dataset.save_to_disk() to save your transformed dataset"
- id: C4
  claim: Hub files are cached under ~/.cache/huggingface/hub.
  source: S2
  evidence: "which is at ~/.cache/huggingface/hub by default"
- id: C5
  claim: HF_HOME changes the cache location.
  source: S2
  evidence: "Change the cache location by setting the shell environment variable, HF_HOME"

## Sources
- [S1] The cache · Hugging Face. Hugging Face (Datasets docs). https://huggingface.co/docs/datasets/about_cache . License: Apache-2.0 (huggingface/datasets docs). Accessed 2026-11-04.
- [S2] Cache management · Hugging Face. Hugging Face (Datasets docs). https://huggingface.co/docs/datasets/cache . License: Apache-2.0 (huggingface/datasets docs). Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).