← All notes

Answer: Results may not be reproducible between CPU and GPU executions, even when using identical seeds. However, there are some steps you can take to limit the number of sources of nondeterministic behavior for a specific platform, device, and PyTorch release. (S1)

Two levers

• First, you can control sources of randomness that can cause multiple executions of your application to behave differently. (S1)
• Second, you can configure PyTorch to avoid using nondeterministic algorithms for some operations, so that multiple calls to those operations, given the same inputs, will produce the same result. (S1)
• torch.manual_seed sets the seed for generating random numbers on all devices. (S2)
• Some applications and libraries may use NumPy Random Generator objects, not the global RNG, and those will need to be seeded consistently as well. (S1)

Trade-off

• Deterministic operations are often slower than nondeterministic operations, so single-run performance may decrease for your model. (S1)
• However, determinism may save time in development by facilitating experimentation, debugging, and regression testing. (S1)

Claims

• id: C1 claim: CPU and GPU results may differ. source: S1 evidence: "reproducible between CPU and GPU executions"
• id: C2 claim: manual_seed seeds all devices. source: S2 evidence: "Sets the seed for generating random numbers on all devices."
• id: C3 claim: Deterministic ops are often slower. source: S1 evidence: "Deterministic operations are often slower than nondeterministic operations"
• id: C4 claim: Determinism helps debugging. source: S1 evidence: "save time in development by facilitating experimentation, debugging, and"

Sources

• [S1] Reproducibility — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/notes/randomness.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.
• [S2] torch.manual_seed — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/generated/torch.manual_seed.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_fm6hrnrk,
  author = {{iinventedlight}},
  title = {{Reproducibility in PyTorch}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/pytorch-reproducibility/},
  note = {Permanent ID iil:note:fm6hrnrk. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/pytorch-reproducibility.md.