---
source: https://iinventedlight.com/library/notes/pytorch-cuda-semantics.md
title: "CUDA semantics in PyTorch"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed; prose restates or closely paraphrases cited source lines"
license_family: pytorch-bsd
rule_check: "needs rule check (family not in auto-publish rule; check.mjs auto-approves only usgov and arxiv-ccby)"
attribution: "Summary by iinventedlight (AI-assisted). Sources: PyTorch docs, CUDA semantics — PyTorch 2.15 documentation, BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed; PyTorch docs, torch.cuda — PyTorch 2.15 documentation, BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: PyTorch"
tags: ['pytorch', 'cuda', 'gpu']
sources:
  - id: S1
    title: "CUDA semantics — PyTorch 2.15 documentation"
    publisher: PyTorch docs
    url: https://docs.pytorch.org/docs/2.15/notes/cuda.html
    accessed: 2026-11-04
    license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed"
  - id: S2
    title: "torch.cuda — PyTorch 2.15 documentation"
    publisher: PyTorch docs
    url: https://docs.pytorch.org/docs/2.15/cuda.html
    accessed: 2026-11-04
    license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed"
---
# CUDA semantics in PyTorch

**Answer:** torch.cuda is used to set up and run CUDA operations. It keeps track of the currently selected GPU, and all CUDA tensors you allocate will by default be created on that device. (S1)

## Devices

- The selected device can be changed with a torch.cuda.device context manager. (S1)
- Once a tensor is allocated, you can do operations on it irrespective of the selected device, and the results will be always placed on the same device as the tensor. (S1)
- Unless you enable peer-to-peer memory access, any attempts to launch ops on tensors spread across different devices will raise an error. (S1)

## Asynchrony and memory

- A consequence of the asynchronous computation is that time measurements without synchronizations are not accurate. (S1)
- Several functions such as to() and copy_() admit an explicit non_blocking argument, which lets the caller bypass synchronization when it is unnecessary. (S1)
- Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU applications. (S1)
- The torch.cuda module can check if peer access between two devices is possible and return the currently selected Stream for a given device. (S2)

## Claims
- id: C1
  claim: torch.cuda can check peer access.
  source: S2
  evidence: "Check if peer access between two devices is possible."
- id: C2
  claim: CUDA tensors go on the selected device by default.
  source: S1
  evidence: "and all CUDA tensors you allocate will by default be"
- id: C3
  claim: Cross-device ops raise an error.
  source: S1
  evidence: "tensors spread across different devices will raise an error."
- id: C4
  claim: Timing without synchronization is inaccurate.
  source: S1
  evidence: "synchronizations are not accurate. To get precise measurements"
- id: C5
  claim: empty_cache releases unused cached memory.
  source: S1
  evidence: "releases all unused cached memory from PyTorch so that those can be used"

## Sources
- [S1] CUDA semantics — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/notes/cuda.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.
- [S2] torch.cuda — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/cuda.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).