---
source: https://iinventedlight.com/library/notes/pytorch-broadcasting.md
title: "Broadcasting semantics in PyTorch"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed; prose restates or closely paraphrases cited source lines"
license_family: pytorch-bsd
rule_check: "needs rule check (family not in auto-publish rule; check.mjs auto-approves only usgov and arxiv-ccby)"
attribution: "Summary by iinventedlight (AI-assisted). Sources: PyTorch docs, Broadcasting semantics — PyTorch 2.15 documentation, BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed; PyTorch docs, Tensor Attributes — PyTorch 2.15 documentation, BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: PyTorch"
tags: ['pytorch', 'broadcasting', 'tensors']
sources:
  - id: S1
    title: "Broadcasting semantics — PyTorch 2.15 documentation"
    publisher: PyTorch docs
    url: https://docs.pytorch.org/docs/2.15/notes/broadcasting.html
    accessed: 2026-11-04
    license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed"
  - id: S2
    title: "Tensor Attributes — PyTorch 2.15 documentation"
    publisher: PyTorch docs
    url: https://docs.pytorch.org/docs/2.15/tensor_attributes.html
    accessed: 2026-11-04
    license: "BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed"
---
# Broadcasting semantics in PyTorch

**Answer:** When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist. (S1)

## Result shape

- If the number of dimensions of x and y are not equal, prepend 1 to the dimensions of the tensor with fewer dimensions to make them equal length. (S1)
- Then, for each dimension size, the resulting dimension size is the max of the sizes of x and y along that dimension. (S1)
- One complication is that in-place operations do not allow the in-place tensor to change shape as a result of the broadcast. (S1)

## Related tensor attributes

- If the type of a scalar operand is of a higher category than tensor operands (where complex > floating > integral > boolean), we promote to a type with sufficient size to hold all scalar operands of that category. (S2)
- Strides are a list of integers: the k-th stride represents the jump in the memory necessary to go from one element to the next one in the k-th dimension of the Tensor. (S2)

## Claims
- id: C1
  claim: Sizes must be equal, 1, or missing.
  source: S1
  evidence: "the dimension sizes must either be equal, one of them is 1"
- id: C2
  claim: The result size is the max along each dim.
  source: S1
  evidence: "the resulting dimension size is the max of the sizes of"
- id: C3
  claim: In-place ops cannot change shape.
  source: S1
  evidence: "One complication is that in-place operations do not allow the in-place tensor to change shape"
- id: C4
  claim: Higher-category scalars promote the type.
  source: S2
  evidence: "we promote to a type with sufficient size to hold"

## Sources
- [S1] Broadcasting semantics — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/notes/broadcasting.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.
- [S2] Tensor Attributes — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/tensor_attributes.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).