← All notes

Answer: When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist. (S1)

Result shape

• If the number of dimensions of x and y are not equal, prepend 1 to the dimensions of the tensor with fewer dimensions to make them equal length. (S1)
• Then, for each dimension size, the resulting dimension size is the max of the sizes of x and y along that dimension. (S1)
• One complication is that in-place operations do not allow the in-place tensor to change shape as a result of the broadcast. (S1)

Related tensor attributes

• If the type of a scalar operand is of a higher category than tensor operands (where complex > floating > integral > boolean), we promote to a type with sufficient size to hold all scalar operands of that category. (S2)
• Strides are a list of integers: the k-th stride represents the jump in the memory necessary to go from one element to the next one in the k-th dimension of the Tensor. (S2)

Claims

• id: C1 claim: Sizes must be equal, 1, or missing. source: S1 evidence: "the dimension sizes must either be equal, one of them is 1"
• id: C2 claim: The result size is the max along each dim. source: S1 evidence: "the resulting dimension size is the max of the sizes of"
• id: C3 claim: In-place ops cannot change shape. source: S1 evidence: "One complication is that in-place operations do not allow the in-place tensor to change shape"
• id: C4 claim: Higher-category scalars promote the type. source: S2 evidence: "we promote to a type with sufficient size to hold"

Sources

• [S1] Broadcasting semantics — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/notes/broadcasting.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.
• [S2] Tensor Attributes — PyTorch 2.15 documentation. PyTorch docs. https://docs.pytorch.org/docs/2.15/tensor_attributes.html . License: BSD-style (pytorch/pytorch LICENSE; Copyright (c) 2016-present, Facebook Inc.; https://github.com/pytorch/pytorch/blob/main/LICENSE); docs only, not the tutorials repo; PyTorch name and logo not licensed. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_ppysnfp8,
  author = {{iinventedlight}},
  title = {{Broadcasting semantics in PyTorch}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/pytorch-broadcasting/},
  note = {Permanent ID iil:note:ppysnfp8. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/pytorch-broadcasting.md.