← All notes

PaLM: Scaling Language Modeling with Pathways

Summary

PaLM is a 540-billion parameter, densely activated, Transformer language model trained on 6144 TPU v4 chips using Pathways. The authors report state-of-the-art few-shot results on hundreds of language understanding and generation benchmarks. They say PaLM 540B outperforms the finetuned state of the art on a suite of multi-step reasoning tasks, and outperforms average human performance on the then-recently released BIG-bench benchmark, with discontinuous improvements from scale on a significant number of BIG-bench tasks. They also report strong multilingual and source-code generation results, and they include analysis of bias, toxicity, and training-data memorization. [S1]

Claims

Verification

When to apply

Use when tracing 540B-scale dense language modeling, Pathways training, or early few-shot reasoning results at that scale.

Facts

Sources

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Aakanksha Chowdhery and 66 coauthors (listed on the arXiv abs page), PaLM: Scaling Language Modeling with Pathways (CC BY 4.0).

Cite this

Permanent ID: iil:note:j2xmd2px

'iinventedlight (AI-assisted)'. 2026. "PaLM: Scaling Language Modeling with Pathways." iinventedlight Library, iil:note:j2xmd2px, version of 2026-11-12. https://iinventedlight.com/library/notes/2204-02311-palm/. License: CC-BY-4.0.

@misc{iil_note_j2xmd2px,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{PaLM: Scaling Language Modeling with Pathways}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-12},
  url = {https://iinventedlight.com/library/notes/2204-02311-palm/},
  note = {Permanent ID iil:note:j2xmd2px. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2204-02311-palm.md.