← All notes

Summary

This Stanford CRFM report names large pretrained models such as BERT, DALL-E, and GPT-3 as foundation models: trained on broad data at scale and adaptable to many downstream tasks. It surveys capabilities, technical principles, applications, and societal impact, and argues that homogenization is powerful but risky because defects can be inherited by downstream adapted models. [S1]

Claims

• id: C1 claim: 'The authors call these large pretrained models foundation models.' source: S1 evidence: "We call these models foundation models to underscore their critically central yet incomplete character"
• id: C2 claim: 'They say defects of a foundation model can be inherited downstream.' source: S1 evidence: "the defects of the foundation model are inherited by all the adapted models downstream"
• id: C3 claim: 'They say we still lack a clear understanding of how the models work and when they fail.' source: S1 evidence: "we currently lack a clear understanding of how they work, when they fail"

Verification

• Code available: No software release is claimed. The authors point to a report page at https://crfm.stanford.edu/report.html, which returned HTTP 200 on 2026-11-11.
• Reproduced: not reproduced (2026-11-11). Code not run.
• Related work: Live notes 2204-05862-helpful-harmless-rlhf and 2212-08073-constitutional-ai. This is the paper-verification note for the foundation-model report itself.
• Limits: This is a survey and framing report, not a new model or benchmark. This note uses the abstract and key framing sections, not every chapter.
• License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 16 Aug 2021 (v1); last revised 12 Jul 2022 (v3). Sources: [S1] https://arxiv.org/abs/2108.07258, v3 (12 Jul 2022). checked_on: 2026-11-11.

When to apply

Use when you need the original, openly licensed account of what 'foundation models' were meant to name, including the warning about inherited defects.

Facts

• id: F1 claim: 'The report introduces the term foundation models for large pretrained models trained on broad data and adapted to many tasks.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F2 claim: 'The authors argue that defects of a foundation model are inherited by downstream adapted models.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F3 claim: 'They say we currently lack a clear understanding of how the models work, when they fail, and what they are capable of.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0

Sources

• [S1] arXiv 2108.07258, https://arxiv.org/abs/2108.07258. License: CC BY 4.0. v3 (12 Jul 2022). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

@misc{iil_note_49j0hndv,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{On the Opportunities and Risks of Foundation Models}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2108-07258-foundation-models/},
  note = {Permanent ID iil:note:49j0hndv. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2108-07258-foundation-models.md.