← All notes

Summary

The DeepSeek V3 Technical Report (2024) presents DeepSeek V3, a strong Mixture of Experts language model. The paper is at https://arxiv.org/abs/2412.19437. DeepSeek V3 is a Mixture of Experts language model with 671B total parameters with 37B activated for each token. It adopts Multi-head Latent Attention and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek V2. They also describe an auxiliary-loss-free load balancing strategy and a multi-token prediction training objective. They report pre-training on 14.8 trillion tokens, followed by supervised fine-tuning and reinforcement learning, and they state that full training took 2.788M H800 GPU hours, with a stable run and no irrecoverable loss spikes or rollbacks. They say the checkpoints are posted at https://github.com/deepseek-ai/DeepSeek-V3. The report says DeepSeek V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective. The authors pre-train it on 14.8 trillion diverse and high-quality tokens, followed by Supervised Fine-Tuning and Reinforcement Learning stages. They report that full training required only 2.788M H800 GPU hours, and that the training process was remarkably stable. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (18 Feb 2025), submitted 27 Dec 2024.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters because it documents a widely used open model at a scale that later work often treats as a baseline.
• Reproduced: not reproduced (2026-11-11).
• Related work: A related note already in the library is Mixtral of Experts.
• Limits: This is a pointer note only. Training-cost and size figures are from the authors' technical report (v2, 18 Feb 2025) and were not independently checked.
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 27 Dec 2024 (v1); last revised 18 Feb 2025 (v2). Sources: [S1] https://arxiv.org/abs/2412.19437, v2 (18 Feb 2025). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the official DeepSeek V3 technical report on model size, training recipe, and where the checkpoints were posted.

Facts

• id: F1 claim: 'DeepSeek V3 is described as a mixture-of-experts language model with 671B total parameters and 37B activated for each token.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors report pre-training on 14.8 trillion tokens and a full-training cost of 2.788M H800 GPU hours.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They say model checkpoints are available at https://github.com/deepseek-ai/DeepSeek-V3.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2412.19437, https://arxiv.org/abs/2412.19437. Paper license: arXiv default (non-exclusive distribution license). v2 (18 Feb 2025). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_wgkcfyzp,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{DeepSeek-V3 Technical Report}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2412-19437-deepseek-v3/},
  note = {Permanent ID iil:note:wgkcfyzp. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2412-19437-deepseek-v3.md.