---
title: 'DeepSeek-V3 Technical Report'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'This note is CC BY 4.0. It links to the paper and does not reproduce it. Paper license: arXiv non-exclusive distribution (verified on arXiv abs page 2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/).'
source: https://iinventedlight.com/library/notes/2412-19437-deepseek-v3.md
source_license: 'arXiv default (non-exclusive distribution license)'
attribution: 'Pointer note by iinventedlight (AI-assisted). Links to: DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, et al. (200 authors), DeepSeek-V3 Technical Report (arXiv). This note does not reproduce the paper.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-11
status: published
approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\""
checked_on: 2026-11-11
as_of: 2026-11-11
confidence: medium
untested: 'Pointer note only. Abs page read 2026-11-11. The paper is not reproduced here.'
category: arxiv
topic_area: 'ai-research: foundations'
tags: arxiv, pointer, foundations
arxiv_id: 2412.19437
alias: 2412.19437-deepseek-v3
format: pointer
type: pointer
sources:
  - id: S1
    title: 'DeepSeek-V3 Technical Report'
    publisher: arXiv
    authors: 'DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, et al. (200 authors)'
    url: https://arxiv.org/abs/2412.19437
    version_read: 'v2 (18 Feb 2025)'
    accessed: 2026-11-11
    license: 'arXiv default (non-exclusive distribution license)'
---
# DeepSeek-V3 Technical Report

## Summary
The DeepSeek V3 Technical Report (2024) presents DeepSeek V3, a strong Mixture of Experts language model. The paper is at https://arxiv.org/abs/2412.19437. DeepSeek V3 is a Mixture of Experts language model with 671B total parameters with 37B activated for each token. It adopts Multi-head Latent Attention and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek V2. They also describe an auxiliary-loss-free load balancing strategy and a multi-token prediction training objective. They report pre-training on 14.8 trillion tokens, followed by supervised fine-tuning and reinforcement learning, and they state that full training took 2.788M H800 GPU hours, with a stable run and no irrecoverable loss spikes or rollbacks. They say the checkpoints are posted at https://github.com/deepseek-ai/DeepSeek-V3. The report says DeepSeek V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective. The authors pre-train it on 14.8 trillion diverse and high-quality tokens, followed by Supervised Fine-Tuning and Reinforcement Learning stages. They report that full training required only 2.788M H800 GPU hours, and that the training process was remarkably stable. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (18 Feb 2025), submitted 27 Dec 2024.

## Claims
Pointer notes do not quote the paper. See Verification and Sources for the link and license.

## Verification
- Format: pointer note only. The paper is not reproduced here.
- Why it matters: It matters because it documents a widely used open model at a scale that later work often treats as a baseline.
- Reproduced: not reproduced (2026-11-11).
- Related work: A related note already in the library is [Mixtral of Experts](https://iinventedlight.com/library/notes/2401-04088-mixtral-of-experts/).
- Limits: This is a pointer note only. Training-cost and size figures are from the authors' technical report (v2, 18 Feb 2025) and were not independently checked.
- License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 27 Dec 2024 (v1); last revised 18 Feb 2025 (v2). Sources: [S1] https://arxiv.org/abs/2412.19437, v2 (18 Feb 2025). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

## When to apply
Use when you need the official DeepSeek V3 technical report on model size, training recipe, and where the checkpoints were posted.

## Facts
- id: F1
  claim: 'DeepSeek V3 is described as a mixture-of-experts language model with 671B total parameters and 37B activated for each token.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0
- id: F2
  claim: 'The authors report pre-training on 14.8 trillion tokens and a full-training cost of 2.788M H800 GPU hours.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0
- id: F3
  claim: 'They say model checkpoints are available at https://github.com/deepseek-ai/DeepSeek-V3.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0

## Sources
- [S1] arXiv 2412.19437, https://arxiv.org/abs/2412.19437. Paper license: arXiv default (non-exclusive distribution license). v2 (18 Feb 2025). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).