---
title: "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2405-04434-deepseek-v2.md
source_license: "arXiv non-exclusive distribution license"
attribution: 'Pointer by iinventedlight (AI-assisted). Source: DeepSeek-AI and 156 coauthors (listed on the arXiv abs page), DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: foundations'
tags: arxiv, deepseek, moe, attention, foundations
arxiv_id: 2405.04434
alias: 2405-04434-deepseek-v2
type: pointer
format: pointer
sources:
  - id: S1
    title: "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model"
    publisher: arXiv
    authors: "DeepSeek-AI and 156 coauthors (listed on the arXiv abs page)"
    url: https://arxiv.org/abs/2405.04434
    version_read: "v5 (19 Jun 2024)"
    accessed: 2026-11-12
    license: "arXiv non-exclusive distribution license"
---
# DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

## Summary
[DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://arxiv.org/abs/2405.04434) (2024) is the report that introduced the efficient DeepSeek architecture later scaled in V3 and R1. The model has 236 billion parameters in total, activates 21 billion per token, and supports a 128K context.

What it introduced: Multi-head Latent Attention, which compresses the key-value cache into a latent vector, and DeepSeekMoE, a sparse expert design aimed at cheaper training. Compared with DeepSeek 67B, the authors report stronger quality, 42.5% lower training cost, a 93.3% smaller KV cache, and up to 5.76 times higher maximum generation throughput. The model was pretrained on 8.1 trillion tokens, then fine-tuned and trained with reinforcement learning.

What it led to: the backbone used in DeepSeek-V3 and in the later DeepSeek-R1 reasoning models. Live notes: [2412-19437-deepseek-v3](https://iinventedlight.com/library/notes/2412-19437-deepseek-v3/), [2501-12948-deepseek-r1](https://iinventedlight.com/library/notes/2501-12948-deepseek-r1/), and [2402-03300-deepseekmath](https://iinventedlight.com/library/notes/2402-03300-deepseekmath/).

Where to read it: https://arxiv.org/abs/2405.04434 (v5, last revised 19 Jun 2024).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
