---
title: "Jamba: A Hybrid Transformer-Mamba Language Model"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is CC BY-SA 4.0 (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2403-19887-jamba.md
source_license: "CC BY-SA 4.0"
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Opher Lieber, Barak Lenz, Hofit Bata, and 19 coauthors, Jamba: A Hybrid Transformer-Mamba Language Model (CC BY-SA 4.0). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: memory-and-context'
tags: arxiv, mamba, hybrid-architecture, moe, long-context
arxiv_id: 2403.19887
alias: 2403-19887-jamba
type: pointer
format: pointer
sources:
  - id: S1
    title: "Jamba: A Hybrid Transformer-Mamba Language Model"
    publisher: arXiv
    authors: "Opher Lieber and 21 coauthors (listed on the arXiv abs page)"
    url: https://arxiv.org/abs/2403.19887
    version_read: "v2 (3 Jul 2024)"
    accessed: 2026-11-12
    license: "CC BY-SA 4.0"
---
# Jamba: A Hybrid Transformer-Mamba Language Model

## Summary
[Jamba: A Hybrid Transformer-Mamba Language Model](https://arxiv.org/abs/2403.19887) (2024) introduced a large hybrid that interleaves Transformer attention with Mamba state-space layers, and adds mixture-of-experts in some layers so capacity can grow without activating every parameter on every token.

What it introduced: a flexible hybrid stack in which those two layer types, plus optional experts, can be mixed for a given compute budget. The configuration the authors implemented is described as fitting on a single 80GB GPU, with high throughput and a smaller memory footprint than a vanilla Transformer, and with reported strength on standard benchmarks and on long context, including results out to 256K tokens. They also discuss which architectural choices mattered at scale and planned to release ablation checkpoints.

What it led to: later hybrid and state-space language models that try to keep long context cheaper than full attention. Live notes include [2312-00752-mamba](https://iinventedlight.com/library/notes/2312-00752-mamba/) and [1706-03762-attention-is-all-you-need](https://iinventedlight.com/library/notes/1706-03762-attention-is-all-you-need/). The authors said the Jamba weights would be released under a permissive license; see the paper for the terms that actually apply.

Where to read it: https://arxiv.org/abs/2403.19887 (v2, last revised 3 Jul 2024). A project page is noted at https://www.ai21.com/jamba.

The paper is posted under CC BY-SA 4.0. This note links to it rather than reproducing it, so the note can stay CC BY 4.0. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: CC BY-SA 4.0; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
