Tulu 3: Pushing Frontiers in Open Language Model Post-Training
Summary
Tulu 3 is a family of fully open post-trained models, released with data, code, and training recipes, built on Llama 3.1 base models. The training algorithms include supervised finetuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning with Verifiable Rewards (RLVR). The authors also introduce a multi-task evaluation scheme with development and unseen evaluations and substantial decontamination of existing open datasets. [S1]
Claims
- id: C1 claim: 'Tulu 3 is a fully open post-training family built on Llama 3.1 base models.' source: S1 evidence: "Tulu 3, which builds on Llama 3.1 base models"
- id: C2 claim: 'Training includes a method the authors call Reinforcement Learning with Verifiable Rewards (RLVR).' source: S1 evidence: "a novel method we call Reinforcement Learning with Verifiable Rewards (RLVR)"
- id: C3 claim: 'The authors release the complete recipe, including datasets, evaluation tooling, and training code.' source: S1 evidence: "we release the complete recipe -- including datasets for diverse core skills"
Verification
- Code available: The HTML names the training codebase allenai/open-instruct (https://github.com/allenai/open-instruct, HTTP 200 on 2026-11-12), the evaluation suite https://github.com/allenai/olmes (HTTP 200), and a demo at https://playground.allenai.org/ (HTTP 200). Not run.
- Reproduced: not reproduced (2026-11-12). Code not run.
- Related work: Live notes 2203-02155-instructgpt and 2305-18290-direct-preference-optimization.
- Limits: Comparisons with Llama 3.1 Instruct, Qwen 2.5, Mistral, GPT-4o-mini, and Claude 3.5-Haiku are author-reported. This note follows the v5 text last revised 14 Apr 2025.
- License: CC BY 4.0 on the abs page (2026-11-12; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 22 Nov 2024 (v1); last revised 14 Apr 2025 (v5). Sources: [S1] https://arxiv.org/abs/2411.15124, v5 (14 Apr 2025). checked_on: 2026-11-12.
When to apply
Use when you need an open post-training stack that combines SFT, DPO, and verifiable-reward RL.
Facts
- id: F1 claim: 'Tulu 3 is a fully open post-training recipe and model family built on Llama 3.1 base models.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F2 claim: 'The training algorithms include SFT, DPO, and Reinforcement Learning with Verifiable Rewards (RLVR).' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F3 claim: 'The authors release data, code, evaluation tooling, and a detailed reproduction report.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
Sources
- [S1] arXiv 2411.15124, https://arxiv.org/abs/2411.15124. License: CC BY 4.0. v5 (14 Apr 2025). Accessed 2026-11-12.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Nathan Lambert, Jacob Morrison, Valentina Pyatkin, and 20 coauthors (listed on the arXiv abs page), Tulu 3: Pushing Frontiers in Open Language Model Post-Training (CC BY 4.0).
Cite this
'iinventedlight (AI-assisted)'. 2026. "Tulu 3: Pushing Frontiers in Open Language Model Post-Training." iinventedlight Library, iil:note:kww5wzkw, version of 2026-11-12. https://iinventedlight.com/library/notes/2411-15124-tulu-3/. License: CC-BY-4.0.
@misc{iil_note_kww5wzkw,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Tulu 3: Pushing Frontiers in Open Language Model Post-Training}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/2411-15124-tulu-3/},
note = {Permanent ID iil:note:kww5wzkw. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2411-15124-tulu-3.md.