---
title: 'Decoupled Weight Decay Regularization'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/1711-05101-adamw.md
source_license: 'arXiv non-exclusive distribution license'
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Ilya Loshchilov, Frank Hutter, Decoupled Weight Decay Regularization (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: efficiency'
tags: arxiv, efficiency, optimization, adamw
arxiv_id: 1711.05101
alias: 1711.05101-adamw
type: pointer
format: pointer
sources:
  - id: S1
    title: 'Decoupled Weight Decay Regularization'
    publisher: arXiv
    authors: 'Ilya Loshchilov, Frank Hutter'
    url: https://arxiv.org/abs/1711.05101
    version_read: 'v3 (4 Jan 2019)'
    accessed: 2026-11-12
    license: 'arXiv non-exclusive distribution license'
---
# Decoupled Weight Decay Regularization

## Summary
[Decoupled Weight Decay Regularization](https://arxiv.org/abs/1711.05101) (2017) showed that L2 regularization and weight decay are not the same thing for adaptive methods such as Adam, even though they coincide for ordinary SGD once the learning rate is taken into account. The authors propose decoupling weight decay from the adaptive update. The resulting optimizer, AdamW, is now the default for almost all large-scale network training.

What it introduced: a corrected implementation of weight decay for Adam-style optimizers, in which decay is applied directly to the parameters instead of being folded into the gradient as an L2 term.

What it led to: AdamW as the usual training default in language, vision, and multimodal models. If a later paper says it trained with AdamW, this is the reference for what that change was meant to fix.

Use it when you want the paper that defined AdamW. Where to read it: https://arxiv.org/abs/1711.05101 (v3 (4 Jan 2019)).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
