---
title: "Large Language Model Unlearning"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: "paper is CC BY 4.0 (verified on arXiv abs page 2026-10-18; icon by-4.0.png, link text view license, http://creativecommons.org/licenses/by/4.0/); note is our own summary"
source: https://arxiv.org/abs/2310.10683
source_license: "CC BY 4.0"
attribution: "Summary by iinventedlight (AI-assisted). Source: Yuanshun Yao, Xiaojun Xu, Yang Liu, \"Large Language Model Unlearning\" (CC BY 4.0)."
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-10-18
date: 2026-10-18
status: published
category: arxiv
arxiv_id: 2310.10683
alias: 2310.10683-large-language-model-unlearning
checked_on: 2026-10-18
sources:
  - id: S1
    title: "Large Language Model Unlearning"
    authors: "Yuanshun Yao, Xiaojun Xu, Yang Liu"
    url: https://arxiv.org/abs/2310.10683
    version_read: v2 (16 Feb 2024)
    accessed: 2026-10-18
    license: CC BY 4.0
    attribution: "Yuanshun Yao, Xiaojun Xu, Yang Liu, Large Language Model Unlearning, arXiv 2023, CC BY 4.0, https://arxiv.org/abs/2310.10683"
---
# Large Language Model Unlearning

## Summary
Yao, Xu, and Liu study unlearning: changing a model so it drops undesirable behavior. The abstract names three uses: stopping harmful replies, erasing copyright-protected content when someone asks, and reducing hallucinations. They argue it needs only negative examples, which are cheaper to collect than the helpful examples used for reinforcement learning from human feedback, and that it is cheaper to run, especially when the training samples behind the bad behavior are known. The abstract says that if the priority is to stop undesirable outputs rather than to produce desirable ones, unlearning is attractive, and that one ablation beat that feedback method on alignment while using 2% of its computational time, with only negative samples. The paper was first posted in October 2023. The abs page's latest version is v2 (16 Feb 2024), and the page lists no journal. This note follows that abstract page only.

## When to apply
- Background for whether a model can be trained to stop producing something it learned, including a request to drop copyrighted text.
- Not a procedure for deleting one person's data, and not a claim that a model has forgotten a fact in a privacy-law sense. The abstract does not evaluate a personal-data request.

## Costs and limits
- The 2% compute comparison is the authors' ablation, reported on the abstract. This note does not re-check the PDF.
- "Unlearning" here means the training setup they study. It is not evidence that weights no longer contain a fact.

## Evidence (detail)
- Three settings named on the abstract: removing harmful responses, erasing copyright-protected content as requested, and reducing hallucinations (Abstract).
- They contrast negative-only examples with the positive examples used in reinforcement learning from human feedback, and they call the method cheaper, especially when the offending training samples are known (Abstract).
- Abstract claim: an ablation reached better alignment performance than that feedback method with 2% of its computational time, using only negative samples (Abstract).
- Submission history on the abs page: v1 Sat, 14 Oct 2023; v2 Fri, 16 Feb 2024. No journal reference on the page.

## Facts
- id: F1
  claim: The paper studies unlearning as a way to stop harmful replies, erase copyright-protected content on request, and reduce hallucinations, using negative examples rather than a full set of helpful demonstrations.
  confidence: medium
  date: 2026-10-18
  sources: [S1 Abstract]
  license: CC BY 4.0
- id: F2
  claim: The abstract reports an ablation in which unlearning beat reinforcement learning from human feedback on alignment while using 2% of its computational time, with only negative samples.
  confidence: medium
  date: 2026-10-18
  sources: [S1 Abstract]
  license: CC BY 4.0
- id: F3
  claim: The arXiv abs page shows CC BY 4.0, a first version in October 2023, and a second version on 16 Feb 2024, with no journal reference.
  confidence: high
  date: 2026-10-18
  sources: [S1 abs page]
  license: CC BY 4.0

## Related
- 2310.07298 Beyond Memorization, about inferring private attributes from text. This paper is about removing trained-in behavior, not about that inference result.
- 2503.13872 Metric Differential Privacy, about calibrating text privacy. Unlearning is a different lever.

## Review notes
- Staging draft. Paraphrase of the arXiv abstract page checked 2026-10-18. The PDF was not re-read.
- License on the abs page: icon by-4.0.png, link text "view license", http://creativecommons.org/licenses/by/4.0/.
- The credit line keeps the venue as arXiv 2023, the year of the first posting. version_read is v2 (16 Feb 2024).
- Fresh-eyes recheck 2026-10-18: the abstract says "2% of its computational time", not a looser "compute" figure. Licence, authors, and dates unchanged.