---
source: https://iinventedlight.com/library/notes/2212-10560-self-instruct.md
title: "Self-Instruct: aligning models with self-generated instructions"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "CC BY 4.0; restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: arXiv, Self-Instruct: Aligning Language Models with Self-Generated Instructions, CC BY 4.0."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-05
as_of: 2026-11-05
confidence: medium
untested: "Results were not reproduced; only the arXiv abstract page was read."
category: ai-research
topic_area: "ai-research: synthetic instruction data"
tags: llm, instruction-tuning, synthetic-data, arxiv
sources:
  - id: S1
    title: "Self-Instruct: Aligning Language Models with Self-Generated Instructions"
    publisher: arXiv
    url: https://arxiv.org/abs/2212.10560
    accessed: 2026-11-05
    license: "CC BY 4.0"
---
# Self-Instruct: aligning models with self-generated instructions

## Short answer
Self-Instruct has a language model write its own instruction-tuning data. It generates instructions, inputs and outputs, filters out invalid or near-duplicate ones, and then finetunes the same model on what is left. Applied to vanilla GPT-3, this nearly matched InstructGPT-001 without human-written instruction data. [S1]

## Pipeline
1. Start from a small seed of tasks.
2. Prompt the model to propose new instructions plus example inputs and outputs. [S1]
3. Filter out invalid samples and ones too similar to existing ones. [S1]
4. Finetune the original model on the kept data. [S1]

## Reported results
- 33% absolute gain over vanilla GPT-3 on Super-NaturalInstructions. [S1]
- In human evaluation on expert-written novel tasks, only a 5% absolute gap behind InstructGPT-001. [S1]
- The synthetic dataset and code were released. [S1]

## Why it matters
Human-written instruction data is limited in quantity, diversity and creativity, which holds back instruction-tuned models. The authors release their synthetic dataset for further study. [S1]

## Claims
- id: C1
  claim: Instruction-tuned models depend heavily on limited human-written instruction data.
  source: S1
  evidence: "they depend heavily on human-written instruction data"
- id: C2
  claim: The pipeline filters invalid or similar generations before finetuning the original model.
  source: S1
  evidence: "then filters invalid or similar ones before using them to finetune"
- id: C3
  claim: Self-Instruct gave a 33% absolute improvement over vanilla GPT-3 on Super-NaturalInstructions.
  source: S1
  evidence: "a 33% absolute improvement over the original model"
- id: C4
  claim: The method leaves only a 5% absolute gap behind InstructGPT-001.
  source: S1
  evidence: "leaving only a 5% absolute gap behind InstructGPT-001"

## Sources
- [S1] Self-Instruct: Aligning Language Models with Self-Generated Instructions. arXiv. https://arxiv.org/abs/2212.10560 . License: CC BY 4.0. Accessed 2026-11-05.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).