← Reference notes

Self-Adapting Language Models

Summary

SEAL trains an LLM (via RL, ReST-EM) to write its own "self-edits" — synthetic finetuning data and optionally hyperparameters/augmentation choices — which are then applied by SFT/LoRA as persistent weight updates. Reward = performance of the updated model downstream (Abstract, §3).

When to apply

• You own an open-weight model and can finetune it, and need to absorb new documents into weights (knowledge incorporation) or adapt per-task from few examples (test-time training).
• Each context must come with a downstream evaluation (QA pairs, held-out example) to compute reward (§Limitations, "Context-dependent evaluation").
• Not applicable to closed API models.

Procedure

1. For a new passage, prompt model to generate "implications"/restatements of the passage (self-edit).
2. LoRA-finetune a copy of the model on that self-edit; evaluate on the passage's questions without the passage in context.
3. Outer loop (if training the policy): sample several self-edits per context, keep the ones whose updated model improved, SFT the generator on them (ReST-EM); repeat ~2 iterations (§4.2, Fig. 4).
4. For few-shot tasks: let the model pick augmentations and hyperparameters (learning rate, epochs, loss on which tokens) as tools, then run TTT (§4.1).
5. Re-test old tasks after each edit to watch for forgetting.

Costs and limits

• Each reward evaluation needs a finetune+eval, ~30–45 s per self-edit (§Limitations, §B.5).
• Catastrophic forgetting: performance on earlier passages declines over sequential edits (§Limitations, Fig. 6).
• Requires paired downstream tasks for reward; does not yet scale to unlabeled corpora.
• Small models, small/curated task sets (ARC subset hand-filtered).

Evidence (detail)

• Knowledge incorporation (Qwen2.5-7B, SQuAD no-passage): base 32.7%, train on raw passage 33.5%, Qwen synthetic 39.7%, GPT-4.1 synthetic 46.3%, SEAL 47.0% after 2 RL iterations (§4.2, Fig. 4).
• Continued pretraining n=200 passages: SEAL 58.2%; here GPT-4.1 synthetic data slightly outperforms SEAL (§4.2, Table 2).
• Few-shot ARC subset (Llama-3.2-1B, 11 train + 8 eval tasks, filtered to be solvable): success 72.5% vs 20% (self-edit without RL) and 0% (ICL); below Oracle TTT (§4.1, Table 1).

Facts

• id: F1 claim: Knowledge incorporation (Qwen2.5-7B, SQuAD no-passage): base 32.7%, train on raw passage 33.5%, Qwen synthetic 39.7%, GPT-4.1 synthetic 46.3%, SEAL 47.0% after 2 RL iterations (§4.2, Fig. 4). confidence: medium # single source: author-reported date: 2025-06-12 sources: [S1] license: CC BY 4.0
• id: F2 claim: Continued pretraining n=200 passages: SEAL 58.2%; here GPT-4.1 synthetic data slightly outperforms SEAL (§4.2, Table 2). confidence: medium # single source: author-reported date: 2025-06-12 sources: [S1] license: CC BY 4.0
• id: F3 claim: Few-shot ARC subset (Llama-3.2-1B, 11 train + 8 eval tasks, filtered to be solvable): success 72.5% vs 20% (self-edit without RL) and 0% (ICL); below Oracle TTT (§4.1, Table 1). confidence: medium # single source: author-reported date: 2025-06-12 sources: [S1] license: CC BY 4.0

Related

• 2610.10310 RSIGym (agents improving target-model weights + harness).
• Test-time training on ARC (Akyürek et al., cited as [36]).
• 2606.01444 self-revising discovery systems.

Review notes

• Built from private assessment notes; social-media hype check and source-post links omitted. Paraphrase only; no verbatim quotes beyond short attributed terms.
• Figures not re-checked against the PDF in this pass beyond the original private notes.

For language models and agents: start at /llms.txt. This page in Markdown: /library/reference/arxiv/2506-10943-seal-self-adapting-language-models.md.