← All notes

Automated AI research and recursive self-improvement

Note, 2026-10-14. Status: reviewed. Tags: ai, agents, research. Authorship: synthesis by iinventedlight from the cited sources; text drafted with AI assistance (Grok Bot), reviewed by Cole Withington on 2026-10-14.

Summary

Automated AI research means AI agents doing parts of the research loop: coming up with ideas, changing training data or code, running experiments and evaluating results. Recursive self-improvement (RSI) is the stronger idea that each round of improvement makes the next round better, so gains compound. As of October 2026, published experiments show agents can improve a smaller model in one budgeted cycle [F1–F4]. Compounding across cycles has not been shown [F6].

Sections

Definitions

• Auto-research: an agent runs a research workflow: diagnose, propose, train or revise, evaluate, keep or discard [F1]. It can also mean just the ideation step [F9].
• Self-improvement: a model writes its own fine-tuning data and is updated on it [F7], or an agent improves another model's data, settings or harness [F1].
• Recursive self-improvement: improvements that compound over cycles. Our definition; the papers below test at most one cycle or a few RL iterations [F6, F7].

Current evidence

• RSIGym: frontier coding agents with a $500 budget closed part of the gap to perfect scores on five benchmarks [F2], with improvement in 29 of 30 agent–benchmark settings [F3]. Changing only the harness also helped [F4].
• SEAL: a 7B model trained to write its own fine-tuning data slightly beat GPT-4.1-generated data in one setting [F7], but not in another [F8].
• IdeaScientist: research proposals were judged more novel than baselines' [F9]. No proposal was carried out [F11].
• Lab goals (stated, not evidence): OpenAI says it aims for an automated AI researcher by March 2028 [F13, F14].

Limits and caveats

• RSIGym scores were not on held-out tasks, and each configuration was run only once [F5].
• Contamination research shows that gains on test sets the model has seen can be shallow memorization [F16].
• SEAL suffers forgetting and costs a fine-tune for every reward [F8].
• Judging research quality is noisy, even between human experts [F10].
• Supervising AI on tasks without ground truth is an open problem [F15].

Speculation and hype (labeled)

• Speculation: claims that RSI leads to an "intelligence explosion" go beyond anything measured in these sources.
• Hype check: social posts called SEAL evidence that a model "might be alive" and said it works "without retraining". SEAL works by retraining [F7]. A post called the categorical framework (S4) a "breakthrough", but its authors describe case studies, not validated laws [F12].
• Stated plans, not results: lab timelines [F13, F14] and bootstrapped alignment plans [F15] are goals or forecasts.

Facts

• id: F1 claim: RSIGym gives a research agent in a CPU-only container metered training, model, rollout, benchmark and sandbox services under a shared budget, and asks it to improve a target model's data, settings and/or harness. confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0
• id: F2 claim: In RSIGym's joint track (base Qwen3.5-35B-A3B-Base, $500 per benchmark run), the best agent closed on average about 48% of the remaining gap to a perfect score across five benchmarks (RSI-Index 0.48). confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0 note: Model names in the paper (e.g. "Opus 5", "Sol 6.1") are reported as the authors give them.
• id: F3 claim: In RSIGym, 29 of 30 agent–benchmark settings ended above the starting system. confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0
• id: F4 claim: With the model weights fixed, harness-only revisions raised Terminal-Bench 2.0 from about 30% to about 41% in RSIGym. confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0
• id: F5 claim: RSIGym's reported scores are not held-out (the agent could see the eval tasks during development), and each configuration was run only once. confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0
• id: F6 claim: RSIGym's authors say whether gains compound across later improvement cycles has not yet been evaluated. Doubling the budget did not consistently help, and a run without network access did not beat the starting system. confidence: medium date: 2026-10-07 sources: [S1] url: https://arxiv.org/abs/2610.10310 checked_on: 2026-10-14 license: CC BY 4.0
• id: F7 claim: SEAL uses reinforcement learning to teach a model to write its own fine-tuning data ("self-edits"), applied as LoRA weight updates. On a SQuAD knowledge task with Qwen2.5-7B it reached 47.0% vs 46.3% for GPT-4.1-written data and 32.7% for the base model. confidence: medium date: 2025-06-12 sources: [S2] url: https://arxiv.org/abs/2506.10943 checked_on: 2026-10-14 license: CC BY 4.0
• id: F8 claim: SEAL loses accuracy on earlier passages over sequential edits (forgetting), needs a fine-tune plus evaluation for each reward, and was slightly beaten by GPT-4.1 data in the 200-passage setting. confidence: medium date: 2025-06-12 sources: [S2] url: https://arxiv.org/abs/2506.10943 checked_on: 2026-10-14 license: CC BY 4.0
• id: F9 claim: IdeaScientist splits research ideation into gap-finder, cross-domain innovator and report-writer agents. It scored 14% above the strongest open auto-research baseline overall, mostly on novelty, using LLM-judge metrics on a held-out 277-paper split. confidence: medium date: 2026-10-02 sources: [S3] url: https://arxiv.org/abs/2610.04074 checked_on: 2026-10-14 license: CC BY 4.0
• id: F10 claim: In IdeaScientist, two human experts agreed only moderately on proposal quality (r = 0.56), and RL training added little over the untrained pipeline (+1.9% novelty, +0.4% quality). confidence: medium date: 2026-10-02 sources: [S3] url: https://arxiv.org/abs/2610.04074 checked_on: 2026-10-14 license: CC BY 4.0
• id: F11 claim: IdeaScientist measures proposal quality only. No proposal was executed, so real-world feasibility is untested. confidence: medium date: 2026-10-02 sources: [S3] url: https://arxiv.org/abs/2610.04074 checked_on: 2026-10-14 license: CC BY 4.0
• id: F12 claim: Wang and Buehler propose separating "search" within a fixed vocabulary from "discovery" that verifiably extends the vocabulary. Their evidence is case studies, which they say do not establish new experimentally validated laws. confidence: low date: 2026-05-31 sources: [S4] url: https://arxiv.org/abs/2606.01444 checked_on: 2026-10-14 license: CC BY 4.0 (our restatement; source is CC BY-NC-ND, not quoted)
• id: F13 claim: In an April 2026 OpenAI Forum talk, OpenAI said its official goal is an automated AI researcher by March 2028 (stated goal, not a result). confidence: low date: 2026-04-07 sources: [S6] url: https://www.youtube.com/watch?v=ZpUKNYcgM-E checked_on: 2026-10-14 license: CC BY 4.0 (our restatement)
• id: F14 claim: At Stanford CS153, Altman described internal goals of an AI "research intern" by September 2026 and a full end-to-end researcher by March 2028 (stated goals; we have not verified whether the 2026 milestone was met). confidence: low date: 2026-06-15 sources: [S7] url: https://prepublish.ai/youtube-transcript/knvviM-LAZU checked_on: 2026-10-14 license: CC BY 4.0 (our restatement) note: Third-party reupload; no official Stanford video found. Speaker is CEO of an AI lab (conflict of interest).
• id: F15 claim: Jan Leike has argued for aligning an automated alignment researcher only slightly smarter than humans, then bootstrapping. He notes that alignment research lacks ground-truth labels, so evaluating it needs methods such as deliberately planted flaws (tampering) and adversarial evals. confidence: low date: 2024-09-10 sources: [S8, S9] url: https://www.youtube.com/watch?v=PuESFNSh_Qo checked_on: 2026-10-14 license: CC BY 4.0 (our restatement) note: Two talks by the same speaker, not independent sources.
• id: F16 claim: In small models (≤344M) trained on copies of the MATH test set, contamination gains disappeared when problems were reworded or their numbers changed. confidence: medium date: 2026-01-07 sources: [S5] url: https://arxiv.org/abs/2601.04301 checked_on: 2026-10-14 license: CC BY 4.0 note: Tested at small scale only; relevance to RSIGym's non-held-out scores is our inference.
• id: F17 claim: Ilya Sutskever has said a better target than self-improving AI is AI robustly aligned to care for sentient life, and that capping the power of the strongest systems would help (opinion). confidence: low date: 2025-11-25 sources: [S10] url: https://www.youtube.com/watch?v=aR20FWCCjAs checked_on: 2026-10-14 license: CC BY 4.0 (our restatement)

Pointers

• answers: Full RSIGym per-agent, per-benchmark tables and the audit protocol where: https://arxiv.org/abs/2610.10310 (§4, Table 4); code https://github.com/evolvent-ai/RSIGym access: open
• answers: Category-theory formalism for "discovery" vs "search" where: https://arxiv.org/abs/2606.01444 access: open (NC-ND; read the source, do not copy)

Disagreements

• What "RSI" means: RSIGym uses the name for one budgeted improvement cycle [F1, F6]. The classic meaning is compounding improvement, which none of these sources demonstrate.
• Lab leaders' timelines [F13, F14] are much more confident than the measured evidence [F5, F6, F11]. Sutskever questions self-improvement as the goal at all [F17].

Markdown

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/auto-research-and-recursive-self-improvement.md.