---
title: "RSIGym: A Flexible Environment for Recursive Self-Improvement"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: "paper is CC BY 4.0 (verified on arXiv abs page 2026-10-14); note is our own summary"
source: https://arxiv.org/abs/2610.10310
source_license: "CC BY 4.0"
attribution: "Summary by iinventedlight (AI-assisted). Source: Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh, \"RSIGym: A Flexible Environment for Recursive Self-Improvement\" (CC BY 4.0)."
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-10-14
date: 2026-10-14
status: reviewed
reviewed_on: 2026-10-14
category: arxiv
arxiv_id: 2610.10310
alias: 2610.10310-rsigym-recursive-self-improvement
checked_on: 2026-10-14
sources:
  - id: S1
    title: "RSIGym: A Flexible Environment for Recursive Self-Improvement"
    authors: "Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh"
    url: https://arxiv.org/abs/2610.10310
    version_read: v1 (2026-10-07)
    accessed: 2026-10-14
    license: CC BY 4.0
  - id: S2
    title: Official code repository
    url: https://github.com/evolvent-ai/RSIGym
    license: "not checked"
---
# RSIGym: A Flexible Environment for Recursive Self-Improvement

## Summary
An environment where a coding agent improves a smaller model by calling metered services (train, rollout, benchmark, sandbox) under a visible budget; scored by RSI-Index (share of remaining gap closed). Big reported gains, but evals were visible during development, single runs, and compounding (true recursion) was not tested.

## When to apply
- Running or evaluating an autoresearch/self-improvement loop where an agent changes training data, hyperparameters and harness code of a smaller model.
- Useful design pattern even without RSIGym: wrap expensive actions as metered services with permission checks and a budget the agent can query (§3.3–3.4).

## Procedure
1. Fix the starting system (base model + minimal harness), allowed changes, eval protocol and budget.
2. Issue one platform key per run; every service call checks permission + remaining budget and logs cost.
3. Agent loop: diagnose failures from dev evals → propose change (data/settings/harness) → train/revise → evaluate → keep or discard.
4. Agent submits one final system; score with official eval.
5. Audit: n-gram overlap of training data vs eval tasks (paper uses 13-gram), checkpoint provenance, LLM-judge review of trajectory (§4.1 Protocol).
6. Report RSI-Index = mean over benchmarks of (s − s0)/(1 − s0).

## Costs and limits
- Scores are NOT held-out: the agent saw the eval tasks during development (§4.1 Protocol). Treat gains as in-distribution optimisation, not generalisation.
- Every configuration is a single run — no variance estimates (§4.1).
- Research-agent inference and final eval are billed separately from the $500 (§4.1).
- Offline (no network) run did not beat the initial system; Qwen3.8 self-improvement run produced no overall gain (§6).
- Whether improvements compound across later cycles (true RSI) is explicitly not yet evaluated (§1).

## Evidence (detail)
- Joint track, base Qwen3.5-35B-A3B-Base, $500/benchmark run: RSI-Index Opus 5 0.4809, Opus 5.5 0.4528, DeepSeek 0.4343, Astra 0.4093, Sol 6.1 0.3321, Sol 6 0.2079; 29/30 agent–benchmark settings improved (§4.3, Fig. 2).
- Opus 5 system: SWE-bench Verified 17.67%→50.33%, AIME 31.67%→97.78% (Abstract).
- Harness-only with Qwen3.6-35B-A3B-Instruct fixed: DSH-harness revisions raise Terminal-Bench 2.0 from 30.34% to 40.82% (tasks solved 3/3 attempts 15→26) (§1, §5).
- Budget $500→$1,000 improved 3 of 4 pairs (§1); conclusion says larger budgets don't consistently help (§6).
- Eval subsets: SWE 100 tasks, TB2 89, AIME 60, GPQA 100, SkillsBench 37; avg@3 (Table 4).

## Facts
- id: F1
  claim: RSIGym exposes Train, Model, Rollout, Benchmark and Sandbox services to a research agent in a CPU-only container under a shared budget and per-run permissions; tracks are Data-only, Harness-only and Joint.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 Abstract; §3]
  license: CC BY 4.0
- id: F2
  claim: RSI-Index is the mean over benchmarks of (s − s0)/(1 − s0).
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §4.2 Eq. 4]
  license: CC BY 4.0
- id: F3
  claim: Joint track at $500 per benchmark run: best reported RSI-Index 0.4809 (Opus 5); 29 of 30 agent–benchmark settings improved.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §4.3, Fig. 2]
  license: CC BY 4.0
- id: F4
  claim: Reported Opus 5 system gains: SWE-bench Verified 17.67%→50.33%, AIME 31.67%→97.78%.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 Abstract]
  license: CC BY 4.0
- id: F5
  claim: Scores are not held out: the agent saw eval tasks during development.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §4.1]
  license: CC BY 4.0
- id: F6
  claim: Each configuration is a single run, with no variance estimates.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §4.1]
  license: CC BY 4.0
- id: F7
  claim: Whether improvements compound across cycles is not yet evaluated.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §1]
  license: CC BY 4.0
- id: F8
  claim: Larger budgets did not consistently help.
  confidence: medium   # single source: author-reported results
  date: 2026-10-14
  sources: [S1 §6]
  license: CC BY 4.0

## Related
- 2506.10943 SEAL (self-generated finetuning data).
- 2610.04074 IdeaScientist, 2606.01444 self-revising discovery systems (auto-research).
- PostTrainBench (Rank et al., 2026), cited as closest prior benchmark.

## Review notes
- Built from private assessment notes; social-media hype check and source-post links omitted. All content is paraphrase; no verbatim quotes.
- Figures not re-checked against the PDF in this pass beyond the original private notes.