A self-improvement loop can generate more training tasks and still fail if it keeps practicing on tasks that no longer teach it anything. A-RSI focuses on that overlooked timing problem: training dynamics decide when to refresh the task pool, while independently checkable answers keep the new supervision from relying on the model’s own guesses.
Key Findings
- The refresh trigger uses reward variation and accuracy from training logs, rather than validation or test performance. This makes task acquisition responsive to learning progress without using evaluation scores to steer training.
- A static library of model-authored programs generates multimodal questions and derives their reference answers. The authors specify and automatically verify properties of the task-generation and refresh rules, then audit generated answers separately.
- The training loop uses no external training examples or human-written answer labels. With 800 fresh tasks, the average across 11 benchmarks rises from 57.77 to 61.05.
Who It’s For—and the Tradeoff
A good fit for researchers studying self-generated curricula, reinforcement-learning training loops, and multimodal self-improvement where held-out scores should not drive task selection. The reported gains are tied to a particular task-generation setup and benchmark suite; reproducing them requires reliable program-generated tasks and answer audits, so the results do not establish that the same refresh rule will help every model or domain.
How the Loop Works
The key separation is between deciding when to seek new experience and deciding what the correct answer is. Training logs govern the timing; verified program execution supplies answers that can be checked independently. This avoids treating model agreement as ground truth, while making the task-refresh mechanism itself an explicit object of verification.