Studies on-policy distillation (OPD) at the data-minimal limit by training on a single query, measuring state coverage and alignment dynamics, and showing OPD is often data-overfed but algorithm-starved.
Compact causal LLM for on-device assistants, coding agents and long-context tool use — ~2.52B parameters with a 131,072-token context, trained with SFT + RL + OPD and released with its UltraData training corpora and multi-format deployment checkpoints.