Most on-policy distillation methods assume teacher token signals are reliable measures of what the student should learn. The paper shows this assumption breaks down when token-level teacher judgments are driven by input-agnostic language priors, formatting conventions, or stereotyped reasoning templates — i.e., spurious signals that produce large gradients but little task-guided learning. The core insight is that input-groundedness (does a token depend on the input?) is a crucial axis for selecting distillation supervision, and that combining an input-groundedness proxy with divergence-based filtering removes high-impact misleading updates without discarding useful teacher signals.
Key Findings
-
Introduces input-groundedness as an explicit supervision-selection criterion, so what? it exposes a failure mode overlooked by confidence/divergence-only filters and enables finer-grained token-level selection.
-
Proposes SA-OPD: a lightweight proxy estimating token input dependence plus a filter that removes tokens with both low input-groundedness and extreme distillation divergence, so what? this targets updates that are large but unlikely to be task-relevant.
-
Demonstrates consistent gains on both large language model and vision-language model benchmarks compared to Vanilla OPD and competitive selective methods, so what? the method generalizes across modalities and teacher/student settings.
-
Keeps computational overhead modest by using an inexpensive proxy and selective token filtering, so what? it is practical to add to existing OPD pipelines without large additional cost.
Who It's For and Tradeoffs
Great fit if you run token-level on-policy distillation for LLMs or VLMs and see limited transfer from teacher supervision or poor downstream improvement despite large distillation gradients. SA-OPD helps when teacher signals are noisy or dominated by input-agnostic priors. Look elsewhere if you have fully supervised task-specific labels, if teacher supervision is already perfectly input-grounded, or if any additional token-level proxy computation is absolutely prohibited in your pipeline. A tradeoff: the approach relies on the proxy correctly estimating input-groundedness — misestimation can remove useful signals — and it adds a modest engineering step to integrate the filter into your training loop.
Where It Fits
Positioned between vanilla OPD and heavy teacher-student selection schemes: it complements confidence/divergence-based selection by adding an orthogonal input-dependence check, making it a practical option when you need better supervision hygiene without redesigning distillation objectives.