Internalizes context-induced performance gains into model parameters by generating prospective queries and distilling context-conditioned outputs into mergeable low-rank or structural feed-forward updates, balancing consolidation and retention across 4B–120B dense and MoE models.
Uses an intermediate teacher checkpoint to gate transfer of RL-induced policy shifts in on-policy distillation by checking directional agreement between early and late teacher changes; suppresses endpoint shifts that reverse and requires one extra frozen-teacher scoring pass.