Reframes Policy Mirror Descent for autoregressive LLM generation into a critic-free, trajectory-level objective using the Bellman equations to avoid per-token value estimation. Derives a practical token-level loss with a smoothed mismatch-correction weight and shows improved math-reasoning accuracy on benchmarks.
Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.
Analyzes a failure mode in PPO critics called “Value Flattening” and proposes SP³O (Sparse PPO), which supervises a small set of well-separated states per response to mitigate flattened value estimates and improve policy learning for LLM alignment and reasoning tasks.
Turns implemented functionality in open-source codebases into executable RL training environments by automatically generating behavioral specifications, execution-grounded tests, and post-rollout filtering; yields 5,545 verifiable coding tasks across 23 languages to train coding agents.
Introduces a rubric-based approach for video reward modeling that generates explicit, query-adaptive evaluation criteria before scoring to reduce scalar drift; proposes RGPO, a two-stage training (seed warm-up + joint optimization) that achieves strong pointwise and pairwise evaluation with high data efficiency.
Converts natural-language SVG instructions into instance-aware six-item rubrics and uses a vision–language judge to score rendered SVGs item-by-item as dense RL rewards. Produces stronger human alignment and lifts benchmark rubric scores substantially on MMSVG datasets.
Provides a unified data and evaluation suite for measuring gameplay capabilities across short, medium, and long temporal horizons. Includes an automated annotator, a 5,000-hour AAA gameplay dataset with temporally aligned frames, actions and multi-horizon instructions, and a reproducible offline + stepwise online benchmark.
Records structural priors with skill-specific policies so a runtime agent can select and compose the version of each skill best suited to new states, improving out-of-distribution and compositional generalization for robot manipulation from few demonstrations.
Normalizes each domain's teacher feedback spread during multi-teacher on-policy distillation so that no domain (e.g., instruction following) overwhelms others, improving student recovery of specialist skills and raising average scores across benchmarks.
Trains a single unified multimodal model with reinforcement learning to perform end-to-end self-reflection and iterative image repair — jointly learning the diagnostic (textual) reflection and the flow-based image revisions so credit propagates across rounds without an external verifier.
Extrapolates RL-induced representation residuals into student hidden states during on-policy distillation: at each layer and token, it regresses the student beyond the teacher along the teacher’s RL-induced direction to improve stability and empirical performance.
Calibrates multi-reward reinforcement learning by adaptively upweighting infrequently active rewards per rollout batch, so sparse objectives provide stronger signals when they matter. Proposes an inverse-square-root density correction and shows faster learning on tool-calling and math-reasoning tasks.