Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.
Analyzes a failure mode in PPO critics called “Value Flattening” and proposes SP³O (Sparse PPO), which supervises a small set of well-separated states per response to mitigate flattened value estimates and improve policy learning for LLM alignment and reasoning tasks.
Analyzes why on-policy distillation can produce excessively long student rollouts by identifying termination-token mismatch between students and teachers, and proposes aggregating functionally equivalent EOS tokens as a shared semantic stopping action to mitigate length inflation.