Xin Wang, Hao Yu +10Affiliation: Alibaba Token Hub, Alibaba Group, Affiliation: Ohio State University
Proposes TRACE, an FP4 quantization framework for RL of MoE LLMs that uses rollout-side FP4 outcomes to guide training-side rounding and caches mantissa/scale from deeper layers to limit overhead. Preserves BF16-level RL performance while enabling up to 5.4× rollout speedup.