Haojun Zhang, Yi Zou +8Affiliation: Xini Ding, Hao Li, Shuchang Zhou, Xianming Liu, Shiyu Huang‡, Affiliation: XPeng Inc.
Progressively prunes and distills audio encoders for speech LLMs to cut inference cost while preserving decoder-facing embeddings, using behavioral probes, representation alignment, cross-scale distillation and LoRA finetuning; reports reduced macro-error on Chinese–English benchmarks.