AIAny
Icon for item

TeleChat to Xing4: Advancing Frontier Agentic Models on Domestic Chips and Frameworks

Details the Xing 4 model architecture, Ascend-scale pretraining systems, and agentic post-training, including a 285B-parameter training run on 8,192 Ascend 910C cards.

Introduction

Training frontier-scale models on domestic accelerators is not just a hardware substitution: it changes how model architecture, memory use, communication, and host scheduling must be designed together. This report’s central contribution is a systems account of that co-design, grounded in a 285B-parameter Xing 4 pretraining run on 8,192 Ascend 910C cards, rather than a direct benchmark comparison with other frontier models.

Key Findings
  • The report studies sparse activation, attention, model depth and width, and speculative decoding as coupled design choices. The open-sourced 29B model activates 4B parameters per token and natively supports 256K-token contexts.
  • Its pretraining stack uses MindSpore HyperParallel for topology-aware hybrid sharding, distributed optimizer execution, and long-sequence memory management; CPU affinity and core isolation help reduce host-side scheduling interference.
  • The larger 285B configuration is reported to activate about 17B parameters per token and support contexts up to one million tokens. Reported system measurements include up to 4.68× prefill throughput over an MLA-only baseline and 2.67×–3.77× decoding throughput from speculative decoding.
  • TeleGym connects agent execution, evaluation, trajectory construction, and policy optimization. Asynchronous execution and partial rollouts address tasks with different durations while retaining trajectory continuity across policy updates.
Who Should Read It

A good fit for researchers and infrastructure teams evaluating large-model training on Ascend, MoE and long-context design, or agent post-training systems. Look elsewhere for a broad head-to-head assessment of model quality: the report focuses on architecture and systems engineering, and its throughput figures are tied to specified system baselines and configurations.

Information

  • Websitearxiv.org
  • OrganizationsXingchen AGI Lab, China Telecom
  • AuthorsBaotian Hu, Bingkai Yang, Boyang Wang, Changzai Pan, Chao Lyu, Chao Wang, Chen Li, Chenshuo Pan, Chenyu Lin, Chenyuan Zhang …
  • Published date2026/11/17

More Items

Studies scaling language models with parallel Transformer branches and introduces Joint-KV to share key-value caches across branches. Experiments report improved modeling quality at comparable training compute and faster long-context decoding.

Examines whether Transformers are necessary when LLMs write model weights without gradient training. When allowed to design the architecture, the tested LLMs compile facts into sparse tables, reaching up to 54.4% accuracy on ARC.

Evaluates how well models and coding agents optimize operators for real Ascend NPU workloads, accounting for both speedups and token cost. Test cases preserve tensor shapes and data types from real model execution contexts.