Combines scientific data understanding with search, coding, and tool use for multi-step research. It checks intermediate results and visual feedback to revise its actions, and accepts more than ten scientific data types.
Studies scaling language models with parallel Transformer branches and introduces Joint-KV to share key-value caches across branches. Experiments report improved modeling quality at comparable training compute and faster long-context decoding.
Handles text and image inputs and generates text, with a context window of up to 1 million tokens. Its hybrid sparse and sliding-window attention supports long-context work, while the MoE design activates about 12B parameters per token.
Provides six open language models from 0.9B to 375B-A23B for reasoning and agentic tasks. The paper also introduces sparse attention and a decoding speedup that needs no separate draft model.
Details the Xing 4 model architecture, Ascend-scale pretraining systems, and agentic post-training, including a 285B-parameter training run on 8,192 Ascend 910C cards.