AIAny
Icon for item

AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs

Evaluates how well models and coding agents optimize operators for real Ascend NPU workloads, accounting for both speedups and token cost. Test cases preserve tensor shapes and data types from real model execution contexts.

Introduction

Kernel benchmarks can reward fast code without showing whether an agent can improve an operator that matters in a deployed workload—or how much it spent finding that improvement. The key shift here is to connect each optimization task to real model execution contexts and score gains alongside token budget, making practical optimization efficiency part of the result.

Key Findings
  • The benchmark covers 48 kernels in three difficulty levels, from basic memory-access optimizations to fused and attention computations. This progression tests more than the ability to generate an isolated kernel.
  • Test cases use shapes and data types drawn from real training and inference tasks, with model, phase, hardware, and parallelism context. Measured kernel gains can therefore be interpreted in relation to actual operator invocations.
  • It supports both model evaluation with a fixed built-in harness and model–harness evaluation with coding agents or custom systems. This helps separate model capability from the effects of an agent’s orchestration and tools.
  • The Speedup–Token Score rewards final speedup while penalizing improvements obtained late in the token budget. It distinguishes systems that reach similar performance with different optimization costs.
Who It’s For—and the Trade-off

A good fit for researchers comparing LLMs and coding agents on Ascend kernel optimization, and for teams assessing whether local kernel gains justify their search cost. It is less useful if you need a hardware-agnostic benchmark or cannot access an Ascend execution environment. The report’s initial results cover Batch 1’s 17 tasks; the remaining batches are described as future evaluation work.

Information

  • Websitearxiv.org
  • OrganizationsAI Training Platform Team, SLAI-AITP
  • AuthorsBingwei Zhang, Jing Li, Amir Abdullin, Sihang Chen, Xin Chen, Liangjun Feng, Yingmeng Gao, Tao Guo, Junjie Huang, Dongfang Li …
  • Published date2026/11/13

More Items

Studies scaling language models with parallel Transformer branches and introduces Joint-KV to share key-value caches across branches. Experiments report improved modeling quality at comparable training compute and faster long-context decoding.

Details the Xing 4 model architecture, Ascend-scale pretraining systems, and agentic post-training, including a 285B-parameter training run on 8,192 Ascend 910C cards.

GitHub
AI Infra2026

Create and manage Google Colab runtimes from the command line, then run local scripts or notebooks on remote CPUs, GPUs, or TPUs. It also handles file transfers and one-off jobs; supported on Linux and macOS.