AIAny
AI Infra2026
Icon for item

深入理解 AI Infra:量化分析与系统设计

Quantitatively analyzes LLM inference and training systems from workload and hardware constraints. Covers accelerators, operators, networks, distributed systems, and edge-to-cloud deployment, with companion calculation tools and experiments.

Introduction

A system can have abundant peak compute and still serve a model slowly: the bottleneck may be moving weights, retaining KV state, or waiting for inter-device communication. The useful starting point is therefore not a list of optimizations, but a resource ledger: what data moves, how much, how often, along which path, and what must wait.

What Sets It Apart
  • It derives system choices from compute, memory capacity, bandwidth, communication, and dependencies, making hidden constraints visible before implementation.
  • Its twelve chapters connect model architecture and workload behavior to accelerators, runtimes, high-speed interconnects, inference, and training. This helps readers trace a service-level symptom back to the hardware or scheduling decision behind it.
  • Reproducible calculations and chapter-organized experiments let readers check estimates against measurements and adapt examples to different models and hardware. The static calculation tools do not require a GPU or model weights.
Who It’s For—and the Trade-Off

Great fit for engineers, researchers, and students who already know basic Python, linear algebra, and computer systems and want to reason about model-serving cost or performance from first principles. Look elsewhere for a quick deployment recipe or a polished, settled textbook: the manuscript is still an evolving draft, and its quantitative approach rewards readers willing to follow the assumptions and calculations.

More Items

GitHub
AI Infra2022

Provides portable tensor operations and low-level compute kernels for machine learning. Its backend support and low-bit quantization help developers build lightweight inference software across varied hardware.

GitHub
AI Agent2023

Orchestrates stateful, long-running agent workflows as explicit graphs, with support for resuming execution, human review, and persistent memory. Best suited to applications that need control over workflow behavior; simpler linear tasks may call for a lighter approach.

GitHub
AI Infra2026

Runs and serves language models locally on Apple Silicon and GPUs using native Metal, CUDA, and other backends. Its Zig engine performs inference without requiring Python or MLX.