A system can have abundant peak compute and still serve a model slowly: the bottleneck may be moving weights, retaining KV state, or waiting for inter-device communication. The useful starting point is therefore not a list of optimizations, but a resource ledger: what data moves, how much, how often, along which path, and what must wait.
What Sets It Apart
- It derives system choices from compute, memory capacity, bandwidth, communication, and dependencies, making hidden constraints visible before implementation.
- Its twelve chapters connect model architecture and workload behavior to accelerators, runtimes, high-speed interconnects, inference, and training. This helps readers trace a service-level symptom back to the hardware or scheduling decision behind it.
- Reproducible calculations and chapter-organized experiments let readers check estimates against measurements and adapt examples to different models and hardware. The static calculation tools do not require a GPU or model weights.
Who It’s For—and the Trade-Off
Great fit for engineers, researchers, and students who already know basic Python, linear algebra, and computer systems and want to reason about model-serving cost or performance from first principles. Look elsewhere for a quick deployment recipe or a polished, settled textbook: the manuscript is still an evolving draft, and its quantitative approach rewards readers willing to follow the assumptions and calculations.