Recasts a scatter of competing graph-network designs as one message-passing recipe — propagate, aggregate, read out — then proves it on QM9, hitting chemical accuracy on most molecular property targets without hand-built descriptors.
Isolates relational reasoning into a tiny plug-in module that scores pairwise object relations, bolting onto CNN/LSTM encoders to hit super-human 95.5% on CLEVR — and proving plain convnets lack this capacity on their own.
The 2017 paper that replaced recurrence with pure self-attention, making sequence models fully parallelizable — and, almost as a side effect, laying the architectural foundation for nearly every large language model that followed, from BERT to GPT.
Embeds multi-head self-attention inside an LSTM-style memory, so stored memories can attend to one another instead of just sitting in separate slots — sharpening relational reasoning and topping WikiText-103, Project Gutenberg, and GigaWord.
Chops any layer-sequence model across accelerators and splits each mini-batch into micro-batches to keep the pipeline busy, hitting near-linear speedup without architecture-specific tricks or fast interconnects.
A 2019 essay arguing that over 70 years of AI, general methods that scale with computation — search and learning — consistently beat hand-coded human knowledge. The short text that crystallized the scaling-vs-priors debate.
Career advice for ML researchers from the creator of TRPO and PPO: how to pick problems worth solving, why goal-driven beats idea-driven research, and the daily notebook-and-review habit that compounds small experiments into breakthroughs.
Graduate-level textbook unifying classical statistics and modern deep learning under one probabilistic framework. Builds from probability, information theory, and optimization up to neural nets, with runnable Python/JAX figure code and exercise solutions.
Builds a single rigorous theory from one question: why some bit strings look random. Defines plain and prefix complexity, the incompressibility method, and Martin-Löf randomness, tying information content to whether a short program can reproduce a string.
Reworks the classic Bishop PRML for the deep learning era, adding dedicated chapters on transformers and diffusion models. Builds each idea from probability up using text, diagrams, math, and pseudocode, aimed at readers new to the field.
Teaches the math behind modern deep learning across 21 chapters, from shallow nets to transformers and diffusion models. Each idea is explained in words, then in equations, then visually. Full PDF, slides, and Python notebooks are free.
An open, intuition-first textbook that teaches the maths, computing, and practical foundations needed for AI engineering. Organized into focused chapters (vectors, matrices, calculus, ML, NLP, CV, GPU/Inference, ML systems) with code-first explanations and interview-ready emphasis.