AIAny

Neural Message Passing for Quantum Chemistry

Recasts a scatter of competing graph-network designs as one message-passing recipe — propagate, aggregate, read out — then proves it on QM9, hitting chemical accuracy on most molecular property targets without hand-built descriptors.

Introduction

The lasting contribution here is not a single model but a vocabulary. Before this paper, "graph neural network" named a loose cluster of independently invented architectures that nobody could easily compare. By showing they were all special cases of one pattern — pass messages along edges, aggregate them at each node, read out a graph-level answer — it gave the field a shared API, and "message passing" became the default way people reason about learning on graphs.

Key Findings
  • Many models, one abstraction. Half a dozen prior graph architectures collapse into three pluggable functions (message, update, readout); choosing variants becomes an engineering search rather than reinventing the wheel.
  • Learned features beat hand-crafted descriptors. On QM9 molecular property prediction the framework reaches chemical accuracy on most of the targets, where chemists had previously relied on engineered molecular fingerprints.
  • The graph itself carries the chemistry. Atoms as nodes, bonds as edges, and a readout invariant to atom ordering let the network respect molecular symmetry without baking in physics by hand.
  • A bottleneck made explicit. The authors argue the benchmark is close to saturated, pointing future work toward larger molecules and more accurate labels rather than fancier models — a rare honest call on diminishing returns.
Why It Still Matters / When to Skip

Great fit if you want the conceptual origin of modern GNNs, or the cleanest mental model for why so many graph-learning papers describe themselves as "message passing." Look elsewhere if you need the current state of the art: attention-based and transformer-style graph models, plus large-scale molecular pretraining, all postdate this — it is the framework that organized the field, not the strongest model running today.

Information

  • Websitear5iv.labs.arxiv.org
  • OrganizationsGoogle Brain, Google, Google DeepMind
  • AuthorsJustin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, George E. Dahl
  • Published date2017/04/04

More Items

Enables multi-head, depth-wise residual retrieval in Transformers by expanding a single residual stream into multiple context-aware streams using causal depthwise convolutions. Improves pretraining convergence and downstream benchmark scores with minimal overhead, yielding notable iso-loss compute speedups in large MoE models.

A concise textbook-style book that explains foundational concepts and techniques for large language models, covering pre-training, generative models, prompting, alignment, inference, and reasoning. Structured as self-contained chapters for readers with some ML/NLP background or those seeking a principled introduction to LLM foundations.

Analyzes why self-evolving reasoning models collapse under repeated self-training and proposes R-Quest: a feedback-driven pipeline that trains solvers to reject invalid questions and uses a frozen base model to detect task-level repetition, filtering training data to sustain multi-round gains.