Category
Explore by categories
Before residual connections, stacking more layers made networks worse, not better — this 2015 paper fixed that by having layers learn a residual F(x)=H(x)-x via shortcut connections, enabling 152-layer nets that won ILSVRC 2015.
Reframes the VAE's tendency to ignore its latent code as a controllable design choice: by limiting a PixelCNN decoder's receptive field and using autoregressive flow priors, the code is forced to keep only global structure and discard local texture.
Recasts a scatter of competing graph-network designs as one message-passing recipe — propagate, aggregate, read out — then proves it on QM9, hitting chemical accuracy on most molecular property targets without hand-built descriptors.
Isolates relational reasoning into a tiny plug-in module that scores pairwise object relations, bolting onto CNN/LSTM encoders to hit super-human 95.5% on CLEVR — and proving plain convnets lack this capacity on their own.
The 2017 paper that replaced recurrence with pure self-attention, making sequence models fully parallelizable — and, almost as a side effect, laying the architectural foundation for nearly every large language model that followed, from BERT to GPT.
Embeds multi-head self-attention inside an LSTM-style memory, so stored memories can attend to one another instead of just sitting in separate slots — sharpening relational reasoning and topping WikiText-103, Project Gutenberg, and GigaWord.
A 2019 essay arguing that over 70 years of AI, general methods that scale with computation — search and learning — consistently beat hand-coded human knowledge. The short text that crystallized the scaling-vs-priors debate.
Argues a single web-scale generative video model handles vision tasks zero-shot the way LLMs handle language. Probes Veo 3 on segmentation, edge detection, image editing, physical and affordance reasoning, and puzzles like maze solving and symmetry.
Omnimodal world model that jointly processes and generates text, images, video, audio, and action trajectories for physical AI. Uses a mixture-of-transformers to combine autoregressive reasoning and diffusion-based multimodal generation; released open-source with checkpoints, datasets and benchmarks for robotics and simulation.
Proposes a router redesign for Mixture-of-Experts (MoE) that aligns each router row with its expert's principal singular direction using Manifold Power Iteration (MPI), improving token–expert affinity. MPI applies a 'power‑then‑retract' step to push router rows toward principal singular vectors while enforcing norm constraints; the paper gives convergence theory and pretraining results on 1B–11B MoE models.
Performs native structural reasoning for proteins, small molecules and inorganic crystals by tokenizing coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary. Treats structural tokens as addressable evidence to produce interpretable prediction traces and improves accuracy across biology, chemistry and materials benchmarks.