Category

Reframes "can machines think?" as a concrete test: the imitation game, now the Turing test, where a machine passes if its typed replies are indistinguishable from a human's. Rebuts nine objections and backs machines that learn like children.

Models the brain probabilistically and proposes the perceptron: weighted threshold units that learn to classify patterns by adjusting connection strengths from examples, rather than storing fixed memories. The 1958 root of trainable neural networks.

Introduces the generalized delta rule — backpropagation — for training multi-layer networks with hidden units by gradient descent on output error, letting hidden layers learn internal representations that solve problems single-layer networks cannot.

Treats a network's weights as a noisy channel and penalizes the bits needed to describe them, formalizing the "bits-back" coding trick — an early variational argument later recognized as a conceptual ancestor of the VAE.

Reframes model selection as data compression: the best hypothesis is the one that lets you describe the data in the fewest bits. Walks through MDL twice — once conceptually, once with full math — turning Occam's razor into a usable inference principle.

The result that kicked off the deep learning era: in 2012 a deep CNN cut ImageNet top-5 error from 26% to 15%, showing that GPU-trained networks with ReLU and dropout could beat decades of hand-engineered computer vision features.

Measures why complexity in closed systems rises then falls while entropy only climbs, using a coffee-and-cream cellular automaton. The key result: only interacting particles produce a transient complexity peak; non-interacting ones never do.

Frames generative modeling as a two-player game: a generator forges data while a discriminator learns to spot fakes, training both by backpropagation alone — no Markov chains, no inference networks. The adversarial pressure yields sharp samples.

A one-line fix that finally made dropout work with LSTMs: apply it only to the non-recurrent connections, leaving the memory path untouched. This let researchers train much larger RNNs without the overfitting that had capped their size.

Bolts a differentiable, addressable memory bank onto a neural network and trains the whole thing with gradient descent, letting it learn algorithms like copying, sorting, and recall from examples — a learned computer rather than a fixed circuit.

Repurposes attention as a pointer that selects positions in the input rather than blending them into a context vector, so the output vocabulary can grow with input length — handling sorting, convex hulls, and TSP that fixed seq2seq cannot.

Demonstrates that the order you feed inputs and outputs into a seq2seq model changes what it learns — even for sets that have no inherent order — and adds an attention-based set encoder plus a training loss that searches over output orderings.