Argues that "interesting" complexity is low in both ordered and fully random states but peaks in between, and proposes "complextropy" — a resource-bounded Kolmogorov-complexity measure — to capture the rise-then-fall pattern entropy can't explain.
Stanford's course teaches deep learning by making you build vision models from scratch — k-NN and linear classifiers up through CNNs, detection, segmentation, and Transformers — with three PyTorch assignments and a self-chosen final project.
Karpathy's 2015 walkthrough of character-level RNNs trained to predict the next character, showing how a tiny model learns to generate convincing Shakespeare, C code, and LaTeX — and what its neurons actually track.
Walks through the LSTM gating mechanism step by step, showing how the cell state and forget/input/output gates let the network carry information across long sequences where plain RNNs lose it to vanishing gradients.
Condenses Stanford's CS 229 into one-page visual cheatsheets spanning supervised, unsupervised, and deep learning, plus probability and linear-algebra refreshers. Available in 10+ languages, with all topics merged into one Super VIP PDF.
Career advice for ML researchers from the creator of TRPO and PPO: how to pick problems worth solving, why goal-driven beats idea-driven research, and the daily notebook-and-review habit that compounds small experiments into breakthroughs.
Teaches classic machine learning through a 12-week, 26-lesson curriculum with quizzes, written lessons, assignments, projects, and multilingual translations.
A 12-week, 24-lesson beginner-friendly AI curriculum with executable Jupyter notebooks, quizzes and labs that teach neural networks, computer vision, NLP, generative models and ethics using PyTorch and TensorFlow examples.
Teaches generative AI app development through 21 lessons covering LLM basics, prompting, chat, search, image generation, agents, RAG, fine-tuning, small models, and responsible AI.
A free, open textbook on engineering ML systems — building efficient, reliable AI from a single GPU up to warehouse-scale clusters. Goes beyond model design and MLOps tooling to the underlying science: scheduling, quantization, data pipelines, serving.
AI Infra2024
A minimal GPU written in under 15 SystemVerilog files to teach how GPUs execute parallel kernels from the ground up. Includes an 11-instruction ISA, multiple cores with ALUs and load-store units, a fetch-decode-execute pipeline, and matrix kernels.
A GitHub repository of learning notes and code dedicated to ML + SYS (machine learning systems). It collects tutorials, code walkthroughs and engineering notes on RLHF, distributed training (FSDP, Megatron), inference and scheduling (SGLang, vllm), quantization, CUDA/GPU optimization, system design, and practical engineering.