Tag
Explore by tags
Argues that "interesting" complexity is low in both ordered and fully random states but peaks in between, and proposes "complextropy" — a resource-bounded Kolmogorov-complexity measure — to capture the rise-then-fall pattern entropy can't explain.
Karpathy's 2015 walkthrough of character-level RNNs trained to predict the next character, showing how a tiny model learns to generate convincing Shakespeare, C code, and LaTeX — and what its neurons actually track.
Walks through the LSTM gating mechanism step by step, showing how the cell state and forget/input/output gates let the network carry information across long sequences where plain RNNs lose it to vanishing gradients.
A 2019 essay arguing that over 70 years of AI, general methods that scale with computation — search and learning — consistently beat hand-coded human knowledge. The short text that crystallized the scaling-vs-priors debate.
Career advice for ML researchers from the creator of TRPO and PPO: how to pick problems worth solving, why goal-driven beats idea-driven research, and the daily notebook-and-review habit that compounds small experiments into breakthroughs.
A line-by-line PyTorch reimplementation of the Transformer paper as a runnable notebook, where each part of the paper sits next to the code that implements it — turning a dense architecture into something you can read and run end to end.
Argues AI has entered its 'second half': a working recipe (language pre-training priors + scale + reasoning) now generalizes RL across tasks, so the bottleneck shifts from inventing methods to defining problems and rethinking evaluation.
Turns content briefs into SEO-optimized long-form articles via chained Claude Code commands and specialized subagents. Pulls live GA4, Search Console, and DataForSEO data to ground keyword research, drafting, and on-page optimization in real rankings.
Frames AI research as a trainable practice of reading, building, debugging, and fast feedback. The essay is most useful for researchers learning how to avoid hype-chasing, benchmark tunnel vision, and agent-induced blind spots.
Converts an academic paper into reusable extracted assets and then produces editable poster, synchronized talk video, and bilingual blog via modular generator skills. Key differentiator: a single Paper2Assets extractor shared by three editable generators plus an interactive Paper2Reel viewer that links slides, video, captions and blog while preserving factual consistency and round-tripable PPT/DOCX output.