Most "train your own LLM" repos stop at a training loop and leave you to glue together tokenizers, eval harnesses, and serving yourself. nanochat collapses the entire pipeline — tokenization, pretraining, finetuning, RL, inference, and a ChatGPT-like web UI — into one readable codebase, then makes the whole thing controllable with a single --depth knob that derives every other hyperparameter for you. The point is not raw quality; it's that a complete LLM lifecycle now fits in a code budget a single person can read in an afternoon.
What Sets It Apart
- One dial, not fifty:
--depthsets transformer layer count and auto-computes the rest, so scaling up or down is a single number rather than a config-tuning project. - Honest cost math: a GPT-2-grade model trains for ~$48 on 8xH100 (~2 hours), or ~$15 on spot — concrete enough to budget a real experiment instead of guessing.
- Transparency over magic: a global
COMPUTE_DTYPEreplaces autocast, so precision behavior is explicit rather than hidden — valuable when you're learning why training behaves the way it does. - A "Time-to-GPT-2" leaderboard reframes the goal as wall-clock speedrunning, turning training efficiency into something measurable and competitive.
Who It's For
Great fit if you want to understand an LLM end-to-end by reading and modifying every stage, or need a clean baseline to fork for research and teaching. Look elsewhere if you want a production-grade model or state-of-the-art quality — the ceiling here is roughly GPT-2 capability, and the design optimizes for legibility and speed, not benchmark wins. Treat it as the spiritual successor to nanoGPT extended across the full chat pipeline, not as a drop-in chatbot.
