AI Model2018
Turns model definitions into a shared layer across training and inference stacks, covering text, vision, audio, video, and multimodal models. Pipelines, Trainer, and generation APIs make pretrained models usable without locking teams to one framework.
A 1.5B-parameter model trained only to predict the next token on diverse web text does translation, summarization, and QA zero-shot, with no fine-tuning. It recast NLP tasks as conditional language modeling and sparked the staged-release misuse debate.
Demonstrated that language model loss falls as a smooth power law in model size, data, and compute across more than seven orders of magnitude — turning "make it bigger" from a hunch into a budget you can plan, and justifying the GPT-3 scale-up.
At 175 billion parameters, this autoregressive model becomes a strong few-shot learner: it handles translation, QA, and reasoning from a few prompt examples with no gradient updates, establishing in-context learning as an alternative to fine-tuning.
A 12-week, 24-lesson beginner-friendly AI curriculum with executable Jupyter notebooks, quizzes and labs that teach neural networks, computer vision, NLP, generative models and ethics using PyTorch and TensorFlow examples.
AI Dataset2022
A 57-subject multiple-choice benchmark for measuring broad language understanding in LLMs; provides per-subject configs and test/dev/auxiliary_train splits for few-/zero-shot evaluation, widely used for model comparison and academic reporting.
AI Dataset2022
Provides cleaned, per-language snapshots of Wikipedia articles (id, url, title, text) packaged as Hugging Face dataset configs (Parquet). Covers 300+ language configs and dated dumps — useful for language modeling, multilingual NLP, retrieval, and RAG pipelines.
Made reinforcement learning from human feedback (RLHF) the standard alignment recipe: collect demonstrations and preference rankings, train a reward model, then optimize with PPO. A 1.3B aligned model was preferred over the 175B GPT-3 by human raters.
AI Dataset2022
Benchmark dataset of ~8.5k grade-school math word problems with step-by-step solutions and calculator annotations for evaluating multi-step arithmetic reasoning in language models. Provided in two configs (main and socratic) and commonly used for chain-of-thought prompting, fine-tuning, and verifier training.
A line-by-line PyTorch reimplementation of the Transformer paper as a runnable notebook, where each part of the paper sits next to the code that implements it — turning a dense architecture into something you can read and run end to end.
AI Model2022
Rust-and-Python toolkit that serves open-source LLMs (Llama, Falcon, Mixtral, StarCoder) over HTTP/gRPC with tensor parallelism, continuous batching, Flash/Paged Attention and quantization. Now in maintenance mode, pointing users toward vLLM and SGLang.
AI Dataset2022
Provides human preference comparison pairs and red-team conversation transcripts collected by Anthropic for training preference/reward models and studying harmful model behaviors; intended for RLHF and safety research, not for supervised fine-tuning of dialogue agents.