Introduced the two-stage recipe behind the GPT lineage: unsupervised generative pre-training on unlabeled text, then supervised fine-tuning per task. A single 12-layer Transformer decoder beat bespoke architectures on 9 of 12 NLP benchmarks.
Pre-trains a deep bidirectional Transformer encoder with masked-language-modeling and next-sentence prediction, then fine-tunes one model on 11 NLP tasks, reaching state-of-the-art on GLUE, SQuAD, and MultiNLI with little task-specific tuning.
AI Model2018
Provides unified model definitions and a single API for pretrained text, vision, audio, and multimodal models for both training and inference. Emphasizes cross-framework compatibility (PyTorch/TF/JAX), pipeline-based inference, and direct access to 1M+ Hub checkpoints.
AI Model2018
Turns model definitions into a shared layer across training and inference stacks, covering text, vision, audio, video, and multimodal models. Pipelines, Trainer, and generation APIs make pretrained models usable without locking teams to one framework.
A 1.5B-parameter model trained only to predict the next token on diverse web text does translation, summarization, and QA zero-shot, with no fine-tuning. It recast NLP tasks as conditional language modeling and sparked the staged-release misuse debate.
Demonstrated that language model loss falls as a smooth power law in model size, data, and compute across more than seven orders of magnitude — turning "make it bigger" from a hunch into a budget you can plan, and justifying the GPT-3 scale-up.
At 175 billion parameters, this autoregressive model becomes a strong few-shot learner: it handles translation, QA, and reasoning from a few prompt examples with no gradient updates, establishing in-context learning as an alternative to fine-tuning.
AI Infra2020
Provides vector search, LLM orchestration and language-model workflows with an embeddings database, RAG pipelines, multimodal indexing and agents. Runs locally or in containers and supports multiple models and language bindings (JS/Java/Rust/Go).
AI API2020
Provides a self-hosted machine translation HTTP API that runs offline using the open-source Argos Translate engine; offers Docker-based deployment and a simple HTTP interface for integration. Suited for privacy-conscious or offline translation deployments.
A 12-week, 24-lesson beginner-friendly AI curriculum with executable Jupyter notebooks, quizzes and labs that teach neural networks, computer vision, NLP, generative models and ethics using PyTorch and TensorFlow examples.
AI Dataset2022
A 57-subject multiple-choice benchmark for measuring broad language understanding in LLMs; provides per-subject configs and test/dev/auxiliary_train splits for few-/zero-shot evaluation, widely used for model comparison and academic reporting.
AI Dataset2022
Provides cleaned, per-language snapshots of Wikipedia articles (id, url, title, text) packaged as Hugging Face dataset configs (Parquet). Covers 300+ language configs and dated dumps — useful for language modeling, multilingual NLP, retrieval, and RAG pipelines.