Introduced the two-stage recipe behind the GPT lineage: unsupervised generative pre-training on unlabeled text, then supervised fine-tuning per task. A single 12-layer Transformer decoder beat bespoke architectures on 9 of 12 NLP benchmarks.
AI Model2018
Provides unified model definitions and a single API for pretrained text, vision, audio, and multimodal models for both training and inference. Emphasizes cross-framework compatibility (PyTorch/TF/JAX), pipeline-based inference, and direct access to 1M+ Hub checkpoints.
AI Train2019
Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.
AI Dataset2022
A 57-subject multiple-choice benchmark for measuring broad language understanding in LLMs; provides per-subject configs and test/dev/auxiliary_train splits for few-/zero-shot evaluation, widely used for model comparison and academic reporting.
AI Dataset2022
Provides cleaned, per-language snapshots of Wikipedia articles (id, url, title, text) packaged as Hugging Face dataset configs (Parquet). Covers 300+ language configs and dated dumps — useful for language modeling, multilingual NLP, retrieval, and RAG pipelines.
AI Train2023
Modular PyTorch-based framework for building, training, and deploying physics-informed ML models (neural operators, PINNs, GNNs, diffusion). Provides GPU‑optimized training, domain-specific datapipes for meshes/point clouds, distributed scaling and a model zoo.
AI API2023
Ensures LLM outputs match precise, schema-defined structures by enforcing Python types, Pydantic models, JSON schemas or grammars at generation time. Provider-agnostic integrations (OpenAI, transformers, vLLM, Ollama, etc.) and function-call style mapping reduce brittle post-processing and make structured generation reliable for production pipelines.
AI Dataset2023
Contains short, small-vocabulary stories synthetically generated by GPT-3.5 and GPT-4 for training and evaluating compact language models. Includes multiple splits, a GPT-4-only V2 subset, and archive files with prompts and metadata for reproducible experiments.
AI Model2023
Multilingual automatic speech recognition and speech-translation model that transcribes and translates audio. Trained on a mix of weakly labeled and pseudo-labeled data (1M + 4M hours), uses 128 Mel bins and adds a Cantonese token, and supports timestamps and long-form chunking for offline ASR and translation.
AI Deploy2024
Applies quantization, pruning, distillation, speculative decoding and sparsity to compress and accelerate deep learning models, producing exportable checkpoints ready for deployment on TensorRT‑LLM, TensorRT, vLLM and similar inference runtimes.
AI Dataset2024
Provides ~1.3 trillion tokens of web pages filtered for educational quality using an LLM-trained classifier; includes per-Crawl configs, smaller random samples (10B/100B/350B tokens), and the classifier code and model for reproducible filtering.
AI Model2024
Provides code, pretrained weights, and tooling for protein language models and structure prediction — including ESMC, ESMFold2, sparse autoencoders (SAEs), and the ESM Atlas. Includes model checkpoints, tutorials, Hugging Face & Biohub integration, and an MIT license.