Tag
Explore by tags
Introduced the two-stage recipe behind the GPT lineage: unsupervised generative pre-training on unlabeled text, then supervised fine-tuning per task. A single 12-layer Transformer decoder beat bespoke architectures on 9 of 12 NLP benchmarks.
Provides unified model definitions and a single API for pretrained text, vision, audio, and multimodal models for both training and inference. Emphasizes cross-framework compatibility (PyTorch/TF/JAX), pipeline-based inference, and direct access to 1M+ Hub checkpoints.
A 57-subject multiple-choice benchmark for measuring broad language understanding in LLMs; provides per-subject configs and test/dev/auxiliary_train splits for few-/zero-shot evaluation, widely used for model comparison and academic reporting.
Provides cleaned, per-language snapshots of Wikipedia articles (id, url, title, text) packaged as Hugging Face dataset configs (Parquet). Covers 300+ language configs and dated dumps — useful for language modeling, multilingual NLP, retrieval, and RAG pipelines.
Modular PyTorch-based framework for building, training, and deploying physics-informed ML models (neural operators, PINNs, GNNs, diffusion). Provides GPU‑optimized training, domain-specific datapipes for meshes/point clouds, distributed scaling and a model zoo.
Contains short, small-vocabulary stories synthetically generated by GPT-3.5 and GPT-4 for training and evaluating compact language models. Includes multiple splits, a GPT-4-only V2 subset, and archive files with prompts and metadata for reproducible experiments.
Provides ~1.3 trillion tokens of web pages filtered for educational quality using an LLM-trained classifier; includes per-Crawl configs, smaller random samples (10B/100B/350B tokens), and the classifier code and model for reproducible filtering.
Provides code, pretrained weights, and tooling for protein language models and structure prediction — including ESMC, ESMFold2, sparse autoencoders (SAEs), and the ESM Atlas. Includes model checkpoints, tutorials, Hugging Face & Biohub integration, and an MIT license.
Provides pre-parsed Parquet snapshots of English and French Wikipedia articles with structured fields (sections, infoboxes, tables, references, images) and credibility signals — optimized for large-scale analysis, retrieval-augmented generation, and model development.
Provides an open platform of omnimodal world models, datasets, and tools to build Physical AI — joint perception, generation, and action reasoning for robots, autonomous vehicles, and smart infrastructure. Supports images, video, audio, and action-conditioned workflows.
Provides end-to-end PyTorch scripts to download/prepare data, implement a transformer from scratch, train LLMs (13M→billion-scale) and generate text. Emphasizes educational clarity and single‑GPU experiments; useful for researchers or hobbyists, but large-scale training still requires substantial compute and engineering.
High-quality, efficiently verified and filtered web corpus for LLM pretraining — supplies ~1 trillion English tokens and ~120 billion Chinese tokens with English/Chinese Parquet splits. Designed for large-scale pretraining experiments and data-filtering research.