AI Train2024
End-to-end framework for running and reproducing foundation-model research workflows — from data curation and tokenization to training and evaluation. Emphasizes reproducibility by recording every step (including failed runs) and expressing experiments as dependency-ordered steps.
Hands-on coding tutorial series for large language models with slides and runnable notebooks covering fine-tuning, prompting, RLHF, safety, steganography, watermarking, multimodal models, GUI agents, and deployment. Community-maintained, free course materials for students and researchers.
AI Dataset2024
Provides a cleaned, deduplicated English web corpus optimized for LLM pretraining—over 15T tokens aggregated from CommonCrawl with per-dump snapshots and smaller sampled configs (10B/100B/350B). Includes the datatrove processing pipeline, MinHash deduplication, and an ODC-By v1.0 license; suited for large-scale model training and ablation studies but not specialized for code.
AI Train2024
Accelerates video generation with a unified framework for inference, finetuning, LoRA, distillation, sparse attention, and distributed execution for research and demos.
AI Train2024
Pretrained foundation model for zero-shot point and quantile forecasting on unseen time series, no per-dataset training. Decoder-only, trained on 100B real-world time-points; v2.5 (200M params) handles up to 16k context and 1k-step horizons.
AI Train2024
Streamlines the full lifecycle of foundation models — data prep, fine-tuning (SFT/LoRA/QLoRA/GRPO), evaluation, and deployment — with ready-to-run recipes, multi-engine inference support, and cloud/CLI workflows for both laptop experiments and large-scale runs.
AI Dataset2024
Provides ~1.3 trillion tokens of web pages filtered for educational quality using an LLM-trained classifier; includes per-Crawl configs, smaller random samples (10B/100B/350B tokens), and the classifier code and model for reproducible filtering.
AI Train2024
A community speedrun to train a 124M GPT as fast as possible on 8 H100s, all chasing a fixed 3.28 FineWeb loss. Successive records cut the run from llm.c's 45 minutes to under 1.4, mostly via the new Muon optimizer rather than more hardware.
AI Train2024
Trains a sub-100M-parameter LLM from scratch — pretraining, SFT, LoRA, DPO/RLHF, and distillation, sized from ~26M up to ~100M-plus dense and MoE. Headline figure: the ~64M minimind-3 variant's SFT stage runs 1 epoch in ~2h and ~3 RMB on one NVIDIA 3090.
AI Image2024
Trains a 65M-parameter vision-language model from scratch in ~2 hours on one RTX 3090, about 3 RMB (~$0.40) of GPU rental. Connects a frozen SigLIP2 encoder to a small MiniMind LLM via a two-layer MLP projector; full PyTorch code for pretraining and SFT.
AI Image2024
Turns PDFs and images into clean Markdown with a 7B vision-language model, keeping tables, equations, handwriting, and multi-column reading order while removing headers and footers. Runs on one 12GB+ GPU at about 1/32 the cost of GPT-4o APIs.
Embodied AI2024
GPU‑accelerated framework for training physically simulated humanoid characters and robots using reinforcement learning and motion imitation. Provides a modular multi‑backend simulator stack, large‑scale multi‑GPU training recipes, built‑in motion retargeting and an ONNX deployment pathway to real robots.