AI Train2024
Streamlines the full lifecycle of foundation models — data prep, fine-tuning (SFT/LoRA/QLoRA/GRPO), evaluation, and deployment — with ready-to-run recipes, multi-engine inference support, and cloud/CLI workflows for both laptop experiments and large-scale runs.
AI Train2024
A community speedrun to train a 124M GPT as fast as possible on 8 H100s, all chasing a fixed 3.28 FineWeb loss. Successive records cut the run from llm.c's 45 minutes to under 1.4, mostly via the new Muon optimizer rather than more hardware.
AI Train2024
Trains a sub-100M-parameter LLM from scratch — pretraining, SFT, LoRA, DPO/RLHF, and distillation, sized from ~26M up to ~100M-plus dense and MoE. Headline figure: the ~64M minimind-3 variant's SFT stage runs 1 epoch in ~2h and ~3 RMB on one NVIDIA 3090.
AI Image2024
Trains a 65M-parameter vision-language model from scratch in ~2 hours on one RTX 3090, about 3 RMB (~$0.40) of GPU rental. Connects a frozen SigLIP2 encoder to a small MiniMind LLM via a two-layer MLP projector; full PyTorch code for pretraining and SFT.
AI Infra2024
Lets Python developers write tile-based parallel kernels for NVIDIA GPUs, generating CUDA Tile IR while staying close to Python syntax for custom GPU operations.
AI Train2024
Parses, generates, and filters training data from noisy sources like PDFs and weak QA, then feeds it into LLM pre-training, SFT, RL, or RAG cleaning. Ships 100+ operators and ready-made pipelines for text, reasoning, Text2SQL, and agentic data.
AI Train2024
Open-source HybridFlow implementation for RL post-training of LLMs. Decouples control flow from compute so PPO, GRPO, GSPO and DAPO share one dataflow; pairs FSDP/Megatron with vLLM/SGLang rollout and reports 1.5-20x throughput over prior RLHF stacks.
AI Train2024
High-accuracy biomolecular structure prediction suite: open-source models (protenix-v2/v1), a benchmark/evaluation toolkit, and a web server for inference. Targets protein/antibody–antigen and ligand-aware predictions with inference-time sampling and constraint support.
AI Train2024
Predicts 3D structures of proteins, nucleic acids, and small-molecule complexes, the first fully open-source model to approach AlphaFold3 accuracy. Boltz-2 adds binding-affinity prediction that nears FEP simulation accuracy at ~1000x the speed.
AI Train2024
A 671B-parameter Mixture-of-Experts language model (37B activated) trained on 14.8T tokens with 128K context, FP8-first training, a Multi-Token Prediction module, and Hugging Face weights—focused on efficient MoE training and long-context use cases.
Provides end-to-end PyTorch scripts to download/prepare data, implement a transformer from scratch, train LLMs (13M→billion-scale) and generate text. Emphasizes educational clarity and single‑GPU experiments; useful for researchers or hobbyists, but large-scale training still requires substantial compute and engineering.
AI Train2025
Bundles a dataset, an interaction harness, and rubric-based reward functions into one RL environment for training and evaluating LLMs — also usable as an eval, synthetic-data pipeline, or agent harness for any OpenAI-compatible endpoint.