Publishes a structured open textbook on large language model foundations, covering language modeling, LLM architectures, prompt engineering, PEFT, model editing, and RAG.
AI Agent2024
Crawls 30+ social platforms (Weibo, Xiaohongshu, Douyin), parses their video and image content, then has five specialized agents debate in a moderated forum to synthesize public-opinion reports. Can fuse public sentiment with a private business database.
AI Dataset2024
Provides pre-parsed Parquet snapshots of English and French Wikipedia articles with structured fields (sections, infoboxes, tables, references, images) and credibility signals — optimized for large-scale analysis, retrieval-augmented generation, and model development.
AI Infra2024
Converts PDFs, Office files, HTML, images and audio into one structured DoclingDocument, with deep PDF layout, reading order, table-structure and formula recognition, OCR, and native LangChain/LlamaIndex/Haystack integrations for RAG pipelines.
AI Dataset2024
A mixed instruction dataset for SFT and RLHF research that combines chat, math, code and instruction-following samples from multiple public datasets under an Apache-2.0-compatible license; intended for instruction tuning and evaluation.
Open-weight Mixture-of-Experts LLM with 671B total parameters but 37B activated per token, trained on 14.8T tokens for 2.788M H800 GPU-hours. Matches leading closed models at a fraction of typical training cost via FP8 and architectural tricks.
Provides end-to-end PyTorch scripts to download/prepare data, implement a transformer from scratch, train LLMs (13M→billion-scale) and generate text. Emphasizes educational clarity and single‑GPU experiments; useful for researchers or hobbyists, but large-scale training still requires substantial compute and engineering.
Shows that LLM reasoning can be incentivized through pure reinforcement learning, with no human-annotated reasoning traces. Self-reflection, verification, and strategy-switching emerge on their own, and the patterns transfer to distill smaller models.
AI Dataset2025
Multi‑modal closed-ended academic benchmark with 2,500 multiple-choice and short-answer exam questions spanning math, natural sciences, and humanities for automated grading. Curated by subject-matter experts, released under MIT, and includes a canary string to help prevent dataset leakage into model training.
AI Dataset2025
Provides professionally translated parallel corpora and a multilingual lexicon across 100+ low-resource languages for training and evaluating multilingual MT and NLP models. Includes SmolDoc, SmolSent, GATITOS, and factuality annotations; licensed CC-BY-4.0.
AI Dataset2025
High-quality, efficiently verified and filtered web corpus for LLM pretraining — supplies ~1 trillion English tokens and ~120 billion Chinese tokens with English/Chinese Parquet splits. Designed for large-scale pretraining experiments and data-filtering research.