AI Dataset2023
A public dataset of one million real-world conversations with 25 LLMs, including conversation text, model name, detected language tags, and OpenAI moderation outputs — useful for studying prompt distributions, safety/moderation, and training/evaluating instruction-following models.
AI Dataset2026
Provides Parquet-backed pretraining subsets of web and synthetic QA text for large-language-model training, including web-high-nltk-qa, web-high-medium, and txt360-qa. Offers streaming access, provenance metadata, and CC BY 4.0 licensing; intended for LM pretraining and research.
Zixuan Fu, Bingxiang He +11Affiliation: University of Chinese Academy of Sciences, Affiliation: Tsinghua University +3
Studies on-policy distillation (OPD) at the data-minimal limit by training on a single query, measuring state coverage and alignment dynamics, and showing OPD is often data-overfed but algorithm-starved.
Builds high-fidelity image generators by pairing a 6B Diffusion Transformer with a frozen LLaDA2.0-Mini vision-language module, relying on extensive image-only pretraining and mid-training; model weights, training code, and recipes are released.