Builds high-fidelity image generators by pairing a 6B Diffusion Transformer with a frozen LLaDA2.0-Mini vision-language module, relying on extensive image-only pretraining and mid-training; model weights, training code, and recipes are released.
Trains a world-action model that predicts future visual states to guide zero-shot robotic manipulation; introduces CoAE, SVP, IDM and KASO to pretrain generative and action components from scratch on manipulation data, scaling up to 30,000 hours and improving zero-shot success.
AI Dataset2026
Provides layered code pretraining corpora (L2 ~400B tokens, L3 ~150B tokens) across 11 languages by filtering ~192M public GitHub repositories into standardized files, algorithmically relevant selections, and implementation-grounded programming exercises. Includes per-file metadata (role, algo relevance, quality) and serialized task records; released under Apache-2.0.
AI Dataset2026
Provides a bilingual Chinese–English corpus for LLM training covering pretraining, capability-oriented midtraining (16K–256K long contexts), and supervised fine-tuning. Includes ~4.2T pretrain tokens, ~600B midtrain tokens, and ~4.57M SFT samples; sources span web, PDFs/OCR, code, math, QA, and agentic trajectories under mixed upstream licenses.
AI Model2026
A 2B-class causal LLM packaged as a GGUF for local inference; offers 131072-token native long context, XML-style tool-calling support, and is tuned with SFT + RL + OPD using the UltraData family for stronger code, math and agentic abilities.
AI Model2026
Compact causal LLM for on-device assistants, coding agents and long-context tool use — ~2.52B parameters with a 131,072-token context, trained with SFT + RL + OPD and released with its UltraData training corpora and multi-format deployment checkpoints.
AI Dataset2026
Provides ~86K verifiable-reward RL training samples across Math, Knowledge (STEM), Long-Context, and Code for post-training LLMs; each sample includes a ground-truth and verifier-friendly JSONL format for stable reward signals. Note: code tasks require an external sandbox to execute tests.
Trains LVLMs to improve 3D spatial reasoning using a synthetic block-stacking dataset of 15,000 problems that cover 3D→2D projection, viewpoint transforms, and structural combination; includes color-anchored cues and provides code and data.
AI Agent Papers2026
Explores a practical mechanism for recursive self-improvement by post-training LLMs: uses a routing harness to record agent executions and convert traces into curriculum-guided supervised fine-tuning and on-policy distillation data, closing an evaluation-selection-update loop and improving benchmark performance.
AI Dataset2026
Selected ~245B-token corpus of algorithmically relevant code across 11 languages from The Stack v3, produced by file-level language-adaptive selection (UltraData-Code-L2-Classifier). Concentrates problem-solving implementations for LLM code pretraining while preserving broad language coverage; released under ODC-By.
Adds a Next Concept Prediction objective that predicts discrete, product-quantized latent concepts spanning multiple tokens alongside next-token prediction, yielding faster pretraining convergence and measurable downstream gains with an 8.9B-parameter latent-space LLM.
AI Dataset2026
Snapshot delivery of arXiv metadata, submission files and rendered documents in multiple Parquet configs (metadata, paper_text, latex, source, pdf, ps). Includes ~3.15M papers, full-text TeX assemblies and indexes to fetch large assets for training, retrieval and analysis.