AI Dataset2026
Provides an L1 filtered English web corpus from recent Common Crawl snapshots for LLM pretraining, including main-text extraction, language and heuristic filtering, sensitive-field replacement, customized cleaning, and MinHash deduplication; contains 1T+ tokens across ~1.14B documents with structured metadata fields.
Separates knowledge storage (a global Memory) from iterative reasoning operators (multiple Reasoners) to improve knowledge compression and inference efficiency; reports a 7B model matching baseline with 62.6% of training data and a 35B Intern-S2-Mobius achieving ~4x end-to-end speedup.
AI Agent Papers2026
Trains a foundation GUI agent using a closed-loop, environment-grounded data stack plus in-context multimodal demonstrations to automate long-horizon desktop workflows. Combines scalable task generation/verification, subtask-level demo guidance, and a 100-task OSWorkerBench benchmark to improve strict success and task progress.
AI Model2026
Injects proprietary news, regulatory and legal data into an open checkpoint via data-centric continual learning to improve performance on legal, tax and journalism tasks while preserving general capabilities and very long context support.
AI Audio2026
Generates and edits speech from natural-language instructions plus optional reference audio, supporting zero-shot TTS, content/acoustic/paralinguistic edits, enhancement, and source separation. Open-source 1.5B-parameter base model with a 4-step distilled AuK‑Flash for faster inference.
Estimates optimal learning rates for large-scale Mixture-of-Experts pretraining using a two-step, compute-efficient transfer: μP-based width transfer from small proxy models, then log-log linear extrapolation across token budgets to trillion-token horizons.
AI Model2026
Provides a GGUF-quantized build of GLM-5.3-Flash for local text-generation and inference. Key features: 320B total / 18B active parameters, hybrid sparse+linear attention, native multimodal pretraining and Unsloth Dynamic quantization. Best for developers running GGUF local inference workflows.
Decides when past post-training updates should be reused for autonomous LLM adaptation by introducing Boundary-Calibrated Intervention Transfer (BCIT). BCIT binds effects to source context, checks applicability and hard conflicts, and runs bounded trials to obtain current-state evidence—reducing harmful updates and improving equal-budget final-model quality.
Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Proposes VLAct, a representation-centric continued pre-training method for Vision-Language-Action models that preserves VLM priors and enforces cross-embodiment action semantics to turn limited robot trajectories into transferable visual-action representations; shows strong gains and sample efficiency on multiple VLA benchmarks using modest compute.
Develops a vision-language foundation model for autonomous driving that unifies 3D BEV perception, visual question answering, and motion planning without changing the pretrained VLM architecture. Key elements include an external BEV perception head for 3D detection and occupancy, a Planning Expert using flow-matching for trajectory prediction, and a staged training recipe combining driving and general VLM data.
Studies looping shared transformer layers in Mixture-of-Experts models under matched budgets and proposes SMELT: loop the middle half twice while matching per-token FLOPs, non-embedding parameters, and KV cache. Shows 6.8–18.0% training-FLOPs savings on the compute-optimal frontier, stronger downstream gains on code and long-context tasks.
Builds high-fidelity image generators by pairing a 6B Diffusion Transformer with a frozen LLaDA2.0-Mini vision-language module, relying on extensive image-only pretraining and mid-training; model weights, training code, and recipes are released.