Tag

TikTok Videos, 4.5 Billion

Provides 4.5 billion TikTok video records with captions, timestamps, music IDs and engagement counts for research; split across 27 zstd-compressed Parquet files (~289 GB) and sampled via TikTok's mobile API; released for research-use only with privacy and ToS caveats.

UltraData-Code

Provides layered code pretraining corpora (L2 ~400B tokens, L3 ~150B tokens) across 11 languages by filtering ~192M public GitHub repositories into standardized files, algorithmically relevant selections, and implementation-grounded programming exercises. Includes per-file metadata (role, algo relevance, quality) and serialized task records; released under Apache-2.0.

ZGCM-1-Data

Provides a bilingual Chinese–English corpus for LLM training covering pretraining, capability-oriented midtraining (16K–256K long contexts), and supervised fine-tuning. Includes ~4.2T pretrain tokens, ~600B midtrain tokens, and ~4.57M SFT samples; sources span web, PDFs/OCR, code, math, QA, and agentic trajectories under mixed upstream licenses.

UltraData-Stack-v3-L2

Selected ~245B-token corpus of algorithmically relevant code across 11 languages from The Stack v3, produced by file-level language-adaptive selection (UltraData-Code-L2-Classifier). Concentrates problem-solving implementations for LLM code pretraining while preserving broad language coverage; released under ODC-By.

arXiv Complete Corpus

Snapshot delivery of arXiv metadata, submission files and rendered documents in multiple Parquet configs (metadata, paper_text, latex, source, pdf, ps). Includes ~3.15M papers, full-text TeX assemblies and indexes to fetch large assets for training, retrieval and analysis.

Eidon Tracker POV

Provides 1,274 hours of head-mounted egocentric video paired with seven-point IMU arm tracking (24 Hz orientation; raw accel/gyro/mag on a subset), packaged for embodied-AI and egocentric-vision research. Key features: torso-relative pose via chest reference, separate Parquet IMU repo for efficient joins, CC-BY-4.0 license; heavy class skew and limited contributor diversity are important constraints.

Typed Decisions

Benchmark for typed probabilistic decisions: given one shared state, a model answers multiple typed questions at once and returns full probability distributions (noul/choice/score). Contains four workflows with train/test splits and soft gold labels from teacher samples, designed to evaluate accuracy, calibration, and latency trade-offs.

HOI-Retarget — Humanoid Human–Object Interaction Motion

Generates humanoid robot motion references that preserve object contact locations/timing by solving windowed trajectory optimizations against contact targets in the object frame. Releases retargeted trajectories for two Unitree robots across 75 objects (≈13.9k robot–motion pairs); CC BY‑NC‑SA 4.0.

ZefanCai/Open-Jev

Maps a state and question to typed probabilistic decisions (choice distributions, yes/no probabilities, or scored/ordinal outputs) across controlled synthetic tasks. Offers multiple frozen configs with train/calibration/validation/test/OOD splits, Parquet + raw JSONL exports, and reproducible manifests and provenance.

SmolDataEnvs

Provides 5.5K+ self-contained data-analysis RL tasks: each row bundles a real tabular dataset, a question, and a deterministically-gradable gold answer. Verified from jupyter-agent notebooks; splits for training, held-out testing, and quick eval; intended for prompting, fine-tuning, and agent RL.

BEV Decision Mix

Provides 150,000 source‑grounded decision examples for training models that pick options, judge yes/no propositions, or assign ordered scores. Each row pairs a 'state' with JEV-style typed questions (CHOICE/NOUL/SCORE); multiple configs and train/test splits are included, license mixed/unknown.

XiaomiMiMo/MiMo-V2.6-RL-oss

Provides mixed-domain, verifiable RL training environments for LLM agents (code, cyber, knowledge work, web dev, music) as Parquet datasets, with domain-specific verifiers, Docker artifacts and links to training code for reproducible agentic RL experiments.