Tag

GLUE (General Language Understanding Evaluation benchmark)

A multi-task English NLU benchmark for evaluating models across nine tasks (acceptability, sentiment, paraphrase, similarity, and various NLI setups), with a diagnostic evaluation set and an online leaderboard to compare generalization and transfer learning.

CLIP (openai/clip-vit-base-patch32)

Learns a joint image–text embedding via contrastive pretraining to enable zero-shot image classification. Uses a ViT‑B/32 image encoder and transformer text encoder; intended primarily for research into robustness and generalization, not untested deployment.

BERT base model (uncased)

Pretrained uncased English BERT base model for masked language modeling and next-sentence prediction. ~110M parameters, pretrained on BookCorpus and English Wikipedia; commonly fine-tuned for classification, token labeling, and question answering.

UnsolvedMath

Provides a machine-readable collection of 5,426 open and historically significant mathematical problems with LaTeX statements, structured metadata and curated per-problem AI-assisted research notes. Includes difficulty labels, canonical problem sets (Millennium, Hilbert, Erdős) and files optimized for benchmarking math reasoning.

Xperience-10M

Provides 10 million synchronized egocentric experience episodes with structured 3D/4D multimodal annotations — 2.88B RGB frames, 720M depth frames, 576M pose/mocap frames and ~1PB total. Designed for embodied AI, robotics, and multimodal pretraining; research-only, gated access.

laion/BVD-V-55M

Provides 55 million scene-level video clips (each with captions, language labels, and timestamps) extracted from an 80M-video, 10-million-hour raw pool to support multimodal pre-training across video, audio, and frames. Access is gated for academic/non-commercial research.

ASI-Bench Generated Instances (Seed 42)

Provides fixed-seed benchmark instances (prompts and agent-visible inputs) for ASI-Bench to run reproducible evaluations of LLM agents on scientific tasks. Includes four matched prompt levels (B1–B4) across 60 project-level tasks in 11 domains; excludes reference answers and private scorers; Apache-2.0 licensed.

ASI-Bench Generated Instances (Seed 31415)

Generated instance set (seed 31415) for ASI‑Bench: includes four matched prompt variants, agent-visible inputs, reference artifacts, and instance metadata for 60 project-scale scientific research tasks across 11 domains; intended for evaluating autonomous research agents. Licensed Apache‑2.0.

DarwinX: Evolving Agent Harnesses Through Natural Selection

Treats agent self-improvement as natural selection over a population of harnesses (prompts, tools, skills, control flow), evolving a frozen-model agent by selecting harness edits that extend capability without regressing others. Uses a preserve-and-extend contract, lineage archive, and verifier-driven fitness (no gold solutions) to recombine complementary edits and transfer across benchmarks.

OpenH-RF

Provides ~39 TB of pre‑beamformed (channel capture) ultrasound RF data and metadata in zea/HDF5 format for reconstruction, flow, and inverse‑problem tasks. Released under CC‑BY‑4.0 and curated for training and evaluating ultrasound/RF foundation models.

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

Frames LLM routing as a sequential decision process and introduces LLMRouter plus the xRouteBench benchmark to develop, evaluate, and deploy learned routing policies across heterogeneous LLMs, optimizing response quality versus inference cost.

Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives

Evaluates whether LLM-driven storyteller agents preserve long-horizon logical consistency under adversarial player interventions. Introduces NCP-Bench (100 movie-derived narrative environments) with structured trajectory/commitments and automatic violation checks; finds strong LLMs often contradict themselves across multi-turn interactions.