Tag
Explore by tags
Blind side-by-side voting site where users send one prompt to two anonymous chat models, pick the winner, and millions of votes become Elo rankings across text, coding, vision, image, and video. Crowd preference, not static benchmarks, decides the order.
Provides leaderboard-ready test splits for the Open ASR Leaderboard: converts unsafe custom loaders to Parquet, sorts samples by audio length, and packages eight ESB test sets (LibriSpeech, Common Voice, GigaSpeech, SPGISpeech, etc.) for reproducible ASR benchmarking.
A human-verified subset of 500 SWE-bench test cases for evaluating models that resolve GitHub issues into PRs using unit-test verification. Contains problem statements and base commits (pre-fix) for reproducible unit-test based evaluation; suitable for benchmarking code-fix and issue-resolution capabilities.
Provides a 1,000-row sample user–item interaction Parquet for the TAAC2026 recommendation task, using a flat column layout with 120 top-level columns (IDs, labels, user/item int & dense features, and four-domain behavioral sequences). Updated 2026-04-10.
Benchmark dataset for evaluating clinician-facing chat assistants: physician-authored conversations plus rubric items, use-case and difficulty labels, specialty metadata, and a built-in canary to reduce benchmark contamination. Hosted on Hugging Face under an MIT license.
A retrieval benchmark suite focused on “oblique queries,” where relevance depends on latent attributes rather than surface keywords. Includes five tasks with large corpora, qrels (and pooled judgments), and task-specific constraints for evaluating embedding-based retrievers and reasoning-augmented retrieval.
Provides task-card metadata for 147 long-horizon professional tasks from the Agents Last Exam benchmark — titles, prompts, taxonomy, and input-file descriptors. This v1.0 release is metadata-only; companion repos host input files and gated reference outputs.
Provides a 289-case (1,058-turn) multi-turn benchmark that evaluates interactive video world models across 22 metrics and five dimensions (quality, setting, interaction, consistency, physics). Includes first-/third-person and navigation splits plus a 20-model leaderboard for head-to-head comparisons.
Proposes TASTE, an automatic pipeline that synthesizes challenging agent benchmark tasks by sampling and evolving valid tool-sequence patterns; uses an adaptive contrastive n-gram model and LLM validity judgments to produce τ^c-Bench with broader tool-use coverage and higher difficulty.
Applies a population-level test-time scaling strategy that uses one model as generator, verifier, refiner, and ranker to search over candidate proofs. Combines generative-verifier RL and a low false-positive verifier with tournament selection to reach competition-level performance on IMO and USAMO.