AI Dataset2026
Provides 100 real-world, open-ended research tasks paired with expert-written rubrics (around 40 weighted criteria per task) to evaluate long-form, web-browsing research agents on factual accuracy, analysis depth, presentation, and citation quality.
AI Video Papers2026
Provides a diagnostic suite that audits video-understanding benchmarks to find samples solvable without visual or temporal input, filters those shortcuts, and produces a distilled video-native testbed that reveals major capability gaps in current Video-LLMs.
AI Dataset2026
A collection of biology-focused 'mystery' tasks for benchmarking model performance on biomedical reasoning, evidence synthesis, and problem solving; curated by Anthropic and hosted on Hugging Face, designed for granular evaluation of scientific decision-making.
AI Dataset2026
A prompt-only mixture of ~478k prompts designed to support antidoom-style generation and preference-data pipelines for reducing model repetition (doom loops). Prompts are stripped of answers and labels and sourced from many public datasets so it’s usable for FTPO/adapter generation but not for supervised QA evaluation.
AI Dataset2026
Provides a county-harmonized corpus of U.S. municipal and county ordinance text (≈2.21M chunks) labeled for function, substantive indicator, and topic to support legal NLP, retrieval, and comparative local-law research. Includes model-assigned labels and continuous scorers (opacity, paternalism, enforcement discretion) plus coverage metadata; not exhaustive or a substitute for legal advice.
Collection of hands-on workshop materials and sample code from Anthropic's "Code with Claude" series, covering Claude Managed Agents, memory (Dreaming Service), eval-driven agent development, and multi-agent patterns. Not maintained and not accepting contributions.
AI Dataset2026
A small image-folder dataset for multimodal/vision model safety benchmarking, containing under 1,000 curated images with annotations to exercise safety-related model behaviours; licensed CC BY 4.0 and hosted on HuggingFace.
AI Agent2026
Trains reusable natural-language 'skills' for frozen LLM agents by optimizing the skill document in text-space — using trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts. Multi-backend, zero inference-time cost at deployment, designed for iterative, validation-led skill improvement.
AI Dataset2026
Provides 40 public Kubernetes incident scenarios (SRE subset) with ground-truth root-cause entities and offline cluster snapshots in JSONL format; designed to evaluate agentic root-cause diagnosis on alerts, events, traces and topology.
Measures how well LLMs and agent-driven workflows prepare supervised training data end-to-end by jointly benchmarking data construction and data-quality evaluation across six domains, using a downstream-grounded protocol and new metrics.
AI Dataset2026
Provides 462 unrestricted long-form chain-of-thought reasoning traces distilled from the full Mythos V2 model (≈104.7M characters); intended for long-context evaluation, trace analysis and process-level supervision. License unknown—verify before reuse.
AI Dataset2026
Provides the gated, official OSWorld 2.0 Python task class files (task_*.py) required to run the benchmark; distributed via a Hugging Face gated dataset to reduce benchmark leakage. Download requires accepting gated access on Hugging Face.