AI Agent Papers2025
Tests research agents on 100 expert-written tasks across 22 fields. Separates report quality from citation reliability, using adaptive reference-based grading and checks that cited pages support individual claims.
AI Agent2025
Evaluates and optimizes AI agents and language models in containerized environments, supporting large-scale parallel benchmarks and RL rollouts. Integrates with third‑party providers for thousands of parallel environments and serves as the official harness for Terminal‑Bench.
AI Agent2025
Open-source companion to a technical book that teaches how to design, evaluate and ship LLM-based AI agents — includes the full Chinese manuscript, community translations, chapter-aligned runnable example projects, and reproducible evaluation harnesses.
AI Agent2025
Automates multi-step web tasks by perceiving webpages as pixels and issuing low-level mouse, keyboard and scroll actions. A 7B-parameter multimodal agent trained on 145K synthetic trajectories (FaraGen), designed for on-device deployment and efficient task completion (~16 steps/task).
AI Agent Papers2025
Trains an 8B model to search, synthesize evidence, and write cited research reports. Its reinforcement-learning rewards use evolving rubrics grounded in retrieved evidence and contrasting model responses.
AI Dataset2025
Collects ~200,000 human responses to 20 visual/semantic association questions (e.g., Bouba–Kiki), with per-response image options and demographic metadata — useful for cross‑cultural perception and evaluation of multimodal systems, but not guaranteed as a rigorously controlled experimental sample.
Reduces object-driven shortcut learning in zero-shot compositional action recognition by enforcing temporal verb cues and regularizing against frequent object-verb co-occurrence priors. Proposes RCORE with Co-occurrence Prior Regularization (treats frequent co-occurrences as hard negatives) and Temporal Order Regularization. Evaluated on Sth-com and EK100-com with improved compositional generalization.
AI Dataset2026
Provides a physical reconstruction benchmark of OmniDocBench v1.5 by producing five real-world photographic variants (Scanning, Warping, Screen‑Photography, Illumination, Skew) for each of 1,355 pages, inheriting original ground-truth to enable controlled, scenario-wise evaluation of document parsing robustness.
AI Coding2026
Feeds coding agents targeted guidance on modern web APIs and practices, helping them choose browser-native solutions, account for accessibility and performance, and adapt to browser support.
AI Dataset2026
Provides 100 real-world, open-ended research tasks paired with expert-written rubrics (around 40 weighted criteria per task) to evaluate long-form, web-browsing research agents on factual accuracy, analysis depth, presentation, and citation quality.
AI Agent Papers2026
Measures whether reusable skills improve agent task completion through paired tests. Across 87 tasks and 18 model–harness configurations, curated skills raise average pass rate by 16.6 percentage points; compact skill bundles work best.
AI Dataset2026
Benchmarks LLM agents on realistic legal work by packaging lawyer-style assignments with client materials and expert, per-deliverable rubrics. Includes an execution harness to run, score, and compare agents across a large, evolving task set spanning multiple practice areas.