AI Dataset2026
Measures how autonomous AI agents learn via long-horizon, feedback-rich executable tasks; publishes 51 public tasks from a 134-task suite and provides SForge, a two-container evaluation harness for iterative 12+ hour runs to track learning trajectories.
AI Dataset2026
Provides 300 adversarial "metacognitive-trap" problems to measure whether LLMs notice and recover from their own reasoning errors. Combines multiple-choice vulnerability tests with free-form adapter-gain evaluation and ships per-model metacognition adapters for frozen-base probing.
AI Dataset2026
Provides 2,165 historical natural-history page scans paired with ~99.95% expert transcriptions and pixel-aligned PAGE XML layout ground truth for OCR and layout evaluation; multilingual (EN/FR/DE/LA), CC-BY 3.0.
Zhilin Wang, Han Song +14University of Science and Technology of China, The Chinese University of Hong Kong +6
Provides a benchmark and protocol to evaluate agents that iteratively edit executable policies under a fixed interaction budget, recording full execution–feedback–revise trajectories. Built from compact RL environments with trajectory-level diagnostics and hidden held-out validation.
Embodied AI2026
Evaluates generalist robot manipulation policies across simulation and real-world settings using 42 sim tasks and 18 real tasks; measures generalization, memory, precision, long-horizon execution and open-vocabulary instruction following, and provides a cloud-accessible real-world evaluation system with XPolicyLab integration and a public leaderboard.
AI Agent Papers2026
A 365-day, order-level simulation benchmark for evaluating long-term coherence of LLM agents in seller-side e-commerce. Grounded in 98,843 real product records and 26 interactive tools, it pairs prompt upstream supplier signals with delayed downstream order outcomes to stress planning, memory, and tool use over long horizons.
Evaluates how large language models fabricate user attributes in personalization and whether model self-monitoring is a reliable signal. Introduces MirageBench (150 personas, 6 personalization tasks, judge-validated faithfulness taxonomy) and a 12-model leaderboard revealing pervasive over-inference and a 'Self-Monitoring Inversion'.