Evaluates how large language models fabricate user attributes in personalization and whether model self-monitoring is a reliable signal. Introduces MirageBench (150 personas, 6 personalization tasks, judge-validated faithfulness taxonomy) and a 12-model leaderboard revealing pervasive over-inference and a 'Self-Monitoring Inversion'.
AI Video Papers2026
Evaluates VLMs' ability to form global spatial awareness from long-horizon egocentric video. Introduces GST-Bench: a VQA benchmark with human-verified questions from 6,790 minutes of synthetic video, reveals a large gap (best zero-shot 42.68 vs human 79.08) and provides GST-Train dataset.
AI Dataset2026
Evaluates schema-guided structured extraction from documents: given a document and a JSON schema, systems must return a schema-valid JSON with page-and-box grounding. Covers 370 documents (4,869 pages) across 8 business domains and 67 document types; scores value accuracy, word/page grounding, and long-list completeness.
Tao Feng, Fangxu Yu +10University of Illinois Urbana-Champaign, University of Maryland, College Park +3
Frames LLM routing as a sequential decision process and introduces LLMRouter plus the xRouteBench benchmark to develop, evaluate, and deploy learned routing policies across heterogeneous LLMs, optimizing response quality versus inference cost.
AI Agent Papers2026
Enables a coding agent to self-develop by evolving its harness, prompts, tools, and core code via reviewed commits — supporting recursive free evolution and experience-driven evolution. Demonstrated a 161-day live lineage and state-of-the-art scores on multiple coding benchmarks while foregrounding operational safety.
Yingpeng Ma, Jianhao Yan +7Affiliation: NLP2CT Lab, University of Macau, Macau, China, Affiliation: Westlake University, Hangzhou, China +3
Evaluates whether LLM-driven storyteller agents preserve long-horizon logical consistency under adversarial player interventions. Introduces NCP-Bench (100 movie-derived narrative environments) with structured trajectory/commitments and automatic violation checks; finds strong LLMs often contradict themselves across multi-turn interactions.
AI Dataset2026
Human-annotated text dataset that labels perceived “AI slop” with a continuous human slop_score (-1 / 0 / +1) plus provenance metadata (source_dataset, source_row_id, content_hash). Collected via Bench Labs SlopFinder from public datasets for training classifiers and studying subjective perception.
AI Dataset2026
Provides a curated benchmark of 170 real-world, multilingual code-refactoring instances to evaluate AI coding agents on large-scale, behavior-preserving, cross-file refactors. Each task includes rewritten issue descriptions and manually reviewed test suites to avoid over- and under-constraining evaluations.
AI Dataset2026
Provides a year-scale multimodal benchmark and evaluation framework for on-device long-term memory in personal assistants, built from real mobile user trajectories. Tests memory construction, retrieval, updating, temporal reasoning, and implicit preference inference, and includes a knowledge-grounded synthesis pipeline to form coherent long-horizon trajectories.
AI Agent Papers2026
Uses a stronger 'builder' model at inference time to construct executable harnesses that boost weaker target models without parameter updates, mainly by turning unstable reasoning into deterministic code, routing, and strict answer-format enforcement.
AI Video Papers2026
Predicts future video frames conditioned on an observed frame, a language instruction, and a sequence of end-effector poses and gripper states for robot manipulation. Uses per-arm SE(3) geometric encoding (PRoPE-style), a lightweight depth branch, SAM3 masks with a frozen V-JEPA teacher, and distribution-matching distillation for efficient, consistent action-conditioned rollouts.
AI Agent Papers2026
Yiwei Li, Wanli Yang +11Affiliation: Meituan, Affiliation: University of Chinese Academy of [email protected]
Systematically evaluates LLM-driven autonomous agents on long-horizon AI research tasks using rule-based within-run metrics (Solution Framing, Execution, Feedback Control). Focuses on experience reuse and harness effects across 36 tasks and seven frontier models, finding agents act more like engineering optimizers than autonomous researchers.