AIAny
Icon for item

EVIDENT: Task Metadata for Evidence-Grounded GUI Evaluation

Provides 220 Korean and English GUI/web-agent tasks with evidence checks to assess task completion, observed evidence, and whether final-answer claims are grounded in that evidence.

Introduction

A task can look successful while an agent never observed the information its answer depends on. EVIDENT makes that gap measurable: it separates completing a request from collecting required evidence and grounding the final claims in what the agent actually saw.

What Sets It Apart
  • The 220 tasks cover semantic grounding, contextual inference, and visual grounding, with 150 Korean and 70 English records. This supports comparisons across both task types and languages.
  • Each record pairs a user request with its intended outcome and explicit evidence checks. Evaluators can judge the agent’s process and answer, not just whether the final result appears plausible.
  • Its evaluation pipeline extracts evidence from screenshots, checks final-answer claims against that evidence, then applies task-specific rubrics. The dataset card reports 91.27% agreement between its Qwen3-VL-235B judge and human labels on 126 trajectories (Cohen’s κ = 0.864).
  • Reported results distinguish task success from evidence success and grounded success. That separation reveals cases where an agent reaches the right outcome but cannot substantiate its claims.
Best Fit—and the Trade-off

Great fit for researchers evaluating GUI or web agents on evidence awareness and answer reliability, especially across Korean and English tasks. Look elsewhere if you need ready-made agent trajectories or screenshots: this release supplies task metadata only, so those must be collected separately. Tasks use live websites, whose content and availability can change over time.

Information

  • Websitehuggingface.co
  • OrganizationsKAIST MLP Lab
  • AuthorsYerin Nam, Minkyung Cho, Minkyu Kim, Seungwoo Song, Hangyeol Yoo, KyungTae Lim
  • Published date2026/11/14

Categories

More Items

Hugging Face

Evaluates whether GUI agents can locate interface elements from Korean instructions and website screenshots. Its 3,319 human-annotated examples span 142 websites, providing a focused test of visual grounding on Korean web interfaces.

Hugging Face

Evaluates document parsers across text, formatting, tables, charts, layout grounding, and forms. Severity-aware scoring prevents a small number of critical errors from being hidden by otherwise high page-level accuracy.

Hugging Face

Provides first-person video of skilled manual work with synchronized motion-sensor data for robot learning and dexterous manipulation research. The first release spans 10,124 hours from 12,227 contributors, with task-level annotations.