A task can look successful while an agent never observed the information its answer depends on. EVIDENT makes that gap measurable: it separates completing a request from collecting required evidence and grounding the final claims in what the agent actually saw.
What Sets It Apart
- The 220 tasks cover semantic grounding, contextual inference, and visual grounding, with 150 Korean and 70 English records. This supports comparisons across both task types and languages.
- Each record pairs a user request with its intended outcome and explicit evidence checks. Evaluators can judge the agent’s process and answer, not just whether the final result appears plausible.
- Its evaluation pipeline extracts evidence from screenshots, checks final-answer claims against that evidence, then applies task-specific rubrics. The dataset card reports 91.27% agreement between its Qwen3-VL-235B judge and human labels on 126 trajectories (Cohen’s κ = 0.864).
- Reported results distinguish task success from evidence success and grounded success. That separation reveals cases where an agent reaches the right outcome but cannot substantiate its claims.
Best Fit—and the Trade-off
Great fit for researchers evaluating GUI or web agents on evidence awareness and answer reliability, especially across Korean and English tasks. Look elsewhere if you need ready-made agent trajectories or screenshots: this release supplies task metadata only, so those must be collected separately. Tasks use live websites, whose content and availability can change over time.