AIAny
Icon for item

Ko-Screenspot

Evaluates whether GUI agents can locate interface elements from Korean instructions and website screenshots. Its 3,319 human-annotated examples span 142 websites, providing a focused test of visual grounding on Korean web interfaces.

Introduction

GUI agents can appear capable on English benchmarks while still missing the labels and conventions users encounter on Korean websites. This benchmark makes that gap measurable: each example pairs a Korean instruction with a screenshot and a target element, testing whether an agent can point to the right place rather than merely describe the page.

What the Evaluation Reveals

The 3,319 examples were derived from 540 human-recorded tasks across 142 websites. Two native Korean annotators contributed about 480 hours of collection and review, including scrutiny of 267 potentially ambiguous questions; 163 were removed. This manual process matters because small wording or target ambiguities can distort point-in-box scores. The remaining examples cover both text targets and icon or widget targets, so results can distinguish reading visible labels from grounding less explicit controls.

The benchmark uses a deliberately narrow, single-screenshot protocol: an agent receives an image and a Korean grounding instruction, then predicts a location. That makes it useful for measuring visual grounding without mixing in navigation planning or task completion. It is one component of a broader Korean GUI evaluation suite, alongside offline action prediction and live website tasks.

Who Should Use It

A good fit for researchers comparing GUI agents on Korean-language interface grounding, or checking whether improvements transfer beyond English-centric screenshots. Look elsewhere if you need to measure multi-step task completion, broad language coverage, or performance on changing live websites: this benchmark evaluates static screenshot grounding, and its viewer exposes only a small illustrative preview. Full-data access is restricted to authorized Hugging Face users.

Information

  • Websitehuggingface.co
  • OrganizationsKAIST MLP Lab, KAIST
  • AuthorsYerin Nam, Minkyung Cho, Minkyu Kim, Seungwoo Song, Hangyeol Yoo, Gyun Lee, KyungTae Lim
  • Published date2026/11/16

Categories

More Items

Hugging Face

Evaluates document parsers across text, formatting, tables, charts, layout grounding, and forms. Severity-aware scoring prevents a small number of critical errors from being hidden by otherwise high page-level accuracy.

Hugging Face

Provides first-person video of skilled manual work with synchronized motion-sensor data for robot learning and dexterous manipulation research. The first release spans 10,124 hours from 12,227 contributors, with task-level annotations.

Hugging Face

Provides 220 Korean and English GUI/web-agent tasks with evidence checks to assess task completion, observed evidence, and whether final-answer claims are grounded in that evidence.