AIAny
Icon for item

Ko-MultimodalMind2Web

Evaluates offline action prediction for Korean web agents across 549 tasks on 144 websites, pairing screenshots and action histories with human-recorded next-step targets.

Introduction

Live websites change constantly, making web-agent results difficult to reproduce. This benchmark takes a more controlled route: agents predict actions from recorded Korean-language browser demonstrations, so evaluation can focus on grounding and decision-making without relying on current site behavior.

What Sets It Apart
  • Human-recorded Korean interactions: Two native Korean annotators created and performed realistic tasks, providing demonstrations across 144 websites rather than machine-translated instructions or synthetic traces.
  • Step-level visual context: Each scored example pairs a Korean instruction and screenshot with prior actions and a target operation, element, and value. This makes it possible to measure both where an agent acts and what it does.
  • Careful handling of ambiguous targets: Manually corrected element boxes and accepted alternative actions—including equivalent form submissions—make scoring better reflect valid ways to complete an interaction. The included scorer reports element accuracy, step success, and task success.
  • Offline by design: The 12,905 scored interaction steps are drawn from 549 trajectories. Navigation, waiting, and task termination are recorded in the full trajectories but excluded from next-action scoring.
Who It’s For

A good fit for researchers comparing screenshot-grounded web agents on Korean interfaces or studying offline action prediction with reproducible demonstrations. Look elsewhere if you need to measure end-to-end success on currently live websites: this release evaluates recorded interactions, and access to the full repository requires an authorized Hugging Face account.

Information

  • Websitehuggingface.co
  • OrganizationsKAIST MLP Lab, KAIST
  • AuthorsYerin Nam, Minkyung Cho, Minkyu Kim, Seungwoo Song, Hangyeol Yoo, Gyun Lee, KyungTae Lim
  • Published date2026/11/16

Categories

More Items

Hugging Face

Evaluates whether web agents can complete tasks on live Korean websites. Its 312 human-written tasks span 104 sites, and success is judged by the final website outcome rather than a single prescribed interaction path.

Hugging Face

Provides paired text and synthetic American English speech in two voices for training lightweight TTS models. The 17,371 clips include phoneme-level annotations, with both voices reading each sentence.

Hugging Face

Evaluates whether GUI agents can locate interface elements from Korean instructions and website screenshots. Its 3,319 human-annotated examples span 142 websites, providing a focused test of visual grounding on Korean web interfaces.