Live websites change constantly, making web-agent results difficult to reproduce. This benchmark takes a more controlled route: agents predict actions from recorded Korean-language browser demonstrations, so evaluation can focus on grounding and decision-making without relying on current site behavior.
What Sets It Apart
- Human-recorded Korean interactions: Two native Korean annotators created and performed realistic tasks, providing demonstrations across 144 websites rather than machine-translated instructions or synthetic traces.
- Step-level visual context: Each scored example pairs a Korean instruction and screenshot with prior actions and a target operation, element, and value. This makes it possible to measure both where an agent acts and what it does.
- Careful handling of ambiguous targets: Manually corrected element boxes and accepted alternative actions—including equivalent form submissions—make scoring better reflect valid ways to complete an interaction. The included scorer reports element accuracy, step success, and task success.
- Offline by design: The 12,905 scored interaction steps are drawn from 549 trajectories. Navigation, waiting, and task termination are recorded in the full trajectories but excluded from next-action scoring.
Who It’s For
A good fit for researchers comparing screenshot-grounded web agents on Korean interfaces or studying offline action prediction with reproducible demonstrations. Look elsewhere if you need to measure end-to-end success on currently live websites: this release evaluates recorded interactions, and access to the full repository requires an authorized Hugging Face account.