AIAny
Icon for item

ParseBench-2

Evaluates document parsers across text, formatting, tables, charts, layout grounding, and forms. Severity-aware scoring prevents a small number of critical errors from being hidden by otherwise high page-level accuracy.

Introduction

A parser can get nearly every character right and still mislead a reader by attaching a value to the wrong table header or inventing a blank form entry. This benchmark makes those high-impact failures visible: scores reflect error severity, not just the fraction of content matched.

What Sets It Apart
  • Six dimensions cover both content and structure, including chart data, visual grounding, and government forms—areas that broad OCR tests can miss.
  • More than 140 error types distinguish changed facts, missing or invented content, and broken relationships. Per-page diagnostics help explain why a score fell.
  • Its 4,317 page-level cases span 2,645 source documents, with targeted collections for difficult tables, meaningful formatting, and forms. Full chart data annotations also make evaluations less dependent on sparse spot checks.
  • Critical and major errors impose calibrated score caps, so a wrong value cannot be masked by a page that is otherwise mostly correct. The overall score averages the six dimensions equally.
Who It’s For—and the Trade-off

A good fit for teams comparing document extraction pipelines or diagnosing failures in OCR, layout analysis, and structured-data recovery. It is less useful as a general-purpose OCR score: its six dimensions and targeted subsets are deliberately specific, and results should only be compared using the same dataset revision and scorer version. Scores are not directly comparable with ParseBench-1.

Information

  • Websitehuggingface.co
  • OrganizationsLlamaIndex
  • AuthorsPierre-Loïc Doulcet, Boyang Zhang, Sebastián G. Acosta, Sacha Bron, Daniel B. Ospina, Antonio Jimeno Yepes, Nutchapol Dendumrongsup, Simon Suo
  • Published date2026/11/11

Categories

More Items

Hugging Face

Evaluates whether GUI agents can locate interface elements from Korean instructions and website screenshots. Its 3,319 human-annotated examples span 142 websites, providing a focused test of visual grounding on Korean web interfaces.

Hugging Face

Provides first-person video of skilled manual work with synchronized motion-sensor data for robot learning and dexterous manipulation research. The first release spans 10,124 hours from 12,227 contributors, with task-level annotations.

Hugging Face

Provides 220 Korean and English GUI/web-agent tasks with evidence checks to assess task completion, observed evidence, and whether final-answer claims are grounded in that evidence.