Structured extraction can look deceptively easy when every test example resembles the training set. The useful twist here is a dedicated split for document types absent from training, making it possible to probe whether a model follows a schema on genuinely unfamiliar tasks—not just new wording of a familiar one.
What Makes It Useful
- The 6,000-example training split and 500-example test split cover the same 30 document types, giving a conventional measure of performance on familiar tasks.
- A separate 300-example
test_unseensplit covers five types excluded from training, providing a direct check of task-type generalization. - Each example pairs source text with its JSON Schema and target object. Standardized conventions for dates, times, amounts, currencies, and missing values make outputs easier to evaluate consistently.
- Six showcase examples preview the unseen types. The dataset spans 35 types and 16 languages, while all people, organizations, addresses, and identifiers are fictional.
Who Should Use It
A good fit for fine-tuning or evaluating models that turn varied documents into schema-constrained JSON, especially when generalization to new extraction tasks matters. It is less suitable as evidence of real-world deployment accuracy: the data is synthetic, and its fictional entities and controlled formats do not capture the full noise, ambiguity, or distribution shifts found in production documents.