AIAny
Icon for item

SVG Generation Benchmark (Static)

Compares 30 frontier LLMs generating static SVG markup from 500 prompts using 1,355,161 human votes across three leaderboards (Preference, Coherence, Alignment); provides raw SVGs, 768×768 rasterized PNGs, and per-comparison human vote records under a CC-BY-4.0 prompt license.

Introduction

Oops! Something went wrong

[next-mdx-remote-client] error compiling MDX: Expected a closing tag for `<svg>` (6:56-6:61) before the end of `paragraph` 4 | - Large-scale human comparisons: 188,754 head-to-head pairs and 1,355,161 annotated votes across three independent questions (Preference, Coherence, Alignment). This emphasizes perceptual quality and prompt fidelity rather than proxy metrics. 5 | - Broad model coverage: 30 contemporary models (including multiple Claude, GPT, Gemini, Mistral, and other variants) evaluated on the same 500 prompts to enable direct cross-model ranking and ELO leaderboards. > 6 | - Reproducible pipeline and raw artifacts: models' raw <svg> markup is stored, repaired where possible, rasterized to 768×768 PNGs, and released with per-vote JSON so analysts can reaggregate or audit human judgments. | ^ 7 | - Prompt curation and licensing: prompt set mixes ~50 human-authored seeds with 450 samples drawn (and attributed) from a CC-BY-4.0 public dataset, selected for semantic diversity via embeddings and farthest-point sampling. 8 | More information: https://mdxjs.com/docs/troubleshooting-mdx

Information

Categories

More Items

Hugging Face

Provides a machine-readable catalog of 117 AI/AX safety and deployment-readiness diagnostic criteria for assessing model intrinsic and serving/infrastructure risks. Includes MODEL-SCAN and AX-SCAN axes, bilingual source fields, per-item evidence guidance, severity/assurance metadata, and a CC BY-NC 4.0 release-candidate.

Hugging Face

Evaluates schema-guided structured extraction from documents: given a document and a JSON schema, systems must return a schema-valid JSON with page-and-box grounding. Covers 370 documents (4,869 pages) across 8 business domains and 67 document types; scores value accuracy, word/page grounding, and long-list completeness.

Hugging Face

Provides 10 million synchronized egocentric experience episodes with structured 3D/4D multimodal annotations — 2.88B RGB frames, 720M depth frames, 576M pose/mocap frames and ~1PB total. Designed for embodied AI, robotics, and multimodal pretraining; research-only, gated access.