AIAny
Icon for item

SVG Generation Benchmark (Static)

Compares 30 frontier LLMs generating static SVG markup from 500 prompts using 1,355,161 human votes across three leaderboards (Preference, Coherence, Alignment); provides raw SVGs, 768×768 rasterized PNGs, and per-comparison human vote records under a CC-BY-4.0 prompt license.

Introduction

Oops! Something went wrong

[next-mdx-remote-client] error compiling MDX: Expected a closing tag for `<svg>` (6:56-6:61) before the end of `paragraph` 4 | - Large-scale human comparisons: 188,754 head-to-head pairs and 1,355,161 annotated votes across three independent questions (Preference, Coherence, Alignment). This emphasizes perceptual quality and prompt fidelity rather than proxy metrics. 5 | - Broad model coverage: 30 contemporary models (including multiple Claude, GPT, Gemini, Mistral, and other variants) evaluated on the same 500 prompts to enable direct cross-model ranking and ELO leaderboards. > 6 | - Reproducible pipeline and raw artifacts: models' raw <svg> markup is stored, repaired where possible, rasterized to 768×768 PNGs, and released with per-vote JSON so analysts can reaggregate or audit human judgments. | ^ 7 | - Prompt curation and licensing: prompt set mixes ~50 human-authored seeds with 450 samples drawn (and attributed) from a CC-BY-4.0 public dataset, selected for semantic diversity via embeddings and farthest-point sampling. 8 | More information: https://mdxjs.com/docs/troubleshooting-mdx

Information

Categories

More Items

Hugging Face

Tests whether AI agents can interpret complex visual files, synthesize information across documents, and produce finished professional deliverables. Its 10 public tasks span distinct domains and require multimodal perception and multi-step tool use.

Hugging Face

A corpus of roughly 366 million audio-text clips, each up to 30 seconds, curated for contrastive language-audio pretraining. Clips are organized by licence, with downloadable audio where redistribution is allowed and links for the rest.

Hugging Face

Evaluates whether AI agents can complete long-horizon engineering deliverables across structural, civil, electrical, and mechanical disciplines. Its 29-task development set combines automated checks with rubric-based grading.