AIAny
Icon for item

ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition

Recovers editable design files from raster images by growing an editable layer hierarchy via an agentic pipeline that selects and composes modality-specific tools. Introduces graceful verification (accept/prune/retry) to prevent error accumulation and presents the Figma Edit Replay Benchmark (909 files, 14,796 edits) to measure editability across layout, color, and text edits.

Introduction

Design files reconstructed from pixels rarely support real edits: visual fidelity metrics miss whether a recovered file can actually be edited in a real design app. ReDesign reframes raster→design reconstruction as a long-horizon, agentic decomposition problem where specialized tool calls build up a layered representation and lightweight local verification prevents errors from compounding.

Key Findings
  • Agentic decomposition for multi-modal recovery: ReDesign composes specialized modules (text, vector geometry, colors, grouping, layer ordering) in a decision sequence, so outputs target editability rather than only pixel fidelity. This means reconstructed files are structured for downstream edits (e.g., text replacement, recoloring).
  • Graceful verification to contain errors: After each expansion step the system either accepts, prunes, or retries a tool output, which avoids large-scale reruns and reduces cascading mistakes — translating to more reliable long-horizon reconstruction.
  • Figma Edit Replay Benchmark and empirical gains: The paper introduces a benchmark of 909 raw Figma files and 14,796 controlled edit instructions that replay edits on reconstructions; ReDesign achieves top editability on layout, color, and text edits while keeping strong visual fidelity compared to layered-decomposition and serial-tool baselines.
Who It's For and Trade-offs

Great fit if you need recoverable, editable design artifacts from images—product designers, UI/tooling teams, and researchers building AIGC→authoring workflows. Look elsewhere if your priority is only pixel-level reconstruction (single-shot image metrics) or if you require a lightweight on-device solution: ReDesign targets editability and relies on multi-step tool orchestration, which increases pipeline complexity and runtime compared with one-shot methods.

Where It Fits

ReDesign sits between reconstruction pipelines that optimize only visual similarity and end-to-end vectorization heuristics: it prioritizes a structured, edit-ready output and introduces a reproducible benchmark to measure the practical utility of reconstructions in real editing scenarios.

Information

  • Websitearxiv.org
  • AuthorsJooyeol Yun, Jintae Park, Hyesu Lim, Junha Hyung, Hyungjin Chung, Jaegul Choo
  • Published date2026/07/28

More Items

Real-time streaming multimodal foundation model that uses a codec-native tokenizer (Mage-ViT) to encode motion- and residual-rich regions from video I/P frames, reducing visual token usage by over 75% and enabling up to ~3.5× wall-clock inference speedup after training on ~560M images and 100M video frames.

Provides a portable, robot-free UMI capture pipeline and shows that policies post-trained only on this high-fidelity data deploy directly on real robots matching teleoperation baselines. Capture achieves ~3 mm end-effector accuracy, microsecond sync, ultra-wide FOV, and releases 2,000h HiFi-UMI-2K.

Drives long‑horizon desktop agents by reading and manipulating program state (files, DOM, backends) instead of relying on screenshots. The main agent uses code for actions and structural verification while a lightweight GUI subagent handles rare screenshot-click steps, improving success rates and lowering per-task cost versus screenshot-only approaches.