Why this matters Most image-generation and editing tools accept vague color descriptors ("blue", "red") but lack a single, prompt-native way to request an exact 24-bit RGB value across both generation and editing. That gap matters for professional design workflows where a precise hex value—what a painter picks from a palette—must be reproduced reliably on specific objects. Paint-Anything shows that a single model can learn a prompt-native hex interface and deliver substantially better object-level color fidelity by combining object-grounded real-image supervision with pure-color anchors.
Key Findings
- Unified hex prompting: The paper demonstrates that raw 24-bit hex strings (e.g., #AABBCC) can serve as a single text interface for both text-to-image generation and targeted editing, removing the need for separate color controllers.
- Data pipeline and Paint-500K: A scalable pipeline converts real images into object-level hex supervision via VLM grounding, segmentation masks, and perceptual clustering (CIELAB MeanShift) to build Paint-500K for training.
- Pure-color anchors + timestep gating: To compensate for illumination and shadow bias in natural images, the method injects pure solid-color images paired with exact hex labels at high-noise timesteps, giving the model clean low-level color grounding while preserving realistic low-noise appearance learning from natural images.
- Evaluation: The authors introduce ACBench (ACBench-T2I and ACBench-Edit) for object-level hex fidelity. On FLUX.2-4B, finetuning with Paint-Anything yields large gains (≈85.3% for generation and ≈28.3% for editing) over the base model and attains top CompColor averages among compared methods.
Who it's for and tradeoffs
Great fit if you need deterministic, object-specific color control in generative or editing pipelines (designers, product mockups, UI/UX assets) and you can incorporate object grounding/segmentation into the workflow. It is especially useful when exact hex fidelity is a primary requirement. Look elsewhere if your primary goal is photorealistic material/lighting simulation rather than exact pixel-wise hex matching—real-world illumination, specularities, textures, and shadows remain sources of approximation and the method relies on segmentation and grounding quality. Also, the approach focuses on color fidelity rather than broader semantic or compositional improvements.
Where it fits
Paint-Anything sits between generic conditional image models and task-specific color controllers: it trades a small amount of model specialization (finetuning on Paint-500K + anchors) for a simpler user interface (plain hex codes in prompts) and direct improvements in per-object color fidelity.
Practical mechanism (brief)
The core recipe is: (1) mine object-level color labels from real images via VLM grounding and segmentation, (2) estimate dominant colors with perceptual clustering, (3) synthesize editing pairs and mix in pure-color anchor samples activated only at high-noise timesteps during diffusion training, and (4) evaluate with ACBench to measure how closely object regions match requested hex values. The timestep gating is the key operational detail that balances noisy synthetic anchors and natural-image fidelity.