Why this matters
Working in pixel space avoids the lossy VAE/latent bottleneck that many image generators use, preserving high‑frequency and texture detail that matter for dense vision tasks. Iris‑3B demonstrates that a pixel-space generative prior can serve both as a text-to-image generator and as a backbone for monocular depth and single-step restoration/upscaling, offering a single model that handles synthesis and dense prediction.
What Sets It Apart
- Direct pixel generation: the model outputs raw RGB pixels (no latent decoder), aiming to retain fine detail that latent models can lose. This matters when downstream tasks rely on subtle image structure.
- Unified backbone for multiple tasks: the same 3B-parameter pixel-space diffusion transformer (with a small pixel head) is fine-tuned for monocular relative log-depth and 4× image restoration/upscaling with no architectural changes.
- Practical release and tooling: pretrained text-to-image weights plus separate depth and upscaler checkpoints are provided, and the model runs at ~1024×1024 with a frozen Qwen3-VL-4B text encoder for conditioning.
Key quantitative notes: pretraining + supervised fine-tuning totals ≈665K steps; depth fine-tune reports AbsRel 0.071 and δ1 0.946 (mean across several datasets, affine-invariant log-depth); 4× restoration reports LPIPS 0.292 and PSNR 20.68 on DIV2K validation.
Who It's For and Trade‑offs
Great fit if you need a single model that both generates high-quality, near-megapixel images and can be repurposed for dense vision tasks (depth, restoration) without changing architecture. It’s useful for researchers exploring pixel-space priors, and for practitioners who want native-pixel outputs without VAE artifacts.
Look elsewhere if you prioritize absolute state-of-the-art depth/restoration scores over a unified pixel-space pipeline: the paper reports that pixel-space pretraining did not yield clear improvements over strong latent-model baselines on the evaluated depth and restoration tasks. Also plan for GPU memory and bandwidth: pixel-space generation at high resolution is more resource-intensive than latent approaches.
Where It Fits
Iris‑3B is a pragmatic attempt to scale pixel-space diffusion to multi-task vision: it matches competitive text-to-image quality at 1024² while serving as a general vision learner. Use it when preserving pixel fidelity is important or when you want a single backbone for both synthesis and dense prediction; use latent decoders or specialized dense models when compute or best-in-class downstream metrics are the priority.
