AIAny
Icon for item

The Mask Is Not the Object: Volumetric Supervision for 3D Gaussian Segmentation

A rendered object mask can look complete even when the selected 3D Gaussians omit occluded parts. VODA uses local depth clusters to supervise those Gaussians directly, and Removal-3D evaluates whether their removal matches the object’s expected 3D removal.

Introduction

A sharp 2D mask can hide a surprisingly incomplete 3D selection: alpha-blended training gives the strongest learning signal to Gaussians that contribute most to visible pixels, while occluded or weak contributors may remain unselected. This work treats that mismatch—not mask quality alone—as the central problem in 3D Gaussian segmentation.

Key Findings
  • VODA groups per-pixel contributors by depth and directly supervises the Gaussians in the dominant depth cluster, rather than relying only on contribution-weighted gradients. This helps recover less-visible primitives associated with the same object.
  • Removal-3D evaluates segmentation by comparing the depth change caused by removing predicted Gaussians with the change from removing the actual object. Its Removal Depth Error (RDE) tests volumetric completeness that rendered-mask metrics cannot reveal.
  • Across four rendering-based baselines, VODA reduces RDE on the synthetic benchmark while maintaining or improving rendered-mask performance on Removal-3D, LERF, and ScanNet. This suggests improved completeness need not come simply from selecting a larger region.
  • Reweighting gradients or propagating features spatially does not reproduce the gains, pointing to depth-aware grouping and direct Gaussian-level supervision as the key ingredients.
Method and Trade-offs

The method uses local depth gaps to separate contributors into clusters, then chooses the cluster with the greatest total rendering contribution. This offers a practical way to extend supervision beyond dominant contributors without requiring ground-truth features for individual Gaussians. However, depth clustering can merge touching objects or split objects with sparse interiors. The volumetric benchmark is also limited to three synthetic indoor scenes, since exact evaluation requires paired depth maps before and after object removal.

Who It’s For

A good fit for researchers building object-level editing, selection, or removal systems on 3D Gaussian Splatting, especially when a convincing rendered mask is not enough. Look elsewhere if your main need is broad real-world volumetric evaluation or robust handling of tightly touching objects; those cases are not fully established by the current benchmark.

Information

  • Websitearxiv.org
  • OrganizationsYonsei University
  • AuthorsJeonghwan Cho, Minsu Kim, Junyoung Hong, Seon Joo Kim
  • Published date2026/11/16

More Items

Tests multimodal models’ spatial reasoning through screen-based clicks and drags. Its programmatically generated CAPTCHA tasks also form a training environment, with fine-tuning gains reported across seven out-of-distribution benchmarks.

Coordinates visual and audio forensic agents to locate and verify signs of AI-generated video. Its dual-timescale training improves both the investigation strategy and the specialist inspectors.

Unifies text, images, audio, and video for Chinese-English multimodal understanding and reasoning, with fast instruct and explicit thinking modes. The paper reports leading open-source results on several omni-modal video benchmarks.