Spatial reasoning—reconstructing 3D structure from a single 2D image—remains a brittle skill for large vision-language models because most training signals are either unlabeled or rely on expensive geometric annotations. This work demonstrates an alternative: a compact, structured synthetic curriculum of block-manipulation tasks that teaches foundational spatial skills and transfers to real-scene spatial tasks.
Key Findings
- Small synthetic curriculum with scale: SpatialBlock-15k contains 15,000 procedurally generated block-stacking problems spanning 3D-to-2D projection, viewpoint variations, and structural composition, enabling focused spatial supervision without dense real-scene labels.
- Two training paradigms: a direct-answer model (predicts answers immediately) and a reasoning-based model (intermediate reasoning/prediction steps). Both improve LVLM spatial performance over baselines.
- Visual anchoring: controlled color modulation is used to encourage block-level anchoring, helping models track relevant objects under visual complexity.
- Transfer despite compactness: models trained only on this synthetic dataset show measurable generalization to real-world spatial benchmarks, indicating the curriculum teaches transferable primitives rather than overfitting visuals.
Who it’s for and tradeoffs
Great fit if you are researching spatial capabilities of multimodal models, building benchmarks for 3D reasoning from 2D images, or exploring data-efficient curricula for embodied AI. Look elsewhere if your application requires out-of-the-box, sensor-grade geometric accuracy (e.g., precise robotics grasping relying on depth sensors) or very large-scale, diverse real-scene annotations. The main tradeoff is domain realism versus annotation cost: synthetic tasks reduce labeling noise and cost but require evaluation and possibly fine-tuning on target real-world data.
Where it fits
This approach complements geometry-augmented methods (pseudo-depth, multi-view geometry, or explicit 3D encoders) by offering a lightweight, curriculum-style supervision signal that elicits spatial priors inside LVLMs without adding complex geometric inputs or modules.