A visual encoder’s final output may be a poor handoff point for robot control: details useful for recognizing a scene are not necessarily the details needed to grasp, align, or track an object. This paper’s central insight is that action policies can benefit from reading visual computation as it evolves, rather than receiving only a finished visual representation.
Key Findings
- With the action expert trained while the backbone stays frozen, the model reaches 90.9% average success on LIBERO, compared with 79.5% for Qwen3-VL and 84.5% for Gemma-3. This suggests its visual features are already more directly usable for control.
- Full fine-tuning raises its LIBERO average to 95.3%. The largest reported backbone-comparison advantage is on LIBERO-Long, where its frozen result of 85.4% exceeds the fully fine-tuned encoder-based comparison models.
- Layer-wise probing finds that action-relevant information can peak before the last visual layer. Simply exposing an encoder-based model to intermediate vision features does not reproduce the benefit, pointing to representation formation and alignment—not encoder removal alone—as important factors.
- In three real-world tasks, it succeeds in 12/20, 17/20, and 13/20 trials after fine-tuning on 50 demonstrations per task. The benchmark is small, so these results are promising evidence rather than broad proof of deployment reliability.
How the Architecture Works
Images enter the language backbone through a lightweight convolutional patch embedder, and visual and language tokens evolve together through its layers. A flow-matching action expert can read representations from different depths and generates continuous actions without discretizing them into action tokens. The 2B-parameter backbone is trained in stages, from image-text perception to multimodal instruction tuning, before action learning.
Who Should Read It
Great fit for researchers studying visual representations, robot manipulation, or the design of vision-language-action policies—especially those questioning whether a conventional vision encoder is the right interface to an action head. Look elsewhere for a ready-to-run robotics system or conclusive evidence across many robots and environments: the reported real-world evaluation covers only three tasks, and the paper notes that changes in spatial configuration or task goal remain challenging.