Why this matters Contact-rich manipulation remains a major bottleneck for general-purpose robot skills: visual cues alone often miss contact subtlety, while tactile signals are sparse and hardware-dependent. This work demonstrates a practical recipe to transfer large-scale visuo-tactile priors into downstream policies and to improve deployed policies from logged experience, producing measurable gains on both simulated and real-robot benchmarks.
Key Findings
- Pretraining on NeoData yields transferable contact priors that the policy can distill into fine motion control via a dedicated latent tactile pathway; this staged integration recovers tactile utility without retraining entire vision backbones.
- ALTER, an advantage‑conditioned offline RL procedure, turns relative progress and event labels into binary advantage conditions, enabling policy improvement from fixed deployment logs and human corrections.
- Empirical gains are substantial: the full system wins all nine NeoReal real-robot tasks, reaches 63.8% mean success on a 20-task simulated suite versus 44.0% for the strongest baseline, and—when combined with ALTER—achieves 75–95% success on three long-horizon real-robot tasks.
Who It's For and Tradeoffs
Great fit if you work on contact-sensitive robot manipulation and can collect or access tactile-enabled datasets and hardware. The approach is valuable for insertion, deformable object handling, and dual-arm contact tasks that require reflexive touch feedback. Look elsewhere if you lack tactile sensors or deployment logs: benefits rely on large visuo-tactile corpora (NeoData), staged tactile integration, and sufficient compute to train/finetune the latent tactile pathway. There are also engineering costs for tactile sensing, data labeling of contact events, and offline-data curation for ALTER.
Where It Fits
Positions itself between purely vision-based policy backbones and end-to-end tactile specialists: it leverages vision-scale architectures while adding a lightweight, predictive tactile stream and an offline RL refinement step, making tactile capability more modular and more amenable to real-world deployment.
