Most embodied tracking systems either rely on heavy perception stacks or trade spatial-temporal foresight for inference speed. This model demonstrates that a compact vision-language-action policy can fuse modern visual features with natural-language conditioning to output short-horizon motion waypoints fast enough for onboard robot control.
Key Capabilities
- Language-conditioned trajectory prediction: accepts a natural-language instruction and predicts eight future [x, y, yaw] waypoints directly, so control stacks can consume smooth, anticipatory actions instead of low-level steering signals.
- Fused modern visual features: uses DINOv3 and SigLIP feature fusion as visual inputs, so it benefits from strong pretrained visual representations without training an end-to-end heavy vision encoder from scratch.
- Quality-driven, self-evolving dataset and DAgger-style interaction: automated filtering plus model-in-the-loop data collection targets hard cases (crossings, occlusions, rapid turns), so the policy improves robustness on challenging real-world scenarios.
- On-device optimization for practical deployment: joint optimizations yield roughly 5+ FPS and ≈180 ms end-to-end latency on Unitree Go2 onboard compute, so it is suitable for real-time embodied experiments on similar platforms.
Who It's For and Trade-offs
Great fit if you are a roboticist or researcher building embodied person-following systems who needs a compact, language-aware motion policy that can run on moderate onboard compute (e.g., Unitree Go2) and you already have or can produce fused DINOv3+SigLIP features. Look elsewhere if you require an end-to-end vision-to-action model (the model expects precomputed visual tokens), need ultra-high frame rates/very low latency beyond the stated measurements, or if your target hardware is far weaker than typical robot onboard GPUs.
Where It Fits
Compared with heavyweight end-to-end VLA agents, this approach trades a standalone visual encoder for precomputed fusion features and model compactness, making it more deployment-friendly on mid-tier robot hardware while retaining language conditioning and short-horizon planning ability.
