AIAny
Icon for item

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

Directly maps visual observations and language instructions to continuous robot actions, replacing LLM-centric V→L→A pipelines. Uses separate visual and language encoders with lightweight bidirectional interaction and a compact decoder to cut inference cost and VRAM, achieving ~31 ms latency and <1 GB VRAM on an RTX 4090; suited for real-time robotic manipulation under tight compute budgets.

Introduction

Most VLA work routes vision into a large language model, then decodes language into actions — an approach that works but pays a heavy price in per-step compute and GPU memory. TurboVLA challenges that norm by building task-conditioned representations directly from visual and linguistic features and exchanging information via a lightweight bidirectional interaction, enabling continuous action chunk prediction with a compact decoder. The result is VLA-level performance at dramatically lower latency and VRAM use, making real-time robotic manipulation feasible on consumer GPUs.

Key Findings
  • Direct V+L→A mapping: Separates visual and language encoders and uses a lightweight bidirectional interaction module so the policy does not require a large LLM as the central interface. This reduces model size and per-step compute.

  • Strong empirical efficiency: On the LIBERO benchmark TurboVLA reports 97.7% average success using only ~0.2B parameters, with 31.2 ms inference latency (~32 Hz) and ~0.9 GB VRAM on an RTX 4090. These numbers show it can match or exceed much larger LLM-centric VLA policies while running in resource-constrained settings.

  • Practical decoder design: Predicts continuous action chunks with a compact decoder instead of autoregressively decoding through a large language space, improving throughput for sequential decision tasks.

Who It's For and Tradeoffs

Great fit if you need a VLA policy that runs at high frequency on consumer GPUs (e.g., real-time robot manipulation or deployment on edge-like rigs) and care about low per-step latency and VRAM. TurboVLA is attractive when end-to-end LLM reasoning is unnecessary and when you can design task-conditioned vision-language interactions.

Look elsewhere if your task requires extensive natural-language reasoning, complex multi-step planning best expressed in language tokens, or heavy reliance on emergent LLM capabilities; TurboVLA trades some of the generality of an LLM-centered interface for efficiency and tighter action-conditioned representations.

Where It Fits

TurboVLA sits between heavyweight LLM-centric VLA policies (higher flexibility, higher cost) and purely reactive visuomotor policies (lower cost, less language grounding). It targets the sweet spot for multimodal robotic control where grounded language instructions matter but LLM-scale reasoning at runtime is not essential.

Information

  • Websitearxiv.org
  • AuthorsHengyi Xie, Chenfei Yao, Xianjin Wu, Xuanyang Xi, Yiping Tang, Di Xu, Yingying Zhu, Dingkang Liang, Xiang Bai, Han Ding
  • Published date2026/07/29

More Items

Evaluates multimodal context learning across grounding, new information application, and knowledge acquisition using a 3,443-instance benchmark spanning science, finance, long documents, spatial reasoning, and web VQA; finds current multimodal models perform poorly (best score 0.2847) and analyzes failure modes.

Evaluates whether vision-language models can make actionable decisions for a physical body by decoupling decision-making from low-level motor execution. Introduces HumanCLAW-Bench with 1,218 long-horizon egocentric episodes across 41 indoor scenes and diagnoses a lack of embodied self-awareness in current VLMs.

Real-time streaming multimodal foundation model that uses a codec-native tokenizer (Mage-ViT) to encode motion- and residual-rich regions from video I/P frames, reducing visual token usage by over 75% and enabling up to ~3.5× wall-clock inference speedup after training on ~560M images and 100M video frames.