What is a vision-language-action (VLA) model?
A VLA takes camera images and an instruction and outputs motor commands directly. It is the robot policy most humanoid and arm makers now build on.
Short answer: a vision-language-action model is a vision-language model (the kind that can describe an image or answer a question about it) trained further so that its output is robot actions instead of, or as well as, text. Given a camera view and an instruction such as "put the cup in the sink", it produces the arm and gripper commands to do it.
Where the idea comes from
Google DeepMind introduced the term with RT-2 in July 2023. RT-2 started from a model pre-trained on web-scale images and text, then was co-fine-tuned on robot demonstrations. It wrote robot actions as tokens, the same way a language model writes words. On scenarios it had not seen in training, RT-2 succeeded 62% of the time, against 32% for its predecessor RT-1. The point was that knowledge learned from the web carried over to robot control.
How actions come out
There are two main approaches today:
- Actions as tokens. Continuous motor commands are turned into discrete tokens and generated one after another, like text. RT-2 did this; π0-FAST does it with a compression scheme (FAST) that turns action sequences into fewer tokens.
- Actions as a continuous output. Physical Intelligence's π0 uses flow matching: it starts from random noise and refines it into a smooth sequence of motor commands, producing trajectories at 50 Hz. π0 was built on the PaliGemma vision-language model and trained on data from 7 robot platforms across 68 tasks.
Why VLAs are hard to compare
A VLA's score depends on the robot, the tasks, the camera setup and who ran the test. The same model can score very differently on two leaderboards, and most models are tested on only one. Our data note on eight policy leaderboards shows six different number ones.
How we test them
Our Robot Policy Index (Phase 2) will score policies on real arms and in simulation under one published protocol, separately from embodied reasoning models and world models. Until then, see the robot policy page for current public results, which we list for reference and do not average.