Last time, we covered NVIDIA teaming up with Japan's robotics giants. But behind every one of those robots is a core technology that makes movement possible in the first place: the VLA model. Today, let's break down what a VLA model actually is, and what happens between the moment someone gives a robot a spoken instruction and the moment its arm actually moves.
VLA Is Eyes, Brain, and Hands Combined
VLA stands for Vision, Language, and Action. Think about what happens when someone asks you to "pick up that apple and put it on the plate." First, your eyes locate the apple and the plate (Vision). Then you understand what the sentence means (Language). Finally, you reach out, adjust the grip strength in your fingers, and move the apple (Action).
Until recently, AI was only good at two of these three things. Large language models like ChatGPT understand and respond to language well, but they have no eyes and no hands. Vision-language models can look at a photo and describe it, but they still can't touch anything. A VLA model adds that missing third piece — action — so a single AI system can see, understand, and physically act, all at once.
From a Spoken Command to a Moving Robot Arm
Here's roughly what happens inside a robot after it hears "pick up the apple and put it on the plate":
Step one: the robot's camera looks at the scene and identifies that the apple is on the left and the plate is on the right.
Step two: it parses the sentence, understanding that "pick up" means closing the hand around something, and "put it on" means lowering it onto a surface.
Step three, and this is the genuinely hard part: it converts that understanding into joint angles and grip force for the robot arm. Squeeze too hard and the apple gets crushed; too soft and it slips out. A VLA model calculates this force control and sends out dozens of joint commands per second.
Humans pick this up naturally from birth — a baby learns how much force to use by dropping things over and over. Robots learn it from massive amounts of simulation data and recorded human demonstrations instead.
Example 1: NVIDIA's Alpamayo — a VLA Model Behind the Wheel
At CES 2026, NVIDIA introduced Alpamayo, a VLA model built for autonomous driving. Think of it as a VLA model that holds a steering wheel instead of a robot arm. It watches the road through cameras (Vision), reasons through situations like "what should I do if the traffic light is out" (Language-based reasoning), and then actually turns the wheel and applies the brakes (Action) — all inside one model.
What stands out is its "chain-of-thought" approach. Rather than jerking the wheel immediately, Alpamayo reasons step by step, the way a human driver might think "that car could cut in, so I should slow down," and it leaves behind a trace of why it made each decision. That matters a lot for building trust in self-driving systems — if something goes wrong, engineers can actually check the reasoning behind the decision. As of May 2026, Alpamayo has scaled from the original 10-billion-parameter Alpamayo 1 up to a larger 32-billion-parameter Alpamayo 2 Super, getting more capable with each version.
Example 2: Google DeepMind's Gemini Robotics — VLA for Arms and Legs
Around the same time, Google DeepMind has been steadily advancing its own VLA family for robot arms and humanoids: Gemini Robotics. It traces back to RT-1 in 2022, moved through RT-2, and is now built on Gemini 2.0.
What's interesting about Google's approach is that it pairs a VLA model with a separate reasoning model (ER). Picture it like a strategist who plans the big picture (the ER model) working alongside someone who actually moves their hands and feet (the VLA model). Also notable is Gemini Robotics On-Device, which runs directly on the robot's hardware without needing a cloud connection, and can adapt to a new task with as few as 50 to 100 demonstrations. That offline capability is genuinely useful in places like disaster zones with no internet, or security-sensitive factory floors.
Why This Actually Matters
Older robots were programmed for exactly one task — "place this part at this exact spot" — and had to be retrained from scratch if the task changed even slightly. VLA models flip that: give the robot a new instruction in plain language, and it can often apply what it already knows to a task it's never explicitly practiced. It's a bit like asking a chef to swap apples for pears in a recipe — they don't need to relearn cooking from zero to improvise.
That said, the limits are real. Research has shown that in situations requiring very precise, hands-on guidance — like a person physically guiding a robot arm — today's VLA models still fall well short of human-level reliability. Self-driving VLA models like Alpamayo are also still closer to the validation and simulation stage than full real-world rollout. But the direction is clear: we're moving away from an era where every robot needed its own custom program, and toward one where a single capable AI model can drive many different kinds of robots and vehicles.
Next time, we'll look at "imitation learning" — how these VLA models are actually trained, and how robots can learn a task just by watching human demonstration videos.
Sources
- TechCrunch, "Nvidia launches Alpamayo, open AI models that allow autonomous vehicles to 'think like a human'" (Jan 5, 2026) — techcrunch.com
- NVIDIA Developer Blog, "Building Autonomous Vehicles That Reason with NVIDIA Alpamayo" (May 2026) — developer.nvidia.com
- NVIDIA, "Alpamayo: Open AI for Robotaxis and Autonomous Vehicles" — nvidia.com
- Google DeepMind, "Gemini Robotics" — deepmind.google
- Encord, "Gemini Robotics: Advancing Physical AI with Vision-Language-Action models" (Apr 2026) — encord.com
Comments
Post a Comment