At the Cambridge, Massachusetts offices of Generalist AI, robotics is taking a leap that resembles human development more than traditional programming. Robotic arms are performing chores like stacking cups and sorting blocks, showing a startling ability to learn almost instantly by simply watching short instructional videos.
Improvisation and Physical Intelligence
The most striking feature of these machines is their capacity to improvise. In one demonstration, a robot instructed to sweep an object used a dustpan as a makeshift brush when the brush was removed from the scene. In another instance, a robot chose to sweep up items with a banana that was placed in front of it. This "physical intelligence"—an intuitive grasp of the world's physics—allows robots to adapt to new scenarios without requiring thousands of training examples for every specific task.
The Generalist AI Approach
Founded by alumni of Google DeepMind and Boston Dynamics, the startup follows a distinct path from its competitors. Rather than relying on open-source language models, Pete Florence, Andrew Barry, and Andy Zeng are building their AI models from scratch. Training involves specialized camera-equipped gloves used by humans—including workers in Mexico—to perform tasks, generating a massive dataset of physical interaction data.
Challenges and Potential
Despite the impressive progress, the technology is not yet ready for mass deployment. The current success rate for task completion stands at approximately 59%, whereas commercial applications typically require upwards of 99%. However, the ability of these machines to transfer knowledge from one scenario to another suggests we are nearing an era where robots can be rapidly deployed in sectors like manufacturing.
