Are brain waves the next unlock for physical AI?

In a San Leandro, California, warehouse, a game of Jenga is being used to explore a new possible ingredient for robots that work in the physical world: signals from the human brain. The experiment is being run by Encord, a company known for tools that organize and prepare data for artificial-intelligence training. Its premise is that a person’s movements and point of view may not be the only useful record of how a task is completed.

During the demonstration, Encord trainer Andrew Ceja removes pieces from an unstable wooden tower. He wears a head-mounted device that records the scene in front of him, a familiar technique for gathering examples that can later guide a machine. But the equipment also monitors his brain activity while he judges which block to touch and how to extract it without bringing the structure down.

The exercise turns an ordinary tabletop challenge into a test of what could be called richer supervision. Video can show a hand approaching a block, the tower shifting, and the final outcome. It cannot directly show the split-second assessment behind a cautious motion: whether the person noticed a risk, changed course, or committed to a choice. Encord is investigating whether neural measurements could add useful context to those visible actions.

That question sits within a broader push to build physical AI, systems intended to perceive and act beyond a screen. Humanoid machines and automated warehouse equipment must deal with objects that vary in position, weight, texture and stability. Their training material therefore needs to reflect real environments, not merely clean digital examples. A task such as dismantling a Jenga tower highlights why: the result depends on perception, fine motor control and continual adjustment.

Encord is among a relatively small but expanding group of startups arguing that data availability, rather than only model design, could become the central obstacle for robotics. The company’s bet is that builders of robots will need far more recordings of people performing hands-on work than currently exist. Instead of limiting its role to cataloguing datasets supplied by customers, Encord is seeking to create the missing material itself.

For robotics developers, that approach could shift attention toward the quality and variety of collection. A single camera angle may miss an operator’s line of sight. Sparse labels may fail to identify the key moment when a task becomes difficult. Combining several viewpoints with detailed annotation can provide a fuller account of what happened; brain-wave readings represent a still more ambitious layer, intended to capture information that is not obvious from an image alone.

The idea also comes with important unanswered questions. A brain signal recorded while one person plays Jenga is not automatically a clear instruction that another person, or a robot, can follow. The value of those measurements will depend on whether they can be reliably matched to actions and outcomes, and whether they improve training compared with less intrusive methods. The demonstration is evidence of an avenue being explored, not proof that neural data has become a standard robotics input.

There are practical questions as well. Producing real-world training data requires people, equipment, careful task design and systems for reviewing what was captured. Adding multiple cameras, dense labeling and physiological sensors can make each example more complex to collect. Yet the cost may be weighed against the difficulty of teaching machines to handle the messy, variable conditions found in warehouses and other workspaces.

What happens next will be determined less by the novelty of a headset than by results in training. Developers will be watching for evidence that data gathered in this way helps robots learn tasks more effectively, adapt to unfamiliar situations or make fewer physical mistakes. For now, the San Leandro experiment illustrates a widening view of the data problem: teaching a machine to act may require recording not just what humans do, but some indication of how they arrive at a decision.

Source: TechCrunch