Encord is testing a new way to build training data for physical AI by recording the brain activity of robot trainers as they work, aiming to capture the exact moment a human recognizes a mistake, hesitates, or makes a judgment call.
The California startup argues that the biggest bottleneck in physical AI is not chips or algorithms, but the lack of enough usable training data. Its latest experiment takes place in a warehouse in San Leandro, where a robot trainer, referred to as a “pilot,” carefully removes a block from a wobbling tower while wearing a helmet fitted with both a camera and EEG sensors.
Camera-equipped helmets are already familiar in robot data collection. What makes this setup different is the added EEG layer, which records brain activity in real time as the operator evaluates the task and reacts to problems.
Encord previously worked mainly with computer vision companies that needed image annotation and model evaluation. The company now says it has run into a more basic issue: in many cases, customers do not have enough training data to annotate in the first place. That has pushed Encord beyond labeling services and into creating datasets itself.
From labeling data to making it
“This data just does not exist,” said Vineeth Velmurugan, Encord’s head of robotics learning. Velmurugan previously worked at OpenAI’s robotics lab and at warehouse automation company Berkshire Grey, and he described real-world robotics data as deeply scarce.
He put a number on the scale of the shortfall. To reproduce in robotics the kind of progress generative AI has delivered in chatbots, the required training data would be roughly five times the size of YouTube’s entire video library. In his view, that is why data generation is turning into a business of its own rather than remaining only a research problem.
Today, robot training mostly depends on two types of data. One comes from workers wearing cameras to capture first-person video. The other comes from teleoperating robots and collecting action data directly. Encord is already doing both, gathering first-person data across multiple factories globally while also operating robots in its San Leandro warehouse.
Still, the company sees the same blind spot in both methods. They can record what happened, but not the operator’s cognitive state in the moment. They do not show when a person became confused, which step caused hesitation, or whether a motion came from instinct rather than explicit calculation.
What EEG is meant to add
To address that gap, Encord partnered with German neuroscience startup Zander Labs. The company has developed an EEG-equipped helmet designed to infer mental states from brain signals, including error perception, execution intent, and surprise response.
In the block tower test, when pilot Andrew Ceja removes a key piece and the structure begins to sway, his brain produces a measurable signal associated with recognizing an error. Zander Labs neuroscientist Lucas Gehrke said changes in the strength of those signals could help model developers decide when a system should deploy its most compute-intensive model.
Put differently, the goal is to let AI reserve full computing power for the moments that matter most rather than using maximum resources all the time.
“We are exploring a solution at the frontier of the physical AI data bottleneck,” Velmurugan said.
An early-stage bet
The Encord-Zander collaboration is still in the experimental phase. The near-term plan is to build an initial dataset labeled with EEG signals, feed it into customer robot models, and test whether the brainwave data improves performance before deciding whether to scale the approach.
That makes this a straightforward early-stage technology bet. The source text notes that the cost of EEG sensors, wearing comfort, and signal stability in noisy warehouse environments all remain open engineering issues.
Even so, Encord appears committed to the direction. Velmurugan said the company plans to recruit 10 more pilots this year to expand collection of EEG-labeled data.
A new data layer for physical AI
If the method works, brainwave signals could become a new layer in the robot training stack. The value would not be limited to what the system sees. It could also include what the human operator was thinking during execution.
While generative AI has rapidly improved chatbots, physical AI is still constrained by the shortage of basic training data. Encord’s EEG experiment remains unproven, but it points to a practical question: when real-world data is in short supply, some of the missing signal may be found in the human brain.
This article is based on TechCrunch’s report, “Are brain waves the next unlock for physical AI?”, as cited in the source text and adapted by BlockTempo editors.