In 2020, GPT-3 shook the NLP field with a simple idea: show the model a few examples and it can handle a new task. A similar pattern is now surfacing in embodied AI, again centered on in-context learning, or ICL.
In mid-August, Skild AI released its robot foundation model S1. According to the public demo, a robot can watch a single task demonstration video and then attempt a previously unseen task without fine-tuning or any extra post-training. The task sequence can run for as long as 10 minutes and include dozens of steps. Skild compared ICL video prompts with traditional language-prompt VLA on both seen and unseen tasks. The reported result showed that when training data was limited to 1,000 hours, language prompts worked better, but ICL pulled ahead as data scale increased. On unseen tasks, adding video context improved performance by about 7x over language-only instructions.
One week before S1, Generalist AI introduced GEN-1.5 and also made one-shot learning a central feature. The company said a robot can learn a new task within seconds after seeing one demonstration, with no gradient update or fine-tuning, while supporting human-to-robot demonstration transfer, compositional generalization, and sim-to-real transfer.
Both developments point in the same direction: robots making use of longer multimodal context.
ICL is already a proven route in large language models. OpenAI laid it out in the 2020 GPT-3 paper Language Models are Few-Shot Learners, describing a capability in which the model gets no parameter update, no fine-tuning, and no backprop-based training, but can still infer and carry out a new task from examples placed inside the prompt during a single forward pass. Over the past three years, context windows in LLMs have expanded from 4K to 1M, letting models process a whole book or an entire codebase in one go. ICL has been one of the main engines behind that shift.
Embodied models face a different setting. They deal with visual observations, language instructions, and action sequences all at once, and real-world tasks are non-Markovian: what the robot should do next depends on what it did minutes earlier. Put more simply, the robot sees dozens of visual frames per second on top of proprioceptive state and action history, which raises the information load by orders of magnitude compared with language alone.
It was not until July 16, 2026 that RoboTTT, a long-context robotic policy model and training scheme co-authored by Fei-Fei Li, Jim Fan, Yuke Zhu and others, systematically brought the idea of context scaling into robot visuomotor policy. In August, Generalist AI and Skild AI followed with their own results. Neither company fully disclosed technical details, and both largely presented outcomes rather than a complete methods paper, but discussion around embodied ICL quickly intensified.
Whether this path can really hold up in embodied AI still depends on a list of unsolved technical problems. How should a robot understand context at this level of complexity? How should demonstrations, language, historical actions, and even human corrections enter the model? Those questions framed a long interview between QbitAI and Yuxiang Gao, founder and CEO of Chinese startup COCO Matrix.
Building ICL into the base layer of embodied intelligence
COCO Matrix was established in April 2026 and is focused on making in-context learning a foundational paradigm for embodied intelligence. Gao said the company was formally launched in April, but he and his co-founder had already agreed in January 2026, during detailed startup discussions, that ICL was the direction to back.
Gao is 30 years old. He came through the Xi'an Jiaotong University Youth Class, later studied for a PhD at Johns Hopkins University, and worked on human-computer interaction before leaving the program. He said that when GPT-3 first displayed context learning, he was still at JHU and had already begun asking whether the ability to solve a new task from examples alone, without parameter updates, could be transplanted into robotics.
Over the next five years, he left academia, returned to China, started companies, worked in industry, and then started again. During his time at Fourier, he led work that connected the full chain from teleoperation and data collection to model deployment for full-size humanoids. Between May and August 2025, he said, that work also produced a world model with a degree of generalization on a full-size biped humanoid at a cost of a little over 1 million.
COCO Matrix was formally founded in April this year to turn a long-running research question into a startup thesis. Unlike Generalist AI and Skild AI, Gao said, COCO Matrix wants to move ICL forward from a post-training capability into the pretraining stage.
Why embodied AI is looking for a new scaling route
For a long stretch, embodied AI followed the same broad scaling logic that worked in large models: gather more data and expect capabilities to emerge. Gao said the mood across the industry was initially optimistic. Once methods such as ACT and Diffusion Policy proved workable, many of the core problems in behavior cloning and imitation learning looked manageable. It was natural, he said, for the field to borrow the success logic of large language models and assume that once data crossed a certain threshold, general capability would emerge and support broad task adaptation.
That assumption fit the industry at the time, so the whole field moved to scale up data collection. The problem came later. Gao said that for a single task, around 200 trajectories can already fit the task well, and moving from demonstration to model adaptation for a new task can take roughly 18 hours. But even as more data was added, the broad generalization people hoped for did not show up.
The data itself also turned out to be hard to scale. Teleoperation data is uneven in quality, object and scene diversity is limited, and that makes it difficult to get the perceptual coverage the models need. Early VLA systems also borrowed pretraining stacks from LLMs and VLMs, but once they were pushed into the physical world, the existing visual representations struggled to cover all the layers of information a robot needs. Many systems ended up stitching together multiple vision modules.
Gao said that route still works in many settings, but over the past year and more, the field has gradually realized that simply stacking more data and more modules under the existing VLA paradigm will make further gains harder to achieve.
That has shifted the question. Beyond scaling models, datasets, and task coverage, can a robot keep learning after deployment? Can it pick up new abilities quickly from demonstrations, past experience, and interaction? In that framing, long context, memory, and adaptation become a new scaling axis.
Why embodied ICL is much harder than language ICL
Gao broke the difficulty into two broad dimensions: model architecture and I/O on one side, and the structural weakness of the data system on the other. He then narrowed that down to three technical bottlenecks.
The first is visual understanding. A robot needs different layers of information at different stages of a task and at different points in an action sequence. It has to know what an object is at the semantic level, but also its position, depth, outline, and spatial relation when action is about to happen. Gao said this is why early VLA systems often had to combine models such as DINO and CLIP. A single visual encoder has struggled to capture the full range of information robotics requires.
The second is multimodality. Language is nowhere near enough for a robot. Vision may carry most of the signal, but touch, sound, and proprioception also shape action. The issue is that those modalities do not come with similar amounts of data. Vision and language can still lean on massive pretraining corpora, while tactile, audio, and body-state data remain far smaller in scale.
The third is long context and memory. Robot tasks unfold continuously, so the system has to know what happened earlier and decide which parts of the past still matter now. That means compressing, filtering, retrieving, and preserving historical information over long horizons. Gao said the basic technologies that support long-term robot memory have only shown more visible progress in roughly the past six months.
Long context is not just stretching a fixed window from one minute to ten
QbitAI asked about the contrast between Skild AI's S1, which supports task demonstrations up to 10 minutes, and Generalist GEN-1.5, whose one-shot ICL demo videos run for only 3 to 12 seconds and complete adaptation inside a 30-second window. Gao said duration on its own is not the point. Information density matters more.
If a 10-minute video is mostly repetitive motion and contains little new task information, then it is not high-quality context for the model. What works better in their own testing, he said, is a full task demonstration from either a first-person or third-person human viewpoint. A person performs a new task once, the robot watches the complete sequence, and then carries out the task in the current scene. That kind of one-shot adaptation packs in the full logic of the task, the action sequence, and the scene state, making the context much richer.
That is why, in Gao's view, long context is not about forcing a fixed window from one minute up to ten. In many real robot tasks, the effective context may only be two or three minutes long. What matters is whether the model can keep filtering and compressing history as new context arrives. The pieces that remain useful might come from 10 minutes ago or even from an hour ago.
He described the target as something closer to a streaming mechanism. The goal is for robots to keep working and keep drawing on past experience, rather than exhausting a fixed context window and forgetting everything learned in the morning by the afternoon.
On the memory and GPU limits that come with long contexts, Gao said COCO Matrix is testing two main directions internally. One is to improve the model's understanding ability first. If the model genuinely understands the task at hand, it has a better shot at knowing which parts of history matter and which can be discarded. In that sense, memory is not only a compression problem. It is also a comprehension problem.
The other direction is to compress the large KV cache produced by long sequences while preserving real-time performance and limiting memory usage. Gao said there are established ideas worth borrowing from large language models and video generation, including linear attention, sparse attention, and routing-based selection of key information. In his framing, the real issue in long context is not how much information can be stuffed into the model, but whether useful historical experience can be retained and called back under finite model capacity.
ICL still needs data, especially for human teaching and correction
Gao stressed that ICL mainly reduces the need for exhaustive task-by-task coverage at massive scale. It does not remove the need for strong base data. The amount and diversity of that data still matter.
Before body-centric datasets such as UMI became more common, the field struggled to obtain high-quality data centered on the robot's own actions and embodiment perspective. As more of that data becomes available, models begin to gain solid baseline manipulation ability. ICL, in Gao's words, means a robot may not need a large dedicated dataset for every new task, but it still requires a strong base capability first.
The clearest gap today, he said, is data around human teaching, real-time correction, and collaborative work. If robots are expected to take live human guidance, get corrected when they make mistakes, and even work together with people, then the models need to see a large number of those interactions during training. Right now, the industry still lacks a mature definition and collection standard for that kind of data.
For example, when a human teaches a robot hand over hand, what information should be logged? At what point does a correction occur? How should human motion, language, and robot feedback be aligned? Gao said COCO Matrix plans to focus on defining that category of teaching and interaction data, including what to collect, how to collect it, and how to use it in ICL training.
COCO Matrix's thesis: strong understanding, light generation
Asked where the company stands now, Gao said progress has been faster than he originally expected. At the start, the team believed that getting the model's understanding ability into shape could take two to three months. After the core team came together, however, they obtained what he called key evidence validating the technical route in less than a month. That part of the roadmap is now largely in place, and the next stage is to scale and refine the data to push model performance higher.
The team's central belief is straightforward: whether a robot can generate strong actions later depends to a large extent on whether it has really understood the physical world first.
To get there, COCO Matrix is building what Gao called a conditional representation, or a conditional latent space. In conventional visual representation, one frame maps to one fixed embedding. No matter what the robot wants to do, the extracted information is largely the same. But real robot tasks do not work that way. The same scene can demand totally different information at different stages and under different goals.
Gao used the example of grasping a cup. At first, the key question is whether the object is a cup at all, which is semantic information. When the hand is actually reaching out, the relevant information shifts to position, depth, contour, and spatial relation. For that reason, he said, a robot-ready visual representation should not be determined only by what is seen, but also by what the robot is trying to do. Their model dynamically extracts different layers of information according to task conditions and action intent.
He said the unified model has already been validated across eight or nine visual tasks, including depth, segmentation, and human pose. On some tasks, the capability is already close to earlier models trained specifically for that single task. The next step is mainly to push toward current SOTA through more data.
Gao's follow-on point is that once understanding is strong enough, action generation can become lighter. Many robot models, he said, suffer because the front-end representation carries too much information that has nothing to do with action. Whether the tabletop is wood grain or marble does not help much in deciding whether to grasp a cup, but traditional embeddings will encode texture and material anyway. That forces the downstream action head to become very large so it can sift action-relevant features from the mixture.
COCO Matrix wants to isolate action-relevant information earlier, during the conditional representation stage. Gao said the team ran what he described as a relatively extreme experiment. Under the same training setup and compute budget, shrinking the action head to around 60M parameters still produced results better than a comparison system with an action head of roughly 1.1B parameters. That result strengthened the team's confidence in the strong-understanding, light-generation path.
No single definition yet for action-relevant information
When asked what exactly counts as action-relevant information, Gao said the industry still does not have a definition he finds fully unified or complete.
One common route today is the latent action model. Since many human videos and generated videos do not come with robot action labels, researchers train a latent variable to represent what changes between frames and treat that as a proxy for action. There are two typical approaches. One assumes that if the first frame can be transformed into the second, then the intermediate change must contain action information. The other uses optical flow and treats pixel-level movement in space as the action representation.
Gao said both approaches have issues. Optical flow is dense, but it is hard to tell whether the motion comes from the camera or from the robot or person in the scene. Human keypoints are more stable and less sensitive to camera motion, but they are sparse.
That is why COCO Matrix has not imposed a rigid hand-crafted definition of what the action representation should be. Instead, it puts multiple prediction tasks related to action into one conditional representation model, including hand keypoints, full-body keypoints, and optical flow, letting those signals complement one another. If new prediction tasks turn out to help action generation later, they can be added. Gao said the company wants progress to come from data and training objectives themselves, rather than from repeatedly complicating model structure.
Beyond success rate, the field may need to track time to first success
On one-shot capability, Gao said the clearest results so far are in simpler tasks. For standard grasping tasks, when the target is changed on the fly and one-shot adaptation is applied, completion rate can stay above 80%.
But the team is not looking at one-shot success rate alone. It also wants to know how far that performance still sits from a model that has gone through dedicated post-training for the same task. Ideally, after seeing one demonstration and adapting through context, the robot should come as close as possible to a task-specific trained model. Complex tasks have not yet been fully evaluated because the relevant models are still being trained.
Even while pushing one-shot learning, the company is also working on self-correction. Gao said people do not always succeed the first time after seeing a new task once, but they can tell what went wrong and adjust the next attempt. He wants robots to have the same trait. After a failed execution, the model should reuse that failure trajectory as context, determine what needs to be fixed, and try again.
In his view, self-correction is also a form of in-context learning. Context does not have to come from someone else's demonstration. It can come from the robot's own recent failure. The target is for the robot to eventually complete a task through a few rounds of autonomous retry and correction even if the first execution falls short.
That changes how performance might be measured. Gao said that if robots can keep trying and keep self-correcting, then asking only for one-shot success rate becomes less informative. A more useful metric may be average time to first success: after one demonstration, how long does it take before the robot genuinely learns the task for the first time?
That time corresponds to teaching cost and learning efficiency. For simple grasping, the team wants first-try success after one demonstration. For more complex tasks, such as folding clothes or even folding clothes into an elephant or a swan, humans may need several hours to learn the skill. Gao said he hopes a robot's learning cycle can gradually approach a normal human training cycle.
If that ability holds up, a new skill learned by one robot in one place could later be reused by other robots. Under that scenario, measuring robot intelligence may no longer mean counting only how many tasks it already knows, but also how long it takes to acquire a new one.
From self-driving to robotics, then out of Fourier to start again
Gao said he shifted from autonomous driving to robotics during his undergraduate years. At Xi'an Jiaotong University, he initially worked on self-driving and then moved into robotics around 2016. His reason was simple: robotics looked more complex, involving hardware, algorithms, and data at the same time, across a broader range of scenarios.
After leaving the PhD track at JHU, he entered industry and worked on humanoids, data, and model deployment. He joined Fourier in 2024. There, he was involved early in connecting teleoperation, data collection, and model deployment for full-size humanoids, and he also took part in some large-scale VLA training.
That experience gave him a direct look at the limits of VLA data scaling. The team later built an internal world-model-based policy for full-size humanoids and, according to Gao, verified that world models could in fact drive full-size humanoid robots while also showing advantages different from traditional VLA in some respects. Taken together, those observations convinced him it was time to reconsider the modeling paradigm itself.
His goal, he said, is to make ICL something much bigger than a feature. He wants it to become a change in training paradigm.
Asked why he chose to leave and start a company even after validating some of the direction inside Fourier, Gao gave two reasons. The first is that this is fundamentally a paradigm-innovation problem. Inside the old organization, he said, it would have been difficult to secure concentrated enough resources or build a sufficiently strong model team around the direction. The second is that his co-founder could not simply join the original team, and once they settled on the direction, he felt she was the only right partner.
He described the pairing as complementary. His co-founder has worked for a long time on robot generalization, learning ability, and ICL from the academic side, while he brings industrial experience across embodiment, data, model training, scenarios, and upstream and downstream resources. That combination led them to start the company together.
In terms of roles, the co-founder is mainly responsible for overall technical direction and core R&D, while Gao handles hardware links, scenarios, data, and compute. He said his past two years in the humanoid robotics industry exposed him to many practical problems in data scaling, teleoperation data, first-person-view data, and deployment, which now gives him a clearer sense of what kind of data fits what kind of model and what each data source should contribute during training.
He added that the company does not plan to become very large. A truly core model R&D team does not need many people, he said. What matters more is intellectual density.
Where the company name comes from
At the end of the interview, Gao explained the name behind COCO Matrix: COEXIST, COWORK, COLLABORATE, and COEVOLVE.
The original piece was published by the WeChat public account QbitAI, written by Heng Yu. The cited reference links include Gao's LinkedIn page, his Google Scholar page, the GPT-3 paper Language Models are Few-Shot Learners, the RoboTTT paper, and the COCO Matrix website.