Study finds GPT-6-Astra nears perfect scores in simulation but still struggles on real robots
A joint research team led by SenseTime, together with Xiaoxiao Robot, Ropedia, Nanyang Technological University and the University of Tokyo, has released a technical report examining GPT-6-Astra’s embodied intelligence across 12 simulation benchmarks and eight real-world robot platforms. The report says the model has largely crossed the threshold of embodied capability, posting standout results on several established benchmarks, including 100% on LIBERO-Long++, 99.3% on LIBERO-Pro, 98.8% on SIMPLE and 98.9% on LIBERO-Ultra-Long. Yet the same model still ran into repeated failures once it was asked to operate physical robots in real environments. The study attributes the strong benchmark performance in part to in-context learning with frozen demonstration packages and to much larger execution budgets, in some cases allowing up to 10 times more action steps. That helped on tasks such as RoboDojo, where success rose from 27.6% to 81.9%, but it did not solve harder long-horizon settings such as BEHAVIOR and LHPR-VLN. Researchers also documented costly inference loops, physical misjudgments, irreversible mistakes, safety interruptions and long decision delays. In one real-world plug insertion test spanning 11 attempts across multiple robots, GPT-6-Astra failed every time. The team argues that benchmark scores alone no longer capture what matters most for deployment: reliability, recovery, time cost and whether a robot can finish tasks without creating new problems.
