OpenAI chief scientist Jakub Pachocki has pushed back on outside interpretations of Astra's recurrent architecture. He said that at OpenAI, the current frontier models, including Astra, have a compute-graph depth at most 2 times that of GPT-4 — not a sudden jump by several orders of magnitude. The remarks are aimed at an earlier report from The Information. That report said Astra relies on recurrent depth, meaning the same set of Transformer layers is applied repeatedly. Under that design, some reasoning can happen inside the model, so not everything has to be written out as a chain of thought. This also raised concerns about whether the model will become increasingly difficult to monitor. Pachocki conceded that CoT monitoring is fragile and is evolving in a harder-to-monitor direction. But he made clear the reason does not depend on changes to the model's architecture. OpenAI has kept and used CoT monitoring since its first generation of reasoning models. The company still lists strengthening that capability as a core research goal.
OpenAI chief scientist Jakub Pachocki pushed back on readings of Astra's recurrent architecture. He framed the debate in numbers: frontier models at OpenAI today, Astra included, sit at no more than twice the compute-graph depth of GPT-4. That is not an orders-of-magnitude leap.
The comment targets a report from The Information. That story said Astra uses recurrent depth, which runs the same group of Transformer layers again and again. With that arrangement, some reasoning can happen inside the model rather than being spelled out as a chain of thought. It also fed concerns about whether the model would become harder to monitor.
Pachocki conceded the underlying worry is real. CoT monitoring is fragile, he acknowledged, and it is heading in a direction that makes supervision tougher. He disputed, however, the causal link to architecture changes. OpenAI has relied on CoT monitoring since its first generation of reasoning models, and strengthening that capability remains a core research target, he said.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
