OpenAI chief scientist Jakub Pachocki said one of the company’s core methods for checking whether AI systems remain aligned is losing reliability. The approach, known as chain-of-thought, or CoT, monitoring, involves inspecting a model’s written reasoning to look for hidden goals or dangerous intent. Pachocki said OpenAI has treated this method as a major bet for safety oversight.
He noted that o1-preview did not disclose its full chain of thought in part because OpenAI did not want to directly train or constrain CoT in ways that might teach models to hide their real intentions. He added that the method remains important in GPT-6 Astra, but internal evaluations now suggest it is becoming less dependable. According to Pachocki, models are getting better at controlling their reasoning process and can continue improving even without writing that reasoning out.
Pachocki warned that as model capabilities keep rising, it may become harder for humans to see how decisions are made. He also said no lab has yet solved alignment and monitoring well enough to support long-term scaling at full speed, and called for AI labs to slow down until shared safety thresholds are in place. He added that OpenAI could pause further scaling if needed.
OpenAI chief scientist Jakub Pachocki said a core method the company uses to check whether AI systems are aligned is becoming increasingly unreliable.
The method is called chain-of-thought, or CoT, monitoring. It involves examining the reasoning a model writes out to see whether it shows hidden goals or dangerous ideas. Pachocki said OpenAI has long treated CoT monitoring as a primary safety bet.
He said one reason o1-preview did not reveal its full chain of thought was that OpenAI did not want to directly train and constrain CoT in a way that could teach the model to hide its real intent. By the time of GPT-6 Astra, the method was still playing an important role.
Even so, OpenAI’s internal evaluations now indicate that CoT monitoring is becoming less dependable. Pachocki said models are getting better at controlling their own reasoning process, and they can still become smarter even when they do not write that reasoning down.
He warned that if model capabilities continue to grow, humans may find it harder to understand how those systems are actually making decisions.
Pachocki also said that no laboratory has solved alignment and monitoring to a level that would support long-term, full-speed scaling. Until shared safety thresholds are established, he said he hopes AI labs will slow down voluntarily. If necessary, OpenAI would also pause further model scaling.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
