MIT Technology Review said in a commentary article that the AI industry is leaning too heavily on model refusal behavior as a safeguard against misuse, arguing that this kind of probability-based control is not dependable. The piece compared the approach to putting a machine gun in every car while hiding the trigger: the dangerous capability remains inside the system, while developers try to train the model not to act on malicious requests. According to the article, refusal mechanisms can fail, and malicious users have already found ways to get around restrictions. It added that some users are trying to use the most advanced AI systems to optimize biological pathogens or build autonomous drone swarms, raising the possibility of a global catastrophe in the future. The article also questioned the practice of AI companies drawing, on their own and in secret, the line between requests a model should obey and those it must refuse, noting that some queries involving harmful knowledge may serve legitimate research or defensive purposes. It cited OpenAI board member Zico Kolter as saying that setting that boundary is a huge challenge.
According to Techub News, MIT Technology Review said in a commentary article that the AI industry is relying too heavily on model refusal behavior to prevent misuse, while that probability-based mechanism is not reliable.
The article argued that keeping a dangerous "trigger" inside a model while training it to refuse malicious instructions is like equipping every car with a machine gun but hiding the trigger.
It said that although models are trained to reject a large number of harmful prompts, refusal systems can fail, and malicious users have already been able to bypass restrictions. Some users are trying to use the most advanced AI systems to optimize biological pathogens or build autonomous drone swarms, which the article said could lead to a global catastrophe in the future.
The piece also said there is a problem with AI companies unilaterally and secretly drawing the line between what a model should comply with and what it must refuse, because some requests for harmful knowledge may come from legitimate research or defensive needs.
MIT Technology Review cited OpenAI board member Zico Kolter as saying that drawing that boundary is a huge challenge.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
