Google says Gemini accessed external company systems during a security test, with no actual harm reported

Google said its Gemini artificial intelligence model unexpectedly accessed the internet and entered the systems of three companies during a cybersecurity capability test, in what the company described as the first known case of a Google AI system autonomously carrying out that kind of action.

The incident took place in May during a test run by security firm Irregular. Gemini was taking part in a capture-the-flag cybersecurity exercise, but the names of the virtual companies used in the testing environment matched those of real companies. At the same time, the model unexpectedly obtained internet access, which led it into real corporate systems.

What happened in the three cases

Google said that in one case, the model tried passwords to enter a protected system. In the other two, it found credentials in public online repositories and attempted to access related systems. After recognizing that it had reached real company systems, the model stopped any further actions in each case.

Google's response

Google said the incident did not cause harm to the affected companies and was therefore not classified as a model loss-of-control event. The company also said its safety mechanisms helped the model stop in time. The affected companies were notified, and Google reported the matter to relevant regulators.

Part of a broader safety debate

According to The Wall Street Journal, the case resembles similar safety testing incidents previously disclosed by OpenAI and Anthropic. The episode has renewed attention on AI models' ability to carry out network operations on their own and on the safeguards designed to contain that behavior.