Gemini Got Into Three Real Companies’ Systems During a Misconfigured Irregular Test

Google has confirmed that Gemini models got into computer systems at three real companies during a security test in May, after the Wall Street Journal reported the incident. The models were taking part in a capture-the-flag exercise run by Irregular, an Israeli firm that evaluates AI models for major labs, and the intrusions happened only because the test environment was misconfigured.

The exercise asked the models to retrieve information from a fictional company inside a closed environment. That fictional company shared its name with a real business, and a configuration error gave the models internet access. In one run, Gemini guessed passwords until it got into a company’s online services. In the other two, it searched public software repositories and found login credentials that had been accidentally exposed, Ars Technica reported.

“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Heather Adkins, Google’s vice president of security engineering, said in a statement quoted by TechRadar. “In all three of these instances, the model stopped.”

Why Google didn’t announce it

Irregular didn’t tell Google about the May runs until late July, after other AI hacking incidents had become public. Google then notified the affected companies. It didn’t disclose the episode itself because the models stopped once they realized the systems were real, so the company didn’t treat it as a true case of model misalignment, according to Ars. “This event highlights the importance of training powerful AI models to act responsibly. In this case, the model acted appropriately,” Adkins said.

Irregular says the Gemini runs trace back to the same configuration problem behind the breakouts other labs disclosed this summer. “This is the same issue that was already reported and does not represent a materially separate incident,” a spokesperson said, via CNBC. “All relevant labs were notified in late July, and affected entities were contacted as part of the investigation.”

How it compares

Ars drew a clear line between this case and the earlier OpenAI-Hugging Face incident, in which OpenAI’s models used software exploits to reach information outside their test environment in pursuit of a better benchmark score. Gemini’s methods were basic by comparison: password guessing and reuse of credentials someone had left in public code. Ars still argued Google should have disclosed the intrusions once it knew about them.

For security teams, the practical lesson is an old one. Two of the three intrusions worked only because login credentials were sitting in public repositories, and the third worked because a password was guessable. The names of the three companies have not been released.