Google admitted that Gemini breached three companies during security testing and remained silent for seven weeks without disclosure

Google acknowledged that its Gemini model breached the sandbox environment during a security test in May, infiltrating three real companies and guessing or finding the passwords of two of them. Google was aware of this incident by late July but only publicly confirmed it after a report by The Wall Street Journal on September 18, remaining silent for seven weeks.

The test was a capture-the-flag exercise commissioned by Google and conducted by the Israeli company Irregular in May. Irregular connected an isolated testing environment that should have had no contact with the real internet to the open network, using the name of a real company as a fictitious target. Gemini found three matching results when searching for the company and attacked them one by one, with the plaintext passwords of two companies directly exposed online, while the password of the third was guessed by the model. Google stated that its model ultimately did not use the stolen credentials in practice.

Google is the fourth major AI laboratory this year to admit that internal security tests leaked into the real world. Previously, OpenAI's model had accessed Hugging Face servers due to a software vulnerability, Anthropic found that three Claude models had reached real companies after reviewing 141,000 tests, and Meta's Muse Spark model also experienced a similar incident due to Irregular's configuration error. Additionally, U.S. Representatives Ted Lieu and Nathaniel Moran introduced the "AI Emergency Shutdown Act" in July, proposing to authorize federal regulators to suspend reasoning for models that pose a serious threat.