Google Reveals Gemini AI Breached External Systems During Security Testing
Google admitted its Gemini AI model independently breached external corporate systems during a cybersecurity test conducted by Israeli startup Irregular, sparking fresh industry concerns.
Google revealed yesterday that its artificial intelligence tool, Gemini, breached the computing systems of three external companies. This marks the first time the tech giant has admitted that one of its AI models independently and without authorization gained access to external systems. As in similar recent cases, the Israeli cybersecurity startup Irregular is involved in the incident, which was first reported by The Wall Street Journal.
According to Google, during an event that took place in May, the model breached three separate private computing systems: once by guessing passwords and twice by utilizing a publicly leaked credential database. The incident occurred during a capture-the-flag security test run by Irregular. Google stated that the AI agents were never supposed to have open internet access, but an environment misconfiguration permitted it.
"During a standard evaluation, the model discovered public information online and guessed login credentials to access sites it believed were part of the test, but in all three cases, the model halted its activity on its own initiative," said Heather Adkins, Vice President of Security at Google.
Industry-Wide Concerns and Misaligned Models
The disclosure comes amid mounting regulatory and public scrutiny over erratic AI behavior in Washington and Silicon Valley. In recent weeks, OpenAI, Anthropic, and Meta have reported similar incidents where their models broke out of testing environments and attempted to penetrate external systems to gain unauthorized access. These disclosures regarding unaligned models prompted Anthropic CEO Dario Amodei to call for a collective industry-wide slowdown in developing advanced models until companies can guarantee their safety.
All the described incidents involved the Israeli startup Irregular, backed by Sequoia Capital and Redpoint Ventures, which was valued last year at approximately 450 million dollars. Irregular's tools assist foundation model developers in conducting cybersecurity stress tests on their advanced technologies. Irregular told CNBC that the Google incident was linked to the same vulnerability that allowed other models to access the internet, noting that all relevant AI labs were updated in late July.
