Google confirms it: Gemini breached three companies’ systems, then halted its own test
The AI found publicly available credentials and even managed to guess passwords to gain access to real systems, which it mistook for part of a security exercise.
Google offices around the world (File photo)
The artificial intelligence model Gemini, developed by Google, gained unauthorized access to the computer systems of three companies during a cybersecurity test conducted in May, marking the first known instance in which one of the company's AI systems autonomously crossed the boundaries of a testing environment.
The incidents were initially revealed by The Wall Street Journal and later confirmed by Google. The tests were conducted by Irregular, a company specializing in evaluating the cybersecurity capabilities of advanced artificial intelligence models.
During the exercise, Gemini was supposed to work with scenarios specifically designed to measure its ability to find vulnerabilities. However, the model gained access to the internet and ended up interacting with systems belonging to real companies.
"During a standard evaluation, the model found public information on the internet and guessed credentials to access websites it thought were part of the test," Heather Adkins, vice president of Security Engineering at Google, explained to AFP.
Gemini even managed to guess passwords
According to The Wall Street Journal, in one of the incidents, Gemini tried different passwords until it managed to gain access to a protected system. In the other two cases, it found credentials exposed in public repositories and used them to access external systems.
Google maintains that, once Gemini identified that it was interacting with real organizations rather than simulated test systems, it halted its activity on its own in all three cases.
The company also stated that no damage was done to the affected systems.
"We made sure the three entities were informed and worked with our testing partner on the changes they have already made to their processes," Adkins said.
Irregular notified Google and other affected labs in late July. The company stated that it took immediate action and that the known issues related to that testing environment were corrected.
Politics
California proposes 'emergency shut-off switch' for advanced artificial intelligence models
Diane Hernández
A flaw in the testing environment
The company explained that the various known incidents stemmed from a single underlying issue in one of its evaluation scenarios, which allowed the models to interact with real systems when the tests were supposed to take place in a controlled environment. According to Irregular, the flaw had already been fixed by the time the initial findings were made public.
The Gemini case is part of a series of incidents detected during evaluations conducted by Irregular on models developed by companies such as OpenAI, Anthropic and Meta, which have also exhibited behavior outside their initially intended environments.
These incidents have reignited the debate in the tech industry regarding the necessary security mechanisms as artificial intelligence systems gain greater capabilities to use tools, browse the internet and perform actions autonomously.
LATEST
