Google sues Chinese network using Gemini for scams

Google has filed a civil lawsuit against a Chinese cybercrime group called Outsider Enterprise that allegedly used its Gemini AI to automate large-scale phishing scams targeting Android users.

Outsider Enterprise offered phishing-as-a-service via Telegram. The group provided instructions on using Gemini to generate nearly 300 scam website templates that imitate Google, YouTube, and government agencies such as New York’s E-ZPass. According to Google’s legal filing, these templates enabled individuals without technical skills to create fraudulent sites and text campaigns.

The operation resulted in more than 2.5 million text messages sent to Android users, with about 55,000 messages sent in a two-week period last month. Google has tracked 9,000 fake websites and 1 million URLs connected to the scam network.

Scam messages tricked users into revealing personal and banking data. Texts typically claimed account problems or package delivery issues. Victims who clicked the links reached Gemini-generated sites designed to look legitimate, where their data was stolen. Google’s filing does not estimate total losses, but its blog post states that hundreds of people have lost money.

Google collaborated with carriers and leveraged its own detection tools. The company worked with AT&T, Verizon, and T-Mobile to block many malicious texts. Its on-device scam detection feature in Google Messages stops 10 billion scam texts every month and likely reduced successful phishing attempts from this network.

This marks the first lawsuit targeting Gemini misuse for scams. Google has sued scammers before but not a group specifically alleged to be using Gemini. The company notes that Gemini’s security measures can conflict with its design to follow user instructions, enabling thousands of scammers to build fake websites.

In addition to the civil suit, Google is assisting the FBI’s cybercrime division with a parallel criminal investigation. However, the perpetrators’ identities remain unknown, and their location in China limits direct enforcement. Google can target fraudulent domains and Telegram accounts to disrupt operations, though the scams may simply evolve.

Google is pushing for new federal legislation on AI-assisted scams. The company renewed its support for seven potential laws, including the National Strategy for Combating Scams Act, the Strategic Task Force on Scam Prevention Act, and the AI Plan Act. Most call for task forces at federal agencies to counter AI-assisted scams and market manipulation, while one focuses on public education to spot malicious AI uses.