Google Gemini for Workspace can be exploited to generate email summaries that appear legitimate but include malicious instructions or warnings that direct users to phishing sites without using attachments or direct links. As many users are likely to trust Gemini's output as part of Google Workspace functionality, chances are high for this alert to be considered a legitimate warning instead of a malicious injection. An example provided by Figueroa shows Gemini following the hidden instruction and includes a security warning about the user's Gmail password being compromised, along with a support phone number. BleepingComputer has contacted Google to ask about defenses that prevent or mitigate such attacks, and a spokesperson directed us to a Google blog post on security measures against prompt injection attacks. A prompt-injection attack on Google's Gemini model was disclosed through 0din, Mozilla's bug bounty program for generative AI tools, by researcher Marco Figueroa, GenAI Bug Bounty Programs Manager at Mozilla. If the recipient opens the email and asks Gemini to generate a summary of the email, Google's AI tool will parse the invisible directive and obey it. "We are constantly hardening our already robust defenses through red-teaming exercises that train our models to defend against these types of adversarial attacks," a Google spokesperson told BleepingComputer. Google has seen no evidence of incidents manipulating Gemini in the way demonstrated in Figueroa's report, the spokesperson said. Another approach is to implement a post-processing filter that scans Gemini output for urgent messages, URLs, or phone numbers, flagging the message for further review.
This Cyber News was published on www.bleepingcomputer.com. Publication date: Sun, 13 Jul 2025 14:40:14 +0000