Researchers Detect AI Disinformation Campaigns Poisoning ChatGPT, Gemini, and Google AI Overview

Security researchers have identified a widespread AI disinformation attack where malicious actors manipulate the answers provided by ChatGPT, Gemini, and Google AI Overview. By flooding the web with Generative Engine Optimization (GEO)-optimized content, attackers inject fraudulent phone numbers, email addresses, and login pages into AI responses for hundreds of major brands.

Scope and Targets

The investigation detected 374 companies affected by these campaigns, including Fortune 100 entities such as Delta, Lufthansa, Chase, Bank of America, Airbnb, and TripAdvisor. The attacks utilize tens of thousands of malicious pages across various platforms, exploiting the fact that AI models rely heavily on web content for factual grounding. The researchers noted that while the attacks are real and happening in the wild, many incidents were statistical, meaning the AI did not cite the fake data every time the query was run.

Attack Methodology

Attackers employ specific techniques to ensure their fraudulent content is cited by LLMs:

• GEO Optimization: Content includes repetitive fake numbers, Q&A formats, and urgency markers like "24/7" or "updated 2026" to boost relevance scores.
• Obfuscation: Phone numbers are hidden using spaces, dots, emojis, or Unicode characters to bypass spam filters while remaining semantically readable to AI tokenizers.
• Authority Spoofing: Malicious posts are embedded in trusted or high-authority sites, including .edu domains, government sites, GitHub Pages, and platforms like LeetCode and Medium.
• Scale: The campaigns are automated, generating hundreds of posts per day per company, overwhelming traditional takedown mechanisms. Even after removal, content persists via archives like archive.org, which remain indexed by search engines.

Industry Response and Vulnerability

The researchers disclosed the findings to both the targeted companies and AI providers. Google classified the issue as out of scope for its Vulnerability Reward Program (VRP), stating that AI-generated misinformation is not covered. OpenAI closed the report as "not reproducible," arguing that submissions must demonstrate proven impact rather than theoretical attacks, despite the presence of live examples. This highlights a significant gap in current security frameworks, as the attack manipulates the output layer rather than breaching the AI vendors' internal systems.