Microsoft Finds “Summarize with AI” Prompts Manipulating Chatbot Recommendations
Summary
Microsoft has discovered a new AI hijacking technique called "AI Recommendation Poisoning" where malicious actors manipulate the "Summarize with AI" button on websites. This technique mirrors classic search engine optimization (SEO) poisoning to influence AI chatbot recommendations. Legitimate businesses are also employing this method, raising concerns about the integrity of AI-generated summaries.
IFF Assessment
The discovery of "AI Recommendation Poisoning" presents a new challenge for defenders as it allows for the manipulation of AI-generated content.
Severity
Defender Context
This new technique presents a challenge for defenders who now need to monitor for potential manipulation of AI-generated content through poisoned recommendations. Defenders should implement checks to ensure the integrity of data fed into AI models, and monitor AI outputs for signs of manipulation. The increasing use of AI in various online services makes this a growing area of concern.