Prompt Injection Via Road Signs
Summary
The article discusses a new class of prompt-based attacks called CHAI (Command Hijacking against embodied AI) that exploit the multimodal language interpretation abilities of Large Visual-Language Models (LVLMs). CHAI embeds deceptive natural language instructions, such as misleading road signs, in visual input to manipulate AI agents. The research demonstrates CHAI's effectiveness against LVLM agents in drone emergency landing, autonomous driving, aerial object tracking, and on a real robotic vehicle.
IFF Assessment
This research highlights a novel attack vector against AI systems, demonstrating how attackers can manipulate them with deceptive visual prompts, which is bad news for defenders.
Severity
Defender Context
This research highlights the emerging threat of prompt injection attacks against AI-powered systems in physical environments. Defenders should monitor the visual inputs and decision-making processes of their AI systems for signs of manipulation and develop defenses that go beyond traditional adversarial robustness. The trend towards integrating AI in safety-critical systems necessitates robust validation and security testing against these novel threats.