Human Trust of AI Agents
Summary
Research indicates that humans expect rationality and cooperation from LLM opponents in strategic games, leading them to choose significantly lower numbers and favor 'zero' Nash-equilibrium choices when playing against LLMs compared to human opponents. This behavior is particularly pronounced among subjects with high strategic reasoning ability, who rationalize their strategies by attributing reasoning ability and even cooperation to LLMs.
IFF Assessment
The findings suggest that adversaries could exploit human trust and expectations of LLM rationality and cooperation, potentially leading to manipulation or unpredictable outcomes in security-sensitive strategic interactions.
Defender Context
Defenders should be aware that human reliance on perceived LLM rationality and cooperation could be exploited. This highlights the importance of understanding how humans interact with AI in critical systems, as adversaries might leverage these expectations for social engineering or to manipulate decision-making processes.