Understanding the Role of Visual Explanations in Human-AI Collaborations in Deepfake Image Detection

Deepfake images pose a growing societal challenge and global concerns. While artificial intelligence (AI) based technologies can support deepfake detection, the opaque decision-making of AI often limits effective human-AI collaboration. This study explores how visual explanations influence human decision-making, trust, and reliance in AI-based deepfake image detection. We conducted a mixed-subject study online with a representative sample of 381 UK residents. Findings show that visual explanations significantly increased human accuracy and trust but failed to improve appropriate reliance on AI. With more information from explanations, participants frequently over-relied on incorrect AI advice and under-relied on correct AI advice. This paper provides empirical evidence of non-experts' decision-making in detecting deepfake facial images with the presence of AI assistance and explanations. Our work contributes to a more nuanced understanding of human-AI collaboration in deepfake image detection.