Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures

Akhila Yerukola, Saadia Gabriel, Nanyun Peng, and Maarten Sap, in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , 2025 .

Download the full text

Abstract

Non-verbal communication, including gestures and poses, varies significantly across cultures, sometimes leading to misinterpretations with serious social and diplomatic consequences. As AI systems become more integrated into global applications, there is a critical need to ensure that they do not inadvertently perpetuate cultural offenses. To this end, we introduce Multi-Cultural Set of Inappropriate Gestures and Nonverbal Signs (MC-SIGNS), a dataset of 288 gesture-country pairs annotated for offensiveness, cultural significance, and contextual factors across 25 gestures and 85 countries. Through systematic evaluation using MC-SIGNS, we uncover critical limitations: text-to-image (T2I) systems exhibit strong US-centric biases, performing better at detecting offensive gestures in US contexts than in non-US ones; large language models (LLMs) tend to over-flag gestures as offensive; and vision-language models (VLMs) default to US-based interpretations, frequently suggesting culturally inappropriate gestures. These findings highlight the urgent need for culturally-aware AI safety mechanisms to ensure equitable global deployment of AI technologies.

Bib Entry

@inproceedings{yerukola2025gesture,
  title = { Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures },
  author = {Yerukola, Akhila and Gabriel, Saadia and Peng, Nanyun and Sap, Maarten},
  year = { 2025 },
  booktitle = { Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) }
}