Conference paper Open Access

Investigating User Perception of Gender Bias in Image Search

Jahna Otterbacher; Alessandro Checco; Gianluca Demartini; Paul Clough

There is growing evidence that search engines produce results that are socially biased, reinforcing a view of the world that aligns with prevalent social stereotypes. One means to promote greater transparency of search algorithms - which are typically complex and proprietary - is to raise user awareness of biased result sets. However, to date, little is known concerning how users perceive bias in search results, and the degree to which their perceptions differ and/or might be predicted based on user attributes. One particular area of search that has recently gained attention, and forms the focus of this study, is image retrieval and gender bias. We conduct a controlled experiment via crowdsourcing using participants recruited from three countries to measure the extent to which workers perceive a given image results set to be subjective or objective. Demographic information about the workers, along with measures of sexism, are gathered and analysed to investigate whether (gender) biases in the image search results can be detected. Amongst other findings, the results confirm that sexist people are less likely to detect and report gender biases in image search results.

This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development. © Author. © ACM 2018. This is the accepted version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of SIGIR 2018, DOI: https://doi.org/10.1145/3209978.3210094, Jahna Otterbacher, Alessandro Checco, Gianluca Demartini, and Paul Clough, "Investigating user perception of gender bias in image search: the role of sexism".In SIGIR '18- The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. https://www.acm.org/publications/policies/copyright-policy
Files (530.0 kB)
Name Size
sp714-otterbacherA-hm.pdf
md5:8a2cdaa785129cafc716de3fdbdd0fa6
530.0 kB Download
  • Amy JC Cuddy, Susan T Fiske, and Peter Glick. 2008. Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology 40 (2008), 61–149.

  • Andrea E Abele and Susanne Bruckmüller. 2011. The bigger one of the "Big Two"? Preferential processing of communal information. Journal of Experimental Social Psychology 47, 5 (2011), 935–948.

  • Andrea E Abele, Mirjam Uchronski, Caterina Suitner, and Bogdan Wojciszke. 2008. Towards an operationalization of the fundamental dimensions of agency and communion: Trait content ratings in five countries considering valence and frequency of word occurrence. European Journal of Social Psychology 38, 7 (2008), 1202–1217.

  • Ayse Göker, Richard Butterworth, Andrew MacFarlane, Tanya S Ahmed, and Simone Stumpf. 2016. Expeditions through image jungles: the commercial use of image libraries in an online environment. Journal of Documentation 72, 1 (2016), 5–23.

  • Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In GoogleWe Trust: Users' Decisions on Rank, Position, and Relevance. Journal of Computer-Mediated Communication 12, 3 (2007), 801–823.

  • Gordon W Allport. 1954. The nature of prejudice. (1954).

  • Jahna Otterbacher, Jo Bates, and Paul Clough. 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6620–6631. https://doi.org/10.1145/3025453.3025727

  • Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 417–432.

  • Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 417–432.

  • Laurie A Rudman and Peter Glick. 2001. Prescriptive gender stereotypes and backlash toward agentic women. Journal of social issues 57, 4 (2001), 743–762.

  • Leif Azzopardi and Vishwa Vinay. 2008. Retrievability: an evaluation measure for higher order information access tasks. In Proceedings of the 17th ACM conference on Information and knowledge management. ACM, 561–570.

  • Matthew Kay, Cynthia Matuszek, and Sean A Munson. 2015. Unequal representation and gender stereotypes in image search results for occupations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3819–3828.

  • Myriam C Traub, Thaer Samar, Jacco Van Ossenbruggen, Jiyin He, Arjen de Vries, and Lynda Hardman. 2016. Querylog-based assessment of retrievability bias in a large newspaper corpus. In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. ACM, 7–16.

  • Peter Glick and Susan T Fiske. 1996. The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. Journal of personality and social psychology 70, 3 (1996), 491.

  • Peter Glick, Susan T Fiske, Antonio Mladinic, José L Saiz, Dominic Abrams, Barbara Masser, Bolanle Adetoun, Johnstone E Osagie, Adebowale Akande, Amos Alao, et al. 2000. Beyond prejudice as simple antipathy: hostile and benevolent sexism across cultures. Journal of personality and social psychology 79, 5 (2000), 763.

  • Robert Epstein and Ronald E. Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences 112, 33 (2015), E4512–E4521.

  • Ryen White. 2013. Beliefs and Biases in Web Search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '13). ACM, New York, NY, USA, 3–12.

  • Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology 29, 1 (2010), 24–54.

24
25
views
downloads
Views 24
Downloads 25
Data volume 13.3 MB
Unique views 22
Unique downloads 23

Share

Cite as