CYENS Centre of Excellence
Published July 1, 2018 | Version Accepted pre-print
Conference paper Open

Investigating User Perception of Gender Bias in Image Search

Creators

  • 1. Open University of Cyprus
  • 2. University of Sheffield
  • 3. University of Queensland

Description

There is growing evidence that search engines produce results that are socially biased, reinforcing a view of the world that aligns with prevalent social stereotypes. One means to promote greater transparency of search algorithms - which are typically complex and proprietary - is to raise user awareness of biased result sets. However, to date, little is known concerning how users perceive bias in search results, and the degree to which their perceptions differ and/or might be predicted based on user attributes. One particular area of search that has recently gained attention, and forms the focus of this study, is image retrieval and gender bias. We conduct a controlled experiment via crowdsourcing using participants recruited from three countries to measure the extent to which workers perceive a given image results set to be subjective or objective. Demographic information about the workers, along with measures of sexism, are gathered and analysed to investigate whether (gender) biases in the image search results can be detected. Amongst other findings, the results confirm that sexist people are less likely to detect and report gender biases in image search results.

Notes

This work has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and under Grant Agreement No 732328 and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development. © Author. © ACM 2018. This is the accepted version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of SIGIR 2018, DOI: https://doi.org/10.1145/3209978.3210094, Jahna Otterbacher, Alessandro Checco, Gianluca Demartini, and Paul Clough, "Investigating user perception of gender bias in image search: the role of sexism".In SIGIR '18- The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. https://www.acm.org/publications/policies/copyright-policy

Files

sp714-otterbacherA-hm.pdf

Files (530.0 kB)

Name Size Download all
md5:8a2cdaa785129cafc716de3fdbdd0fa6
530.0 kB Preview Download

Additional details

Funding

European Commission
RISE – Research Center on Interactive Media, Smart System and Emerging Technologies 739578
European Commission
FashionBrain – Understanding Europe’s Fashion Data Universe 732328

References

  • Andrea E Abele and Susanne Bruckmüller. 2011. The bigger one of the "Big Two"? Preferential processing of communal information. Journal of Experimental Social Psychology 47, 5 (2011), 935–948.
  • Andrea E Abele, Mirjam Uchronski, Caterina Suitner, and Bogdan Wojciszke. 2008. Towards an operationalization of the fundamental dimensions of agency and communion: Trait content ratings in five countries considering valence and frequency of word occurrence. European Journal of Social Psychology 38, 7 (2008), 1202–1217.
  • Gordon W Allport. 1954. The nature of prejudice. (1954).
  • Leif Azzopardi and Vishwa Vinay. 2008. Retrievability: an evaluation measure for higher order information access tasks. In Proceedings of the 17th ACM conference on Information and knowledge management. ACM, 561–570.
  • Amy JC Cuddy, Susan T Fiske, and Peter Glick. 2008. Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology 40 (2008), 61–149.
  • Robert Epstein and Ronald E. Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences 112, 33 (2015), E4512–E4521.
  • Peter Glick and Susan T Fiske. 1996. The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. Journal of personality and social psychology 70, 3 (1996), 491.
  • Peter Glick, Susan T Fiske, Antonio Mladinic, José L Saiz, Dominic Abrams, Barbara Masser, Bolanle Adetoun, Johnstone E Osagie, Adebowale Akande, Amos Alao, et al. 2000. Beyond prejudice as simple antipathy: hostile and benevolent sexism across cultures. Journal of personality and social psychology 79, 5 (2000), 763.
  • Ayse Göker, Richard Butterworth, Andrew MacFarlane, Tanya S Ahmed, and Simone Stumpf. 2016. Expeditions through image jungles: the commercial use of image libraries in an online environment. Journal of Documentation 72, 1 (2016), 5–23.
  • Matthew Kay, Cynthia Matuszek, and Sean A Munson. 2015. Unequal representation and gender stereotypes in image search results for occupations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3819–3828.
  • Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 417–432.
  • Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 417–432.
  • Jahna Otterbacher, Jo Bates, and Paul Clough. 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6620–6631. https://doi.org/10.1145/3025453.3025727
  • Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In GoogleWe Trust: Users' Decisions on Rank, Position, and Relevance. Journal of Computer-Mediated Communication 12, 3 (2007), 801–823.
  • Laurie A Rudman and Peter Glick. 2001. Prescriptive gender stereotypes and backlash toward agentic women. Journal of social issues 57, 4 (2001), 743–762.
  • Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology 29, 1 (2010), 24–54.
  • Myriam C Traub, Thaer Samar, Jacco Van Ossenbruggen, Jiyin He, Arjen de Vries, and Lynda Hardman. 2016. Querylog-based assessment of retrievability bias in a large newspaper corpus. In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. ACM, 7–16.
  • Ryen White. 2013. Beliefs and Biases in Web Search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '13). ACM, New York, NY, USA, 3–12.