Published October 26, 2020 | Version Published
Conference paper Open

OpenTag: Understanding Human Perceptions of Image Tagging Algorithms

  • 1. Research Centre on Interactive Media, Smart Systems and Emerging Technologies (Nicosia, CYPRUS)
  • 2. Research Centre on Interactive Media, Smart Systems and Emerging Technologies (Nicosia, CYPRUS)and Cyprus Center for Algorithmic Transparency, Open University of Cyprus (Latsia, CYPRUS)
  • 3. Research Centre on Interactive Media, Smart Systems and Emerging Technologies (Nicosia, CYPRUS) and Cyprus Center for Algorithmic Transparency, Open University of Cyprus (Latsia, CYPRUS)

Description

Image Tagging Algorithms (ITAs) are extensively used in our information ecosystem, from facilitating the retrieval of images in social platforms to learning about users and their preferences. However, audits performed on ITAs have demonstrated that their behaviors often exhibit social biases, especially when analyzing images depicting people. We present OpenTag, a platform that fuses the auditing process with a crowdsourcing approach. Users can upload an image, which is then analyzed by various ITAs, resulting in multiple sets of descriptive tags. With OpenTag, the user can observe and compare the output of multiple ITAs simultaneously, while researchers can study the manner in which users perceive this output. Finally, using the collected data, further audits can be performed on ITAs.

Notes

This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy.

Files

AAAI_HCOMP_2020__OpenTag_Demo.pdf

Files (369.4 kB)

Name Size Download all
md5:761352ed3e5a85a01b8ae864382b131a
369.4 kB Preview Download

Additional details

Funding

CyCAT – Cyprus Center for Algorithmic Transparency 810105
European Commission
RISE – Research Center on Interactive Media, Smart System and Emerging Technologies 739578
European Commission