ZERO-SHOT OPEN SET OBJECT DETECTION IN MUSIC ALBUM COVERS

We applied zero-shot object detection to music album cover art enabling a quantitative iconographic analysis of visual Pop music culture. For this we constructed and partly manually annotated a dataset of decades of USA Billboard chart music. We first used automatic image captioning to yield candidate object classes. Next we input these object classes and the album cover images to a pretrained zero-shot object classification model, allowing detection of objects and their classes without any re-training. We confirmed accuracy of our approach on a manually labeled sub sample of our dataset. Our results give an overview of what different objects are depicted on album covers belonging to different music genres and types of artists.