Published March 1, 2020 | Version Accepted pre-print
Journal article Open

Image annotation: the effects of content, lexicon and annotation method

  • 1. Research Centre on Interactive Media, Smart Systems and Emerging Technologies (RISE), Nicosia, Cyprus
  • 2. Department of Communication and Internet Studies, Cyprus University of Technology, Limassol, Cyprus


Image annotation is the process of assigning metadata to images, allowing effective retrieval by text-based search techniques. Despite the lots of eorts in automatic multimedia analysis, automatic semantic annotation of multimedia is still inefficient due to the problems in modelling high level semantic terms. In this paper we examine the factors affecting the quality of annotations collected through crowdsourcing platforms. An image dataset was manually annotated utilizing: (i) a vocabulary consists of pre-selected set of keywords,(ii) an hierarchical vocabulary, and (iii) free keywords. The results show that the annotation quality is affected by the image content itself and the used lexicon. As we expected while annotation using the hierarchical vocabulary is more representative, the use of free keywords leads to increased invalid annotation. Finally it is shown that images requiring annotations that are not directly
related to their content (i.e. annotation using abstract concepts), lead to accrue annotator inconsistency revealing in that way the diculty in annotating such kind of images is not limited to automatic annotation, but it is generic problem of annotation.


This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.



Files (1.4 MB)

Name Size Download all
1.4 MB Preview Download

Additional details


RISE – Research Center on Interactive Media, Smart System and Emerging Technologies 739578
European Commission