Deep image annotation: making a difference in knowledge organization

Visual images are a powerful medium for communicating ideas and information, and they provide a valuable complement to textual content. A vast amount of information resides inside photographs, paintings, diagrams and drawings, which is comprehensible to the human eye but relatively inaccessible to machine queries. Well established techniques exist to support searching and browsing images based on the metadata that has been applied to whole images, but search and browse access to specific features within images is a relatively immature field.
This paper will present a detailed methodology for deep image annotation that has been co-developed by the author and a team of high-definition imagery and Linked Data practitioners. The methodology is based upon three key principles: (i) images must be rendered as multi-resolution pyramids to enable ultra high definition images to be explored via the web; (ii) all image and sub-image metadata must be expressed as Linked Data; and (iii) sub-image visual features, which may be single point coordinates or bound areas, must be directly addressable via HTTP-URIs.
The paper will discuss how Knowledge Organization Systems (KOS) can support search and browse access to the informational content deep within images. Among the methods discussed will be: using hierarchies to decompose and organize the spatial or thematic structure of an image; classifying and indexing visual features to the subjects and named entities they represent; linking from visual features to concepts and from concepts back to visual features; supporting faceted search and graph-based queries.