5548438
doi
10.1145/3485844
oai:zenodo.org:5548438
user-ai4media
GANDON, FABIEN
Université Côte d'Azur, Inria, CNRS, I3S, France
PRECIOSO, FREDERIC
Université Côte d'Azur, CNRS, Inria, I3S, France
Learning and Reasoning for Cultural Metadata Quality
BOBASHEVA, ANNA
Université Côte d'Azur, Inria, CNRS, I3S, France
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
artificial intelligence
knowledge representation
machine learning
reasoning
image recognition
knowledge graph
semantic Web
<p>This work combines semantic reasoning and machine learning to create tools that allow curators of the visual art collections to identify and correct the annotations of the artwork as well as to improve the relevance of the content-based search results in these collections. The research is based on the Joconde database maintained by French Ministry of Culture that contains illustrated artwork records from main French public and private museums representing archeological objects, decorative arts, fine arts, historical and scientific documents, etc. The Joconde database includes semantic metadata that describes properties of the artworks and their content. The developed methods create a data pipeline that processes metadata, trains a Convolutional Neural Network image classification model, makes prediction for the entire collection and expands the metadata to be the base for the SPARQL search queries. We developed a set of such queries to identify noise and silence in the human annotations and to search image content with results ranked according to the relevance of the objects quantified by the prediction score provided by the deep learning model. We also developed methods to discover new contextual relationships between the concepts in the metadata by analyzing the contrast between the concepts similarities in the Joconde’s semantic model and other vocabularies and we tried to improve the model prediction scores based on the semantic relations. Our results show that cross-fertilization between symbolic AI and machine learning can indeed provide the tools to address the challenges of the museum curators work describing the artwork pieces and searching for the relevant images. </p>
Zenodo
2021-10-01
info:eu-repo/semantics/article
5548437
user-ai4media
award_title=A European Excellence Centre for Media, Society and Democracy; award_number=951911; award_identifiers_scheme=url; award_identifiers_identifier=https://cordis.europa.eu/projects/951911; funder_id=00k4n6c32; funder_name=European Commission;
1647845656.647185
1697518
md5:e4c3a50195d053f8256d927a45357196
https://zenodo.org/records/5548438/files/Learning and Reasoning for Cultural Metadata Quality.revision2.pdf
public
ACM Journal on Computing and Cultural Heritage
2021-10-01