Eindrapportage Automatische beeldherkenning als instrument voor museumcollecties: Innovatieproject in de Nederlandse natuurhistorische musea

AI for registration and publication of museum collections
With the project Automatic recognition as a tool for museum collections, Naturalis set out to investigate how Artificial Intelligence (AI), and specifically automatic image recognition, can contribute to the registration and publication of museum collections. This was done by working closely with various project partners and stakeholders to develop a number of image recognition models for various types of specimens and artifacts. These recognition models were made accessible to man and machine through a web-based interface and an API: https://museum.identify.biodiversityanalysis.nl/.

Disappearing identification expertise
Objects (specimens) in natural history collections are traditionally identified by taxonomists. Being able to name and interpret species is a crucial prerequisite for biodiversity research and capacity building. Knowledge of popular species groups is often widespread, but for other species groups specialists and knowledge are scarce resources. Virtually all natural history museums have to deal with extensive and ever-growing collections on the one hand, and a decreasing number of available species specialists on the other hand. As a result, often large numbers of objects (specimens) in these types of collections have not yet been
identified. Automatic image recognition could contribute to the naming of (a part of) these specimens, thus allowing for more efficient use of the specialist knowledge of taxonomists. Their expertise could be better put to use in cases where automated identification on the basis of a photograph alone cannot provide a conclusive answer (e.g. because microscopic features are relevant) or when the specimen is in fact an undescribed species.

What did we learn from the pilots?
In this project we investigated which type of objects or collections are well suited for automatic image recognition, how image recognition can be deployed in the process of collection registration and publication, and how AI can answer identification questions from the public. We developed image recognition models for a variety of objects from the collections of Naturalis, Museon-Omniversum, Het Natuurhistorisch and a handful of other institutions. These models can be used to identify specific object types or species. The project provided a number of learned lessons for the museum sector about how to develop and deploy image recognition models. Important lessons concern the requirements for the images and associated data that are needed to train models, the amount of training data needed for different types of objects and how to best define classes in your model.

Public interface and additional information
The public web interface launched in this project provides the user with the identifications generated by the different image recognition models and additional information about the species or object type (i.e. concise chunks of textual information, either with or without additional reference images). The additional information is both embedded in the interface itself as well as provided through links to existing additional online sources. With this information, users can verify the AI identifications with validated knowledge about the specific object type or species.The additional content was developed by Naturalis and project partners. Also, through collaboration with partners, we made good (re)use of content created within other ongoing projects. All content is sustainably managed in the information infrastructure of Naturalis and is suitable for reuse in other contexts.

Symposium for and with the Dutch heritage sector
We concluded the project with a symposium titled ‘Museum Collections & AI’ in Naturalis. During this symposium we shared the project results and lessons learned with colleagues from various museums, libraries and archives, with biodiversity researchers and with other people interested in the topic of AI. During this afternoon, the public web interface was launched: a website on which the developed recognition models can be used by both the public and heritage institutions. A number of other AI initiatives in the Dutch heritage sector were also discussed and presented during the symposium.

We are very grateful to NLBIF, the Mondriaan Fonds and Prins Bernhard Cultuurfonds for their support. Thanks to their financial contribution, this project was able to come to fruition.