FAIR4ML, a vocabulary to describe Machine/Deep Learning models

In recent years, we have seen an increase in the creation and usage of Machine/Deep Learning (abbreviated as ML in the text) models in most, if not all, scientific disciplines. ML models, like software, have become research artifacts themselves, with the need to provide rich descriptions and documentation to facilitate reuse, transparency, and reproducibility, while also adhering to common good practices for research. Despite their importance and applicability (e.g., predictions, classification, clustering), machine-readable metadata describing these models is not often provided. Some efforts such as the ML Model Cards [1] and the Data, Optimization, Model, and Evaluation (DOME) [2] recommendations promote good reporting practices. However, there is still a gap when it comes to support rich structured semantic metadata as descriptions are still buried in text-based documents such as readmes, tables, reports and scholarly articles. To overcome this challenge, the Research Data Alliance (RDA) FAIR for Machine Learning Interest Group (FAIR4ML-IG) took over the task to create FAIR4ML [3], an extension of the schema.org [4] vocabulary, designed to describe ML model metadata. We decided on schema.org as it has been successfully extended by other research-related initiatives [5], including Bioschemas [6] to describe Life Science data in web pages and its sister-project Schemas.Science to describe research artifacts; Science on Schema.org [7] and RDA Research Metadata [8] to describe datasets and data repositories; Croissant ML [9] to describe datasets to be used in Artificial Intelligence applications; and CodeMeta [10] and machine-actionable Software Management Plans [11] to describe research software. By extending schema.org, our vocabulary FAIR4ML becomes compatible with these efforts. We have followed a collaborative approach to create the FAIR4ML vocabulary. Initially proposed as a task force in the RDA FAIR4ML-IG, members of the IG contributed with use cases, competency questions, and analysis of descriptions provided by ML model hosting platforms (e.g., Hugging Face [12], OpenML [13]). Two groups proposed independent schemas that were later combined and harmonized, resulting in v0.0.1 [3]. The vocabulary has been further refined during hackathons, where we have worked on crosswalks to other schemas and ML model hosting platforms, the resulting version has not been yet published. We will continue tuning the vocabulary until reaching a stable version that fits most of the cases. Although FAIR4ML vocabulary is still under development, it has already been adopted by two of the involved partners. InesData [14] uses FAIR4ML to describe a small set of models. MLentory [15], part of NFDI4DataScience service portfolio, uses FAIR4ML to harmonize metadata extracted and aggregated from multiple platforms, including Hugging Face and OpenML. Further adoption is already in consideration. For instance, FAIR4ML will be integrated into the NFDI4DS in-silico metadata protocol for reproducibility, a comprehensive view of core metadata needed to facilitate the reproducibility of ML models. Other collaborators like the Oak Ridge National Laboratory (ORLN) have shown interest in contributing to and extending FAIR4ML to describe provenance and results of their trained ML models.