identifier: identifierType: DOI 10.5281/zenodo.3885753 creators: Jason Armitage (University of Bonn, Germany), Golsa Tahmasebzadeh (TIB, Germany), Endri Kacupaj (TIB – Leibniz InformationCenter for Science andTechnology, Germany), Swati (Jožef Stefan Institute, Slovenia) title: MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities publisher: Zenodo publicationYear: 2020 subjects: Machine learning, Multitask learning, Multimodal data, Multilingual data language: en resourceType: Dataset version: version 1.0.0 formats: application/zip rights: rightsIdentifier: CC-BY-4.0 rightsURI: https://creativecommons.org/licenses/by/4.0/legalcode Creative Commons Attribution 4.0 International Public License descriptions: descriptionType: Abstract In this paper, we introduce the MLM (Multiple Languages and Modalities) dataset - a new resource to train and evaluate multitask systems on samples in multiple modalities and three languages. The generation process and inclusion of semantic data provide a resource that further tests the ability for multitask systems to learn relationships between entities. The dataset is designed for researchers and developers who build applications for digital humanities projects that perform multiple tasks on data encountered on the web and in digital archives. A second version of MLM provides a geo-related subset of the data with weighted samples for countries of the European Union. We demonstrate the value of the resource for digital humanities applications with a motivating use case and specify a benchmark set of tasks to retrieve modalities and locate entities in the dataset. Evaluation of baseline multitask and single task systems on the full and geo-related versions of MLM demonstrate the challenges of generalising on diverse data.