D2.3 A report on the features and nature of novel data which are needed within online benchmarking experiments in different subareas

The objective of Deliverable 2.3 is to report on the different types of data sets involved in the management of scientific benchmarking and their main features. Understanding the nature of these data sets is crucial to properly include them into the OpenEBench’s data model.

Importantly, understanding those data sets will contribute to interact with the different scientific communities facilitating their migration into OpenEBench. Deliverable 2.3 updates the initial report presented at ( D2.1: Creation of a database warehouse infrastructure for storing and organizing data for online performance assessment experiments ) and offers detailed real world examples on how the different proposed data sets types map to a broad set of benchmarking initiatives.

As OpenEBench becomes an ELIXIR integrated platform for both scientific and technical benchmarking, it is important to identify commonalities across scientific communities which can be used at the platform level, but also specific divergences that should be considered to give a better support to the communities. The effort made to identify novel data types and map existing ones into OpenEBench is crucial to offer specific actions for each data type e.g.accessibility modes, data ownership, etc. Moreover, the nature of data sets dictate how workflows can be articulated at the platform and beyond; e.g. communities can choose to run their evaluation workflows at their own platforms, for privacy reasons, and deposit only the results into OpenEBench using the APIs developed for that end.

As data is the key component of the infrastructure, this report directly touches on most of the objectives associated to the ELIXIR-EXCELERATE WP2 (1 to 5). The remaining one, will be addressed indirectly through specific hackathons that will be organized to identify, link and/or deposit different data types for the benchmarking communities in OpenEBench.