D2.2 Blue Data Infrastructures - Services Analysis Report

The Blue Cloud data discovery and access service will be one of the components of the Blue-Cloud technical framework. It will serve federated discovery and access to blue data infrastructures for external users and also interact with the Blue-Cloud Virtual Research Environment(the component federating computing platforms and analytical services)for populating the VRE data pool. The pilot Blue-Cloud project aims at federating initially in total 10 blue data infrastructures. Each of these existing infrastructures have been described earlier in deliverable [D2.1 -Blue Data Infrastructures –Services Description Report](https://data.d4science.org/ctlg/Blue-CloudProject/blue_data_infrastructures_-_services_description_repor), in particular with a focus on their current data discovery and access mechanisms. While a further analysis of the architecture and concept for the Blue Cloud data discovery and access service has been described in deliverable D2.6 –Blue Cloud Architecture (1stRelease). The Blue Cloud data discovery and access service will provide a common interface, both by web services as by GUI, for discovery and retrieval of data collections from the federated blue data infrastructures. Thereby, conceptually it is planned to set up the query mechanism as a two-step approach, whereby the first step will focus on identifying interesting data collections and products using a common set of search criteria, while the second step will focus on drilling down and sub-setting within the identified collections, in order to get more specific datasets. For the second step, geographic and temporal criteria will be instrumental, next to additional criteria which could be specific per blue data infrastructure. There are also cases, when one step can be sufficient, such as in case of specific data products, that a user wants to download as a complete file. Finally, users should be able to download and store the retrieved data collections on their own machines or in a data pool as part of the Blue Cloud VRE. Implementing this conceptual approach Blue Cloud data discovery and access service will largely depend on the machine-to-machine interfaces of blue data infrastructures, that should be supportive. For that purpose, further analysis activities have been undertaken by MARIS and CNR-IIA for each of the blue data infrastructures concerning the functioning of the existing web services, and in how far these web services are already fit for purpose for the Blue Cloud data discovery and access service. The analyses have been undertaken by try-outs of the earlier given GUI's and web services, and by a series of bilateral web meetings and e-mail communication with the technical contact persons of each of the blue data infrastructures. Moreover, the draft findings and further actions have been presented and discussed at the third TCOM meeting which took place 20-21October 2020. As a follow-up, this draft deliverable D2.2 -Blue Data Infrastructures –Services Analysis Report has been prepared by MARIS and CNR-IIA, describing the results of the analyses and formulating actions for each of the blue data infrastructures, where required. The draft D2.2 has been circulated to the TCOM in order to gather further feedback, and to finalise the draft for submission. On short term, the formulated actions as listed in Chapter 4 should be followed up by technical staff of relevant blue data infrastructures for making their web services fit for purpose, while the technical team of the Blue-Cloud data discovery and access service should continue their planned developments. The ultimate aim is to establish an operational Blue-Cloud data discovery and access service by M18 (end March 2021). This is only feasible if the web services / APIs are well-functioning and fit-for-purpose at each of the blue data infrastructure.