AGINFRA PLUS D5.4 - Agro-climatic and Economic Modelling Pilot Evaluation Report

In the past three years, the AGINFRA project has introduced virtual research environments as a collaborative tool for open data science in three communities in agri-food research. For the agro-climatic modelling community that was targeted in WP5 of the project, it was for most members the first time they learned about virtual research and the opportunities of using VREs for collaborative data science. In the past years, many community members have heard about the backgrounds, seen demonstrations of the VRE and its agro-climatic modelling pilot applications and had the opportunity to provide their opinions and thoughts through evaluation and validation surveys. A group of evaluators also had the chance to get a more in-depth view, did guided hands-on work in the VRE and responded to the evaluation survey in the three evaluation events that were organised.

This document describes the evaluation procedures and evaluation outcomes performed in the AGINFRA+ project to assess the agro-climatic modelling Virtual Research Environment (VRE) and the pilot applications developed for the agro-climatic modelling community.

Analysing the evaluation feedback, we can conclude that in general the impressions and opinions about the AGINFRA+ VREs and the developed pilot application are overall positive. We observe that community members state that VREs can be a useful tool for digital open science, and we see a broad willingness and enthusiasm to start using such environments for research. There is also a lot of interest for the developed pilot applications, both from the perspective of novel technologies used in the modelling pilots and from the more scientific perspective of used methodologies. The developed pilot applications show very well the potential and opportunities of collaborative, open science for the agro-climatic modelling community and how to create valuable applications and scientific workflows. Because we have demonstrated the VRE and the pilot applications to a broad audience that also includes data scientist that (partly) work on the broader agro-environmental domain, we can also conclude that concepts and applications are well received and useful beyond the agro-climatic modelling community. Many seem to acknowledge that virtual research offers unique options to collaborate, co-develop and share, responding to the demand for working together over different domains with heterogeneous data that is required to tackle the grand interdisciplinary challenges in agri-food and environment.

Nevertheless, we also need to state that there are clear signs of shortcomings and request to improve features, both from evaluators and from the developers and modellers that were involved in developing the pilot applications. These are reported and have been expressed as recommendations for future improvements where possible.

For the agro-climatic modelling community we can conclude that through AGINFRA+ a number of valuable steps have been taken to introduce virtual research environments to groups of scientists that were generally not aware of the opportunities of virtual research. As a project, supported by previous outcomes from infrastructural and innovation actions, we were able to show the general concept of virtual research, and how VREs can support collaborative teams in performing data science. Moreover, we have demonstrated through the developed community driven pilot applications how a VRE and its components can be used to develop scientific workflows and catalyse open science. Steps like this could only be made because of the collaborative and iterative process implemented in AGINFRA+, where infrastructure developers collaborated with developers and data scientists from scientific communities in agri-food.