SnR Evaluation Framework

Purpose of this SnR evaluation framework is to provide guidance on a structured evaluation process for all use cases to evaluate all technology components developed under the SnR project. In order to achieve this, a set of KPIs has been formulated for each individual technology component. Subsequently an iterative definition and selection process has been established to identify relevant KPIs for each use case. The evaluation criteria take account of the technologies on the one hand, but on the other hand also consider context specific needs of each use case. The deliverable takes stock of the objectives of all use cases that have already been defined in the respective pilot plans. These are important to understand the different contexts in which the technologies will be tested and from which different needs and requirements for the different technologies may result. This Framework intends to provide guidance for the overall set up of the evaluation process applying a common approach. However, the variety of technologies and use cases makes it impossible to define one process that fits every situation and hence needs to leave room for some flexibility in adapting the evaluation tools, while still leaving possibilities for comparison. D8.9 elaborates on the development process of the SnR evaluation framework, under involvement of end-users and technical partners, also providing indications for further research and next steps to be taken for the evaluation phase of the project. Furthermore, it details the data collection methodologies, listing and summarizing all relevant methods needed to plan and execute the trials. These have also been matched with the preliminary set of KPIs for the technology components which have been selected for use cases. The deliverable concludes with a checklist for all use case organizers in order to ensure that planning prior, during and after completion of the use cases is timely and adequate. The framework will be core for this next phase of the project when it comes to testing the technologies in the use cases. Considerations on the evaluation need to be incorporated in the overall planning and set up of the use cases to have a clear understanding of the tasks that need to be performed and by whom, to have high quality data for the evaluation.