BigDataGrapes D7.3 - Experimental Report on Projected Datasets

The deliverable D7.3 “Experimental Report on Projected Datasets” consists of a report describing the outcomes of the experimentation performed on the projected datasets provided by each pilot, utilizing the Big Data Grapes stack.

This report starts by outlining the methodology and metrics we employ for our experimentation. Moreover, data generators are described emphasizing on State of the Art. We focus our experimentation on a pilot level, initially analysing the provided datasets and evaluating them against the Big Data Vs, as suggested and described in detail in D7.1 and described the data usage patterns.

For three of the pilots, we move on to identify and document the data flows each pilot adheres to throughout the Big Data Grapes stack, highlighting the frameworks and components involved in the process. Moreover, applying data generation techniques create projected datasets per dataset. We then experiment on each of the identified data flow steps by describing 3 usage scenarios for each. During the execution of each scenario for each step, we monitor the chosen performance metrics, showing the respective diagrams and analysing the outcomes.

For the two remaining pilots we perform large scale estimates of the data growth patterns.

This deliverable also presents an end to end report for each of the pilot’s data flows, identifying bottlenecks and making suggestions to improve the performance where needed.