Intelligent system for auto-tuning of big data analytics deployment properties

In the field of machine learning applied to big data, in this thesis work has been implemented an intelligent system for auto-tuning of big data analytics deployment properties, and which will be integrated on a cloud platform with a model-based BDA-as -a-service (MBDAaaS) approach. A readapted solution proposed by Guolu Wang et al. was first implemented. The original method consists of optimizing a single metric on the Spark platform (an engine for big data processing), readapting it to the use case and aimed to the optimization of multiple metrics. Once the solution proposed for the optimization of a single metric was implemented and tested, the method was extended and tested for the optimization of multiple metrics. A general deployment optimizer method has therefore been proposed, applicable not only to the Spark framework, but also to other frameworks such as Cassandra and Kafka. These frameworks share the same problem: the tuning of deployment properties to optimize application performance. The proposed deployment optimizer then aims to solve these problems automatically, transparently, and with faster times than the manual tuning acted by domain experts and data engineers.