D2.1 - First release and evaluation of the AI-SPRINT design tools

The aim of the AI-SPRINT project is to implement a design and runtime framework to accelerate the development of AI applications whose components are spread across the edge-cloud computing continuum.
AI-SPRINT tools will allow trading-off application performance (in terms of end-to-end latency or throughput), energy efficiency, and AI models accuracy while providing security and privacy guarantees.
This document, in particular, describes the first release and evaluation of the design tools of the AI-SPRINT platform, while the second release and evaluation are due at M24.
The first release includes programming abstractions that hide the communications across components and transparently implement the parallelization of the compute-intensive part of the application; quality annotations to enrich applications with constraints on performance and security; performance models to support AI components execution time prediction (both for inference and training tasks); AI Models Network Architecture Search providing solutions to enable developers with limited Machine Learning (ML) expertise to train high-quality models specific to their needs also in terms of Quality of Service (QoS) requirements; 
Applications design space exploration tools to evaluate multiple alternative candidate deployments for complex applications involving many components.
The main results of this release include:

• The implementation of a prototype application for the AI-SPRINT Healthcare use case that adopts a parallel version of the Cascade Support Vector Machine (CSVM) algorithm to train a model that classifies a dataset to identify atrial fibrillations possibly related to the heart stroke. This implementation adopts the programming model of AI-SPRINT using PyCOMPSs and the dislib distributed ML library, reaching an average f1-score of 0.667 in discriminating atrial fibrillation from normal recordings using the time-frequency features of a balanced selection of the PhysioNet CinC Challenge 2017 available data with shuffled 80/20 training and test sets splits repeated 10 times.
• The design of QoS annotations corresponding to performance and security constraints. The AISPRINT API will include parsing tools which will extract the decorated functions from the code of the application and produce semi-automatically1 the input required by the AI-SPRINT toolchain (including design, deployment and monitoring).
• Tools for the definition of performance models and profiling tools based on a ML library, that allows to predict the execution time of inference and training jobs, through feature selection, and hyperparameter tuning.
• A component for automatic neural architecture search (POPNAS) to automate the search for the best Deep Neural Network (DNN) architecture for a given classification/regression task. Also leveraging the performance models, the algorithm searches for the best network configuration to achieve the higher accuracy in the lowest possible time by searching the Pareto front of the time-accuracy trade-off.
• A tool (SPACE4AI-D) for the automatic design space exploration in order to minimize the execution cost of the AI application while providing response time guarantees. The output of this tool determines the optimal component placement, resource selection and the optimal number of nodes/VMs which helps the developer to find the optimal placement.

For each of the above-mentioned components, an evaluation section is also provided to discuss preliminary results on the adopted technologies and to evaluate the integration of the components according to the project’s milestones.