Published May 16, 2022 | Version v1
Preprint Open

Resource Allocation for Distributed Machine Learning at the (Edge-Cloud) Continuum

Description

Edge computing has emerged as a paradigm for local computing/processing tasks, reducing the distances over which data transfers are made. Thus, an opportunity is presented for data transfer-intensive, distributed machine learning. In this paper, we develop a solution for serving distributed Machine Learning (ML) training jobs at the edge–cloud continuum. We model the specific requirements of each ML job and the features of the edge and cloud resources. Next, we develop an Integer Linear Programming algorithm to perform the resource allocation. We examine different scenarios (different processing and bandwidth costs) and quantify
tradeoffs related to the performance and cost of edge/cloud bandwidth and processing resources. Our simulations indicate that even though there are many parameters that determine the allocation, the processing costs seem to play on average the most important role. The cloud b/w costs can be significant in certain scenarios. Finally, in certain examined cases, significant monetary benefits can be achieved through the collaboration of both edge and cloud resources when compared to using exclusively edge or cloud resources.
 

Files

1570775178 (1).pdf

Files (794.2 kB)

Name Size Download all
md5:a9d7ac99746ebe58f810fd7637cef77a
794.2 kB Preview Download

Additional details

Funding

5G-IANA – 5G Intelligent Automotive Network Applications 101016427
European Commission