Iterative MPC for Energy Management and Load Balancing in 5G Heterogeneous Networks

Multi-Access Heterogeneous Networks introduced a step forward in modern communication networks allowing the provision of reliable and efficient broadband services. However, heterogeneous networks imply a burden of complexity in the integration, coordination and QoS management processes thus complicating the satisfaction of users’ requirements. The aim of the present work is to address the above-mentioned issues by developing a mathematical framework for optimizing resource usage in 5G heterogeneous networks. More in detail, the optimization will take into account both the network’s load and energy consumption simultaneously. The proposed approach, based on Model Predictive Control, will be compared with other control strategies for validation and performance comparison.

Among the various problems that need to be addressed for the design of a fully operable Multi-Connectivity system, a fundamental role is played by resource management algorithms, as controlling the amount of resources dedicated to the various connections over the available APs directly impacts on the network efficiency and performances. In fact, optimally balancing the amount of resources used over the available APs allows the network operator to reduce power consumption, enhance the system resiliency to faults or malicious attacks and may also provide higher QoS levels to the network users.
This paper presents a control strategy to optimize the transmission power in a 5G heterogeneous network, by defining a dynamic algorithm for queue management. The methodology selected for this purpose is based on Model Predictive Control and the problem is modelled as a linear dynamical system with a quadratic cost function to be optimised. The main contributions of this work are the following: • The modelling of a heterogeneous network with multiple RATs as a dynamical system in which the controller of the UEs regulate their transmission powers to sustain the connections; • The design of a centralized Iterative Model Predictive Control (MPC) control system for the optimal network resource management in a Multi-Connectivity setting; • The validation of the proposed approach with standard solutions to evaluate its performance. The rest of the paper is organised as follows: Section II reviews adopted approaches and methodologies for load balancing present in the literature; in Section III the load balancing and energy management problem is formalized from a mathematical point of view and the proposed approach is presented; in Section IV illustrative simulations are presented in order to validate the proposed approach also with respect to the performances of other control strategies; finally, Section V summarizes the contributions of the present work suggesting future research lines.

II. RELATED WORKS
Several methodologies were studied in the field of multiconnectivity for resource management and traffic steering, such as Multiple Attribute Decision Making (MADM) [8], [9], Game Theory [10]- [12] and Reinforcement Learning [13]- [15]. Multi-Access Networks allow the "Always Best Connected" concept presented in [16], defined as the capability to provide to users with the best connection experience, exploiting several radio accesses characterized by heterogenous technologies. The management of these multiaccess networks shall be carried out considering both users' requirements and network condition. Many methods are proposed in literature with several advantages or disadvantages in terms of performances as well as problem tractability. A comprehensive comparison between available solutions in the literature is provided by [17], where the authors show that tractable methods, in terms of implementation and execution, are weak from the performance point of view. Indeed, methods such as utility theory, MADM and fuzzy logic are typically static. On the other hand, methods able to provide a dynamic solution, such as optimization, game theory and Markov chain, have some disadvantages considering implementation and execution effort. This paper proposes a dynamic and model-based solution, able to satisfy both performances as well as tractability, considering users and network requirements.

III. PROBLEM FORMULATION
In the following, the Energy-Management and Load Balancing problem for Multi-Access Networks will be detailed and formalized.

A. Problem Definition
The problem addressed in this article consists in fairly distributing loads in multi-access networks with the aim of optimizing the network's energy consumption.
In the considered scenario, users can transmit and/or store the traffic produced by their applications (e.g., chats, browsing, video streaming) by means of different technologies (e.g., Wi-Fi, LTE, etc.). These two actions are characterized by different costs defined in terms of the consumed energy which depend on the user's (i) storage technology and on the (ii) transmission technology and distance from the AP. Users are also characterized in terms of a maximum storage capacity and a maximum transmission bandwidth due to constraints imposed by the physical transmission channel used and the APs' limited processing capabilities.
The objective is thus to allocate to the users the APs' available bandwidth respecting users' storage capacity while minimizing the energy consumption of the overall network. To solve this problem, an Iterative Model Predictive Control (I-MPC) problem is set up in order to optimize the network's energy consumption in a centralized way. The main advantage of using I-MPC compared to standard MPC solutions is the ability of the controller to determine, at each time step, the optimal prediction window length. We will show in the simulation chapter that this characteristic will translate in having a better queue management with no significant overhead in power consumption.

B. Mathematical Modelling
The problem can be modelled as a quadratic optimization problem where the cost function represents the network's consumed energy. As already mentioned, the consumed energy can be defined in terms of the total storage ( ) and transmission ( ) costs. The former models the storage energy consumption and is a constant depending on each user's storage technology. The transmission cost, instead, depends on (i) the transmission technology, (ii) the distance , between the -th user and the -th AP and (iii) the medium used for the transmission. Hence, the transmission cost associated to the -th user and the -th AP, referred to as , , can be defined as where is a constant depending on the -th user's transmission technology and , function of the distance between the user and the AP, depends on the medium used for the transmission. The resulting cost function can be defined as: where is the number of users, is the number of APs, ( ) is the traffic stored by the -th user at time , and , ( ) is the traffic transmitted by the -th user towards theth AP. The optimization problem described can be formulated as a Model Predictive Control (MPC) problem as follows: subject to where is the time horizon over which the optimization is performed and (3) specifies that the transmission energy of all the users is the objective function to be minimized (4) guarantees that, for each user and each time instant , the stored traffic ( ) does not exceed the maximum storage capability which can be solved by means of several methods (e.g., see [18]- [20]). According to the MPC paradigm, the controller shall compute the solution to the constrained optimisation problem and then apply the first element of the optimal control vector to the system and discard its remaining entries. At the following sampling time, the MPC controller shall re-evaluate the optimal control actions, in what is called a receding horizon approach. This iterative procedure is what gives MPC both the characteristics of optimal, open loop, control and feedbackbased, closed loop, control systems.
With respect to equation (6), it should be noted that, for the considered problem, the traffic ( ) generated by the user represents a disturbance which should be estimated for each > 0. Errors in the estimation of such variable translate in a poor description of users' storage level evolution and, in turn, of the network's dynamics. Furthermore, a fixed time horizon may lead to slower network's performances. Indeed, by tailoring the time horizon at each time step, it is possible to find the optimal load distribution while minimizing the time at which users' queues are empty.
To address the two above-mentioned issues, an iterative implementation of the Q-MPC described by (3)-(6) has been developed and will be described in the following section.

C. Proposed implementation
An The proposed implementation is aimed at exploiting the available knowledge about the users' generated traffic ( ) in order to improve the quality of solutions generated by the model (3)- (6). To achieve this purpose, at each instant of time the disturbances ( ) are measured and the optimization is performed by taking in consideration only the measured values. In other words, at each given time , the status of each user is gathered and their generated traffic = is measured. For exploiting such knowledge, the model (3)-(6) can be extended as follows. Let be the traffic generated by the -th user at time and, without loss of generality, assume = 0. Then, the above-mentioned issues can be addressed by considering the following additional constraints: (7) allows to take in consideration the measured values of the disturbances (8) specifies that the optimization is performed without taking in consideration future unknown values of the disturbances (9) specifies that at the end of the time horizon users' queues should be empty It should be noted that the formulation (3)-(9) is heavily impacted by the choice of the time horizon. Indeed, if the time horizon is too small, then the problem may result unfeasible since there would not be enough time to empty users' queues (constraint (9)). On the other hand, large time horizons do not allow to reduce the time at which users' queues are empty.
To address this problem, it is possible to adopt an iterative solution for finding the most suitable time horizon at each time step. The pseudo-code of the proposed implementation is reported in the table below.   TABLE II. PSEUDO-CODE OF THE PROPOSED IMPLEMENTATION

Inputs.
≔ initial time horizon; = ( , … , ) ≔ measured disturbance at time = 0 for each user . Outputs. ( , , ) ≔ optimal load distribution and sent traffic for each user i over the time window . Algorithm: 1.   return empty solution // Problem is unfeasible or is too small 18. end

IV. SIMULATIONS AND RESULTS
To validate the proposed I-MPC several simulations have been performed focusing on (i) the fairness of the load distribution among the users and (ii) the energy consumption.
The performances of the proposed approach, i.e. the model described by (3)-(9) implemented according to the algorithm reported in TABLE II. , are compared with those of a standard MPC, i.e. the model described by (3)-(6), and a Load Balancing Controller based on the Most Loaded (ML) principle (at each time step , users with higher queue levels are assigned with higher bandwidth).

A. Scenario
Simulations have been performed assuming 6 users and 3 APs. The storage costs (expressed in energy unit) have been assumed equal for all users: The maximum storage levels and APs' capacities (both expressed in packets) are = 15, 15, 15, 15, 15, 15 , = 19, 7 , 10 Finally, the traffic (expressed in packets per time step) to be sent by each user has been considered as a random, piecewise continuous signal uniformly distributed between 1 and 7.

B. Results
As already mentioned, simulations are aimed at studying the performances of three control strategies with respect to the fairness of the (i) load distribution and (ii) energy consumption.
Concerning the first aspect, in Fig. 2 it is shown the mean storage level per users. As expected, the ML approach guarantees the fairest load distribution. However, the amount of traffic stored is higher with respect to the I-MPC and MPC approaches. In presence of traffic spikes, this aspect could translate in network congestion or higher packets' waiting times. On the other hand, the I-MPC performs better than the MPC with respect to the fairness of load balancing. More in detail, the MPC tends to allocate the APs' to users with lower transmission costs, without considering the storage level. The I-MPC, instead, is able to take in consideration also users' loads due to constraint (9) and the adopted algorithm (see  TABLE II. ), i.e. the imposition of emptying users' queues in the minimum time steps.
Concerning the performances of the three approaches with respect to the energy consumption problem, in Fig. 3 it is shown the total consumed energy per user. It can be observed that, as expected, the ML approach performs very poorly since it does not take in consideration energy-related aspects. On the other hand, the performances of the I-MPC and MPC approaches are quite similar though the latter behaves slightly better at the expenses of a significantly higher storage level.
Furthermore, as highlighted in Fig. 4, the I-MPC guarantees lower peaks of energy consumption for all users. This aspect is particularly relevant in Internet of Things (IoT) scenarios which envisage a multitude of connected devices with limited processing and battery capabilities, that may have limitations on their maximum transmission power output. The paper presented an Iterative MPC strategy for load balancing and energy management in multi-access communication networks. Resource management in such networks is very important for exploiting the opportunities provided by the different RATs while matching the heterogeneous set of users. In this respect, a fair distribution of loads and energy consumption allows not only to improve networks' performances but also to empower the adoption of a wide set of UEs with different computation capabilities and battery life.
The performances of the proposed approach have been compared with those of a standard Load Balancing algorithm based on the Maximum Loaded principle and of a standard quadratic MPC. Simulations proved that the proposed I-MPC allows to achieve good performances with respect to load distribution and at the same time offering the best performances with respect to the energy management problem. While the performances of the ML approach from the energetic viewpoint proved to be very poor (as expected), the MPC proved to have good performances both for the load balancing and energy management problems. However, with respect to the proposed I-MPC, the quadratic MPC presents spikes in the energy consumption profiles which may be limiting in some cases, for example in IoT scenarios.
Next research directions include the comparison between the I-MPC strategy and canonical optimal control methods such as Minimum Time Controller as well as the implementation of the I-MPC solution in distributed fashion.