Optimal Control of an Energy Storage System and Plug-in Electric Vehicles Fast Charging in a Grid-connected Service Area

As a result of the increasing charging rate implemented by car manufacturers in the new generation of plug-in electric vehicles (PEVs), charging point operators are continuously adjusting the charging infrastructure accordingly. In order to maximize the charging operator’s return of investment and minimize the impact on the electricity grid, a key aspect is finding technical solutions which allow to downsize the nominal power flow at the point of connection between the charging station/charging area and the electricity grid, as the operating expenses are significantly affected by this parameter. In this regard, this study discusses the optimal control of an energy storage system (ESS) and PEVs fast charging for reducing the impact on the grid of the charging load in a charging area. A trade-off is achieved between the objectives of keeping limited the charging power withdrawal from the grid and the one of keeping limited the fluctuation of the state of charge of the ESS around a given reference, while keeping the charging power near to the nominal one. We present a deterministic solution, under the realistic assumption that the charging operator knows a piece-wise constant estimate of the aggregated charging power demand over the control period. Numeric simulations are provided to validate the proposed approach.

Francesco LIBERATI * , Roberto GERMANÀ, Emanuele DE SANTIS, Alessandro DI GIORGIO Abstract-As a result of the increasing charging rate implemented by car manufacturers in the new generation of plug-in electric vehicles (PEVs), charging point operators are continuously adjusting the charging infrastructure accordingly. In order to maximize the charging operator's return of investment and minimize the impact on the electricity grid, a key aspect is finding technical solutions which allow to downsize the nominal power flow at the point of connection between the charging station/charging area and the electricity grid, as the operating expenses are significantly affected by this parameter. In this regard, this study discusses the optimal control of an energy storage system (ESS) and PEVs fast charging for reducing the impact on the grid of the charging load in a charging area. A trade-off is achieved between the objectives of keeping limited the charging power withdrawal from the grid and the one of keeping limited the fluctuation of the state of charge of the ESS around a given reference, while keeping the charging power near to the nominal one. We present a deterministic solution, under the realistic assumption that the charging operator knows a piece-wise constant estimate of the aggregated charging power demand over the control period. Numeric simulations are provided to validate the proposed approach.

I. INTRODUCTION
Among the main barriers to the large scale adoption of electromobility, the time needed for charging represents a key aspect for many drivers planning to buy a new vehicle without having the availability of a private parking for slow charging during the night or a charging station (CS) available at the workplace [1] [2]. This is the reason why car manufacturers and charging infrastructure operators are working in the direction of technically enabling diversified charging services, and offering increasing level of charging power [3] [4]. In the context of an electricity system subject to unbundling, a charging infrastructure operator may be a different player than the distribution system operator, so that the connection of its CSs to the grid is subject to installation and operational costs which depend on the nominal power flow at the point of connection (POC). Consequently, finding technical solutions which allow to downsize the power flow at the POC is a key aspect to maximize the charging This work has been carried out in the framework of the 5G-SOLUTIONS project, which has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 856691. The content of this paper reflects only the author's view; the EU Commission/Agency is not responsible for any use that may be made of the information it contains.
The operator's return of investment (and indirectly minimize the impact on the electricity grid).
This paper deals with the problem of controlling a gridconnected microgrid equipped with a set of CSs providing the fast charging service to plug-in electric vehicles (PEVs), an electric energy storage system (ESS) and potentially a source of power generation. We call such microgrid a "service area". The service area concept is relevant both for fast charging in urban scenarios and for fast charging during long-range trips. Since, in a fast charging scenario, there is limited possibility to modulate the single PEV charging sessions (as the priority is to serve customers as close as possible to the maximum power and in the minimum possible time), in this paper we take into account the ESS as an additional degree of freedom providing the flexibility needed to control the aggregated service area power exchange with the grid. More specifically, the control objectives for this work are: 1) To keep as low and smooth as possible the power flow at the point of connection of the service area with the grid. This lowers the operation cost of the service area, which is nowadays one of the main barriers for the deployment of the service area concept (the higher the power flow is, the higher the grid connection fees that the service area operator pays); 2) To keep the evolution of the state of charge (SOC) of the ESS as close as possible to a desired reference value (usually, 50% of charge), to make sure the ESS has always a reserve of energy to charge or discharge; indeed, there is limited flexibility in the control actions if the ESS is operated close to the 0 or 100% SOC levels; 3) To keep the aggregated PEVs charging power in the service area as close as possible to a reference power curve, typically representing the nominal power, so as to minimize the PEVs dwelling time at the CS.
The above mentioned requirements work in opposition each other, giving raise to the need of formalizing an optimal control problem. Optimal control has been successfully applied in many applications dealing with resource management problems, see, e.g., [5], [6]. Control problems related to PEVs and microgrids equipped with several forms of controllable loads, storage devices and local generation are the subject of a wide literature. The contributions mainly differ for the application requirements, the control methodology and specific mathematical issues arising when modeling the control problem. Among the application scenario, [7] studied the problem of minimizing the voltage deviation in the microgrid in presence of distributed generation units, while [8] tackles the problem of reconfigurable microgrids to enhance security and reduce operation costs. Moreover, [9] faced the problem of energy management of smart homes in a microgrid environment. Several control methodologies, such as model predictive control (MPC) [10], [11], [12], [13],w sliding mode control [14], machine learning [15] have been investigated in several application contexts. Also the calculus of variations and the Pontryagin minimum principle (PMP) have found numerous applications in energy management, especially in the optimal control of hybrid vehicles (see, e.g., [16], [17], [18]) and microgrids (see, e.g., [19], [20]). One of the main obstacles in the adoption of these techniques in such problems is that it is not always possible to derive a causal or a closed loop solution. To address this, in literature some simplifications on the calculation of the costate from the necessary optimality conditions are discussed. In [16], the authors show that for the given hybrid vehicle control problem, the costate of the system can be assumed constant, simplifying the derivation of the optimal solution. In [18], in the context of the energy management of a plug-in hybrid electric bus, the authors combine PMP and MPC. In [19], a similar approach is used to optimize the energy flows in a microgrid hosting ESSs and renewables, with the objective of minimizing the ESS control effort and the deviation of the SOC from a reference value; also in [19], the costate is constant. However in many applications it is not possible to assume a constant costate.
The distinctive features of this work are as follows. First, we consider a deterministic formulation of the service area fast charging optimal control problem, which assumes the knowledge of the charging power demand in the service area over the control window. In this regard, charging infrastructure operators offering a booking service for charging are typically able to build a prediction of the near future value of the charging demand, which takes the form of a piece-wise constant signal and here represents a time-varying reference to be tracked by the actual aggregated charging power. Though quite realistic, the limitations resulting from this assumption are discussed in view of a future stochastic formulation of the proposed optimal control problem. Second, we model the above mentioned power tracking requirement through a cost term whose weight is dependent on the power demand, in order to mitigate the difference in the tracking performances resulting from different level of congestion (i.e. power demand) in the service area. Such a modeling choice gives raise to a two-point boundary value problem referred to a non-stationary dynamic system, for which an iterative procedure is proposed with the aim of explicitly calculating the initial costate. Third, we design an optimal control for a system subject to an exogeneous non-controllable signalthe short term prediction of locally generated power -which we assume to be known to the controller. Finally, to the best of the authors' knowledge, this work presents one of the first applications of the theory of calculus of variation to the problem of ESS and PEVs charging control in a smart charging area subject to the above mentioned requirements.
The reminder of the paper is organized as follows. Section II presents the reference service area architecture and the proposed optimal control problem. Section III discusses the calculation of the optimal control. Section IV presents and discusses simulations in a simplified but still realistic charging scenario. Section V concludes the work and discusses future directions of the work.

II. PROBLEM FORMALIZATION
The reference service area architecture considered in this paper is presented in Fig. 1. In this setup, the ESS, the CSs and the power generation are both directly connected to the POC. Let p(t) denote the power flowing at time t at the POC, u ess (t) the ESS charging/discharging power, x(t) the deviation of the ESS SOC from a reference value (typically, half of the full charge), u ev (t) the actual cumulative power absorbed by the PEVs,û ev (t) the nominal cumulative PEVs power demand, w pv (t) the power generation in the area. The ESS dynamics is modeled asẋ(t) = f (x(t), u ess (t), t) = u ess (t). The service area is subject to the balance equation p(t) = u ess (t)+u ev (t)−w pv (t). In order to keep simpler the analysis of the problem, the obvious box constraints on p(t), u ess (t) and x(t) are neglected. Several works in literature (see e.g., [21], [22]) show how they can be handled, by including additional auxiliary state variables and accordingly expanding the Hamiltonian of the system.
Based on the control objectives introduced in Section I, the following optimal control problem is defined. Problem 1. (Service area fast charging optimal control problem). Given an initial and a final time of problem definition (respectively, t i and t f ), given subject to with s, q, r > 0. The weight c(û ev (t)) depends on the charging power currently requested. The idea is that it should be very high (ideally infinite) when the requested charging power is low (since there is not difficulty in this case to serve immediately the charging requests, at the requested power level), while it should progressively decrease as the requested charging power approaches the nominal maximum power available at the charging area (which denotes a state of congestion). In the following, for brevity, we write c(t) in place of c(û ev (t)).

III. THE OPTIMAL CONTROL
Problem 1 can be solved via standard techniques from calculus of variations. The Hamiltonian of the system is Problem 1 is convex, and the resulting sufficient optimality conditions areλ where the last step in (10) is achieved by noticing from (9) that ru ess (t) + ru ev (t) − rw pv (t) = −λ(t). Equation (8) is the costate equation, (9) and (10) the equations of the control, and (11) the transversality condition. By plugging (10) into (9), we rewrite (9) as: where r (t) : After plugging (9) into the state dynamics, the state and the costate dynamics define the following two-point boundary value problem with boundary conditions x(t i ) = x i and λ(t f ) = sx(t f ), and where we have defined w(t) :=û ev (t) − w pv (t).
Notice that (14) is a time varying system, because r depends onû ev (t). Its explicit solution can be however computed easily in the present case, by exploiting the fact that the referenceû ev (t) in practice is piece-wise constant.
First of all, partition [t i , t f ] into sub intervals over which the reference is constant, i.e., consider the time instants t i = t 1 < t 2 < ... < t N = t f , with N ≥ 2, such that, for any n ∈ {1, ..., N − 1},û ev (t) is constant over the time interval [t n , t n+1 ).
Consider a time instant in the n-th time interval of the above-defined partition, i.e., t ∈ [t n , t n+1 ), with n ∈ {1, ..., N − 1}. Then we have where we write A n to remind that it is the matrix A computed based on the value which the aggregate charging reference takes on the n-th interval (i.e.,û ev (t), t ∈ [t n , t n+1 )). In turn, [x(t n ), λ(t n )] T can be computed based on the same formula, and starting from [x(t n−1 ), λ(t n−1 )] T . The process can be iterated backward up to the initial state [x(t 1 ), λ(t 1 )] T . Based on this consideration, it is not too difficult to see that [x(t n ), λ(t n )] T , with n ∈ {2, ..., N } (being n = 1 a trivial case), can be written as 1 where we have defined for simplicity ∆t j := t j+1 − t j . Equation (16), substituted back into (15) provides the explicit solution for the optimal state and costate trajectories, as a function of λ(t 1 ), which in turn is found by evaluating (16) for t n = t f and substituting x(t f ) and λ(t f ) into the transversality condition (11), and finally solving for λ(t i ). These calculations, though conceptually simple, are quite involved, and can be executed with a computer, through symbolic computation routines. An alternative approach allowing to explicitly calculate the initial costate, which is a key information to determine the optimal control and trajectories of the system, is as follows. Consider the transition matrix characterizing the free evolution of system (14) in the generic time interval [t n , t n+1 ). After simple calculations, it takes the form Evaluating (15) for t = t f = t N and t n = t N −1 , substituting x(t f ) and λ(t f ) into the transversality condition (11) and finally solving for λ(t N −1 ) gives where Equations (18)(19) establish, with little abuse of nomenclature, a new "transversality" constraint linking λ and x at time t N −1 . In turn, [x(t N −1 ), λ(t N −1 )] T can be computed using (15) and starting from [x(t N −2 ), λ(t N −2 )] T , which allows to establish a new constraint between λ and x at time t N −2 . Iterating the procedure backward up to the initial time t 1 , the initial costate can be written as or equivalently, posing h = N − k, in the more compact form The above procedure completely determines the optimal solution. In particular, the optimal state and costate trajectories are determined by (15) (as said, after embedding (16) and the initial costate (21)); the optimal control of charging power w ev is found from (10); the optimal ESS control from (9); Fig. 2. Shape of weight c(û ev (t)) used in the simulations. and the optimal trajectory of the power at the point of connection with the grid from (6).

IV. NUMERICAL SIMULATIONS
Simulations were performed in Matlab 2018b and span 12 hours of operation of the service area. We empirically selected q = 1, r = 2 and s = 10, resulting in good performance of the control system. The weight c(û ev (t)) of the charging power error term in (3) was selected as in Fig. 2. It remains close to the maximum value for low charging power demands, then it drops to a minimum value as the charging demand approaches the maximum allowed in the service area (a congested state of the service area, in which curtailment of the power delivered to the charging stations is admissible). The simulation was performed considering fast-charging at a maximum supply rate of 50kW . Problem 1 and the associated optimal control handle the presence of renewable generation in the service area. In this simulation we consider the presence of photovoltaic (PV) power production at the service area; the forecasted PV power output is here assumed to have the form reported in Fig. 3, which is representative of an almost clear sky day, with two small perturbations approximately around 10:00 and 14:00. Figure 4 displays the charging reference considered in the simulation (dashed line), and the resulting aggregated controlled charging curve u ev (solid line). The charging reference reflects a test scenario, with different peaks of charging requests during the day. This simulation is aimed to study the behaviour of the system under high and  low load conditions (the proposed control system naturally also works in conditions with no/reduced renewable energy, a scenario not discussed in this paper; in that case, the charging energy will be provided only by the ESS and the POC). The reference is tracked with high fidelity when there are only 3 charging sessions active (see Fig. 4). As the number of simultaneous charging sessions increases, the tracking requirement is progressively relaxed, accordingly to the weight c(û ev (t)), allowing for an incremental mismatch between the reference charging profile and the controlled one. The zoom in Figure 4 highlights this behavior. Figures  5 and 6 report the ESS charging/discharging power and the ESS SOC evolution, respectively. The piece-wise nature of the charging sessions reference signal is reflected on the power contribution of the storage. The ESS SOC evolution well reflects the strategy adopted by the storage, to smooth the power flow at the POC. The ESS recharges in advance, in prevision of the large peaks of demand, and later discharges to contribute during the peaks-periods, thus reducing the overall SOC error. Between the two charging peaks, the ESS recharges and balances the charging requests, thus smoothing their effect on the POC. Figure 7 reports the power flow at the point of connection with the grid. The resultant power flow is smooth, in line with the desired requirement. The charging peaks in particular are significantly reduced, thus contributing to keeping low the power flow at the POC, as desired.
The results presented above show the effectiveness of the proposed control under the assumptions reported in Section I, in particular on the knowledge of the charging demand in the service area. Though quite realistic in a scenario where the charging point operator offers the charging service through a booking system, some practical remarks and limitations have to be considered. For example, from Figures 4 and 5 it can be seen that the ESS power profile has fast changes at the times in which a charging session starts or stops; as a consequence, in order to avoid undesired perturbations of the power flow at the POC, it is fundamental in practice to start and stop the delivery of charging power at the scheduled times, instead of synchronizing it with the actual times of connection and disconnection of the PEV at the CS. Further, the performances of the proposed control are clearly affected by PEVs connecting at the CSs after the time indicated in the booking system, as well as by drivers booking the charging service when the control window is already active. In order to properly take into account these limitations, and more in general to solve the same control problem in a more general scenario, where PEVs connect to the CSs without booking the service, it appears natural to formalize the same control problem under milder assumptions on the charging demand. In this regard, this work can be considered instrumental to the formalization of a future stochastic version of the proposed optimal control problem, in which the knowledge of the power demand is replaced by that of parameters characterizing the corresponding stochastic process (e.g. expected value, variance, etc). Also, re-optimization and the adoption of a receding horizon approach can be considered, in order to take into account the progressive update of the information available to the controller (not only the forecast of the charging demand, but also of the renewable power output, to which similar considerations apply). Finally the simulation results previously presented may be used as a benchmark for the evaluation of the performances achieved by the controller under the mentioned milder assumptions.

V. CONCLUSIONS AND FUTURE WORKS
This paper has presented an optimal control strategy for controlling the recharging of plug-in electric vehicles (PEVs) in a service area equipped with an energy storage system (ESS). The objective is to control the ESS and the PEV recharging process in order to serve the PEV users as fast as possible, while flattening the power profile at the point of connection with the grid (which results in lower operation costs of the charging area).
Future works will focus on the detailed modelling of the charging demand as a stochastic process, and consequently on the extension of the problem to stochastic optimal control [23], also considering a probabilistic formulation of constraints. Another line of research will address the inclusion into the problem of all the applicable constraints (on ESS charging power, POC, ESS SOC, etc.), as well as of the PEVs' charging dynamics, for better capturing the recharging process. Also, re-optimization schemes to address uncertainties in the start and the duration of the PEV charging sessions will be analysed. Finally, in future works, an assessment of the economical benefits arising from the flattening of the power profile at the POC will be carried out.