Application of genetic algorithms and the cross-entropy method in practical home energy management systems

: Home energy management systems (HEMSs) are important platforms to allow consumers the use of flexibility in their consumption to optimise the total energy cost. The optimisation procedure embedded in these systems takes advantage of the nature of the existing loads and the generation equipment while complying with user preferences such as air temperature comfort configurations. The complexity in finding the best schedule for the appliances within an acceptable execution time for practical applications is leading not only to the development of different formulations for this optimisation problem, but also to the exploitation of non-deterministic optimisation methods as an alternative to traditional deterministic solvers. This study proposes the use of genetic algorithms (GAs) and the cross-entropy method (CEM) in low-power HEMS to solve a conventional mixed- integer linear programming formulation to optimise the total energy cost. Different scenarios for different countries are considered as well as different types of devices to assess the HEMS operation performance, namely, in terms of outputting fast and feasible schedules for the existing devices and systems. Simulation results in low-power HEMS show that GAs and the CEM can produce comparable solutions with the traditional deterministic solver requiring considerably less execution time.


Nomenclature
Subscripts and superscripts t time (h) i period i for the appliance use θ temperature restriction PMax power cap restriction Variables/constants n p number of control periods n number of appliances c d appliance cycle duration of operation (h) P power of appliance (kW) x binary variable-condition λ algorithm penalty C energy cost (€/kWh) θ temperature (C) θ e exterior temperature (C) C thermal capacity (kWh/C) R thermal resistance (C/ kW) η coefficient of performance S real-value function γ threshold or level parameter γ* minimum limit x values in state space x* minimiser TC total cost of all appliances (€)

Introduction
Electric power systems have undergone significant changes in recent years due to the progressive integration of distributed energy resources (DERs), namely, in low-voltage systems. There has been a significant increase in distributed renewable energy sources, namely in residential buildings, creating the need to implement demand-side strategies capable of contributing to the improvement of the grid operation.
The smart grid concept [1] is creating a bidirectional flow of electricity and communication between consumers and the grid, through the deployment of Information and Communication Technologies. Consumers will soon manage their consumption to implement intelligent control actions with the objective of reducing energy costs, which can, at the same time, help the grid with peak reduction, consumption levelling, and integrating local renewable generation.
Home Energy Management Systems (HEMSs) are designed to optimise energy use and increase the efficiency of residential buildings. These platforms allow the optimal scheduling of different appliances to reduce energy costs and CO 2 emissions. As decision support tools, HEMS can handle all the relevant information like price tariffs, characteristics of appliances, user preferences, and carry out automated actions to ensure an optimal energy use or even support new energy services. These devices can contribute to behavioural changes by providing discriminated information of energy use (e.g. per appliance, per room etc.), and insights on problematic equipment (e.g. fault detection, aging etc.). HEMS can also convey important information to potential service providers about the flexibility of energy use considering the likelihood of specific loads being activated at different periods of time. When properly deployed, aggregated HEMS allow stable participation in retail and ancillary services markets [2].
The use of energy optimisation schemes for households has been explored in previous work [3,4], where mixed-integer linear programming (MILP) formulation was implemented and demonstrated in real buildings to reduce costs and maximise photovoltaic (PV) integration, as part of the AnyPLACE H2020 project . However, results have shown that it is not always possible to find the optimal solution in a suitable time for practical applications, especially in the case of domestic scenarios with considerable complexity. This means that other optimisation techniques are necessary, especially in low-power computational platforms, to rapidly produce close-to-optimal solutions. This paper reports the application of two heuristics developed for fast optimal scheduling: a genetic algorithm (GA) and a crossentropy method (CEM) optimiser. These allow a lightweight implementation of a modular energy optimisation framework that ensures feasible, understandable and useful actions for the existing appliances in residential buildings with considerably less execution time.
The paper is organised as follows: Section 2 presents the stateof-the-art and the related work; Section 3 introduces the energy management methodology implemented in the HEMS for producing optimal schedules of appliances for the next day. Also, the HEMS platform used is presented and tested in a European project; Section 4 presents the algorithms formulations: the MILP algorithm and the heuristics formulations; Section 5 defines the scenarios that are likely to be found in households for which energy optimisation procedures are expected to be used and evaluated; the results are presented and discussed in Section 6; the main conclusions from work presented in this paper are drawn in Section 7 along with some considerations for future work.

Demand response (DR)
DR is a concept dating back to several decades ago being the first automated implementations developed in the 1970s through radio and ripple control systems. These systems aimed to manage highconsumption devices such as air conditioning and water heating systems [5]. With the growing amount of DER, especially PV, the concept of DR has evolved beyond the use of peak shaving techniques. Current programs are capable of providing services with direct benefits to the distribution network, reflecting the transformation of DR from a way to shave peak demand to an increasingly valuable tool to manage smart grids. The flexibility on the demand side, namely the ability to increase or decrease the load over a given period of time [6], allows consumers to participate in a wider set of energy services via HEMS.

Energy management schemes
Energy management schemes, which can be based on dynamic or time-of-use tariffs, aimed at encouraging consumers to run their devices out of peak hours and benefit from lower prices. It is also possible to develop schemes considering PV production. Apart from cost savings, such schemes can also maximise the remuneration associated with self-consumption.
The effectiveness of energy management schemes is deeply related to self-consumption regulation. Policies have been devised in countries such as Germany and Portugal to encourage instant consumption when the renewable generation exists. Recent legislation in Portugal [7] ended the feed-in tariff incentives and, consequently, the remuneration for PV generation, which translated into self-consumption being more profitable than the injection of power into the grid [8].

Device modelling
Effective HEMS require a preliminary analysis of the household consumption, the characteristics and configurations of the existing appliances, as well as consumption habits. This assessment is fundamental for understanding the overall behaviour of the devices, including inter-dependencies and interaction with other devices. This process can be done based on historical data retrieved from the household or based on default consumption patterns. However, these methods do not provide results with sufficient accuracy, especially when dealing with thermal loads. Thus, HEMS need to be connected with sensors to monitor energy consumption or be equipped with suitable interfaces to allow the user to insert related information.
Two different categories of appliances are usually considered in the literature -controllable and non-controllable loads. Controllable loads can be further distinguished between thermal and deferrable loads. Deferrable appliances operate on a predefined cycle with a known duration and power consumption. Note that the operation of these devices can be shifted along the planning horizon (24 h of the next day). Smart deferrable appliances can be remotely monitored and controlled by the HEMS while the manual ones are activated by the user upon request or through smart plugs.
Thermal device modelling, like electric water heater (EWH) or air conditioner (AC), is more complex than deferrable load modelling. In [9] Kupzog and Roesener described a thermal process of an AC based on comparison with a resistor-capacitor (RC) circuit. The objective of this approach consists of establishing relations between the electrical circuits and the thermal balance inside the room in which the AC device is operating. Similar differential equations are used to characterise other types of thermal loads, namely refrigerators (RFs) and EWH. In the case of refrigerators, the thermal model is equivalent to the AC representation. EWH models are very diverse; each one accounting for different constructive characteristics of this type of appliances [8].

Optimisation algorithms
Deterministic approaches can be seen as general mathematical methods to obtain the global optimum or close-to-optimal solution. This approach takes advantage of the mathematical formulation of the problem to generate a sequence of solutions until the optimum or a satisfactory solution is found. Deterministic optimisation assumes that perfect information about the cost function is available and relies on that information to determine the search direction classically at every step of the algorithm [10]. Linear programming (LP) [11][12][13][14], and MILP [15][16][17][18] are typical deterministic methods.
Heuristic approaches are more flexible approaches than deterministic ones but with no guarantee that the obtained solution is the optimal one. Like deterministic optimisation methods, there is no single method that works well for all problems. Therefore, structural assumptions -such as limits on the size of the decision and outcome spaces, or convexity -are needed to make the optimisation problems tractable [19]. Moreover, the probability of finding the global optimum solution is inversely proportional to the size of the problem. Meta-heuristics, such as GA [20][21][22][23], have been successfully applied to solve classification, decision, simulation and optimisation problems. By searching over a large set of feasible solutions, heuristics can find good solutions with reduced computational effort [24][25][26][27]. Recently, optimisation based on the cross-entropy metric between probability distributions has been explored [28] as a simple, efficient and general method for solving complex estimation and optimisation problems. It offers a generic approach to combinatory and multi-extremal optimisation and rare event simulation -where very small probabilities are needed to be estimated (e.g. reliability analysis, or performance analysis of telecommunication systems) [29].

Related work
Several works have been carried out on detailing different types of models that can be used to optimise energy consumption inside a building or at the aggregator level. In [30,31] the authors presented a multi-scale multistage stochastic optimisation model for HEMS formulated as a model predictive control algorithm for cost minimisation and peak-power reduction. The model includes plugin hybrid electric vehicles (HEVs) charging, thermal dynamics, temperature measurement and real-time pricing signals. In both works, a partition of energy management into slow and fast time scales is made to reduce computation complexity. Other stochastic dynamic programming framework was proposed in [32] for the optimal management of a smart home with plug-in electric vehicle (PEV) energy storage. The model aims to minimise electricity cost while satisfying home power demand and PEV charging requirements. In [33] Bahrami and Amini formulate a centralised energy trading as a bi-level optimisation problem, which is nonconvex and includes the entities' optimal strategy to the price signals.
Regarding the use of LP in energy scheduling problems, the work in [34] describes an optimal and automatic residential electricity consumption scheduling framework that aims to achieve a trade-off between minimising the payment and minimising the waiting time for the operation of each household appliance based IET Renew. Power Gener., 2019, Vol. 13 Iss. 9, pp. 1474-1483 © The Institution of Engineering and Technology 2019 on the needs declared by users. In [35] Rahmani-andebili and Shen proposed a combination of LP with GA aiming to reduce the electricity consumption costs of a smart home. Their approach separates the discrete variables from the continuous variables by addressing the non-linearity of the problem with the GA (discrete) and boosting the search for the global optimum with LP (continuous).
Other GA applications on energy scheduling can be found in the literature. In [36] a GA and a polynomial function-based solution are proposed with the aim to minimise the combination of generation cost and the inconvenience caused to the customer. Also, a multi-objective GA is used in [37] to address the optimisation problem in smart grid by shifting the controllable devices to start working at an appropriate time.
The CEM has been successfully applied to a variety of problems of combinatorial optimisation and rare-event estimation. References and more details on applications and theories can be found in [29,38]. In [39] a performance comparison of two metaheuristics and the CEM presented. The superiority of the CEM over the other two in terms of both solution quality and computation time is highlighted.
The proposed models in the literature include different types of optimisation formulations for dealing with different goals and constraints. However, there is a lack of studies focusing on developing improved optimisation models with the goal of providing viable solutions on a HEMS platform already developed and tested on the field. In this paper, the aim is to develop optimisation algorithms using previously tested methods in other contexts (GE and CEM), and to assess their ability to produce close-to-optimal solutions in a very short time on a real HEMS platform.

Problem description
The methodology presented in this work includes internal inputs related to the home domain and external inputs received from service providers that are interested in shaping the energy demand of certain households. Internal inputs comprise the operation of appliances according to a specific set of preferences and configurations, and the way users manage these devices. These inputs are used to determine the flexibility of the energy use in each household. Also, the behaviour of the appliances and their influence on end-user consumption is carried out to identify the ones more suitable to be rescheduled.

Energy management
3.2.1 Controllable appliances: As previously mentioned, two different types of appliances were considered for optimisation: deferrable and thermal. The deferrable device model is based on three parameters: power (kW), duration (h) and a number of activations during the day. The number of time frames for the appliance is determined according to how many times the appliance is activated. For example, if the appliance operates only once, then there is only one-time frame; if it operates four times, then four-time frames are considered. Each time frame can be defined as a time interval for the operation of the device. The best time for activation of the devices within the time intervals is decided by the optimisation method. Some important points must be considered concerning time frames: • If there is only one time frame, the user can define an interval for the operation (e.g. between [9 am and 6 pm]) or an interval to exclude operation (e.g. between [12 am and 6 pm] meaning that it can operate between [0 am and 11 am], it cannot operate between [12 am and 6 pm], and can operate again between [7 pm and 12 pm]). • If there is more than one-time frame, then an interval to exclude operation cannot be defined. Also, the starting time of the next operation cannot overlap the previous one (e.g. two-time frames: if the first one has a deadline at 9 am, then the second only can start after 9 am, including).
Note that this is the only type of appliances that can be freely added to the HEMS by the user, since only three parameters are required. The modelling approach adopted for thermal devices is based on the physically-based load models (PBLMs) developed in [3]. PBLMs are based on energy balances that occur inside a thermal chamber, which can be a room -for space heating devices -, an RF cavity or cylinder for hot water. Two separate models are considered, one for AC and another for EWH. Note that the model for the AC can also be applied to the RF. However, the RF was not considered in this study since this appliance has tighter limits in terms of temperature and fast variations when powered having little impact on the cost minimisation. Moreover, these devices consume less power when compared to other appliances and are currently being designed to be as efficient as possible.

User preferences:
The preferences set by the users in a HEMS platform can be quite complex, raising additional challenges to the optimal schedule. To overcome this issue, the HEMS was designed to consider several possible scheduling parameters that can be modified. User preferences are treated as restrictions in the formulation of the cost minimisation problem. They need to be carefully considered to avoid making the problem unfeasible. For deferrable appliances, several deadlines can be set depending on how many times an appliance is expected to operate during that day for a maximum operation horizon of 24 h. A time window is available between the configuration time and the deadline during which the operation of the appliance may start. If the appliances are manual (without remote control), then a time interval must be set during which the appliance can be manually operated. For thermal appliances, comfort requirements such as the desired hot water temperature and the usual number of baths per period for the following day (e.g. two baths at 7 am and one bath at 8 pm) can be set. Further restrictions like power cap or contracted power limits may be introduced. Also, information on the consumption baseline and the inflexible loads is necessary, either through direct measurement or estimation, so that feasible solutions are computed.

External interaction:
The HEMS can also incorporate functions to deal with external information. This information includes tariff publishing (load and generation) and other online services that can provide local forecasts for the day-ahead. Hence, three sets of data are included in the optimisation problem, namely: (i) price tariffs from energy providers and remuneration for the PV production injected into the grid; (ii) temperature for the next day; (iii) PV forecast, which includes the expected PV production.

Optimisation objectives:
The formulation of the optimisation problem depends on the goal to be achieved. Among the wide diversity of optimisation criteria for a HEMS, the cost minimisation is probably the most appealing goal since it focuses on finding the least expenditures with the energy use. If price discrimination follows an efficiency goal from the grid perspective, then the participants will be participating in a DR program as well. The objective function still consists of minimising the electricity costs even if PV is considered for the optimal day-ahead scheduling. In this case, the remuneration associated with PV based self-consumption for the day-ahead operation is computed based on the forecast for the PV generation and the domestic inflexible load.

HEMS platform
The implementation of the HEMS tested in this research work was firstly achieved in the AnyPLACE H2020 project. This project specified the functional, technical and technological requirements of such a solution. The main outcome was a software and hardware platform capable of interconnecting with other devices and systems. The HEMS developed is currently being upgraded with new features and a dynamic and easy-to-use interface. It will also be tested in a real demonstrator of the InteGrid H2020 project in Portugal and Sweden.
The HEMS developed is capable of storing the configuration and comfort preferences of users to produce optimal schedules of appliances (legacy and smart) for the next day. It is a highly customisable platform that incorporates functionalities from other existing systems as well as innovative ones related to the energy management purpose. The software modules were designed, implemented and embedded into a Raspberry PI 3 computation core to produce a cost-effective solution that is flexible and adaptable to different contexts.

Optimisation tools
The computational capacity required at the HEMS processing unit to run the energy optimisation problem may be significant, especially with the growth of the problem size due to time parameters or the number of devices. For example, the calculation of the schedule for the 24 h ahead considering 15 min time step results in 96 periods to be optimised. Therefore, it is necessary to find methods to be implemented in affordable and low-end performance computational platforms, such as the one presented in the previous section, that can consistently provide good solutions in a useful time.

Deterministic approach
A deterministic optimisation method was implemented to establish a baseline for comparison of the other optimisation methods. The goal of the optimisation problem implemented consists in minimising the total daily cost considering dynamic price tariffs, peak-power limit, deferrable appliances with different duration, power consumption and a number of activations, as well as the models and external data for thermal appliances.
This multi-period optimisation problem is implemented and solved as a scheduling problem using binary variables that represent the decisions of activation of the domestic appliances. The binary decision variables in each period represent the on/off control: 0 and nominal power consumption.
An objective function that can be performed by the HEMS platform is presented in (1), where the aim is to find the lowest total daily cost for the energy consumed from the grid The objective function presented in (2) also consists in the minimisation of the total daily cost considering the remuneration associated with PV self-consumption for the next day taking into account the PV generation forecast and the domestic inflexible load Both objective functions can be considered if PV is installed. The objective is chosen by the end user regarding his preferences towards a more economic or ecologic target. Equations (3)-(5) are constraints in the formulation necessary to enable the use of a MIP solver. In this formulation, the decision variables Y + and Y − represent the case when PV is greater than or less than the residential consumption, respectively. The binary variable u t and the positive constant M are used to impose either/or constraints There are technical limitations linked with the country's regulation and must not be disregarded. In fact, there are contracted limitations to the power consumed from the grid, namely for LV consumers. This constraint is considered in the following equation: Deferrable appliances can have multiple deadlines to be operated during the day. A time window is available between the allowable start time and the deadline, during which the control method can choose the optimal start time considering the appliance cycle duration. Hence, the constraint defined in (7) is added per appliance and the time frame for each activation set by the user Each thermal appliance has two comfort constraints (maximum and minimum temperature) as well as an equality constraint. The linearisation of these constraints was developed in other research work (see [3]), which consists in the representation of the appliances' physical temperature models defined in (8) for the air conditioner and in (9) for the electric water heater. The integration of these models contains a recurrence relation between the decision variables, i.e. the consumption in each period depends on the consumption in previous periods

Non-deterministic
A non-deterministic approach was developed based on two different meta-heuristics: GA and CEM. GA is based on the theory of evolution and relies on mutation, recombination and natural selection to obtain solutions with high fitness. CEM is based on converting the optimisation problem in a rare event estimation problem. It is solved by an adaptive sampling process: (i) generation of random samples; (ii) update the parameters of random generation, based on sampled data, to improved samples over iterations.
In both meta-heuristics, a single-objective function to be minimised is set. This function is expressed in (12) determines the performance per sample. The function benefits the samples that minimise the total cost of operation and comply at the same time with temperature and power limit constraints. The fourth element (λ DA ) is only added to the equation when using the CEM, since it is necessary a penalty guarantee the correct number of activations of the deferrable appliances. Constraints are imposed to achieve the cheapest solution that considers consumption patterns and comfort requirements set by the user (e.g. set the dishwasher after lunch and/or dinner, the washing machine during the dawn, room temperature). This allows algorithms to be personalised according to a time frame of operation and user's consumption patterns. If any of the restrictions are violated, a penalty is applied • TC, total cost -equals to the total power consumed by the appliances multiplied by the respective value of the price tariff. • λ θ , the penalty for the temperatures limits -if the temperature is outside the lower or upper limit, equals to the sum of all the surpluses between the respective temperature and limit. • λ PL , power cap penalty -if the power being consumed by all the appliances is greater than the power limit, equals to the difference between both values. operating outside the time limits, equals to the sum of the total excess. This penalty is only necessary for the CEM due to the nature of this heuristic.

GA optimisation:
GA is a probabilistic search algorithm based on the procedure of natural selection and genetics proposed by Charles Darwin. The fundamental characteristics of GA are the principles of survival of the fittest and adaptation to the environment. This paper contains a GA version specifically designed to minimise the electricity bill of the user in their households, suggesting an optimal solution for the activation of some devices.
i. Initialisation • Initial population: number of random individuals. It is important to ensure the diversity of the population and for the convergence speed of the algorithm. • Number of elite parents and elite sons. To certify that the best individuals will be in the next generations. • Number of parents and sons, chosen randomly, until the population size is reached. • Mutation probability. ii. Parents selection strategy: A selection strategy is used to decide the parents of the individuals that will become part of the next generation. This strategy selects all the individuals of the current population as parents regardless of the fitness value. Basically, the first parent is selected by its order in the population, while the second one is picked randomly from the remaining individuals. iii. Application of genetic operators: The genetic operators are used to produce new individuals (offspring). The optimisation problem has two categories of devices (deferrable and thermal) with unique constrains, so genetic operators must be adapted. Deferrable appliances use mutation. The process consists of randomly selecting a new period of the day (one day is composed of 96 periods of 15 min) to use the appliance. Thermal appliances can undergo crossover and/or mutation. The crossover operator starts by randomly selecting the crossover point (cxpoint) to generate two new offspring. The first one keeps the genes of parent one until the cxpoint while the other offspring inherits the genes of the second parent after the cxpoint. The opposite process is used to obtain the other offspring. In this case, mutation consists of switching the device for randomly chosen operating points (if on, then turn off and vice-versa). iv. Selection: The selection process in GA must be set carefully: too strict a selection results in a reduction of diversity while a week selection can result in slow evolution. To prevent these problems, diverse selection methods are available, such as roulette wheel, elitism, rank selection and tournament selection. In this optimisation problem, two methods were used: elitism selection to choose a percentage of the fittest individuals and tournament selection to pick the remaining individuals. This hybrid selection is helpful to maintain diversity. In GA, there are successive generations for the population and by consequence, the average fitness tends to improve until some stopping criterion is reached. In this problem, the fitness was computed based on the minimisation of the electricity expenditure. v. Stopping criterion: Since GA is an iterative method that computes successive populations to converge, a stopping criterion is needed to get the final solution. This algorithm has two stopping criteria that can be set: define a maximum number of iterations or a maximum number of repeated solutions after a predetermined fitness value (e.g. all violations cleared). This allows the algorithm to output quick sub-optimal solutions.

Cross-entropy method:
The CEM is a heuristic first proposed by Rubinstein [28] as an adaptive importance sampling (IS) procedure for the estimation of rare-event probabilities that uses the Kullback-Leibler divergence as a measure of similarity between sampling distributions. A variation introduced to the Rubinstein formulation allowed the translation of rare-event estimation in an optimisation heuristic, by associating the optimisation problem in (13) with the estimation of a probability l = P(S(X) ≤ γ), where γ is close to the unknown γ* and x is a random variable parameterised by a finite-dimensional real vector v and with some density function f ( ⋅ ; v) on X [38] S(x*) = γ* = min Considering l as a rare-event probability, a multilevel CE estimation approach can be used to find an IS distribution that concentrates all its mass in a neighbourhood of the point x*. Sampling from such distribution will produce optimal or nearoptimal states. In contrast to rare-event simulation, the final level γ = γ* is generally not known in advance. Still, the CEM for optimisation produces a sequence of levels γ^t and reference parameters v^t such that ideally the former tends to the optimal γ* and the latter to the optimal reference vector v* corresponding to the point mass at x*. The optimal energy scheduling is formulated as a discrete multi-stochastic minimisation problem with a finite state space represented by binary vectors. The final result is a binary vector, with the time(s) where devices are to be activated during the next day. This is done iteratively ii. Random sample generation: A random number generator is used to draw samples from a set of parametric densities adequate to the problem. Given that only vectors of discrete variables are considered, the sampling is based on the multivariate Bernoulli distribution. iii. Calculate sample performance: The performances are determined using the function presented in (12). The number of performances being calculated in each t iteration is equal to N, i . e ., a single performance per k sample. iv. Order samples: For minimisation problems, the performances are ordered from largest to smallest: S(0) ≥ ⋯ ≥ S(N − 1) and the limit (γ^t) being estimated is the (1 − q)-quantile of the performances: γ^t = S (N − N e − 1) . In this step, the goal is to obtain the limit γ^t that represents the last/worst performance to be considered. Moreover, the range of performances is directly affected by N e , which itself depends directly on ϱ. v. Update sampling distribution parameters: The probability v^t − 1 is updated to v^t using the set of samples (X 0 , …, X N − 1 ) and the respective performances according to where n is the number of binary components of each sample and X K, j is the jth component of the kth random binary vector X. Notice that I is an indicator that takes the value 1 if the performance of the kth sample satisfies the condition, and the value 0 if the opposite. After the probability is updated, a smoothing process based on the parameter α can be applied Stopping criterion: The algorithm stops when the sampling distribution has degenerated enough or a maximum number of iterations is reached. In the case of the multivariate Bernoulli, the parameter v^t , j can be used as follows: This means that the CEM stops if all the probabilities in the vector v^t are greater than (1 − ϵ), or are less than ϵ.

Scenarios of operation
The energy management methodology, presented earlier, was tested in scenarios likely to be found in typical daily-life applications. The formulations include the internal inputs related to the home domain and the external inputs received from service providers that are interested in shaping the energy demand of certain households. These settings were tested using real data sets concerning two distinct countries, namely Portugal and Spain, to evaluate the algorithm with two different types of external data.

Usage profiles
Several experiments have been carried out for the energy scheduler. In all scenarios, the appliances considered were the more likely ones to be found in households, clothes dryer (CD), dishwasher (DW), washing machine (WM), AC and EWH. The first three appliances are included in deferrable's group. This category includes all appliances that operate on a predefined cycle with a known duration and consumption. The other two belong to the category of thermal appliances where the only thing predefined is power consumption.
To obtain scenarios, the duration and consumption of the appliances were estimated. Tracebase [40] dataset is an appliancelevel power consumption data set that was used to obtain appliances plug data. For this purpose, more than one month of data was collected. Namely, each time the device activates the average power consumption and duration. With all the statistics obtained before, a new general average power consumption and duration were achieved by calculating the mean value for each parameter. These values are presented in Table 1. Table 2 presents the characteristics of thermal appliances, as well as the consumption patterns (baths information) and the comfort requirements (temperatures) used in the study. Typical parameters for commercial appliances were considered. Realistic comfort temperatures, as well as consumption patterns of the enduser, were included. The EWH analysis comprised two hot water usages (38°C): 50 l at 07:00 and 17:00.
Thermal parameters can be calculated considering the temperature readings obtained during the regular operation of the appliance [41].

External values
• Climate conditions: Realistic thermal models must also consider ambient temperature variations. Obviously, the activation of the air conditioning and even the heating of water differ regarding the external thermal conditions. Therefore, different summer and winter temperatures for the two countries were used. The temperatures chosen for Portugal correspond to a typical summer/winter day in a hot Mediterranean climate in Alentejo (southern Portugal). In this region, temperatures can reach 45°C in summer and fall to a maximum of 11/12 in winter. In Spain, as weather can be quite different depending on the region. For this analysis, the temperatures of the Coruña region with a cold Semi-Arid climate were chosen. • Energy prices: Dynamic pricing is a strategy in which businesses set highly flexible prices for products or services based on current market demands. In the case of electricity pricing, dynamics prices are becoming more attractive due to its ability to influence consumption behaviour of customers. Households consumption patterns can be adjusted according to electricity prices. Different energy prices were assumed by different countries. For the Spanish case, a particular type of dynamic pricing referred to as the day-ahead hourly pricing [42] was considered. The Spanish tariff is calculated for each hour of the day being the prices published at 8:15 pm of the previous day. For the Portuguese case, a bi-hourly TOU tariff [43] is used where workdays, Saturdays and Sundays are treated separately and different time-of-day periods are set. Tariffs can have two different energy prices (accordingly to the tariff period and with different time schedules in each type of day). The PV remuneration price used was €0.045/kWh, which is based on the average Portuguese values. • PV production: In Portugal, the policy that is being implemented is to promote instantaneous consumption. For instance, in the case of small PV installations, there is a lower remuneration for PV energy fed into the grid in comparison with the retailing energy price, thus ending the high feed-in tariffs that were in use. On the one hand, this makes self-consumption always more profitable than injecting PV generation into the grid. On the other hand, daily profiles of PV generation are usually not correlated with the consumption, namely in the residential sector, since a significant part of the population is not at home during daylight hours. Therefore, some form of flexible consumption is required at the local level so that it can be shifted to the periods when PV generation is higher. To validate the effectiveness of the scheduling methodology including the PV variability by season, two different types of data were collected for summer and winter.

Results and discussion
The scenarios discussed in this section are based on the data presented in previous sections. Eight residential cases were tested, four in Portugal (PT) and four in Spain (ES), to evaluate savings and the profile behaviour for each tariff system, and season of the year: • PT summer RC -bi-hourly energy prices and Warm Mediterranean climate (Alentejo) in the summer. • PT summer PV -PV production and Warm Mediterranean climate (Alentejo) in the summer. • PT winter RC -bi-hourly energy prices and Warm Mediterranean climate (Alentejo) in the winter. • PT winter PV -bi-hourly energy prices and Warm Mediterranean climate (Alentejo) in the winter.  • ES summer RC -bi-hourly energy prices and cold Semi-Arid climate (Coruna) in the summer. • ES summer PV -PV production and cold Semi-Arid climate (Coruña) in the summer. • ES winter RC -bi-hourly energy prices and cold Semi-Arid climate (Coruña) in the winter. • Spain winter PV -bi-hourly energy prices and cold Semi-Arid climate (Coruña) in the winter.
The presented MILP formulation was implemented in C++ language to allow the performance-oriented implementation. To solve the optimisation problem, two different solvers were used, CPLEX and SCIP. Despite producing faster results, CPLEX solver cannot be used in platforms like raspberry PI where the HEMS is installed. Therefore, SCIP was also used as a workaround. In any case, the results of the CPLEX solver are presented as a benchmark to compare other solvers and methods since the probability of reaching the optimum solution is greater with CPLEX. Moreover, the SCIP has to be implemented with a time or a GAP limit for reaching useful results given its slower performance.
The results of these simulations were the optimised cost of equipment scheduling for the next day. This specific problem has 576 integer variables for the first objective function and 768 (576 binary, 192 continuous) for the second objective function and both have 580 (linear) constraints. An example of this output for the next day is presented in Fig. 1 where an optimised schedule for the third Spanish case is presented.
Note that all the consumption is shifted to the lowest price hours and all the deferrable appliances comply with the deadline constraints. AC and EWH are activated more often at lower prices periods to maintain the required temperature during the time of day when prices are higher.
However, this type of scheduling can cause problems due to the overload at the lowest price period. This can activate the protection systems when there are power limits. Therefore, a restriction of 4.6 kW can be added to avoid this problem.

Computer simulation
All eight scenarios were solved with the CPLEX to obtain the optimal result that can be achieved with the MILP formulation. Note that it was not possible to reach the optimal value in the required time for two tested scenarios -the third and fourth scenario of the Portuguese case. They both quickly reached a 1% gap. However, it takes a considerable amount of time (more than one week) to reach a 0% gap. Hence, only the best-achieved value for these two specific scenarios is presented.
For the SCIP solver, after some tests were verified that in some scenarios it took too long to achieve the optimum value, so a GAP (admissible error) of 2% was defined.
A crude Monte Carlo method is applied to analyse the behaviour of the heuristics (GA and CE) in terms of achieving the optimal result. For that purpose, 100 independent simulations of each scenario are performed. All solvers are compared in terms of total operation cost, the total running time of the simulations and total electricity consumption.
As can be seen in Tables 3 and 4, the averages of the total daily electricity consumption are very similar for all algorithms despite a better average total daily cost in the deterministic case. Usually, CE and GA have higher costs because the AC is activated more times or in a period where the price is higher. This means that the heuristic optimisation process could have been stuck in a local optimum.
After analysing the total running time, it is possible to conclude that both heuristics present faster results and GA shows faster convergence than the CEM. However, the CEM gives fast and optimal results in a matter of a few seconds when considering fewer appliances, namely if all are deferrable. Note that the increase in running times and daily cost is mainly related to the thermal appliances, since it is necessary to obtain the temperatures for each period, of each sample, in each iteration, with the thermal  equations. Furthermore, thermal appliances also have a larger range for possible activations than the deferrable ones, due to the type of constraints associated (maintain the temperature inside limits). Another factor that tampers with the running time is the type of tariff being used. With more dynamic tariffs, such as the one used in the Spanish cases, the algorithm converges faster and usually with much less probability of being stuck in a local optimum.
Regarding the average daily consumption, both Portugal and Spain present quite identical results during winter. In the summer, however, Portugal has higher daily consumption than Spain because its summer temperatures are higher and consequently the AC is activated more times. Moreover, as can be perceived from the Spanish scenarios, the average of the total daily consumption can vary slightly between scenarios with and without the PV. This difference is only caused by the thermal appliances -since the deferrable ones have a predefined number of operations -, that are activated with zero or near zero cost during periods with higher PV production, even if the temperature is within the fixed limits.
To understand the advantages of choosing deterministic or nondeterministic methods, it is important to compare the algorithms running time as well as the price increase introduced by the heuristic variation. Table 5 shows the summary average time of all solvers.
As expected, from the previous subsections, the average values of running time from CPLEX are exaggerated, showing the need to limit the gap for low-power computation platforms. Even SCIP with a GAP of 2% took an average of 20 min for all scenarios. These times can be limited in the HEMS platform not to disturb the user, however, the associated error will increase.
The performance of each algorithm is illustrated in Fig. 2 by presenting the loss that each algorithm can present when it is compared with the optimal solution from CPLEX.
The results indicate some similarities between methodologies. SCIP losses have an average of €0.016 for all scenarios, making it almost not visible on the figures. Both heuristics have the worst results, specially CEM. However, on average none of the heuristics have losses higher than €0.50 per day.
All algorithms presented good results suitable for HEMS integration and proved that it can be used in real-life scenarios. Although some savings are lost with GA and the CEM, these values are residual for the user and do not make that much difference in the total achieved savings.

HEMS simulation
To test and validate the heuristic algorithms in the HEMS platform, the same Monte Carlo method is applied to analyse the behaviour of the heuristics (GA and CE) in terms of achieving the optimal result. For that purpose, also 100 independent simulations of each scenario are performed. The average running times for each scenario are presented in Tables 6 and 7. As expected, the heuristics present average times less than ∼10 min in the HEMS platform. Even in cases where the average time has passed above 10 min, we can observe in the standard deviation   that there are also very good results for that scenario. These results validate the use of heuristics in low computation platforms to solve energy management optimisation problems.

Conclusions and future work
This paper compares the use of GA and the CEM in low-power HEMS to optimise the total energy cost. Different scenarios were considered to assess the HEMS capacity of outputting fast and feasible schedules for existing devices and systems. The formulation of the deferrable appliances posed some challenges, namely to guarantee that the appliance was activated inside predefined time frames. Conversely, the formulation for the thermal appliances was simplified with the use of the PLBMs.
The convergence of all algorithms to optimal or sub-optimal results was assured for computer simulation and HEMS integration to be used in real-life scenarios. Although some heuristic results were not quite as good as the optimal solution, the price deviation is residual to the user, and the achieved savings are approximately the same.
Simulation results in low-power HEMS show that GA and the CEM can produce comparable solutions with the traditional deterministic solver requiring considerably less execution time.
Considering the algorithms developed and implemented in this work, the next approaches would be to develop multi-objective optimisation strategies that determine the optimal operational schedules of residential appliances -including thermal constraints, occupant's routines and consumers preferences -to meet objectives such as reductions in cost, CO 2 emissions, energy use, and grid export. Also, optimisation strategies will be developed to define the control actions necessary to provide the requested flexibility to the grid. The optimisation algorithms will be implemented on existing SW and HW of HEMS.