Hybrid Artificial Neural Network with Meta-Heuristics for Grid-Connected Photovoltaic System Output Prediction

ABSTRACT


INTRODUCTION
In a Grid-Connected Photovoltaic (GCPV) system, one major question among consumers is how much output energy can be obtained from the system throughout the operation. In many cases, customers would want to know the performance of the associated GCPV system under different climatic conditions. Therefore, many studies have been conducted in order to predict the output of the system. One of the popular techniques used for prediction is by using Artificial Neural Networks (ANN) [1]- [4]. While this study had discovered many findings in PV output prediction using ANN, the ANN design was heavily dependent on past experience with the similar application and also subjected to trial and error process [5]. The trial and error process during ANN training can be very complex and time consuming. Therefore, meta-heuristic algorithms are presented in this paper to facilitate the ANN training such that the prediction of the AC output from the GCPV system is optimized.
In order to solve many types of optimization problems, meta-heuristics are used as they have better potential in solving complex optimization problems when compared to traditional algorithms [6]. Examples of meta-heuristics are Genetic Algorithms (GA), Simulated Annealing (SA), Ant Colonies Optimization (ACO), Particle Swarm Optimization (PSO), Bee Colonies Optimization (BCO), Harmony Search Algorithm (HS), Firefly Algorithm (FA), Bat Algorithm (BA) and Krill Herd (KH) [7]. Besides that, several attempts were made to facilitate the ANN design by combining the ANN models together with an meta-heuristic algorithms, such as Evolutionary Programming-Artificial Neural Network (EP-ANN) [8]- [10], Artificial Bee Colony [11], Harmony Search-Artificial Neural Network (HS-ANN) [12] and Particle Swarm Optimization-  [13]. Most of them are inspired by the nature, by mimicking the successful characteristics of the physical, biological and sociological systems.
Nevertheless, some of these algorithms can give better solutions to some particular problems. There are no specific algorithms in order to solve all kind optimization problems [6]. If two algorithms produce similar results but one is significantly simpler than the other, then the simpler of the two is a superior algorithm. Algorithms with a low degree of complexity have a number of advantages, including being simple to implement in an industrial setting, being simple to re-implement by researchers, and being simpler to explain and analyze [14]. The algorithmic structure of a meta-heuristic algorithm is desired to be simple enough to allow for its easy adaptation to different problems. Also, it is desired that the meta-heuristic algorithm has no algorithmic control parameters or very few algorithmic control parameters excluding the general ones, i.e. size of population, total number of iterations, problem dimension of the population based optimization algorithms. If a meta-heuristic algorithm has algorithmic control parameters, the related algorithm must not be too dependent on the initial values of the mentioned algorithmic control parameters [7].
In this paper, selected meta-heuristics, i.e. Cuckoo Search algorithm (CSA), Firefly algorithm (FA) and Evolutionary Programming (EP) were used to optimize the ANN training for predicting the power output of a GCPV system.

RESEARCH METHOD
This study was implemented in several stages. Firstly, a hybrid ANN models were developed for predicting the AC power output of a GCPV system. A multi-layer feedforward neural network was proposed as the architecture of the ANN. In addition, the inputs to the ANN were Solar Irradiance (SI), Ambient Temperature (AT) and Module Temperature (MT) and the output of the ANN was the AC power output. These input and out data were obtained from a GCPV system located at Green Energy Research Centre (GERC), Universiti Telknologi MARA, Malaysia. The selected meta-heuristics, i.e. Cuckoo Search Algorithm (CSA), Evolutionary Programming (EP) and Firefly Algorithm (FA) were then used separately to determine the optimal number of neurons in hidden layer, learning rate and momentum rate during the training of the ANN such that the RMSE of the prediction was minimized. Upon completion of the training process, testing process was subsequently performed to confirm the training process. The performance of these hybrid ANN models using different meta-heuristics was compared based on RMSE and computation time. Later, Mutated Cuckoo Search Algorithm (MCSA) was introduced with the aim of improving the prediction performance of hybrid ANN using CSA. Gaussian mutation was introduced as part of the optimization process durng the ANN training. The implementation of Cuckoo Search (CSA), Firefly Algorithm (FA) and Evolutionary Programming (EP) for the prediction were briefly explained in the following sections.

Cuckoo Search Algorithm (CSA)
Cuckoo Search Algorithm (CSA) is inspired by the way of laying eggs from cuckoo species [15]. A cuckoo normally lays eggs in the nest of a bird from other species. The basic principles of CSA and the conceptual implementation of the ANN using CSA are described as follows: (i) The female cuckoo bird lays egg one at a time, and put it randomly in a host nest. Thus, each nest initially contains an egg from the host bird and an egg from the cuckoo bird. In this study, the cuckoo egg was represented by a set of decision variables that need to be optimized in predicting the AC power from the GCPV system. The decision variables used were the learning rate, momentum rate and number of neurons in hidden layer of the ANN model. (ii) The nest with the most quality egg will survive without failure. The quality of the cuckoo egg is compared with the quality of the host egg in a particular nest. In this study, each egg also carried information on quality, i.e. the RMSE of the prediction using ANN. However, RMSE can only be obtained after simulating the ANN with the set of decision variables for that particular nest. If the quality of the cuckoo egg was better than the quality of the host egg, the cuckoo egg was set to survive in the host nest, and vice versa. (iii) The number of host nests is fixed with the probability of cuckoo's egg being discovered by the host bird, Pa is from 0 to 1 [16]. If the cuckoo egg is discovered by the host bird, the cuckoo egg is destroyed or thrown away by the host bird. This event was expected to occur by chance with probability Pa. If the cuckoo egg was found to be destroyed, new nest locations are identified using where xt t+1 is the new egg at a new nest location for a cuckoo i and α denoted as a step size. The product ⊕ means entry wise multiplications [17]. The L´evy flight essentially provides a random walk while the random step length is drawn from a L´evy distribution: where λ denoted as the random walk and the value was set between 1 to 2. The flowchart of CSA was illustrated in Figure 1. First, cuckoo search parameters and the initial host nest are defined. Next, the fitness of each cuckoo is evaluated before it was ranked. After evaluation, the host nest is modified using Levy flight equation as shown in Equation 1 and the fitness for each modified cuckoo is evaluated. Next, if the condition is not satisfied, the cuckoos are moved towards the best nest environment and the process will be repeated from the beginning. On the other hand, if the condition is satisfied, choose the current best nest as the best cuckoo until the population exceeds the maximum generation, and lastly, the evolution process will be stopped.

Evolutionary Programming (EP)
EP is one of meta-heuristic technique which is used to perform random search in optimizing an objective function. It had been used in many numerical and combinatorial optimization problems in recent years. It is a combination of several main processes namely initialization of parents, evaluation of the fitness value, mutation process to produce an offspring from their parents, evaluation of fitness for the offspring, a combination process, selection and lastly the convergence test. In brief, there are two major steps to be summarized in optimization by EP [18]: where q'1j were the offsprings generated from each parent by Gaussian mutation . Nj(0,1) represents a normal Gaussian random variable with mean 0 and standard deviation 1. In this study, each parent contains information of the decision variables that need to be optimized, i.e. learning rate, momentum rate and number of neurons in hidden layer.
(ii) Select the next generation from the parents and mutated parents (offspring). At this stage, each parent is mutated to produce an offspring. Then, the next generation of candidates for potential solution is selected by first ranking the pool of parents and offspring according to their fitness values. Subsequently, a population of candidates is transcribed to the next generation for the next evolution. In this study, the fitness value was RMSE. The flowchart of EP was illustrated in Figure 2. First, initialize the parent population of the generation. Second, the fitness of each parent generated is evaluated using the selected equation or function. Next, perform the mutation process by using Gaussian distribution operator as shown in Equation 3 to produce the new population. The new population is known as the offspring. After that, combine all parents and offsprings in order to find the best result by undergoing the selection process. The process is stopped when the convergence is achieved. If not, the process will be repeated by performing the mutation process again.

Firefly Algorithm (FA)
Another type of meta-heuristic algorithm selected in this study was Firefly Algorithm (FA). FA is a meta-heuristic optimization algorithm which was introduced at Cambridge University in 2008 by Xin-She Yang [19]. The algorithm was inspired by the flashing characteristics of fireflies at night. The algorithm was formulated based on three idealized rules [15]: (i) All fireflies are unisex. Thus, one firefly will be attracted to the other fireflies regardless of their sex. In this study, besides being unisex, every firefly generated contains a set of decision variables that need to be optimized in predicting the AC power from the GCPV system. The decision variables used were the learning rate, momentum rate and number of neurons in hidden layer of the ANN model. (ii) Attractiveness is proportional to the brightness of fireflies. However, the brightness decreases as the distance increases. For any two flashing fireflies, firefly with less brightness will fly towards the brighter one and if there is no brighter one than a particular firefly, the firefly will fly randomly. Therefore, the lower the distance between two fireflies, better agreement on the solution is obtained. On the other hand, random flight by a firefly was implemented by randomly adjusting the value of each decision variables. (iii) The brightness of firefly represents the quality of solution and it is associated to the fitness value of the objective function. In this study, the brightness of the firefly was represented by the fitness value, i.e. the RMSE. Therefore, the firefly with less brightness had higher RMSE while the firefly with more brightness had lower RMSE. The flowchart of FA was illustrated in Figure 3. First, generate the initial population of the firefly and evaluate the fitness of each fireflies generated by using the selected equation or function. After that, update the light intensity of the firefly by using light intensity equation. Next, the fitness of all the fireflies is evaluated and ranked acoording to the corresponding fitness value. If the maximum iteration was reached, the process will proceed by finding the global best throughout all the generations of the firefly. The new population of firefly from the best generation will be used to represent the best population.

Mutated Cuckoo Search Algorithm (MCSA)
In this study, the performance of CSA was improved by modifying the optimization process in CSA. The conventional CSA commonly employs Levy flight to update the location of the nest during the evolution of cuckoo. However, the random walk of cuckoo using Levy flight is very random in nature and thus causing the cuckoo to divert far away from the possible optimal solutions during the search for optimal solution. This random walk can be improved by introducing a characteristic of inheritance such that the movement of cuckoo is still dependent on the group performance of cuckoos in the population.
As a result, mutation was introduced such that the cuckoo was mutated after a random walk using Levy flight. In MCSA, Levy flight will be combined with the Gaussian distribution operator to perform the mutation process. Meanwhile, the rest of the processes are similar with the process of CSA. Hence, the group

RESULTS AND DISCUSSIONS
This section presents the results and discussion of the study. The performance comparison of hybrid ANN models, i.e. CSA-ANN, EP-ANN and FA-ANN is shown in Section 3.1. After that, the performance of MCSA-ANN is shown in Section 3.2. The results achieved by MCSA-ANN was evaluated and benchmarked with CSA-ANN.

Performance Comparison of Hybrid Artificial Neural Network Models
After all the ANN training parameters had been determined for the models, the performance of the hybrid ANN models was compared in terms of prediction accuracy based on RMSE and R 2 as shown in Table 1. CSA-ANN was discovered to be the best model as it produces the lowest RMSE during both training and testing. The RMSE during training produced by CSA-ANN is approximately 43.60% lower than FA-ANN and 38.26% lower than EP-ANN. Meanwhile, the RMSE during testing produced by CSA-ANN is approximately 47.55% lower than FA-ANN and 42.86% lower than EP-ANN. Moreover, CSA-ANN also yields the highest R 2 during both training and testing when compared with FA-ANN and EP-ANN. Besides that, CSA-ANN was also approximately 5.3 and 5.6 times faster than EP-ANN and FA-ANN respectively.

Performance Comparison of Mutated Cuckoo Search Algorithm
Although CSA was found to be the best meta-heuristics for the hybrid ANN model, the performance of the conventional CSA depends on random walk using Levy flight which is very random in nature. Thus, it may cause the cuckoo to divert far away from the possible optimal solutions during the search for optimal solution. Therefore, a Mutated Cuckoo Search Algorithm-Artificial Neural Network (MCSA-ANN) was developed to improve the performance of the conventional CSA. The MCSA-ANN was later used to improve the hybrid ANN for the prediction task. The performance of MCSA-ANN and CSA-ANN was compared in Table 2. MCSA-ANN had outperformed CSA-ANN by producing lower RMSE and higher R 2 during both training and testing. The MCSA-ANN was also slightly faster than CSA-ANN. In short, the introduction of MCSA-ANN for the prediction was justified shown by the superior performance obtained during the testing and training process. The MCSA-ANN had exhibited the optimal number of neurons in hidden layer of 4, while the optimal values for learning rate and momentum rate are 0.6740 and 0.7965 respectively. This algorithm had also yielded lowest RMSE of 11.0534 W during training and 13.1348 W during testing. Other than that, it had produced maximum R 2 with 0.9994 and 0.9913 for both training and testing, with the least computational time of 17.9616 minutes when compared to other hybrid ANNs.

CONCLUSION
The performance of selected hybrid ANN models was compared. The results showed that CSA-ANN had outperformed EP-ANN and FA-ANN by producing the lowest RMSE and computation time. Apart from that, the introduction of MCSA-ANN showed that MCSA-ANN had outperformed CSA-ANN by exhibiting the lowest RMSE during training and testing as well as the highest R 2 during training and testing. In terms of computation time, MCSA-ANN needed less computation time than CSA-ANN. As a conclusion, the cability of MCSA-ANN for the output prediction was justified.