Artificial Neural Network Application in Prediction of Concrete Embedded Antenna Performance

Artificial Neural Network (ANN) has been extensively applied to microwave device modeling, design and simulations. In the present paper, the prediction of concrete embedded antenna performance using ANN is presented. The ANN model takes antenna embedded depth and concrete dielectric constant as inputs and gives antenna radiation efficiency, gain and input impedance as outputs. The Particle Swarm Optimisation (PSO) is employed to search the global optimal weights and bias for ANN, then Bayesian Regularisation (BR) is used to train the ANN for overcoming the overfitting issue. It is found that the PSO computation iteration for optimal network weights and bias searching is less than gradient descent algorithm. A PSO-BR neural network (PSO-BRNN) and back-propagation neural network (BPNN) are trained to compute and predict the antenna performance. The PSO-BRNN performance is better than BPNN in terms of accuracy and generalisation.


I. INTRODUCTION
Mobile operators have experienced an exponential traffic growth in their network in the last decade. Ultra dense cell deployment is considered as a promising way to fulfil the large amount of traffic demand that takes place indoors, and the dense deployment of small cells in buildings facilitates the improvement of throughput in the next generation of cellular communication [1]. However, the physical dimension of indoor small cells can lead to the extra space occupation and disfunction of the building [2]. A feasible solution for these issues is to integrate antennas with the building materials, such as embedding antennas into concrete walls. In fact, due to the strong coupling between antennas and concrete, it is challenging to calculate and predict the antenna performance once antennas are embedded. The numerical method based on full-wave simulations (e.g. method of moments, finite element analysis) can provide rigorous solution to the antenna performance in concrete wall. However, the full-wave simulations are computation-intensive, and requiring a large amount of computation time and computer memory. As a result, timesaving and fast surrogate models are required to address highdimensional and nonlinear electromagnetic problems.
Artificial neural network (ANN) has already been recognised as a feasible tool for microwave modelling and simulation in recent years [3], which can learn and solve the  [5] and antenna analysis and synthesis [6]. In [7], the ANN based models were presented to compute the resonant frequency of antenna with lower error. Generally, the gradient based training algorithm such as back-propagation (BP) algorithm is used in ANN training process. However, the slow convergence ratio and local optimum issues are the main drawbacks of gradient based algorithm [8]. As a result, the optimisation algorithm such as particle swarm optimisation (PSO) [9] and genetic algorithm (GA) [10] are combined with ANN and can significantly improve the performance of ANN. In addition, the PSO could also be applied to optimise the topology and parameters in the ANN [8]. On the other hand, the overfitting issue and generalisation capability are main concerns for ANN model. In order to improve the accuracy of ANN, Early-Stopping strategy and Bayesian Regularisation (BR) [11] are introduced to address these issues, which prevents the overtraining occurring and effectively improves the generalisation capability of network.
In the present work, ANN is utilised to compute and predict the performance of a concrete embedded antenna for indoor An structurally integrated antenna with multi-layer configuration that proposed in [12] is selected because of its excellent mechanical and electrical performances. The antenna is fully embedded in a solid concrete slab as shown in Fig. 1, and the concrete has a dimension of 1000 mm × 1000 mm × 200 mm. The embedding depth d of antenna is measured as the distance between the top concrete-air interface and the top surface of the antenna. The effect embedding depth d and concrete dielectric constant r on antenna performance are going to be investigated, thus other electrical property such as loss tangent is fixed to 0.03 (tan δ = 0.03). The proposed antenna is optimised to operate at 3.5 GHz, and it is sandwiched among lower facesheet (LF), a honeycomb (HC) structure and an upper facesheet (UF) for obtaining better electrical and mechanical characteristics in the concrete wall, the electrical properties and thickness of each layer are listed in Table I.

A. ANN construction and data preparing
Neural network is a powerful tool to map the nonlinear and complicated relationship between inputs and outputs. In the present work, ANN model is used for the prediction of proposed antenna performance while embedded in the concrete wall. For the concerned input variables, the antenna embedding depth d and the concrete dielectric constant r are selected, while the antenna's radiation efficiency η rad , gain G, input resistance R in and input reactance X in are considered as outputs. Therefore, the suggested network architecture consists of one hidden layer, 2 input neurons and 4 output neurons as shown in Fig. 2. The hidden layer consists of 75 neurons which are fully connected to the output layer that gives the desired values of antenna performance. The activation function used in hidden layer is tangent sigmoid, while simple linear function is used in the output layer.
Given a data set D = [x j , y j ] T consists of inputs vector x j and outputs vector y j , a supervised nonlinear regression task is going to be solved by ANN. The relationship between the inputs vector and the outputs vector could be written as: where the corresponding inputs and outputs of ANN model are: The range of inputs are 0.001 m to 0.189 m with step width of 0.002 m for the embedding depth d, and 4 to 9 with step width of 1 for the concrete dielectric constant r . The data set D is generated using Computer Simulation Technology (CST) Studio. The length of D is 570, all the obtained data have been normalised between 0 and 1 for avoiding the error caused by different order of magnitude. The data set D is randomly divided as training set and testing set, wherein, 85% data are classified as training set and the rest of 15% are testing set. All the optimisations and ANN trainings are performed on an Intel Xeon W2135 3.70 GHz machine with 32 GB RAM.

B. PSO and ANN
The conventional ANN utilises the gradient based method to train generally, and the convergence of ANN strongly depends on the initial guess of weights and bias. BP algorithm is a well known training method for neural networks, it is based on the gradient descent algorithm. Hence, the initial point of weights and bias is essential for the BP training, if the weights and bias are not initialised properly, the results are likely to get stuck in a local optimum and consequently the solution is not the best.
PSO is a random search algorithm based on group cooperation, which is developed by emulating the foraging behavior of birds. It is an effective evolutionary algorithm that can find the global maximum or minimum of target function. In this study, the mean square error of neural network is taken as the evaluated fitness in PSO, which is calculated as: where y j is the network outputs vector, y j is the outputs vector of data set D, N is the total number of data, k is the total number of output. In the present work, the N and k are 570 and 4, respectively.
(4) is the target function that needs to be optimised in PSO. Since the neural network learning process is mainly to update the weights and bias, thus the location of the particles in PSO are corresponding to all weights and bias in the network. E is taken as the fitness function of the PSO algorithm, and all the weights and bias are optimised by PSO algorithm in order to obtain the global minimum of the fitness function. In each iteration, the fitness function of each particle is calculated, and the corresponding position P i and velocity V i are updated according to the calculated value of fitness function, personal best p best and global best g best , and the updated regulations are: where c 1 and c 2 are acceleration coefficients, φ 1 and φ 2 are random and positive number with uniform distribution ranged between 0 and 1, p best is the personal best position of particle, and g best is the global optimum position of particle. ω is the inertial weight, the linear decline weight (LDW) strategy is used to manipulate ω for the optimum solution search. The larger ω facilitates global searching, while the smaller ω is beneficial to precise local searching. The LDW strategy is expressed as: where ω max is the maximum inertial weight, ω min is the minimum inertial weight, t is current iteration, and t max is the maximum iteration of PSO. At the beginning, the ANN is building with specific topology, thus the dimension of particle can be determined. The dimension of each particle equals to the total number of weights and bias in the network, in this work we implement a PSO to optimise 529 weights and bias in total. 500 particles are employed and the computation iterates 700 times. Firstly, the number of particle is selected, then the particle positions and velocities are randomly initialised, each particle i is characterised by its position vectors X i and velocity V i . The position boundry [−X max , X max ], velocity boundry [−V max , V max ], inertial weight range [ω min , ω max ], acceleration coefficients c 1 and c 2 , and the maximum iteration are defined, and these parameters are presented in Table II.
In each iteration of PSO, the value of fitness function of individual particle is calculated, and the velocity and position of all particles is updated using (5) and (6). Once the optimisation criteria meets, the PSO iteration terminates. g best stores the global optimum solution for the network weights and bias, then the g best is reshaped and assigned according to the topology of network which is prepared to be trained.

C. Bayesian regularisation of ANN
The Bayesian regularisation (BR) is used to mitigate the potential overfitting problem that may occurs in ANN training process. The overfitting and overtraining can lead to the loss of regression accuracy and generalisation of the network. For the purpose of overcoming overfitting issue, the BR adds an addition regularisation term to the object function as: where the F is the objective fucntion, E D is the sum of squared errors of network, E ω = 1 m m i=1 w 2 i is the sum of squared errors of the weights in network, m is the total number of weights. α and β are the hyperparameters that need to be estimated in the training process. Network weights ω are regarded as random variables and the density function could be written as: where D represents data set, and M is the ANN topology; P (w|D, α, β, M ) is the posterior distribution of ANN weights, P (D|w, β, M ) is the likelihood function represents the training data occurrence probability with given weights, P (w|α, M ) is the prior density of weights before data is fed. The BR algorithm is explained in detail in [10]. In general, all the noise in data is assumed to be Gaussian additive noise, with this assumption the probability density function of weights in (9) could be estimated. Then the hyperparameters α and β are determined by solving the Hessian matrix of F at the minimum point. Gauss-Newton approximation is used to solve Hessian matrix while the Levenburg-Marquardt (LM) training algorithm is used to search the minimum point, the training process terminates once the training goal is met. The flow chart summarizing major step of PSO-BRNN training is shown in Fig. 3.
This paper's copyright is held by the author(s). It is published in these proceedings and included in any archive such as IEEE Xplore under the license granted by the "Agreement Granting EurAAP Rights Related to Publication of Scholarly Work."  IV. RESULT AND DISCUSSION PSO are used for ANN learning process, weights and bias are adjusted in order to reach the minimum of the error between ANN estimation and actual values. The training effect of PSO is compared with the BP algorithm, the comparisons of convergence and regression accuracy are exhibited. Fig. 4 presents the comparison of PSO and BP algorithm in terms of convergence rate. It can be observed that PSO performs better than the BP algorithm. PSO converges faster than BP algorithm, and the iteration is terminated with a lower mean square error (MSE) and mean absolute percentage error (MAPE) which is calculated in (10). The training metrics such as performance, accuracy and training time are illustrated in Table III. It apparently shows that the MSE of PSO-BRNN is much lower than the classic BPNN, with 0.0002 to 0.15, as well as the error ratio, 1.79% to 3.92%. In addition, by applying the PSO and Bayesian regularisation, the iteration times of convergence is lower than BPNN, thus result in the reduction of training time of PSO-BRNN (151 seconds) than BPNN (270 seconds). It can be observed that the learning accuracy of PSO-BRNN is better than the BPNN, the BPNN cannot map the fluctuation as the embedded depth increases. This problem is caused by the local minima issue, once a network is trained with gradient descent based algorithm, the local minima is likely to be considered as the best result by network. Therefore the error between network estimation and actual value cannot be further minimised, then the weights and bias in the network stop adjusting and maintain in a plateau. While the PSO-BRNN is trained with optimum weights and bias, so it can map the nuanced fluctuation of the antenna performance, which indicates the learning ability of PSO-BRNN is better.
The generalisation capability is essential for networks, and the performance of network is mainly measured by its gener-

V. CONCLUSION
In this paper, the ANN-based method has been presented to predict the performance of concrete embedded antenna. A hybrid ANN (PSO-BRNN) is trained to predict the performance of concrete embedded antenna, and the training metrics are compared to the BPNN. The PSO algorithm is utilised to search for the global optimum weights and bias for ANN, and the BR algorithm is employed to overcome the overfitting issue of ANN. Compared to BPNN, PSO-BRNN exhibits an more accurate and efficient manner in computation and prediction, and leads to a reduction in MSE and iteration times. The generalisation capability of different networks is tested with the new inputs vector, the outputs of PSO-BRNN reveals an excellent generalisation capability, and its learning ability excels BPNN. The result indicates that the PSO-BRNN is an effective method for the concrete embedded antenna performance prediction for indoor communication.