Optimized Kernel Extreme Learning Machine for Myoelectric Pattern Recognition

Received Oct 22, 2017 Revised Jan 5, 2018 Accepted Jan 20, 2018 Myoelectric pattern recognition (MPR) is used to detect user’s intention to achieve a smooth interaction between human and machine. The performance of MPR is influenced by the features extracted and the classifier employed. A kernel extreme learning machine especially radial basis function extreme learning machine (RBF-ELM) has emerged as one of the potential classifiers for MPR. However, RBF-ELM should be optimized to work efficiently. This paper proposed an optimization of RBF-ELM parameters using hybridization of particle swarm optimization (PSO) and a wavelet function. These proposed systems are employed to classify finger movements on the amputees and able-bodied subjects using electromyography signals. The experimental results show that the accuracy of the optimized RBF-ELM is 95.71% and 94.27% in the healthy subjects and the amputees, respectively. Meanwhile, the optimization using PSO only attained the average accuracy of 95.53 %, and 92.55 %, on the healthy subjects and the amputees, respectively. The experimental results also show that SW-RBF-ELM achieved the accuracy that is better than other well-known classifiers such as support vector machine (SVM), linear discriminant analysis (LDA) and knearest neighbor (kNN). Keyword:


RESEARCH METHOD 2.1. Kernel Extreme Learning Machine
ELM is a learning algorithm for single layer feedforward networks (SLFNs). In classical SLFNs, network parameters are tuned iteratively while in ELM; most of these parameters are determined analytically. Hidden parameters can be independently calculated from the training data, and output parameters can be determined by the pseudo-inverse method. As a result, the learning of ELM can be carried out fast compared to the other learning algorithms [25].
As described in [25], the output of ELM is defined by: where g(x) is the feature mapping in the hidden layer, T is the target and C is the regulation parameter of ELM. The feature mapping in the hidden layer of ELM can be replaced by a kernel function. Therefore, the formulation of the kernel based ELM is defined by: where and K is a kernel function as shown in Equation (4) to Equation (6).

Particle Swarm Optimization (PSO)
Particle swarm optimization (PSO) is a population-based stochastic optimization algorithm that has been applied widely in many optimization areas [29][30][31]. PSO is inspired by the social behaviors of animals like fish schooling and bird flocking [29]. The particle swarm does not use selection. It means that all population members survive from the beginning until the end [32]. In the PSO, a swarm of interacting particles moves in an n-dimensional search space of the problem's possible solution. Four elements that are a position ⃗ , a velocity ⃗ , the best previous (local) position ⃗ and the best global position ⃗ represent a particle in the swarm. Some generations are generated to update the particle's positions and velocities. The particles explore the promising domain to find the best solutions, which spread throughout the swarm. The parameter adaptations are given by: Where In the above equations,

PSO with Wavelet Mutation
PSO typically converges in the early stage of the searching process. This indicates that PSO tends to be trapped in the local optima. This shortcoming may influence the performance of the myoelectric finger classification. One of the solutions of the local optima is by injecting a wavelet function inside the PSO. The wavelet mutates the swarm particle in small probability to create a possibility for the swarm particle to get out from the local optima.
The wavelet mutation in PSO was proposed by Ling et al. [29]. A mutation chance is driven by a mutation probability p m  [0 1]. If x i (t) is selected to be mutated then a new position is given by: where par max and par min are the maximum and minimum position, respectively. As for , it is the Morlet wavelet function defined by: The variable "a" in the Morlet wavelet is determined by equation: The objective of the optimization using the wavelet-PSO is to find the optimum parameters of the kernel ELM that minimize the classification error of the finger motion recognition. A 3-fold cross validation was employed to measure the error. Moreover, the fitness function of particle x is defined by where N v is the number of cross validations, E n is the error in each validation process. The pseudo code of the wavelet mutation for optimizing the parameters of the kernel based ELM is presented in Figure 1.    Figure 2 shows the diagram block of the experiment conducted in this section. The EMG data was collected from eight able-bodied subjects, two females and six males aged 24-60 years old. Two EMG MyoScan™ T9503M sensors or electrodes were placed on the forearm of the subject to collect myoelectric signal from flexor policis longus (FPL) and flexor digitorium superficialis (FDS) muscles, as shown in Figure 3. The FlexComp Infiniti™ System from Thought Technology acquired the EMG signals with a sampling frequency of 2000 Hz and then amplified the signals with a total gain of 1000. The data collection and the myoelectric pattern recognition process were conducted using the Matlab 2012b installed in the Intel Core i5 3.1 GHz desktop computer with 4 GB RAM running on Windows 7 operating system. Digitally, the EMG data is filtered using a band-pass filter which filters the signals in the frequency range between 20 and 500 Hz and a notch filter was used as well to remove the 50 Hz line interference. The collected data was down-sampled to 1000 Hz.

The Experimental Setup
In this paper, the experiment considered ten classes of the individual and combined finger movements. The individual fingers consist of the flexion of thumb (T), index (I), middle (M), ring (R), and little (L) fingers, while the combined finger consists of the pinching of thumb and index fingers (T-I), thumb and middle fingers (T-M), thumb and ring fingers (T-R), thumb and little fingers (T-L), and closing the hand (HC). During the data collection, the subjects were asked to perform one finger movement for 5 s and then take a rest for 5 s. The subject repeated each movement six times. The data collected were divided into training data and testing data using 3-fold cross validation.
In the experiments, the myoelectric pattern recognition (M-PR) extracts features of waveform length (WL), slope sign changes (SSC), number of zero crossings (ZCC), sample skewness (SS), mean absolute value (MAV), mean absolute value slope (MAVS), root mean square (RMS), some parameters from Hjorth time domain parameters (HTD) and 6-order autoregressive (AR6) model parameters are included. Moreover, SRDA will project and reduce the dimension of the feature extracted. The experiment involved the steady state signal only and removed the transient state of the myoelectric signal. The majority vote with four previous states may be used to refine the classification performance.
As depicted in Figure 2, PSO mutated by wavelet is used to optimize the parameters of radial basis function extreme learning machine (RBF-ELM). This hybridization is called swarm-wavelet based RBF-ELM or SW-RBF-ELM. Some parameters should be determined at the beginning of the experiment. Two parameters of RBF-ELM are C and g (see Equation (4)). They are in the range of [2 -7 , 2 10 ], and [2 -7 , 2 10 ] for C and g, respectively. Then, the parameters of PSO (see Equation (7) and Equation (8)) are set as follows.
Parameter c 1 and c 2 are set at 2.05, and  is 0.9. Parameters r 1 and r 2 are random functions in the range of [0-1]. In addition, the optimization was done until 150 generations were completed with 30 particles in each generation. As for the parameter of the wavelet, the work in this section will vary the value of the wavelet parameters, as seen in Equation (9) and Equation (10)) except for α; it is determined randomly, according to [33].To test the efficacy of the proposed system, some experiments will be conducted. They are: a. The experiment on the influence of the mutation probability p m b. The experiment on the shape parameter ( Equation (11)) c. The experiment on the parameter g (Equation (11)) d. The experiment on the pattern recognition performance

Experiment on the Able-bodied Subjects 3.1.1. Mutation Probability pm
This section tested the influence of the mutation probability p m to the SW-RBF-ELM performance. The p m value is varied from 0 to 0.6. The parameter p m = 0 means no wavelet mutation in the PSO. Besides, is equal to 0.2 and g is equal to 10000. The experimental results are presented in Figure 4. Figure 4 indicates that on the parameter p m = 0, the fitness value of the PSO is larger than that with p m more than 0, even when it is the largest value. The lower the fitness value, the better the system, so the PSO with wavelet mutation is better than without wavelet mutation. Therefore, the wavelet mutation can enhance the optimization process. Moreover, in general, the figure also shows that the more the mutation probability, the less the fitness value. However, the p m = 0.5 is the optimum value among the tested values.  Table 1 gives more information regarding the mutation probability p m across different subjects. InTable 1, the underlined value indicates the minimum value for each subject. This table emphasizes the fact in Figure 4 that p m = 0.5 is the most accurate PSO across seven subjects, out of eight. Although the accuracy of the parameter p m =0.6 is the highest, it occurred in five subjects only. Another interesting fact is also found in the Table. The mutation wavelet does not provide a benefit to the optimization process on two subjects, S5 and S8 because the accuracy of the system with wavelet mutation and without is very similar. This fact shows that the wavelet mutation in the PSO does not fully ensure the improvement in the classification performance. However, there is a high probability that the optimization process will be improved. Finally, the parameter p m = 0.5 is selected for the rest of the experiment. The higher value of the parameter p m increases the searching space of the optimization in PSO. If the number of elements in a particle is small, it is preferable to increase the value of the parameter. Figure 4 implies that the higher value of p m tends to give good optimization performance. This phenomena matches with the fact suggested by Ling et al. [29]. They recommended a higher value of p m in between 0.5 -0.8 for a small number of elements in a particle. In this research, the number of elements is two.
To examine the benefit of wavelet mutation statistically, an analysis of variance (ANOVA) test was conducted on the fitness value of the PSO without wavelet mutation and with wavelet mutation p m = 0.5. The confidence level p is set at 0.05. ANOVA test produced p = 3.69 x 10 -7 . This result concludes that the enhancement produced by wavelet mutation is statistically significant.

Shape-parameter
This section varied the value of shape parameter in Equation (11). The shape parameter is varied among 0.1, 0.2, 0.3, 0.5, 2 and 5. The value of the parameter p m is 0.5 following the result in section 0. Furthermore, g is equal to 10000. The experimental result is presented in Figure 5. Figure 5 indicates that = 2 converged earlier than the others did. The final fitness value of it is the second worst after = 5. On the other hand, the small value of gave a good optimization process. These facts imply that the high value of is not a good option for optimization of SW-RBF-ELM. The best optimization process is shown when = 0.2.  Table 2 draws different finding from Figure 5. The table shows that SW-RBF-ELM with  = 0.1 achieved the highest average accuracy, not  = 0.2. Besides, it attains the highest accuracy across four subjects, which is similar to  = 0.2. By considering the fitness value and the average accuracy performed,  = 0.2 is selected as the optimal shape parameter.

Parameter g
The previous two experiments have selected two optimum parameters, p m = 0.5 and  = 0.2. This section tries to get the optimum g parameter. The parameter g (Equation (11)) is varied from 100, 1000, 10000 and 100000. The experimental results are presented in Figure 6. Figure 6 depicts the fitness values of four different g values. This figure indicates that the big number of g value give better accuracy than the small one. The g = 10000 exhibits the best performance. This fact is supported by the accuracy of SW-RBF-ELM in Table 3. Although the accuracy of the parameter g = 10000 is the lowest one on average across eight subjects, it is the highest in the over half of the subjects, which is five out of eight. These results confirm the recommendation of Ling et al. [29]. They found that by setting the parameter g in the high value, the other parameter could be chosen by trial and error.

Pattern Recognition Performance across Subjects
The previous sections conducted some experiments to determine the optimum parameters of the wavelet. They are p m =0.5,  = 0.2 and g = 10000. This section applied those values to SW-RBF-ELM and did analysis on the results especially on the comparison between PSO with wavelet mutation and without mutation. The result is shown in Figure 7.  Figure 7 depicts the average accuracy of radial basis function ELM (RBF-ELM) that is optimized by PSO with wavelet mutation (SW-RBF-ELM) and without mutation (SRBF-ELM). The figure indicates that SW-RBF-ELM achieves better accuracy than SRBF-ELM across seven subjects. SRBF-ELM is as accurate as SW-RBF-ELM in one subject only, which is subject S8. Therefore, the probability of the improvement of the performance using wavelet mutation is 7/8 x 100 % = 87.5 %. On average, SW-RBF-ELM attained an accuracy of 95.71 % while SRBF-ELM achieved the accuracy of 95.54 %.

Pattern Recognition Performance on the Movement
This section investigates the performance of both systems, SRBF-ELM and SW-RBF-ELM, in classifying finger movements. The myoelectric pattern recognition classifies ten finger movements. They include thumb (T), index (I), middle (M), ring (R), and little (L) finger movements. The other movements are thumb-index (TI), thumb-middle (TM), thumb-ring (TR), thumb-little (TL), and the hand close (HC) movements. Figure 8 presents the classification results of SRBF-ELM (without wavelet mutation) and SW-RBF-ELM (with wavelet mutation). Figure 8 shows that SW-RBF-ELM is better than SRBF-ELM in classifying two individual finger movements (T, and M), and four combined movements (TI, TM, TR, and TL). On the other hand, SRBF-ELM is better than SW-RBF-ELM in two movements only: L and HC. As for finger movement I and R, both systems exhibited a similar performance. Overall, the SW-RBF-ELM is better than SRBF-ELM. In other words, the wavelet mutation in PSO enhances the classification performance of the pattern recognition system. However, the analysis of variance test (ANOVA) set p = 0.05 yields p is equal to 0.96. Therefore, the improvement is statistically not significant. This result confronts the ANOVA test result in Section 0 that proved the significance of the existence of the wavelet in PSO. These two results can be accommodated by saying that the enhancement of wavelet mutation in the optimization process is statistically significant, but it is not significant in the classification performance.  Another fact found in Figure 8 is that SRBF-ELM and SW-RBF-ELM exhibit relatively bad performance in classifying all combined movements and little finger movement. The phenomena can be investigated through the confusion matrix in Table 4 and Figure 9. Table 4 shows that the SW-RBF-ELM mostly misclassified the little finger movement (L) to thumb-index motion (TI) with the accuracy of 2.19 %. Besides, the system also misclassifies L to movement R and TL. As for the combined movement, SW-RBF-ELM generally misclassified them to the individual movement they belong to. For instance, the movement  Figure 9 helps the reader the get a visual graph of the confusion matrix. Figure 9. The confusion matrix plot of the classification result of SW-RBF-ELM

SW-RBF-ELM and other well-known Classifiers
In this experiment, the performance of SW-RBF-ELM is compared to other well-known classifiers such as original ELM using sigmoid activation function (Sig-ELM), SRBF-ELM, SVM, LDA, and kNN. The experimental results are depicted in Figure 10. Figure 10. The accuracy of SW-RBF-ELM and other well-known classifiers for finger movement recognition using 3-fold cross validation Figure 10 shows that SW-RBF-ELM is the most accurate classifier among seven different classifiers in recognizing ten finger movements using EMG channels across eight able-bodied subjects. This finding is supported by Table 5 that presents the average accuracy achieved by each classifier. SW-RBF-ELM achieved the accuracy of 95.71 %. Furthermore, SW-RBF-ELM achieved the highest accuracy on four subjects, while it attained the second lowest accuracy on the subject S3 and S4.

493
The comparison of SW-RBF-ELM and the others can be made more obvious using the one-way ANOVA test, as described in Table 6. The table shows that the performance of SW-RBF-ELM and the other classifiers is not significantly different for the majority of subjects, except for subject S6. With this subject, most classifiers could not classify the ten finger movements as well as SW-RBF-ELM and SRBF-ELM. As for the subject S3, SW-RBF-ELM could not achieve a good accuracy, it is even worse than the other classifiers; more significantly it is worse than Sig-ELM, SVM, and LDA.

Experiment on the Amputee Database
This section tested the performance of SW-RBF-ELM and SRBF-ELM to classify 12 finger movements on the EMG signals collected from the amputee subjects. The data collection is presented in [34]. The myoelectric pattern recognition used in this experiments is the same as the system used in section 3 and Figure 2. For wavelet parameters, the values of the parameters are p m = 0.1,  = 2 and g = 10000, following the work of Anam and Al-Jumaily [35]. Figure 11 depicts the experimental results of SRBF-ELM and SW-RBF-ELM on five amputee subjects. Figure 11 show that the SW-RBF-ELM achieved better performance than SRBF-ELM across five amputees except on amputee A5. On the amputee A5, SRBF-ELM is better than SW-RBF-ELM. Overall, SW-RBF-ELM outperformed SRBF-ELM. Probably, the optimization process in the PSO influences the superiority of SW-RBF-ELM over SRBF-ELM. Figure 12 gives clearer information about this assumption. It is shown in Figure 12 that after 30 th generation, the PSO did not change the fitness value. Meanwhile, the wavelet mutation helped the PSO to avoid the local optima.
Furthermore, a statistical test on the accuracy using one-way ANOVA (p set at 0.05) was also done. The performance of the SW-RBF-ELM is significantly different from swarm ELM (p = 0.036). The SW-RBF-ELM achieved the average accuracy of 94.27 %, while SRBF-ELM produced the average accuracy of 92.55 %. In addition, the classification performance in regards to the finger motion was observed. As shown in Figure 13, the SRBF-ELM was able to classify the flexion motions with the average accuracy more than 90%. In contrast, the extension motions were classified with the average accuracy less than 90%. Similarly, the SW-RBF-ELM recognized the flexion motions better than the extension motions, but with the average accuracy that is better than the SRBF-ELM. Figure 13. The accuracy of different finger motions across five amputees The confusion matrix in Table 7 provides information about the misclassified finger motions. According to the Figure 13, SW-RBF-ELM poorly classified the Little extension (Le), Middle extension (Me), and Ring extension (Re). Me was mostly misclassified to Thumb abduction (Ta) and Middle flexion (Mf). Furthermore, the system mostly misclassified the little extension (Le) to Re and vice versa. Even though the misclassified motions were present, arguably the SW-RBF-ELM has succeeded in recognizing different finger motions on five amputee subjects with the accuracy of about 94%.   To conclude, the proposed pattern-recognition system, which employs PSO mutated using a wavelet function to optimize the kernel based ELM (SW-RBF-ELM), was able to recognize eleven imagined finger motions on five trans-radial amputees with the high accuracy of 94.27 % even though it employed only two EMG channels. The proposed system performed better than standard PSO-ELM (SRBF-ELM).

DISCUSSION
The previous research [36] has shown that RBF-ELM is an promising classifier for myoelectric pattern recognition. However, the parameters of RBF-ELM should be selected properly. In this article, two kinds of PSO are employed to optimize the parameters of RBF-ELM, PSO and wavelet-PSO, that produce SRBF-ELM and SW-RBF-ELM, respectively. Both classifiers have been tested on the healthy and amputee subjects. In general, SW-RBF-ELM is better than SRBF-ELM and RBF-ELM. To show more general performance of SW-RBF-ELM, the comparison of the proposed methods and other well-knowns have been conducted, as shown in Figure 10 and Table 5. The results imply that the parameter optimization on RBF-ELM using wavelet-PSO can improve the performance of RBF-ELM. In addition, the results support the result in [22] and [23] that the optimization is needed in ELM to look for the optimized parameters for ELM. However, these two publications optimized the number of the units in the hidden layer. Meanwhile, in this article, the optimization is conducted for radial basis function parameters.

CONCLUSION
This paper proposed the optimization of radial basis function extreme learning machine (RBF-ELM) using particle swarm optimization (PSO) and the hybridization of wavelet and PSO. The former is called SRBF-ELM and the later is named SW-RBF-ELM. The role of the wavelet in SW-RBF-ELM is to increase the searching space of the PSO in order to avoid the local optima that possibly occur in the PSO process. The experimental results show that the wavelet mutation improves the optimization process of the PSO. Consequently, the wavelet mutation in PSO also enhances the classification performance of the system. Both classifiers have been tested on the able-bodied subjects and amputees. On the able-bodied subjects, the accuracy of SW-RBF-ELM is 95.71 % while SRBF-ELM is 95.53 %. The improvement of wavelet mutation on the amputees is more significant than that on the able-bodied subjects. On the amputees, the SW-RBF-ELM achieved the average accuracy of 94.27 %, while SRBF-ELM produced the average accuracy of 92.55 %. The experimental results also show that SW-RBF-ELM achieved an accuracy that is better than well-known classifiers such as SVM, LDA, and kNN.