Contradictory of the Laplacian Smoothing Transform and Linear Discriminant Analysis Modeling to Extract the Face Image Features

Laplacian smoothing transform uses the negative diagonal element to generate the new space. The negative diagonal elements will deliver the negative new spaces. The negative new spaces will cause decreasing of the dominant characteristics. Laplacian smoothing transform usually singular matrix, such that the matrix cannot be solved to obtain the ordered-eigenvalues and corresponding eigenvectors. In this research, we propose a modeling to generate the positive diagonal elements to obtain the positive new spaces. The secondly, we propose approach to overcome singularity matrix to found eigenvalues and eigenvectors. Firstly, the method is started to calculate contradictory of the laplacian smoothing matrix. Secondly, we calculate the new space modeling on the contradictory of the laplacian smoothing. Moreover, we calculate eigenvectors of the discriminant analysis. Fourth, we calculate the new space modeling on the discriminant analysis, select and merge features. The proposed method has been tested by using four databases, i.e. ORL, YALE, UoB, and local database (CAI-UTM). Overall, the results indicate that the proposed method can overcome two problems and deliver higher accuracy than similar methods.


Introduction
Appearance-based method to extract the image features is still conducted by many researchers such as Principal Component Analysis (PCA).PCA is the oldest method to extract the image features using statistical methods [1]- [5].The method also has been developed into many methods, i.e. multi linear PCA [6], Non-Linear Principal Component Analysis, Improved principal component regression [7], and kernel subspace [8].
The problem of the image computational is high dimensionality, for both video and still images.Many researchers have tried to obtain the best solution related to dimensionality reduction.However, many methods have been improved, but they just leave another problem.The oldest appearance method to extract the features is Principal Component Analysis (PCA).PCA projects new space into eigenvector based on decreasing ordered-eigenvalues, where the results can reduce meaningfully the image samples as the training sets, where the dimensionality of the image as the training has same size or smaller compared to the number of training samples.In this case, PCA only performs efficiently when the number of training sets is fewer than image dimensionality.Unfortunately, PCA will fail to project the training sets when the number of training sets is more than the image dimensionality.In addition, PCA also represents global manifolds of the object as the training samples, whereas local manifolds cannot be well represented by PCA.
LDA can overcome the limitation of PCA, where LDA is able to extract features up to a number of classes.Another result of a researcher is Linear Discriminant Analysis (LDA) [8]- [11].LDA maximized between class scatter and minimized within class scatter.LDA projects eigenvectors based on the ordered-eigenvalues of between and within class scatter.LDA is better than PCA, because LDA can reduce more significant than PCA, i.e. maximum features for LDA is number of classes, while maximum features for PCA is number of training samples, where training samples are greater than number of classes [11], [12].In addition, LDA has also maximized the distance between classes to each another, and minimized the distance of ISSN: 1693-6930  Contradictory of the Laplacian Smoothing Transform and Linear Discriminant … (Arif Muntasa) 1795 member on the same class, therefore the member of class will be easly separated.However LDA also leaves a new problem, the inability to reduce the dimensions of the image, when the number of classes exceeds the dimensions of the image.Another approach is also conducted by using laplacian smoothing, where the Kronecker matrix operation is employed and selected to earn the optimum values.As we know that laplacian smoothing uses Neuman discrete and Kronecker matrix to obtain the new space, where the value used is -1 the first or the last diagonal and -2 for other diagonal elements.The negative values will generate negative new spaces, and the negative new spaces can remove the dominant characteristics.The Kronecker matrix result usually cannot be solved to obtain the eigenvalues and eigenvectors because of singular matrix, therefore it is necessary to appear the features by replacement the negative into positive emenets.
In this research, we improve two limitations of the laplacian smoothing.The first, we propose contradictory of the laplacian smoothing transform to generate the positive diagonal matrix.The second we propose to overcome singularity matrix on the laplacian smoothing transform.We also propose to combine between the contradictory of the laplacian smoothing and the linear discriminant to earn the best features before classification applied.
The rest of the paper is composed as follows, proposed method, experimental material and scheme, results and discussion, compare to similar methods, and conclusions.On the proposed method, we explain the framework of the proposed approaches, i.e. calculate contradictory of the laplacian smoothing matrix, calculate the new space modeling on the contradictory of the laplacian smoothing, calculate eigenvectors of the discriminant analysis, calculate the new space modeling on the discriminant analysis, select and merge features.

Research Method
The Laplacian transform modeling has been implemented to extract the image features.However, this method can be also improved by Neuman Discrete, where the original of the method uses the negative values, which are -1 and -2.The results showed that the resulting features can eliminate the dominant characteristics.Therefore, we proposed to reverse the values (as the first contradictory) to appear the dominant characteristics.Another weakness of the laplacian smoothing is singularit, where it usually appears when eigenvalues and eigenvectors are calculated.We proposed the model to overcome the singularity (as the second contradictory).The proposed approach can be displayed in Figure 1.Suppose, training image sample is symbolized by F w, h , while testing image sample is represented by G w,h .The image samples have size with w and h as width and height.To process an image, the size of image must be resized into one dimensional model, an original image size will be changed into row matrix model, which is matrix with width wh and height 1.The row matrix model of an image can be shown as follows If number of training samples is m, then the matrix training sample can be composed into matrix with width wh and height m as follows The testing sample must be also represented by one dimensional matrix, which is row matrix model in Equation (3).Equation ( 2) and (3) will be applied to earn the features of the training and the testing samples.

Calculate Contradictory of the Laplacian Smoothing Matrix
Contradictory of the Laplacian Smoothing Matrix is a new approach to produce the new projection based on the opposite of the Laplacian Smoothing Transform.We have developed the new model based on the Laplacian smoothing matrix.At first, we compose the matrix with zero values except on last position, where an index of j applies to ∀,  ∈ 1, 2, , ⋯ , ℎ ×  and index of k instructs to ∀,  ∈ 1, 2, , ⋯ , ℎ ×  as shown as follows Furthermore, the value 1 will be inserted on the above matrix using Equation ( 5) and (6), where the index of j is ∀,  ∈ 1, 2, 3, ⋯ ,  − 1 The Equation ( 4), ( 5) and ( 6) produces the sparse matrix, where only the several positions have element with value 1.The results of Equation ( 4), (5), and (6) are added for each column as shown in the following equation Equation ( 7) produces row matrix model, where it has hw columns and a row.The result of Equation ( 7) is changed into diagonal matrix.The matrix diagonalization result is subtracted by Equation (5) as shown as follows In this case  denotes the formation of the diagonal matrix, where all matrix elements are 0 values except the diagonal elements is filled by L 2 (1,k).The results of Equation ( 8) can be also represented as follows In this case, ∀,  ∈ 1, 2, , ⋯ , ⌈ ℎ× 2 ⌉, the results of Equation ( 9) is applied to compute the Newman Discrete matrix as follows Output of Equation ( 11) is employed to calculate ∆ ℎ, through Kronecker operation as shown in the following equation The second contradictory of the Laplacian smoothing matrix can be used to avoid singularity matrix through as the follows The result of Equation ( 13) is applied to compute the Eigenvalues and eigenvectors by using characteristic equation The Eigenvalues generated by Equation ( 14) must be ordered decreasingly and followed by a change of column index of the eigenvectors as follows

Calculate the New Space Modeling on the Contradictory of the Laplacian Smoothing
The results of equation ( 16) are required to earn the new space as object features, for both on the training and testing sets.The new space of the training sets can be calculate by multiplication of the training sets  ,ℎ× and eigenvectors of the contradictory of the laplacian smoothing  ℎ×, using equation While the new space of the testing set can be easily obtained using multiplication between the testing sets  1,ℎ× and eigenvectors of the contradictory of the laplacian smoothing  ℎ×, using equation

Calculate Eigenvectors of the Discriminant Analysis
The discriminant analysis is produced by rationalization of the fisher's discriminant analysis [13]- [15].The result of the discriminant analysis can be used as linear classifier or well known as dimensionality reduction.It can be earned to maximize between scatter and to minimize within scatter.Between and within scatter can be defined as follows The Eigenvalues and eigenvectors of the discriminant analysis can be easily obtain by using equation The calculation results of the Equation ( 21) deliver non-ordered eigenvalues and eigenvectors.In order to the best characteristics, the ordered-eigen values must ob ordered decreasingly and supported by column index of the eigenvector [1], [6], [16].The results are follows

Calculate the New Space Modeling on the Discriminant Analysis
The new space as features can be earned by multiplication between the training sample  ,ℎ× and the eigenvector  ℎ×, (Equation ( 23)).In this case, the matrix results have a size of m, where m is the number of training sets While the new space of the testing set can be easily obtained by multiplication of the testing sets  1,ℎ× and eigenvectors  ℎ×, of the discriminant analysis as follows

Select and Merge Features
Before classification process, the features delivered by contradictory of the laplacian transform and discriminant analysis must be selected first.Based on the Equation (17), we can re-write it into the matrix form as follows At first, the features of the contradictory of the laplacian transform are selected u features, where u=m-q, and ∃,  ∈ 1, 2, 3, ⋯ ,  − 1.It means that, we remove q features.The feature selection results can be written as follows The feature extraction of the discriminant analysis as shown in Equation ( 24) can be also expressed on the matrix form with m columns and n rows as shown follows: Moreover, the features generated by discriminant analysis are also selected v features, and v=m-s, where ∃,  ∈ 1, 2, 3, ⋯ ,  − 1.In this case, s features are removed as shown follows The results of the feature selection are merged into a matrix, where the features of the discriminant analysis are put on the right of the contradictory of the laplacian transform features as shown follows: After selection and merging of the features on the training sets, the similar process is also employed on the testing sets.The Equation (18) as the features of the testing sets of the contradictory of the laplacian transform and discriminant analysis are formulated follows: Furthermore, the number of feature selection on the testing sets must take the same size with training sets.In this case, we have selected u features for the contradictory of the laplacian transform and v features for the discriminant analysis.The selection and merging results are:

Classify the Testing Sets
The feature selection and merging results are compared between the Equation (33) and Equation (30).In this research, we employ the Euclidian distance to obtain the nearest neighbor as follows  = ‖ − ‖ (34)

Experimental Material and Scheme
In this research, four different databases have been utilized to test the robustness of our proposed method, which are ORL [13], YALE [14], UoB [15], and CAI-UTM.The ORL has four hundred images, where ten people were taken with ten different accessories, poses, and expressions.In this case, an image of the ORL has 112 and 92 pixels for height and width [13].The YALE only involves fifteen persons, and for each person was captured from eleven different models, they are lighting, view angle, expressions, and also pose.The size of the image on the YALE has been changed into 136 pixels for height and 104 pixels for width [14].The third database is UoB face image, where thirty persons have been involved to be created the image samples with ten poses, where the size of image is 140 pixels for height and 120 pixels for width.This size is the result of resizing from the original size [15].The last database is CAI-UTM, University of Trunojoyo Madura.We have taken a thousand images from a hundred persons, where for each person has support ten different models, which are expression, accessories, angle of view, and pose.Four databases are shown in Figure 2. The ORL, YALE, and UoB samples are shown on the first, the second, and the third row respectively, whereas on the fourth row is sample of the CAI-UTM database.
Figure 2. Samples of Face Image, ORL [13], YALE [14], UoB [15], and CAI-UTM To proove the robustness of the proposed method, we employed four schemes for each database as shown in Table 1.We use P(c, t, e) as scheme models for the testing, where c represents c-fold cross validation.Variable of t depicts number of training sets used, while e symbolizes number of testing sets employed.For example P(4, 3, 7) means that cross validation is conducted four times with four different indexes, number of training sets employed are three images, and number of testing sets performed are seven images.

Results and Discussions
We have randomized the training set index for each scheme based on Table 1.For all schemes, cross validation utilized is 4-folds, whereas number of features selected and number of experiments conducted are depend on number of images on the database image applied and k-fold cross validation implemented as shown in Table 2.The results will be discussed sequentially, which are the results of the ORL, the YALE, the UoB, and the CAI-UTM face image database.

The Results of the ORL Database
We have carried using four schemes on the ORL database, which are experimental using two, three, four, and five training samples.For each scheme, we conducted different eighty times.The results are demonstrated in Figure 3.It indicates that the different results are delivered from all schemes, where the worst accuracy occurred on the fourth scheme (using two images as training sets), while the best accuracy is obtained by using four images as training sample.
From the experimental results it can be seen that, the difference in outcomes between the first, second, third, and fourth schemes.In the first scheme twenty experiments have been performed and the results of each experiment are not much different, therefore the accuracy deviation on the first scheme is also very small, that is 0.009.
Experimental results in the second scheme show that all experiments are better than the first scheme.The addition of an image as the training sample has provided an increase in accuracy across all experiments.It can be proved by the best, the average, and the worst accuracies also increase for all experimental, which are 87.86%,85.71%, and 86.70% for the best, the average, and the worst accuracy respectively.The standard deviation obtained is also better than previous scheme that is 0.006.This shows that the result difference of the experiments in the first scheme is smaller than the first scheme.In the third and fourth schemes also occurred the same phenomena as in the first and second scheme.The results of all schemes show that the best results have been achieved by the last scheme, i.e. 97.50% for maximum accuracy, 94.50% for minimum accuracy and 96.18% for average accuracy, while the best standard deviation appeared on the second scheme as demonstrated in Table 3. Errors in facial image recognition occur because the training data selected does not represent all models of the tested image, this is evidenced by the addition of training images with different models can improve the accuracy of face image recognition as shown in Figure 3 Figure 3. Accuracy on the ORL

The Results of the YALE Database
The images on the YALE database are influenced by lighting, therefore the face images are harder recognized than face images on the ORL database, though the YALE database has fewer images than the images on the ORL database.Because the YALE database has fewer images than the ORL database, therefore we only conducted 144 experimental to evaluate the robustness our proposed method.We take six until fourteen features to characterize an object.In this case, the YALE also is evaluated by using four schemes, and for each scheme is conducted thirty six experiments, i.e. nine experiments is repeated by four-folds cross validation.The performance results of our proposed method on the YALE database can be depicted in Figure 4.The anomaly results are delivered on the second scheme as shown in Figure 4, the results decreased accuracy when number of features is more.It appeared when features implemented are ten until fourteen features, whereas for other schemes have produce higher accuracy for more features.The best accuracy is delivered by using five images as the training samples, i.e. 97.78%, 85.56%, and 92.84% for the maximum, minimum, and average accuracies respectively as shown in Table 4. Similar phenomenon also occurs in the YALE database, where lower accuracy has been delivered by the experimental results with smaller samples for the training process.This can be proven in Figure 4, where the average accuracy for 5 training data is higher than the others.In general, the used features also affected to the recognition results.Accuracy has a tendency to rise along with the number of features used, although there are some anomalies in some points.

The Results of the UoB Database
The ORL database is rare utilized to analyze the image, but in this research we apply this database to evaluate the robustness our proposed method.Experiments conducted are same with the YALE database, i.e. 144 experiments.Our proposed method delivered clear results, where the difference of the 1 st , 2 nd , 3 rd , and 4 th schemes are very clear, i.e. the use of two samples as training is not better than three, four and five samples as training sets.Similar to other schemes, three samples as training has delivered worse results than four and five samples as training, where the results of four and five samples as training are similar, but it is better to use five samples instead of four samples as training as demonstrated in Figure 5.For each scheme, the difference of experimental results is not significant.It indicates that the use of features as parameter in similarity measurements does not bring effect with accuracy results.It can be also shown from standard deviation delivered is very small, especially on the fourth scheme, i.e. 0.005.The best accuracy of all schemes is found on the last scheme, i.e. 94% for the maximum, 93.00% for the minimum, and 92.25% for the average accuracy.It indicates that the use of samples as the training has played the essential role to produce the accuracy of the recognition.Table 5 clarifies clearly the results in the maximum, the minimum, and the average.
In this database, it is clear that the difference of facial image recognition results in each of the planned schemes.The addition of images to the training process has contributed to the accuracy of the recognition process, although the addition of features does not impact the increase in recognition accuracy as shown in Figure 5.This proves that the features used have the same level, where the associated eigenvalue has almost the same value

The Results of the CAI-UTM Database
The last schema of experiments is to evaluate our proposed method using the CAI-UTM database.It is the local database taken from our university, i.e.University of Trunojoyo Madura.Figure 6 presented in detail the results, where the first scheme has delivered anomaly results, i.e. the less features measured, the higher resulting accuracy, though the difference not significant.The same phenomenon is also found in the second scheme, the result also decreases when the measured feature is added, although the decreasing of accuracy is extremely small.The second scheme, the increasing of the features measured does not show significant increasing in accuracy, and even the accuracy tends to decrease even though the decline is extremely small.Unlike experimental results on the third and last scheme, the similarity measurement results tend to be stable, in which the result difference of each experiment is very small.Table 6 is summary of the experiments on the CAI-UTM database, where the best performance is delivered by the last scheme, i.e. the maximum accuracy is 91.40%, the minimum accuracy is 90.60%, while the average accuracy is 90.97%.
In this database, Experimental results look different when compared to the previous three databases.In the first scheme, it appears that the addition of features actually decreases the accuracy, although the decreasing of accuracy is not significant, this is because the training image used has the same expression.While in the second until the last scheme shows an almost stable accuracy despite the number of features added.Overall, the results demonstrated that the little difference between the maximum, minimum, and average accuracies produced very small standard deviation, i.e. 0.002.This indicates that all experiments produce the similar accuracies as shown in Figure 6.

Compare to Similar Methods
The results on the ORL, the YALE, and the UoB of our proposed method are also compared to similar methods, i.e.PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE.The first, the results of our proposed method on the ORL database The second comparison is conducted on the YALE database.Our proposed is also compared to similar method, such as PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE.The results show that experiments for all schemes are better than similar method, i.e.PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE as shown in Figure 8.Only LST+LDA delivers accuracy close to our proposed method results, but our proposed method is still better than LST+LDA method.Our proposed method is suitable for face images with occlusion effects.The last comparison is performed to UoB database as shown in Figure 9.The results indicates that our proposed method also delivered higher accuracy than others, i.e.Principal

Conclusion
Proposed method has been evaluated by using four face image databases, and it can be concluded that the contradictory of the laplacian smoothing transform has extract the object features through positive diagonal elements.The experimental results also prove that singularity cases never appear when the eigenvectors and eigenvalues are computed.The performance results displayed that the maximum accuracies for the ORL, the YALE, the UOB, and CAI-UTM are 97.50%,97.78%, 94.00%, and 91.40% respectively, whereas the best standard deviation are 0.006 for the ORL, 0.027 for the YALE, 0.005 for the UOB, and 0.002 for the CAI-UTM database.
The proposed method also efficienly worked on occlusion condition.It can be proved when the proposed method was evaluated by the YALE database.It has delivered higher accuracy than the others.The evaluation results showed that our proposed method is better than others, i.e. as PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE.

Figure 1 .
Figure 1.Framework of the proposed Method this case N j describes the number of training samples on the j class,  j is the average of training sample on the j class, where ∀,  ∈ 1, 2, 3, ⋯ ,  and m states the number of class.The average for all training samples is depicted by , while  ∈   is training samples on the j class.

Figure 4 .
Figure 4. Accuracy on the YALE

Figure 5 .
Figure 5. Accuracy on the UoB

Figure 8 .
Figure 8.Comparison Results on the YALE Database

TELKOMNIKA
Vol. 15, No. 4, December 2017 : 1794 -1807 1806Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locality Preserving Projection (LPP), and Orthogonal Locally Preserving Projection (O-LPP) for all schemes, and even our proposed method produces much higher accuracy than PCA, LDA, LPP, and O-LPP.The comparison results indicate that our proposed method can be accepted as feature extraction method on the face image.

Figure 9 .
Figure 9.Comparison Results on the UoB Database

Table 1 .
Testing Scheme Used in This Research

Table 4 .
Experiments Conducted for each Scheme on the YALE

Table 5 .
Experiments Conducted for each Scheme on the UoB

Table 6 .
Experiments Conducted for each Scheme on the CAI-UTM Contradictory of the Laplacian Smoothing Transform and Linear Discriminant … (Arif Muntasa) 1805 are compared to PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE.The comparison results displayed that the method of S-LPP, LST+LDA, S-MFS, S-NPE, and S-LDA outperformed to our proposed method for two, three, and four training samples, but our proposed method is better than the PCA, LDA, combining LST+LDA, S-LDA, LPP, Simplification of LPP, S-MFS, and S-NPE when five images are applied as training samples as shown in Figure7.It shows that our proposed method is suitable for more images than fewer images as the training samples Figure 7.Comparison Results on the ORL Database