Validation procedures
For all classification methods, cross validation can be performed with venetian blinds or contiguous blocks (cross-validation type). Regarding venetian blinds, with 3 cross-validation groups the split of the first group in venetian blinds will be [t,0,0,t,0,0,....,t,0,0], while the second one will be [0,t,0,0,t,0,....,0,t,0], and so on, where t are the samples included in the cross-validation groups. On the other hand, the split of the first group with contiguous blocks will be [t,t,t,t,0,0,....,0,0,0] and so on. If cross-validation is performed, the number of cross-validation groups must be defined. Moreover, bootstrap with resampling or validation based on random sampling (montecarlo) of 20% of samples can be calculated. If bootstrap or montecarlo validation are selected, the number of iterations must be defined.
[-> top]
Discriminant Analysis (DA)
In order to calculate classification models based on Discriminant Analysis (DA), select "calculate->Discriminant Analysis->fit DA". The form to select the DA options will appear. Here you can select the type of DA (Linear or Quadratic) and the type of validation.
Multinormal distribution for data can be analysed by clicking "calculate->Discriminant Analysis->check multinormality". In this case, a new figure will appear with results of the multinormality test based on squared generalized distance and chi-square percentiles. Data can be assumed to be multinormally distributed if: 1. the plot of the ordered squared distances and the chi-square percentiles is nearly linear; 2. roughly half of the distances are less then or equal to chi-square percentile of 0.5 [Johnson,R.A. Wichern,D.W., Applied Multivariate Statistical Analysis, 2008].
[-> top]
Discriminant Analysis coupled with Principal Component Analysis (PCA-DA)
Discriminant Analysis can be calculated on the scores produced by Principal Component Analysis, select calculate->Discriminant Analysis->fit PCA-DA". The form for settings PCA-DA options will appear. Here you can select the number of components to be retained in the model, the row and column pre-processing, the type of DA model to be calculated (Linear or Quadratic), and the type of cross validation. To select the proper number of PCs, click the "optimal PC" button: the cross validation procedure for selecting the optimal components for PCA-DA will produce a plot of the error rate in classification as a function of the number of components retained in the model. See the example provided in this help to better understand.
[-> top]
Partial Least Square - Discriminant Analysis (PLSDA)
To fit Partial Least Square - Discriminant Analysis (PLSDA), select "calculate->Partial Least Squares DA". The form for settings PLSDA options will appear. Here you can select the number of Latent Variables, row and column pre-processing, the type of assignation criterion (e.g. how each sample is assigned to classes, see the theory section for further information) and the type of cross validation. To analyse the optimal number of LV to be retained, click on the "optimal LV" button. The cross validation procedure for selecting the optimal components for PLSDA will produce a plot of the number of components retained in the model versus the error rate in classification. See the example provided in this help to better understand.
[-> top]
Classification Trees (CART)
In order to calculate classification models based on Classification trees (CART), select "calculate->CART". The form for settings CART options will appear. Here you can select the type of validation you want to apply.
[-> top]
Random Forest (RF)
In order to calculate classification models based on Random Forest (RF), select "calculate->Random Forest ". The form for settings Random Forest options will appear. Here you can select the number of trees to be included in the Random Forest and the type of validation. To evaluate the optimal number of trees, click the "optimal trees " button. The cross validation procedure for selecting the optimal number will produce a plot of the number of trees versus the error rate in classification and a plot of the number of trees versus the classification error obtained on out-of-bag samples, that is, samples not included in the temporary traing set selected by boostrap. The cross validation procedure is repeated 10 times to evaluate variability of results. Error bars shown in the plot report the dispersion of error rate in cross valdiation along the 10 repetitions.
[-> top]
K-Nearest Neighbors (kNN)
In order to fit K-Nearest Neighbors (kNN), select "calculate->K-Nearest Neighbors". The form for settings kNN options will appear. Here you can select the number of neighbours (k), row and column pre-processing, the type of distance and the type of validation. To evaluate the optimal number of neighbours (k), click the "optimal k" button. The cross validation procedure for selecting the optimal k will produce a plot of the k values versus the error rate in classification. See the example provided in this help to better understand.
[-> top]
Potential Functions (Kernel Density Estimators)
In order to calculate Potential Functions, select "calculate->Potential functions". The corresponding setting form will appear. Here you can select the target class and its smoothing parameter, row and column pre-processing, and the percentile to define the class boundary, as well as the type of validation. Potential Functions can also be calculated on scores obtained by means of Principal Components Analysis. In this case, the number of PCs to be used can be selected in the setting form; the "automatic" option will selects PCs with eigenvalue higher than the average eigenvalue. Note that the PCA model is calculated only on the samples of the target class. Sampels of other classes will thus be projected into the PCA class model. When dealing with Potential Function, in order to evaluate the optimal smoothing parameters (for a given target class), click "optimal smoothing ". The cross validation procedure for selecting the optimal smoothing parameter will produce a plot of the smoothing values versus the class error rate, specificity and sensitivity.
[-> top]
Support Vector Machines (SVM)
In order to calculate the SVM model, select "calculate->SVM". The corresponding setting form will appear. Here you can select the kernel type, cost value, kernel parameter (if dealing with RBF or polynomial kernels), row and column pre-processing, and type of validation. SVM can also be calculated on scores obtained by means of Principal Components Analysis. In this case, the number of PCs to be used can be selected in the setting form; the "automatic" option will selects PCs with eigenvalue higher than the average eigenvalue. When dealing with Support Vector Machines, in order to evaluate the optimal cost value (as well as the optimal kernel parameter when using polynomial or RBF kernels), click "optimal parameters". The cross validation procedure for selecting the optimal smoothing parameter will produce a plot of the cost values versus the error rate in classification (as well as the average number of support vectors), when dealing with a linear kernel; when using RBF or polynomial kernels, it will produce a surface plot of the cost values and kernel parameters values versus the error rate in classification (as well as the average number of support vectors).
[-> top]
Soft Independent Modeling of Class Analogy (SIMCA)
In order to calculate the SIMCA model, select "calculate->SIMCA". The corresponding setting form will appear. Here you can select the target class, the number of components to be retained, row and column pre-processing, and type of validation. When dealing with SIMCA, in order to evaluate the optimal number of components to be retained for the PCA target class model, click the "optimal PC" button. The cross validation procedure for selecting the optimal components for SIMCA will produce a plot of the number of components retained in the PCA class model versus the class error rate, specificity and sensitivity.
[-> top]
Unequal class models (UNEQ)
In order to calculate the UNEQ model, select "calculate->UNEQ". The corresponding setting form will appear. Here you can select the target class, the number of components to be retained, row and column pre-processing, and type of validation. When dealing with UNEQ, in order to evaluate the optimal number of components to be retained for the PCA target class model, click the "optimal PC" button. The cross validation procedure for selecting the optimal components for UNEQ will produce a plot of the number of components retained in the PCA class model versus the class error rate, specificity and sensitivity.
[-> top]
Backpropagation Neural Networks (BPNN)
In order to calculate Backpropagation Neural Networks, select "calculate->Backpropagatipn NN". The corresponding setting form will appear. Here you can select the number of hidden layers, the number of neurons for hidden layers (for more flexible options model your data through the MATLAB routines), learning rate, alpha (momentum term), number of iterations and type of validation.
[-> top]