Training of Convolutional Neural Network using Transfer Learning for Aedes Aegypti Larvae

The flavivirus epidemiology has reached an alarming rate which haunts the world population including Malaysia. World Health Organization has proposed and practised various methods of vector control through environmental management, chemical and biological orientations. However, from the listed control vectors, the most crucial part to be heeded are non-accessible places like water storage and artificial container. The objective of the study was to acquire and compare various accuracies and cross-entropy errors of the training sets within different learning rates in water storage tank environment which was essential for detection. This experiment performed transfer learning where Inception-V3 was implemented. About 534 images were trained to classify between Aedes Aegypti larvae and float valve within 3 different learning rates. For training accuracy and validation accuracy, learning rates were 0.1; 99.98%, 99.90% and 0.01; 99.91%, 99.77% and 0.001; 99.10%, 99.93%. Cross-entropy errors for training and validation for 0.1 were 0.0021, 0.0184 whereas for 0.01 were 0.0091, 0.0121 and 0.001; 0.0513, 0.0330. Various accuracies and cross-entropy errors of the training sets within the different learning rates were successfully acquired and compared.


Introduction
The flavivirus epidemiology has reached an alarming rate which haunts the world population including Malaysia. From the record, Malaysia suffered twice increment of dengue cases reported from the last 2 years [1]. With the rapid growth of Malaysian population either from newborn or migration [2], this matter should be taken seriously in order to curb this problem. As the population grows, technologies and artificial intelligence also occupy most of our daily routines. Machine learning is one of the artificial intelligence subtopics which has empowered many aspects of modern life [3], [4].
World Health Organization has proposed and practised many methods of vector control through environmental management, chemical and biological orientations [5]. However, from the listed vector control, the most crucial part to be heeded are non-accessible places like water storage and artificial container [6]. This should be taken into account where the maintenance and eradication work of eliminating Aedes Aegypti larvae are difficult to implement. The elimination of Aedes Aegypti during the larvae stage is very important because when it turns into adult mosquito and is able to fly, the population control becomes more complicated.
Studied on classification of Aedes Aegypti larvae inside the water storage tank are scarce. Hence, the discovery and exploration are crucial in this field to offer a better living to the human population. Previous studies, only offer reviews on identifying and counting Aedes Aegypti larvae [7]. This paper hence, proposed a classification of Aedes Aegypti larvae in water storage tank through machine learning. Therefore, the objective of the study was to acquire and compare various accuracies and cross-entropy errors of training sets within the different learning rates. Convolutional neural networks are the latest and efficient recognition methods especially for data that have known-like topology like the human visual cortex. This network is trainable multistage architecture and has a lot of advantages compared to the neural network traditional method. This network has abundant important steps of traditional pre-processing which reduce its complexity because it has built-in pre-processing inside its architecture. Hence, the machine is able to be fed with raw data or original images directly as inputs [8].
Convolutional neural networks stand out as examples of neuroscientific principles. In general, they are an operation of a real-valued argument within two functions. The convolutional operation can be formally described as [9] ( ) ∫ ( ) ( ) [1] Therefore, where s(t) is convolutional operation in real time, x is referred as the input, a is the age of measurement and w is probability density function or weight. This network has used pooling layer technology, receptive field, weights sharing and training parameter reduction compared with the traditional network. It has made prodigious progress in the field of image classification and localization [10].
With the rapid development of computer, people start moving toward digitalization and everything goes automated and automatic. With the use of convolutional neural networks, a study on land-cover classification using high-resolution imagery has been carried out [11]. By applying transfer learning, the study has successfully compared the classification accuracies among SafeNet, GoogLeNet and ResNet which are 97.8±2.3%, 97.6±2.6% and 98.5±1.4% respectively. The convolutional neural networks have also been applied in medical studies to classify derma infection between melanoma and benign, where the trained model of VGGNet architecture is implemented in the study with 95.95±1.2% of training accuracies [12].
Convolutional neural networks have attracted attention from a lot of computer vision research communities. A study regarding granite tiles classification has been made [13]. To classify granite tiles in stone industry is a challenge as it has similar visual appearance. By implementing CifarNet, the study has successfully classified the granite tiles with 87.26% training accuracy. In terms of safety issue, a trained model of AlexNet has been used in order to classify firearms in x-ray baggage for security imagery [14]. As a result, the trained models have obtained 95.26% accuracies.
Vehicle type classification has played an important role in intelligent traffic light system [15]. Convolutional neural networks have been used in a study where the architecture has 2 convolutional layers, 2 pooling layers and 2 fully connected layers. Besides, to improve the performance of the network, non-linear activation function has been added to the model. The training accuracy of this application is 97.88%. Besides training the convolutional neural networks, it also performs a hybrid with salient feature successfully [16]. The study has successfully trained for facial expression with more than 90% accuracies when using the public database.
This paper is organized as follows; the research method regarding the classification of Aedes Aegypti larvae using transfer learning in water storage tank is presented in Section 2. Section 3 presents the result and analysis of the performance for the classification process in different parameters of learning rates. Finally, the conclusion will be drawn in Section 4.

Research Method
This section focuses on the development process of Aedes Aegypti larvae classifier inside the water storage tank. The architecture of this network had skipped the pre-processing steps where the only important step of this network was to label the data which was executed by sharing the directory of the created subfolder that contained images to be classified.
In this experiment, Tensorflow was used as a tool in the experiment. Tensorflow is a framework or system for large-scale machine learning which has been developed by the Google team [17]. Tensorflow is a dynamic control flow where it supports convolutional neural network, Inception-V3 was implemented in this experiment. Inception-V3 is a state of the art or network model of machine learning. This experiment performed transfer learning which has wide applications [18], [19], [20]. In this context, transfer learning used a previous train model for a new task where it successfully trained a large number of images dataset. Back-propagation algorithm as the weight parameter which was adjusted by the cross-entropy cost function. Figure 1 shows the Inception-V3 architecture.

Figure 1. Architecture of Inception-V3
To classify the Aedes Aegypti larvae and other materials inside water storage tank, a subfolder between both of the classes was created first. The dataset of Aedes Aegypti larvae were obtained based on experiments conducted in the lab. As the data were fed in the network, the last layer of the model would be replaced and retrained with the new categories of intended classes as shown in Figure 1. As the dataset had 2 classes, the final layer has 2 output nodes. This experiment was run on Linux 17.10 of the virtual machine within 3.6GHZ of quad i7 processor and 10GB 3601MHz DDR2 memory. Figures 2 and 3 show examples of 2 classes of the dataset used to feed in the Inception-V3. In the experiment, the manipulated variables would be the learning rates of train networks. The network was trained within 3 different learning rates namely 0.1, 0.01 and 0.001. Other parameters were set as default as the Inception-V3 set. The significance of the experiment was to show the effect of the size of the learning rates on the accuracies and training speeds. The images used to train in the network were 534 where 380 of them were Aedes Aegypti larvae images and others were float valve images. In the procedure, all Aedes Aegypti larvae images were taken in rest position and without any intersection between two larvae.

Results and Analysis
This section discusses the results from the retrained model. Figures 4, 5, 6 and 7 show the training accuracies, training cross-entropy errors, validation accuracies and validation crossentropy errors based on different learning rates. The results showed the filtered digital signal processing. The digital filter was used to smoothen and reduce the ripples in the graph so that the best result can be obtained. In the experiment, moving average filter was applied in the analysis. This filter was an optimal filter for this application where it is often used for common tasks without involving complex mathematical such as frequency domain analysis. This filter was implemented in convolution where the equation form is written as [21] [3] where is the input signal, is the output signal and is stated as the number of window size or also known as the number of points in the average. The value for the window size was 5.
To measure the accuracy and training speed, all experiments were conducted in the same training steps which were 4000. The results is shown in Figure 4. The training accuracy of retrained model with 0.1 of learning rates was the fastest retrained model which only needed 250 steps to complete, followed by 0.01 of learning rates which needed 800 steps and 0.001of learning rate which needed at least 4000 steps to fully complete the training. However, the validation accuracy in Figure 6 has shown the opposite result where the smallest learning rate has higher precision on randomly-selected group tests. It is proven that the smaller value of learning rates has the higher accuracy on recognition. Cross-entropy error is representing loss function which gives an indication into how good the learning process is progressing. The purpose of training was to make an error as small as possible. The training cross-entropy error and validation cross-entropy error are shown in Figures 5 and 7. The training with the smallest learning rate has bigger different values between training loss function and validation loss function. This difference signified that the training has a lesser error tendency when tested on randomly-selected group of images. The results (Figure 7) also showed that the smallest learning rate had continuously declined as the training step went on. Nevertheless, at the end of the training set, the smaller learning rates had smaller value of loss function during training compared with the bigger learning rate. The performance summary is shown in Table 1.  Table 1 shows the overall performance in various indexes and learning rates. The smallest learning rate had higher accuracy even though it only had small changes from 0.1 to 0.01. In fact, inception-V3 architecture could classify more than hundreds of classes within millions of images, the small changes in the result were due to small application where only 2 classes were classified. The results of the changes signified the different learning rates applicable in the experiment.

Conclusion
At present, there are only few studies with regard to Aedes Aegypti larvae machine learning in entomology field. The results show various accuracies and cross-entropy errors of the training sets within different learning rates. Future study should focus on retrained model with higher accuracy to be applied in Aedes Aegypti larvae detection.