Performance Evaluation of Automatic Number Plate Recognition on Android Smartphone Platform

ABSTRACT


INTRODUCTION
Due to the increasing number of vehicle nowadays, the modern city needs to establish the effective and efficient automatic traffic system for the management of the traffic law enforcement. Automatic number plate recognition (ANPR) leads the significant role in this condition. The ANPR is an image processing technique to extract the image of license plate on vehicle taken by digital camera or taken by either a color or a grayscale digital camera, as well as an infrared camera in order to identify the vehicles using their number plate through optical character recognition (OCR) [1].
The ANPR system recognizes characters on license plate through the combination of various techniques and algorithms, including image pre-processing, object detection, character segmentation and recognition. It consists of a camera to detect the number plate object and processing unit to process and extract the characters and interpret the pixels into numerically readable characters [2]. It became much exciting in the last decade along with the improvement of digital camera technology and the computational processing [3]. Nowadays, the ANPR system has been used in traffic law enforcement, including speed camera, traffic light camera, stolen car detection, and border monitoring. It can be used also for the building management, such as parking management and gate control [4], [5]. Android smartphone operating system (OS) has captured more than 82% of the total market-share as of 2015 [6]. Android is the most popular mobile operating system due to its open architecture and the popularity of its application programming interface (APIs) in the developer community. Due to its open source nature, Android OS can be found from low end device to the high end device. Android can be divided into roughly three layers below the application layer, including the application framework layer, the runtime and libraries layer, and the kernel layer. The proposed ANPR system was implemented in the application framework layer.
Although many researches have been conducted on the ANPR system. However, the implementation platform is normally a high resolution camera with high computing power and not many researches have been conducted on the ANPR implementation on the smartphone with lower resolution camera and lower computing power. Therefore, the objective of this paper is to proposed ANPR system implemented on Android smartphone and to evaluate its performance in recognizing Malaysian number plate. The design of the ANPR system, including GUI design, process design, and database design on Android platform has been presented in [7], while this paper will focus on the performance evaluation of the proposed ANPR system. The rest of the paper is organized as follows. Section 2 introduces the proposed ANPR system, with the improvement or enhancement to the original ANPR algorithm by Tesseract [8]. Section 3 discusses the implementation and construction of Malaysian number plate image database. Section 4 evaluates performance of the proposed algorithm compared to the original algorithm [8], while Section 5 concludes the paper.

PROPOSED ANPR ALGORITHM
The ANPR system generally consisted of the following five stages [9] as shown in Figure 1. The first stage is image acquisition, capturing the image of license plate using a camera by considering the camera resolution, orientation, shutter speed and light. The second stage is image pre-processing, such as normalization, brightness and contrast adjustment, and skewness correction of the captured image. The third stage is localizing the license plate to extract the license plate from the whole vehicle image based on some features, such as the boundary, the color, or the presence of the characters. The fourth stage is character segmentation to segment the characters on the license plate by locating and identifying the individual character on the license plate image. The final stage is optical character recognition to recognize the segmented characters by pattern matching or classifiers, such as artificial neural networks (ANN), fuzzy logic, Hidden Markov Model (HMM), template matching [10]. The ANPR system can be connected to other applications or databases to further enhance its functionality. Based on previous design presented in [7], the proposed algorithm can be illustrated in Figure 2. It shows the difference between the original ANPR algorithm developed by [8] and our proposed ANPR system. Note that, the original algorithm was developed mainly for optical character recognition which can recognize various alphanumeric characters. The input image is acquired using Android smartphone's camera which has low resolution. The noticeable difference can be found in the image preprocessing stage. We add several enhancements, including edge enhancement using unsharp masking, color inversion, and Otsu thresholding with fraction of 0.1. The performance improvement by adding additional unsharp masking and color inversion will be further explored in Section 4. Meanwhile, fixed pitch detection and chopping was used in the character segmentation stage, while ANN or template matching was utilized in the OCR stage.

IMPLEMENTATION OF ANPR ALGORITHM
In this section, the implementation of ANPR algorithm on Android smartphone will be discussed. First, the actual graphical user interfaces (GUI) for the ANPR system on Android smartphone was constructed based on the GUI design as described in [7]. The hardware specification of Android smartphone used will be described. Finally, the Malaysian number plate image database was constructed, including 15 one row and 15 two rows number plate types.

Software and GUI implementation
For the software implementation, the Leptonica library will be used to implement image processing algorithm [11], while Tesseract library will be used as the core engine for OCR [8]. For the GUI implementation, the actual ANPR GUI for Android smartphone is shown in Figure 3. The actual GUI contains four main menus including camera menu, map menu, manual input menu, and searching menu. The main page shows the list of number plates along with the date taken in the right side of the number plate. The GPS coordinate is related to the Google map to show the actual location. Furthermore, there is one screenshot of email notification from the server informing the number plate recognized by the Android ANPR system. Note that, the GUI design, process design and database design was explained in [7]. The actual GUI illustrated a law enforcement scenario, where a police captured the wrongly park vehicle number plate, in which the system later on sends the notification email to the owner of the vehicle.

Hardware Implementation
The Android mobile phone used in this research is Samsung GT-S5830 known as Galaxy Ace or Galaxy Cooper which is mid-end smart phone family released by Samsung in February 2011. It has 800 MHz 1 Core ARM v6 of processor with 278.34 MB of RAM. The screen resolution is HVGA (480 x 320) ~165 ppi pixel density and 5 mega pixels of camera resolution. This Android mobile phone model is installed Android OS version 2.3. The requirement in this research is the Android version should be 2.3 or later and should have camera.

Malaysian Number Plate Image Database
For the purpose of the research, it required to set up the experimental data of the number plates images which were taken under various time and condition. In this research, the image database are divided into two groups of Malaysian vehicle number plates. First group is one row number plates as shown in

RESULTS AND DISCUSSION
In this section, the performance evaluation of the proposed ANPR system implemented on Android smartphone will be elaborated. First, the performance measures were discussed, including the accuracy or recognition rate and the processing time. Next, the effect of unsharp masking and color inversion was explored. Lastly, the performance comparison between original Tesseract OCR algorithm and the proposed ANPR system using either ANN or template matching was discussed in more details.

Performance Measures
In this research, the method to measure accuracy of recognition rate is based on the objective evaluation as in [12]. The number of error can be defined as the total characters to be recognized subtracted with the characters that do not match with the correct character or original character, such as character insertion, deletion and substitution. If there are n total characters to be recognized, then the accuracy is stated as [12]:  (1) The processing time in millisecond was also recorded in our experiments. Additional complexity will be justified if the additional computational time is negligible.

The Effect of Unsharp Masking and Color Inversion
As shown in Figure 2, the proposed ANPR system includes additional algorithms, including edge enhancement (unsharp masking) and color inversion (negative image). Unsharp masking was used to sharpen the number plate image, while the color inversion was used to reverse the color of alphanumerical characters with its background. Both algorithms reduce some noises produces during the image acquisition step so that it will be better recognized by the next stage. Furthermore, the Otsu method was set with fraction parameter value of 0.1. For the testing purposes, template matching algorithm was utilized in the OCR stage. Table 1 shows the results in terms of processing time (ms) and accuracy (%) for the proposed ANPR system without unsharp masking, without color inversion, and without both unsharp masking and color inversion.  Table 1 shows that the average accuracy of the proposed ANPR system without unsharp masking, without color inversion, and without both algorithms are 91.1%, 60.5%, and 57.7%, respectively. The processing time for each scenario are 960, 1027, and 946, respectively. Note that, the accuracy and the processing time for the proposed ANPR system with template matching are 97.5% and 1136.5 ms, respectively, as will be discussed later in Table 5. The results showed that unsharp masking has positive effect and the color inversion has significant effect on the accuracy, while adding negligible additional processing time. It can be concluded that the combination of both unsharp masking and color inversion algorithms have significant contribution to the proposed ANPR system.

1979
In this section, the original algorithm used the default parameters in Tesseract library [8] without modification, optimization, or inclusion of additional processes (see Figure 2). Both the original algorithm and proposed ANPR system used ANN in the OCR stage. Table 2 shows the performance comparison between original ANPR algorithm with the proposed ANPR system for both one row and two rows Malaysian number plate, in terms of accuracy and processing time. It is noted that the result for one row number plates was significantly different to two rows number plates as can be seen in Table 2. It shows that the accuracy for two rows was better than one row number plate, in which the score was 51.8% compare to 43.8%. However, the processing time of two rows number plate is longer than one row number plates, i.e. 54.1 seconds and 46.1, respectively. The proposed ANPR system shows similar results, in which the accuracy is 62.4 and 59.5 for two rows and one row number plates, while the processing time is 59.6 and 48.3 seconds, respectively.  Table 3 focuses on the comparison between the original algorithm and the proposed algorithm in terms of accuracy and processing time. On average, the proposed ANPR algorithm increased the accuracy by 13.2% while additional processing time is only increased by 7.7%.  1973 -1982 1980 In this section, the original used the default parameters in Tesseract library [8] without modification, optimization, or inclusion of additional processes (see Figure 2). Both the original algorithm and proposed ANPR system used Template Matching in the OCR stage. Table 4 shows the performance comparison between original ANPR algorithm with the proposed ANPR system for both one row and two rows Malaysian number plate, in terms of accuracy and processing time. It is noted that the result for one row number plates was significantly different to two rows number plates for original algorithm as can be seen in Table 4. It shows that the accuracy for two rows was better than one row number plate, in which the score was 91.3% compare to 76.0%. However, the processing time of two rows number plate is longer than one row number plates, i.e. 1.04 seconds and 0.92, respectively. The proposed ANPR system shows different results, in which the accuracy is 96.8 and 98.1 for two rows and one row number plates, while the processing time is 1.2 and 1.06 seconds, respectively. Table 5 focuses on the comparison between the original algorithm and the proposed algorithm in terms of accuracy and processing time. On average, the proposed ANPR algorithm increased the accuracy by 13.8% while additional processing time is only increased by 16.5%. From Table 3 and Table 5, the results show that the accuracy for the proposed algorithm using ANN and Template Matching is 61.0 and 97.5, respectively. The processing time is 54.0 and 1.1 seconds, respectively. It can be concluded that the proposed ANPR system using template matching is better than using ANN. Therefore, template matching is more suitable to be utilized in ANPR system in Android smartphone due to its accuracy and lower computational time.

CONCLUSIONS AND FUTURE WORKS
The performance evaluation of proposed ANPR system implemented on Android smartphone platform has been presented. The original ANPR algorithm using Tesseract library was compared to the proposed ANPR system. The proposed ANPR system has additional processing steps, such as unsharp masking and color inversion. Results showed that our proposed ANPR system as higher accuracy with negligible additional processing requirement. On average, the proposed ANPR algorithm using ANN increased the accuracy by 13.2% while additional processing time is only increased by 7.7%. On average, the proposed ANPR algorithm using Template Matching increased the accuracy by 13.8% while additional processing time is only increased by 16.5%. Future works includes using higher resolution and higher processing smartphone, and improve the accuracy by adding additional enhancement algorithms.