Gaze Tracking Algorithm Using Night Vision Camera

Received Jun 9, 2016 Revised Nov 20, 2016 Accepted Dec 11, 2016 Nowadays, the advancement of medical technology has given birth into many innovative machines and devices to improve our health life, especially to those who are disabled or gifted. On some people with severe disabilities such as quadriplegia, the human eye-ball does not only serve as a vision system, but also a means of conveying information and intention to other people. This is because although quadriplegic patients suffer the loss of motor sensory functions from the neck and below, upper neck functions such as the vision system is normally spared. This enables the patient to control the movement of his/her eyeballs to convey desired information. Although many similar researches have been done, this paper proposes the use of image processing on image captured using webcam with its Infra-Red (IR) filter removed (a.k.a night vision) to achieve robustness. This allows the algorithm to properly track the location of the iris despite of its and the pupil color variations. Two image processing algorithms are then used, each with owns tradeoff between speed and accuracy. Analysis on both algorithms shows good tracking performance despite of the mentioned tradeoff


INTRODUCTION
Our freedom to move freely is invaluable to us. However not everyone shared the same taste of freedom. For some, such as a quadriplegic patient, or a paralyzed person, unable to move independently is dementing. To overcome this matter many researches have been done in the medical field to improve the quality of life of a quadriplegic. The fact that most quadriplegic retain most of its upper neck function, give way for researcher to leverage this into assisting them to move independently again [1]- [3]. In most researches an electric wheelchair is used to achieve this objective [1], [4]. Bio-signals such as the movement of the eyeball [5], [4], EEG signals and EMG signals are translated in command to move the wheelchair [4]. This project proposes the use of image processing technique to track the movement of the eyeball, by locating subject's pupil. The location can later be translated into commands to move the wheelchair to left or right. This leads the researchers to develop a system that can help the disable person by moving the eye balls.
Generally, gaze detection system is a system that captured image data of eye pupil, detecting and tracking movements of the user's eyes, approximating the line-of-sight vector, and move the electric wheelchair in desired direction based on the person's eyes and it is a system based on eye biometrics [6]. Eye tracking system/gaze tracking system use the image processing technique to convert the input data into digital form and some mathematical operations are applied to the data to create more image enhancement [7]. Using image processing technique, the image of the picture will then be identified on colors, shades, contrasts and shapes that cannot be seen by normal eye. Subsequently, after performing image processing technique, information on the location of the pupil will then be sent to a microprocessor that will control the directional movement of a wheelchair motor [16]. Edge detection is an image processing technique for finding the boundaries of objects within images [8]. This can be achieved by detecting discontinuities in brightness. Edge detection is castoff for image segmentation and data extraction in areas such as image processing, computer vision, and machine vision. Common edge detection algorithms include Sobel, Canny, Prewitt, Roberts, and fuzzy logic methods. But in this case, the use of canny method is essential as the input frame is processed for digital images [9]. This method is not finalized since it is in the middle of process. The result of edge detection method will be continued with some morphology fundamental for pupil extraction.
This paper proposes two type of method in order to detect the iris. The first step is to detect the user face by using skin color segmentation and find the face of the person over the image. This method categorized the image into facial and non-facial region since human skin color is dissimilar from the nature. After the face of the person is detected, the proposed used of circular Hough Transform permitted to locate the iris in the face region. This project will focus more on eye detection. The Hough transform is a feature abstraction technique used in image analysis, computer visualization, and digital image processing [10]. The purpose of this transform is to discover imperfect occasions of object inside a certain class of shapes by a voting procedure. This method is considered to be suitable since the circular shapes can detect the pupil region [11], [12].
Regionprops is also used in this project, which it can used to calculate object properties from the binary image. Every regionprops has some specific properties that can be used to measure the binary image. Some of example properties that can be use is bounding box properties. This is represented as a 4-vector where the first two entries are the x and y coordinates of the upper left corner of the bounding box, and the two last entries are the width and the height of the box [13]. The suitable properties that can be use are centroid properties that calculate the center coordination of binary object. The centroid will be used gaze detection purposed.

RESEARCH METHOD
This system consists of 3 parts which is the hardware development for the first part. After the hardware configuration is ready, the next step will be the algorithm to track the pupil region. The second part will shows the process under taken to extract pupil boundary. The last part is to formulate the gaze tracking direction.

Headset Configuration
This project consists of several devices that need to be considered for the accuracy of webcam placement. Therefore, a head set is design by using SOLIDWORK so that the webcam can be placed on top of the head to identify the gaze position.

Night Vision Camera
The contour and color for both pupil and iris of Asian people is more to dark brown, unlike European people which its eyes intensity between pupil and iris can be seen by clear vision or normal colored webcam [14]. To overcome this problem, the use of night vision camera can be used to overcome the problems since it can clearly differentiate between pupil and iris of the human eyes.
The use or real Night Vision Camera is expensive. However this can be achieve by removing the IR filter of a normal cheap webcam such as shown in Figure 3. By removing the IR filter, it allows the infrared light to be detected by the night vision camera.
Normal LED tends to make the patient or user feels uncomfortable due to light intensity that immerse towards the eyes. The function of the IR LED is to provide an additional light towards the human vision so that it can be detected by night vision camera. The advantages of this camera are it detect the iris and provide additional light that cannot be seen by normal vision.

Algorithm for Gaze Tracking
For both algorithms, the process of Image Acquisition, Edge Detection, Image Filled and Creates a Structural Element are identical [15]. The different comes when the pupil is detected and it needs to be centralized. Both Algorithms will use different medium in order to identify the coordinates and to highlight the binary region.

Image Acquisition
First part of this project is by capturing the image which will be in an RGB image. The image acquisition will frame these images from recorded video and stored in the MATLAB directory. The image after that will be converted into a gray scale image since the pixel stored in the images is mostly bright and dark. After that, the gray scale image is converted into binary images that represent 0 for black and white for 1. This will result in black and white images. As we know that our pupil is mostly black, thus it wills easily getting the region as we convert it into binary.

Edge Detection
After the binary image is obtained with desired threshold level, the next process would be detecting the edges of the area of interest in the image which is the pupil portion, in order to remove the portion of the pupil out from the enhanced image. The chosen filter which is the canny filter has been implemented to the images in order to get the noticeable edges. The canny edge detector first smooths the image to eliminate and reduce noise. It then finds the image gradient to highlight regions with high spatial derivatives which mean the sharp changes in image brightness to get important information from the image. Canny filter been chosen due to the characteristic of the eyes which is round and curved, thus it is can produced the desired result.

Fill Holes
After the final result we will get the desired edge which is the pupil center, it is an essential requirement to detect the movement of the human eyes. As we know, pupil portion has been used for tracking the gaze movement either right or left. In order to make the pupil portion to be more noticeable, dilation process has been applied towards the image. Then the morphological operation is implemented to the image. Fill holes act as a function that fill an image with white pixel. It fills all the holes that present in the image thus make the image full in white pixels. This process using structural element in order to creates a large shapes of binary image from the pupil region. Since the system using a night vision camera, the region that shows the darkest color is only the pupil region. Thus the use of structuring element together with dilation can help to creates image with more prominent and larger. Structuring element comes with many parameters such as diamond, rectangles, octagon and disk structuring elements. All of the parameters come with specific setting for the structural elements. For this project, the suitable parameter is disk structuring element. Disk structuring elements will creates a disk-shaped of binary image with desired radius. But the desired radius must satisfy the non-negative integer. The larger the value of radius, the structuring elements will become larger.

Pupil Localization using Region Props and Circular Hough
When the pupil portion has been detected, the next step is to track this portion from the rest of the image. It can be obtained by detecting the comparison of the black and white of the image. To identify the portion of white pixels, the system for Algorithm 1 use regionprops function to creates properties that can measures and highlight the image region with bounding box, while algorithm 2 will use Circular Hough Transform to detect the round image and centralized the image. The Image will be highlighted with circle outline that actually the properties of Circular Hough Transform.

Gaze Tracking Direction
The input image given to the MATLAB for processing will produce an output image with the coordinates of the pupil. The output image will be a fixed size and the size will be divided into six blocks and the pupil coordinates in the center will indicate it as straight. Where the block in the second row and third column will indicate the position of the pupil as left and similarly the block in the second row and first column will indicate the pupil position as Right. [1] By setting the coordinates of each block. The output images will be considered as a rectangle with certain length and of breadth.
But a modification of this method would be proposed instead of using the coordinates for image size, the proposed method would be identifying the direction of the gazing by calculating the angle of sight. The distance from user eye can be calculated from the webcam and the distance of the user eye can also be calculated. With these two values, the value of θ can be determined. This can be understood by looking at the Theorem Pythagoras concept. But, the webcam itself has been provided with the viewing angle. The viewing angle take place with 60 degree total as we take an example of 320*240 output images, the 320 is the x-axis of the frames and the viewing angle of the webcam will be synchronize with it. The condition would be like this:a. Block

RESULT AND DISCUSSION
The first step of the localization of the pupil is by capture the image of the human eye in RGB color for the image acquisition process. The image format will be JPG format that can be read by MATLAB command. The image also must be stored in MATLAB directory so that the command window can read the image. Figure 6: Normal RGB image into night vision image Figure 6 shows the image of an eye in original RGB color captured from night vision webcam. The night vision camera allows the infra-red light to immerse to it. Therefore, the pupil region can be clearly identified. Based on the Figure 9, it can be shown that the blue line process the image faster compared to the red plotted line. This is due to the function used by the blue line is taking less time to process.

Accuracy of Gaze Detection
The last is the test for accuracy, Table 3, Table 4 and Table 5 shows the accuracy of gaze detection by moving the eyeball to the left and right. Both technique produce accurate gaze and the detection can be clearly seen. But the problem occurs for both techniques when the eye gazes to the most right which it indicates the angle of 36 degree. This is due to the surrounding light that immerse towards the eyes.
For this project, the requirement is for fast computational with high accuracy. For both algorithms, it uses canny edge detection which detects the edge boundary of pupil region. But for time computational, Algorithm 1 process faster compared to Algorithm 2 with time different of 7.91 seconds. Therefore for robotics application that requires fast computation, Algorithm 1 is more suitable and can be implemented into a robotics wheelchair system.

CONCLUSION
This paper has presented the development and analysis of an algorithm that can detect the pupil region by using regionprops and Circular Hough. The pupil region is then formulated to give a reference angle that can be used by robotic wheelchair. Detection of pupil region is done by using a night vision camera that is modified from a normal webcam. The use of night vision camera can help to maximize the edge boundary region since it can differentiate between iris and pupil. The webcam is attached with 4 IR LED that can give extra illumination towards the eyes so that the pupil will be more prominent. This is an advantage for the gaze detection because the pupil can still be detected even in a dark room. More important, The IR LED does not make the patient or user feels uncomfortable since the IR LED can only be detected by night vision camera. For future improvement, the webcam can be replaced with a camera that can zoom out from wide distance. The current camera is placed in front of the user and this might make the user feel uncomfortable.