A New Method for Ball Tracking Based on α-β, Linear Kalman and Extended Kalman Filters Via Bubble Sort Algorithm

Received Jan 19, 2018 Revised Mar 27, 2018 Accepted Apr 11, 2018 Object tracking is one of the challenging issues in computer vision and video processing, which has several potential applications. In this paper, initially, a moving object is selected by frame differencing method and extracted the object by segment thresholding. The bubble sort algorithm (BSA) arranges the regions (large to small) to make sure that there is at least one big region (object) in object detection process. To track the object, a motion model is constructed to set the system models of Alpha-Beta (α-β) filter, Linear Kalman filter (LKF) and Extended Kalman filter (EKF). Many experiments have been conducted on balls with different sizes in image sequences and compared their tracking performance in normal light and bad light conditions. The parameters obtained are the root mean square error (RMSE), absolute error (AE), object tracking error (OTE), Tracking detection rate (TDR), and peak signal-to-noise ratio (PSNR) and they are compared to find the algorithm that performs the best for two conditions.


INTRODUCTION
Computer vision is the advancement in science and technology that include methods for acquiring, processing, analyzing, and understanding images. As a scientific discipline, computer vision is fretful with the theory behind artificial systems that excerpt information from images. The image data can take various forms, such as video sequences, views from many cameras, or multi-dimensional data from a medical scanner. As a scientific discipline, computer vision pursues to relate its theories and models to the design of computer vision systems. Many research extends to include: uses in security and surveillance, video communication and compression, Navigation, display technology, High-Level Video Analysis, traffic control, Metrology, video editing, augmented reality and Human-Computer interfaces to medical imaging [1][2]. Given, the most important state (e.g., position and velocity) of a target object in the leading image, the objective of tracking is to estimate the states of the target in the subsequent frames. Even though object tracking has been studied for more than a few decades and significant evolution has been made in recent years [3][4][5][6][7], it remains a challenging problem.
Object illustration is one of the key works in any pictorial tracking algorithm, and many outlines have been proposed [8]. Since the primary work of Lucas and Kanade [9], based on the raw concentration of values have been broadly used for tracking [10]. The Lucas and Kanade approach [11], do not take great occurrence changeability into account and thus, do not succeed due to the dissimilarity of the chromatic properties of a target object. Matthews et al. [12] established a template update method by exploiting the information of the first frame to accurate points. Towards better account for changes in the presence of the object, subspace-based tracking approach [7,13] have been proposed. In [14], Hager  recommended a proficient Lucas and Kanade algorithm and used low dimensional designs for tracking changing illumination conditions. Several techniques have been anticipated for video object segmentation. Commonly, these methods can be approximately classified into two types [15,16]: background structure based video object segmentation and foreground extraction based video object segmentation.
In recent times, a number of tracking methods based on sparse representations have been proposed [17][18][19][20][21]. To expand tracking robustness, a local sparse presence model was proposed in [22] using the mean shift algorithm to detect objects. By the presumptuous representation of particles as together, H. Lu et al [17] expressed object tracking as a multi-task sparse learning problem. J. Fiscus, J. Garofolo, T. Rose, and M. Michel, [23] provides an experimental evaluation of several tracking algorithms on the advanced video and signal based surveillance, in a conference on advanced video and signal based surveillance, using dataset for following various people. The effort of such reviews is of better-quality still, as in [24] which discuss tracking specific targets only, for example, sports players. The survey of J. C. McCall and M. M. Trivedi [25] is on tracking paths for driver aid.
In this paper, initially, a moving object is selected by frame differencing method and extracted object by segment thresholding. The bubble sort algorithm (BSA) arranges the regions (large to small) to make sure that there is at least one big region (object). To track the object, a motion model is constructed to set the system models of the α-β filter, LKF and EKF. Many experiments have been conducted on balls with different sizes in image sequences and compared their tracking performance in normal light and bad light conditions. The parameters obtained are RMSE, AE, OTE, TDR, and peak signal-to-noise ratio (PSNR). Experimental results indicate that the proposed algorithms perform well for the detection and tracking of different sizes of balls in video sequences captured by using a static iphone6 camera. The organization of the paper is as follows. The moving object detection and tracking methods are proposed in Section 2. Section 3 presents experimental results and discussions. Finally, the conclusion is given in Section 4.

THE PROPOSED METHOD 2.1. Simple Background Subtraction
Simple Background Subtraction (SBS) is the base for several developments in object detection [26][27] which detects moving objects through calculating the absolute difference between the reference image , [28] and incoming video frame , . Notice that the reference image , must have only static background; the incoming frame features possible moving objects along with background information. After the reference image , and the incoming video frame , are taken from a video sequence, the detected binary motion detection mask , is calculated as follows: where, τ is an empirically a selection of threshold, which is used to distinguish pixels of moving objects from those of the background in an image frame. The existence of moving objects is indicated if the absolute difference between the reference image , and the incoming video frame , go beyond τ these pixels of the detection mask , are then labeled with 1 and those that are non-active are labeled with 0.

Bubble Sort Algorithm
The bubble sort algorithm is based on a comparison of consecutive elements of a list. It uses an object detection process. Bubble sort algorithm arranges the regions (large to small) to make sure that there is at least one big region (target object) in object detection process. It performs a series of passes over the sequence: a. Compare each element (not including the last one) with its neighbour to the right-if they are out of order, swap them. Once the largest element is reached, it keeps on swapping until it goes to the last position. b. Compare each element (not including the last two) with its neighbour to the right-if they are out of order, swap them. Once the second largest element is reached, it keeps on swapping until it goes to the second last position. This process continues until there are no unsorted elements are left.

Mathematical Modelling
The main objective of object tracking is to estimate the state trajectories of an object a moving or movable object. It is well-known that a point moving in our 2D space can be described by its 2D position and velocity vectors.
X k x , y , x , y , ω For instance, "Equation (2)" can be used as a state vector of such a point in the Cartesian coordinate system, where ′ denotes matrix transpose, x and y are the position coordinates along x, y axes, respectively, and x and y are the velocity vectors, and ω is the angular velocity. The system used in this work is modelled using the "constant velocity" (CV) model, or more precisely "nearly-constant-velocity model". The Process state and dynamic measurement models are assumed to be linear in discrete time. It can be written as, where x is the N 1 vector state at discrete time k , z is the M 1 vector measurement at discrete-time k, H is the constant measurement matrix of size M N, A is the constant state transition matrix of size M N, w is an N 1 zero mean Gaussian distributed processing(or system) noise vector at time having constant covariance matrix of , and is an 1 zero mean random measurement noise vector at time having a constant positive definite covariance matrix of . , are assumed to be independent of each other for all and . It is assumed that the period between each observation is a constant ∆ seconds.

Tracking Algorithms 2.4.1 α-β filtering
The α-β tracking [29] algorithm can be expressed as follows. Prediction step (time update): where the vector x and s = x are the predicted position matrix and predicted velocity matrix, respectively. Correction step (measurement update): where the vector , ̂ are the estimated position matrix and estimated velocity matrix, respectively, α and β are tuning constants between number 0 and number 1 to smooth the position and velocity estimates, respectively.

Linear Kalman Filter
The LKF is a time-domain recursive filter with the capability to estimate the state of a dynamic system by using a series of measurements. Considering the system model equations "Equation (3)" and "Equation (4)"; the Linear Kalman filter equations are given by [30], is the a posteriori estimate of the state specified measurements up to and including time .
is the Kalman gain, is the covariance of the a priori estimation error , and is the covariance of the a posteriori estimation error . The Kalman filter is initialized by: where . is the expectation operator.

Extended Kalman filter
The EKF 31 is the nonlinear version of the Kalman filter. The nonlinear models for EKF are given by, The algorithm for the extended Kalman filtering is essentially similar to that of Linear Kalman filtering, except following modifications,

EXPERIMENTS 3.1. Implementation details
In this work, we implemented our tracker in MATLAB R2013a PC machine with Intel i7-3770 CPU (3.4GHz) with 2GB memory, which runs 29fps on this platform. In addition, the self-made video consisting of JPEG image sequence with 720 x 1280 pixels per frame is taken using the iPhone6 camera. The environment for the bad light condition is created by switching off all the lights in the room (improper illumination), while the environment is considered as a normal light condition when the room is properly illuminated.

Experimental Setup
The distance considered for tracking the target object is approximately 6meters with target objects being balls of different sizes (small, medium and big) whose radii are 3.325cms, 4.9cms and 8.5cms, respectively. The video captured using iPhone6 as following specifications: Image Type: RGB colour space, Image Dimension: 720 x 1280 pixels (Height= 1280 and Width= 720), Frame Rate: 29 fps, Number of frames: For Normal Light condition (For small size ball =54 frames, For medium size ball= 60 frames, For big size ball=58frames) and For Bad Light condition (Only for small size ball=70 frames). Table 1 shows the initial values of the basic parameters for the tracking algorithms.

Performance Measures
a. Absolute error (AE): AE is the magnitude of the difference between the true value and the tracked value of the object.
where x the true value of object parameters and x is the tracked value of the object parameters. b. Root Mean Square Error(RMSE): RMSE is one of the most widely used full-reference quality assessment metric, which is computed by square root of the average of squared intensity differences between tracked (x ) and true image pixels (x ).  (21) where N and M are the image dimensions. c. Peak Signal-to-Noise Ratio (PSNR): The value of Peak signal-to-noise ratio can easily be obtained by using the mean squared error.
PSNR 10 * log Mean square error: where, L is the dynamic range of the image (8 bits per pixel gray scale images L = 255). d. Tracking detection rate (TDR): Tracking detection rate is the ratio of a number of frames in which the object is detected to the total number of frames in which the object present.

TDR * 100
e. Object tracking error (OTE): Object tracking error is the normal inconsistency in the centroid of the tracked object from its true value. It is given by, where, x and y are the actual 2D coordinates of the object, x and y are the tracked 2D coordinates of the object.

Tracking results of small size ball in normal light condition
The RMSEs of the LKF, α-β filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value of the RMSE was reduced after applying the LKF to small ball data. On an average, the minimum RMSE reduction was 0.0330 and the maximum was 0.4478 for small size ball data. This result indicates that the predicted output of the LKF is closer to the desired output. The best average values are emphasized in Table 2. Similarly, the PSNRs of the LKF, α-β filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased after applying the LKF to small ball data. On an average, the maximum PSNR was 48.44dB and the minimum was 33.10dB for small size ball data. Figure1 shows the error of centroid, error of radius, RMSE, centroid and radius of x-y coordinates, AE and OTE in each frame for small size ball based on LKF under normal light conditions. Figure 1 shows the error of centroid, error of radius, RMSE, centroid and radius of x-y coordinates, AE and OTE in each frame for small size ball based on LKF under normal light conditions. The OTEs of the LKF, α-β filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value of the OTE was reduced after applying the LKF to small ball data. On an average, the minimum OTE reduction was 0.0476 and the maximum was 1.3562 for small size ball data. This result indicates that the predicted output of the LKF is closer to the desired output. Similarly, the AEs of the LKF, α-β filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE was reduced after applying the LKF to small ball data. On an average, the minimum AE was 0.1276 and the maximum was 0.9898 for small size ball data.

Tracking results of medium size ball in normal light condition
The OTEs of the LKF, α-β filter and EKF for medium size ball dataset are given in Table 2. As observed in Table 2, the average value of the OTE was reduced after applying the LKF to medium size ball data. On an average, the minimum OTE reduction was 0.0591 and the maximum was 0.6972 for medium size ball data. The best average values are emphasized in Table 2. Similarly, the AEs of the LKF, α-β filter and EKF for medium size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE was reduced after applying the LKF to small ball data. On an average, the minimum AE was 0.2146 and the maximum was 1.2519 for small size ball data. This result indicates that the predicted output of the LKF is closer to the desired output.  The RMSEs of the LKF, α-β filter and EKF for medium size ball dataset are given in Table 2. On an average, the minimum RMSE reduction was 0.0489 and the maximum was 0.2869 for medium size ball data. Similarly, the PSNRs of the LKF, α-β filter and EKF for medium size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased after applying the LKF to medium ball data. On an average, the maximum PSNR was 45.16dB and the minimum was 28.82dB for medium size ball data. Figure 2 shows the error of centroid, error of radius, RMSE, centroid and radius of x-y coordinates, AE and OTE in each frame for medium size ball based on LKF under normal light conditions.

Tracking results of big size ball in normal light condition
The OTEs of the LKF, α-β filter and EKF for big size ball dataset are given in Table 2. On an average, the minimum OTE reduction was 0.2997 and the maximum was 1.0102 for big size ball data. Similarly, the AEs of the LKF, α-β filter and EKF for big size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE was reduced after applying the LKF to small ball data. On an average, the minimum AE was 0.63999 and the maximum was 1.3412 for big size ball data.
The RMSEs of the LKF, α-β filter and EKF for big size ball dataset are given in Table 2. On an average, the minimum RMSE reduction was 0.2112 and the maximum was 0.4545 for big size ball data. The best average values are emphasized in Table 2. Similarly, the PSNRs of the LKF, α-β filter and EKF for big size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased after applying the LKF to big ball data. On an average, the maximum PSNR was 45.00dB and the minimum was 27.91dB for big size ball data. Figure 3 shows the error of centroid, error of radius, RMSE, centroid and radius of x-y coordinates, AE and OTE in each frame for big size ball based on LKF under normal light conditions.

Tracking results of small size ball in bad light condition
The OTEs of the LKF, α-β filter and EKF for small size ball dataset in bad light condition are given in Table 2. As observed in Table 2, the average value of the OTE was reduced after applying the LKF to small ball data. On an average, the minimum OTE reduction was 0.0227 and the maximum was 0.2275 for small size ball data. This result indicates that the predicted output of the LKF is closer to the desired output. The best average values are emphasized in Table 2. Similarly, the AEs of the LKF, α-β filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE was reduced after applying the LKF to small ball data. On average, the minimum AE was 0.1194 and the maximum was 1.0528 for small size ball data.
The RMSEs of the LKF, α-β filter and EKF for small size ball dataset in bad light condition are given in Table 2. As observed in Table 2. On an average, the minimum RMSE reduction was 0.0218 and the  Table 2. As observed in Table 2, the average value of the PSNR was increased after applying the LKF to small ball data. On an average, the maximum PSNR was 45.17dB and the minimum was 27.89dB for small size ball data. Figure 3. The experimental results of big size ball tracking using LKF Figure 4 shows some of the test results for small size ball under normal light condition. We have evaluated our approach using self-made Database, with the frame size being 720 x 1280. We have selected some frames from the video while tracking the object continuously during its movement. The target object is marked by the bounding box (red circle). Figure 4 shows 4 frames out of the tracking results based on LKF in Figure 4(a), α-β filter in Figure 4(b) and EKF in Figure 4(c). In frames 20 and 30 shown in Figure 4(a), the target object tracked using LKF is dominated by the red bounding box as the camera is static and the size of the object is reduced with distance. The same phenomena is observed in frames 30, 45 and 20, 26 shown in Figure 4(b) and Figure 4(c), using α-β filter and EKF, respectively. Even though the target object is small, the proposed tracking algorithms can detect the object (small ball).

Tracking frame results of medium size ball in normal light condition
The Figure 5 shows each 4 frames out of the tracking results based on LKF in Figure 5(a), α-β filter in Figure 5(b) and EKF in Figure 5(c). In spite of the reflections from the target object on the tiles of the floor which is consider as noise, the LKF, α-β filter, and EKF algorithms are effective in tracking the object.

Tracking frame results of big size ball in normal light condition
The test results for big size ball under normal light condition are shown in Figure 6. We have selected some frames from the video while tracking the object continuously during its movement, where the target object is marked by a bounding box (red circle). Figure 6 shows each four frames out of the tracking results based on LKF in Figure 6(a), α-β filter in Figure 6(b) and EKF in Figure 6(c).
The values of covariance matrices and plays an important role in the estimation of LKF and EKF parameters. In this paper, the values of and were determined using trial error which forms the delicacy of the proposed method. Choosing incorrect values of Q and R may lead to increase in RMSE and decrease in the accuracy.

Tracking frame results of small size ball in bad light condition
The results for small size ball under bad light condition are shown in Figure 7. The target object is marked by the bounding box (red circle). Figure 7 shows 4frames out of the tracking results based on LKF in Figure 7(a), α-β filter in Figure 7(b) and EKF in Figure 7(c). In frames 43 and 50 shown in Figure 7(a), the target object tracked using LKF is dominated by the red bounding box as the camera is static and the size of the object is reduced with distance. The same phenomena is observed in frames 34, 48 and 45, 50 shown in Figure 7(b) and Figure 7(c), using α-β filter and EKF, respectively.

CONCLUSION
In this paper, a new method is developed for tracking of different sizes of balls using α-β, LKF and EKF. In proposed tracking framework, three different tracking algorithms were used to track the target object under the normal light condition for different sizes of balls and also under bad light conditions for the small size of the ball. Experimental results demonstrate that LKF algorithm is more effective and efficient than α-β and EKF algorithms for linear system model applications. In spite of the reflections from the target object on the tiles of the floor which is considered as noise, the LKF, α-β filter, and EKF algorithms are effective in tracking the object. For future work, an optimization method could be considered in determining the covariance matrices Q and R, to obtain an optimal LKF and EKF parameter estimator more accurately.