Human Gait Model for Automatic Extraction and Description for Gait Recognition

The gait signature is extracted directly from the evidence gathering process. This is possible by using a Fourier series to describe the motion of the upper leg and apply temporal evidence gathering techniques to extract the moving model from a sequence of images. The improved classification capability of the phase-weighted magnitude information is verified using statistical analysis of the separation of clusters in the feature space. Furthermore, the technique is shown to be able to handle high levels of occlusion, which is of especial importance in gait as the human body is self-occluding. As such, a new technique has been developed to automatically extract and describe a moving articulated shape, the human leg, and shown its potential in gait as a biometric.


INTRODUCTION
Most people can recognize acquaintances by the way they walk, although it is not just their gait that identifies them for example, their hair style or clothing is usually recognizable.This research investigates the possibility of recognizing people by way of a gait signature as obtained by computer vision.We first review the field of biometrics and present current approaches to gait recognition by computer vision.There has been considerable study of gait in a number of fields, although there has been little cross-fertilization of ideas between these fields: psychological gait cues have yet to find deployment elsewhere.Amongst this work, there are emergent techniques aimed at recognizing people by their gait.None of these techniques use a known mechanical topology or medical studies, but concentrate more on heuristic and statistical metrics.It is however possible to develop a model-based gait extraction technique from which a metric directly applicable to the mechanics of walking can be generated.Before the performance advantages associated with such an approach are detailed, current research in gait and its allied fields shall be reviewed.

MODELING HUMAN GAIT
Gait was considered by as a total walking cycle the action of walking can be thought of as a periodic signal, with an associated frequency spectrum.

Medical Studies
The goal of most gait research has been to classify the components of gait.[7] And [6] produced standard movement patterns for normal people that were compared to the gait patterns for pathological patients [6].No statistical or mathematical analysis was performed on the collected data.Gait was considered by [6] as a total walking cycle the action of walking can be thought of as a periodic signal, with an associated frequency spectrum [1,2].

Gait Description
The following terms are used to describe the gait cycle, as given in [7].Fig. 1 illustrates the terms described.A gait cycle is the time interval between successive instances of initial foot-tofloor contact (heel strike) for the same foot.Each leg has two distinct periods; a stance phase, when the foot is in contact with the floor, and a swing phase, when the foot is off the floor moving forward to the next step.The cycle begins with the heel strike of one foot, the left foot for example.This marks the start of the stance phase.The ankle flexes to bring the left foot flat on the floor (foot-flat) and the body weight is transferred onto it.The right leg swings through in front of the left leg as the left heel lifts of the ground (heel-off).As the body weight moves onto the right foot, the supporting left knee flexes.The remainder of the left foot, which is now behind, lifts of the ground ending the stance phase with toe-off.The start of the swing phase is when the toes of the left foot leave the ground.The weight is transferred onto the right leg and the left leg swings forward to strike the ground in front of the right foot.The gait cycle ends with the heel strike of the left foot.Stride length is the linear distance in the plane of progression between successive points of contact of the same foot.
Step length is the distance between successive contact points of opposite feet.A step is the motion between successive heel strikes of opposite feet.A complete gait cycle is comprised of two steps.

Characteristics of human gait
From the work carried out by [7] and [6] it can be concluded that if all gait movements were considered, gait is unique.In all there appear to be 20 distinct gait components, some of which can only be measured from an overhead view of the subject.Murray [6] found the pelvic and thorax rotations to be highly variable from one subject to another.These patterns would be difficult to measure even from an overhead view of the subject, which would not be suited to application in many practical situations [6].As such, they would not appear suited to an automated computer vision-based biometric system.In [7] and [6], ankle rotation, pelvic tipping, and the spatial displacements of the trunk (vertical oscillation, lateral oscillation, and forward displacement) were shown to possess individual consistency in repeated trials.Naturally, given the resolution of most general purpose cameras, the ankle is difficult to extract consistently, let alone its rotation.Equally, the pelvis can easily be obscured by clothing, making a measurement of its inclination easily prone to confusion and error.The spatial displacements of the trunk are measured from the neck.As such, these components would be difficult to extract accurately from real images.Again, these would appear unsuited to an automated system.Since many features established by medical studies appear unsuited to a computer vision-based system, the components for this investigation have been limited to the rotation patterns of the hip and knee.

Rotation pattern of the hip
Fig. 2 shows the rotation angles for the hip and knee, as measured by [6].The normal hip rotation pattern is characterized by one period of extension and one period of flex-ion in every gait cycle.The gait cycle, the hip is in continuous extension as the trunk moves forward over the supporting limb.In the second phase of the cycle, once the weight has been passed onto the other limb, the hip begins to flex in preparation for the swing phase.This flexing action accelerates the hip so as to direct the swinging limb forward for the next step.The angle of rotation is measured as the angle between the line joining the hip and knee, and the line passing through the hip point parallel to the ground.

Model of legs for gait motion
The potential of the periodic nature of gait for an analytic approach was first investigated by [5], who performed a feasibility study into using gait as a biometric.An analytic approach was used, describing the legs and the motion of walking as a model based on medical and perceptual studies.The human leg was modeled as two pendula joined in series (Fig. 3).The upper pendulum modeled the thigh and was suspended between the hip and the knee.The lower pendulum modeled the lower leg suspended from the knee to the ankle.This pendulum model is backed by [6] for normal gait, the duration of successive temporal components and the length of successive steps are rhythmic.Kuan [5] extracted the hip rotation pattern for three subjects from a sequence of images using computer vision techniques.The rotation patterns were curve fitted manually to in-fill for missing data points.Fourier analysis was performed on the rotation patterns, and the magnitude and phase spectra for each subject were examined.The magnitude plots showed some variation between subjects, whilst the phase plots exhibited greater variation between subjects.The greater inter-individual variation of the phase spectra makes the phase information an attractive measure for recognition.The manner in which the hip inclination changes is of as much interest as the actual angle itself, and as such both the magnitude and phase information were of use.Kuan concluded that this model-based approach looked promising, but as yet insufficient for an automated non-invasive technique.

GAIT SIGNATURE BY EVIDENCE GATHERING
The preliminary study described in [3] demonstrated positive results in the use of gait as a biometric measure.A gait signature was extracted, but not automatically, using computer vision techniques and produced a high correct classification rate on a small database of subjects.However, the techniques used in [3] had inherent problems which would be likely to affect more general use of the technique, especially on a larger database.These are discussed in this section.Novel vision techniques have been developed specifically to overcome these problems and their motivation, as well as their implementation, is described.Further experimentation shows how these novel techniques can extract and describe gait, and results are presented showing how they can be used to recognize people by their gait.Furthermore, the new technique extracts the gait signature automatically from the image sequence, without human intervention, one of the major aims of this work.

Previous work
The study performed in [3] described a novel model-based approach to gait recognition, using the notion of gait as a periodic signal to create a gait signature.The lower limbs were modeled as two inter-connected pendula and gait was considered as the motion of these pendula.Using image processing techniques, lines representing legs in a sequence of images were extracted using the standard Hough transform (SHT).The inclination of the line representing the leg in each frame was collated to create the hip rotation pattern for the subject.In-filling for missing data points was done by least-squares analysis of the collected data points to an eighth-order polynomial.This could, in hindsight, have been achieved by Fourier interpolation.Fourier analysis was performed on the extracted hip rotation pattern using the Discrete Fourier transform to find magnitude and phase information.The magnitude data and the phase weighted magnitude data were classified using the k-nearest neighbor rule.The phase-weighted magnitude data was found to give a better correct classification rate than just the magnitude data.This model-based approach uniquely gave a signature which could be directly related to the original image sequence.However, images were analyzed singly, without reference to the whole sequence.As such, the technique would be unable to handle occlusion, except by interpolation such as by least squares.
Although the extracted hip rotation patterns in [3] concurred with those presented in medical research [6,7], the idea of gait as a periodic function was not reflected in the use of a polynomial to model the motion of the thigh.Any periodic signal, with period T, can be represented by a Fourier series (FS).The motion of the thigh is better represented as an FS rather than by polynomial fitting to the extracted data.Also, greater noise immunity can be achieved when extracting temporal features in a sequence of images by including the entire sequence in the evidence gathering process [8].Described a Velocity Hough transform (VHT) technique that enables the concurrent determination of structural and motion parameters of moving parametric shapes in an image sequence.Essentially, the VHT includes motion within the parametric model.The polar representation of a circle radius r with co-ordinates x0, y0 in the first frame and moving with horizontal and vertical velocity, vx and vy , respectively, has x and y co-ordinates at time t as where h is an index to points on the circle's perimeter.Votes are accumulated in a 5dimensional accumulator (x0; y0; vx; vy; r) from edge images of each image in the sequence.By combining VHT techniques with the FS representation of the hip rotation, a feature-based human gait model can be extracted from a sequence of images.This feature-based model has a high fidelity to the data, with a clear analytic justification.Evidence gathering using the VHT offers greater immunity to noise and occlusion, and produces a maximum likelihood estimate of the model parameters.By modeling the hip rotation as an FS, the gait signature described in [3] can be extracted directly, without intervention.Lee et al., (2009) proposed for efficient gait recognition with carrying backpack.They have been constructed gait energy image (GEI) to apply recursive principle component analysis technique.This method is aim to remove subject backpack without losing subject original shape and information.They applied to their method to normal walk, slow walk, and fast walk for experiment.They have used CASIA C dataset for conducting the test.They achieved better recognition rate after comparing others result [9].
Shingh and Biswas (2009) are approached gait energy image (GEI) method for human identification.They selected normal walk, wearing a coat or jacket or carrying a bag for recognition purposes.They informed that normal walk sequence is obtaining better recognition rate compared to carrying a bag or wearing a jacket or coat.They focused on subject body alignment with bottom and upper part of the body as feature.They also reported that gait recognition rate can be improved by applying GEI method.They selected large CASIA gait database for the experiment [10].Ju and Bir (2006) proposed gait energy image (GEI) method for person recognition individually.They created statistical gait features from actual and artificial gait templates for the experiment.They selected USF Human ID gait database for gait recognition purposes.They also used others gait database to compare recognition rate with current method with selected gait database.The GEI method is obtained better recognition rate after comparing published gait result [11].
Okumura et al., (2010) described a large scale gait database that can use widely for vision based gait recognition.They focused on gait energy image method for recognition on gender or age groups.From the experiment, female subjects are achieving better recognition rate compared to male.For the age grouping, it's evaluated according to maturity of walking ability and also physical strength.They have got different fluctuation from different age groups.They also compared with several gait databases to evaluate their method performance [12].Cheng et al., (2006) proposed gait recognition based on PCA and LDA.PCA is mainly used for dimensional reduction technique and LDA is performed to optimize the pattern class.For the experiment, they used their own database and achieved better recognition rate from PCA compared to LDA [13].

Implementation by genetic Algorithm
Extracting model instances from images is effectively a problem in optimising the defining function of the model for a given set of edge points.A Genetic Algorithm (GA) [4] is an optimisation method that is shown to consistently outperform many other search methods in solving hard optimisation problems.For a satisfactory sized parameter space, the VHT implementation took the order of days to run on a P75 PC.The GA based VHT to extract the human gait model takes approximately 20 min on the same parameter space, offering a speed-up factor of approximately 100.As such, all further experimentation was performed using a GA based implementation.In this GA implementation, each individual's chromosome was the binary coding of the parameters of the gait model.Each parameter was represented by n bits which gave an integer index to a position within a specified range.Naturally, the value of n controlled the resolution for each parameter.The fitness was derived from the number of edge points matching those calculated for the (moving) template described by the current values within the chromosome.The fittest individuals were selected as those that had a greater probability that a spin of a biased roulette wheel would select them.Crossover was set to occur with a probability of 0.7 and mutation with a probability equalling the reciprocal of the maximum population (the effect of which was to complement the mutated bit).

GA Performance for Occluded Features
A characteristic of the VHT is that it can extract temporal features in scenes where the feature has been occluded.This is attractive in extracting human gait models as the human body is selfoccluding in almost all its motions.As such, the GA based VHT for gait analysis, GAVHT, was tested for the presence or absence of this characteristic, at increasing levels of occlusion.A major advantage of the VHT method of moving feature extraction over the traditional static, frame-by-frame moving feature extraction is its improved performance in sequences where the feature has been occluded in some frames.As such, an analysis of the GAVHT's performance in extracting the gait model for an occluded subject was performed to verify that this occlusion immunity characteristic of the VHT extraction method had been maintained in its translation to a GA.Fig. 5 shows the extracted model for the unoccluded sequence for subject CM using the GAVHT.The model used only a first-order FS to represent the hip rotation.As Fig. 5 shows, the inclination of the hip is tracked throughout the cycle although without much precision.This was due to the low order of FS used to model the motion of the hip rotation, but for the purposes of this exercise it was sufficient.The subject CM in Fig. 5 was occluded by simulating a column in the centre of the field of view of the camera, between the subject and the camera.The illusion was created by setting the pixels with the column to zero (black).Fig. 4 shows an example of this process, with the column width set to 90 pixels, representing almost two fifths of the image width.A series of tests was then conducted on the sequence in Fig. 5.

Result Analysis.
For the classification analysis, two measures were compared; the Fourier magnitude and the phase-weighted Fourier magnitude.Walking sequences for ten subjects were used, each subject having four walking sequences; three training sequences and one test sequence.The measure for each test sequence was compared against those for the training sequences.The k-nearest neighbour rule was used to classify the differences in these measures for k=1 and k=3.Table 4summarises the correct classification rates (CCR) for the two measures.Unlike the earlier study [5], the nearest neighbour classification led to the same classification performance as the 3nearest neighbour rule.Classification analysis showed that the phase-weighted Fourier magnitude offered a better classification rate (100%) than just the Fourier magnitude (80%), verifying earlier work in [5].This suggests that subjects are recognised not only by flexion, but also by the time when it occurs; both the phase and the magnitude of the oscillatory motion would intuitively appear to describe a particular pendulum better.Direct generation of the Fourier information from the evidence gathering process was possible using the FS coefficients, and as such no further transform processing was required.Using this evidence gathering technique, improved classification rates, of 100% for both k=1 and k=3, were achieved compared to those obtained using the computer vision techniques in [5], on the same data being 80% and 90% for k=1 and k=3, respectively.

Comparisons.
Identification of people by gait is a challenging problem and has attracted growing interest in the computer vision community.However, there is no baseline algorithm or standard database for measuring and determining what factors affect performance.The unavailability of an accredited common database (e.g., something like the FERET database in face recognition) of a reasonable size and evaluation methodology has been a limitation in the development of gait recognition algorithms.A large number of papers in the literature reported good recognition results usually on a small database, but few of them made informed comparisons among various algorithms.To examine the performance of the proposed algorithm, here we provide some basic comparative experiments.
Tables The first comparative experiment is to test our method on the early SOTON gait database [15].This database collected six subjects and four sequences of each subject.Walkers are required to move frontal-parallel to the image plane.The gray images were captured by a fixed camera with a stationary indoor background at a rate of 25 fps, and the original resolution is 384x288.The length of each sequence is about 60 frames except that the sequences of two subjects have only 30 frames.Fig. 6 gives several samples in the SOTON gait database.Nixon and his research group have made one of the first attempts on gait recognition and have developed many algorithms [21], [22], [14], [15], [16], [23], [24], [25], [26], most of which evaluate performance on the whole or a subset of the SOTON database.Hence we evaluate the proposed algorithm on such a database so as to make a direct quantitative comparison with some of their recent methods.Table 2 shows the comparison results of several different approaches, where we directly select the best recognition accuracy reported in [14], [15], and [16] without reimplementing them.From Table 2, we can see that the recognition performance of our method is superior to others.Also, we plan to test the proposed method on their new larger dataset if available.
Tables 3. Comparison of several different approaches on NLPR database (0°).Another comparative experiment is to compare the performance of the proposed algorithm with those of five recent methods which are from Maryland [20], [27] CMU [17], MIT [18] and USF [19] respectively, and to some extent reflect the best work of these research groups in gait recognition.BenAbdelkader et al. [27] used image self-similarity plots as the original measurements to recognize gait based on the idea that the image self-similarity plot of a moving person is a projection of its planar dynamics.Reference [20] is a slight extension of [27].Based on body shape and gait, Collins et al. [17] established a template matching method based on body silhouettes in key frames for human identification.Lee et al. [18] described a momentbased representation of gait appearance features for the purpose of person identification and classification.Phillips et al. [19] proposed a baseline algorithm for human identification using spatio-temporal correlation of silhouette images.Here, we re-implement these methods using the same silhouette data from the NLPR database with a lateral viewing angle.The results are summarized in Table 3.

Methods
The above only provides preliminary comparative results and may not be generalized to say that a certain algorithm is always better than others.Algorithm performance is dependent on the gallery and probe sets.Some similar-size [17], [18], [20] or larger [19] databases have concurrently emerged, so further evaluations and comparisons on a larger and more realistic database are needed in future work.

5.FUTURE WORK
The aims of the research initially outlined were fulfilled.In this process, several areas were identified for further study.Firstly, there is the problem of how to handle the ever increasing dimensionality of the model.This will allow the variability of signatures for a given individual to be assessed to establish possible class bounds.It would also appear worthwhile to assess the potential effect of background, though the simulation tests in noise have indicated good ability to handle background.Essentially, the human gait model describes a moving line whose inclination is constrained by a periodic signal and velocity governed by some initial conditions and characteristics.Further work could explore the effect on the model parameter extraction when the evidence gathering process is performed on line images rather than edge images.These line images can be produced by SHT for lines.Due to the nature of the voting method in the SHT, in complex noisy scenes containing many various sized lines the shortest lines are unlikely to be detected.
The HHT could be used as a second stage of pre-processing to the GA for gait analysis.Accordingly, this promotes the investigation into a technique that incorporates HHT methodology for extracting temporal feature a Hierarchical Velocity Hough transforms (HVHT), perhaps.A CCD array camera on a tripod without a shutter was used to collect data, and its output was recorded on a video recorder.The camera was situated with a plane normal to the subject-path in an environment with controlled illumination.However, the presence of the stripe allows clearer assessment of extraction accuracy.Gait recognition, a characteristic word which is applied to computer vision field, is a process in which human motion features are extracted automatically and applied to recognize passerby's identity.
Comparing the line extraction of the HHT with the results of the SHT applied to the same image (Fig. 6), shows that the HHT line extraction produces a less noisy image.

6.CONCLUSION
Previous work in [5] showed that a feature-based method could be used for gait recognition.Greater immunity to moderate noise and feature occlusion when extracting temporal features in a sequence of images was achieved by using VHT evidence gathering techniques.All the frames in the image sequence were used in the evidence gathering process, allowing the concurrent extraction of both structural and temporal parameters of the feature.An improved human gait model was described, having both a structural and temporal description of the upper leg.The hip rotation was modelled by a FS, paralleling earlier medical studies that described gait as a periodic signal.This FS description of the thigh motion allowed the generation of the gait signature directly from the evidence gathering process, via the FS coefficients.

Figure 1 .
Figure 1.Relationship between temporal components of the walking cycle and the step and stride lengths during the cycle.

Figure 3 .
Figure 3. (a) Leg outline.(b) Pendulum model of a leg

Figure 6 .
Figure 6.Walking sequence with extracted thigh model for subject DC.Frames run from left to right, and top to bottom.

Figure 7 .
Figure 7. Hip rotation pattern extracted with the thigh model for the sequence in Figure 5.

Figure 8 .
Figure 8. Magnitude and phase plots for hip rotation as described by FS coefficients for gait cycle of subject DC in Figure 5.

Figure 9 .
Figure 9. Magnitude and phase plots for hip rotation as described by FS coefficients for gait cycle of subject IM in Figure 6.

Figure 10 .
Figure 10.image involve in optical flow information
2. Comparison of several different approaches on SOTON database.