IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012 543

The Effect of Time on Gait Recognition PerformanceDarko S. Matovski, Mark S. Nixon, Sasan Mahmoodi, Member, IEEE, and John N. Carter

Abstract—Many studies have shown that it is possible to recog-nize people by the way they walk. However, there are a numberof covariate factors that affect recognition performance. The timebetween capturing the gallery and the probe has been reported toaffect recognition the most. To date, no study has isolated the ef-fect of time, irrespective of other covariates. Here, we present thefirst principled study that examines the effect of elapsed time ongait recognition. Using empirical evidence we show for the firsttime that elapsed time does not affect recognition significantly inthe short-medium term. This finding challenges the existing viewin the literature that time significantly affects gait recognition. Weemploy existing gait representations on a novel dataset capturedspecifically for this study. By controlling the clothing worn by thesubjects and the environment, a Correct Classification Rate (CCR)of 95% has been achieved over the longest time period yet consid-ered for gait on the largest ever temporal dataset. Our results showthat gait can be used as a reliable biometric over time and at a dis-tance if we were able to control all other factors such as clothing,footwear etc. We have also investigated the effect of different typeof clothes, variations in speed and footwear on the recognition per-formance. The purpose of these experiments is to provide an in-dication of why previous studies (employing the same techniquesas this study) have achieved significantly lower recognition perfor-mance over time. Our experimental results show that clothing andother covariates have been confused with elapsed time previouslyin the literature. We have demonstrated that clothing drasticallyaffects the recognition performance regardless of elapsed time andsignificantly more than any of the other covariates that we haveconsidered here.

Index Terms—Aging, elapsed time, gait biometrics, gait recogni-tion, object recognition, personal identification, time.

I. INTRODUCTION

T HEway we walk, combined with our posture, is known asgait. Many studies have demonstrated that gait has the po-

tential to become a powerful biometric for surveillance and ac-cess control [1]–[3]. There is a continuous growth in the numberof surveillance systems in efforts to improve safety and secu-rity. These systems have yet to include recognition capability.The major advantages of gait are: noninvasive, hard to concealand capable of being acquired at a distance. Gait can be detectedat a very low resolution and therefore it has a unique advantageover all other biometrics for visual surveillance systems. Re-

Manuscript received January 21, 2011; revised October 10, 2011; acceptedOctober 13, 2011. Date of publication November 15, 2011; date of current ver-sionMarch 08, 2012. This work was supported by the National Physical Labora-tory (NPL UK). The associate editor coordinating the review of this manuscriptand approving it for publication was Dr. Fabio Scotti.The authors are with the School of Electronics and Computer Science

(ISIS Research Group), University of Southampton, SO17 1BJ U.K. (e-mail:dm08r@ecs.soton.ac.uk; msn@ecs.soton.ac.uk; sm3@ecs.soton.ac.uk;jnc@ecs.soton.ac.uk).Color versions of some of the figures in the paper are available online at http://

ieeexplore.ieee.org.Digital Object Identifier 10.1109/TIFS.2011.2176118

cently, gait has been used as a forensic tool [4], [5] and therehave already been convictions based on the evidence producedby gait analysis. There are a number of situations in which gaitis the only biometric available from a crime scene.However, there are a number of covariates (also known as ex-

ploratory variables) that affect gait recognition. Studies quanti-fying the effect of covariates on the performance of gait recogni-tion algorithms include [6]–[8]. Covariate factors can be relatedeither to the subject itself (e.g., different clothing) or to the en-vironment (e.g., different walking surface). The research in [9]considers five covariates that affect recognition: viewing angle,shoe type, walking surface, carrying objects and elapsed timebetween sequences being compared. Understanding these fac-tors is crucial to developing robust and accurate gait recognitionalgorithms. The effect of a particular covariate on the recog-nition performance depends on the algorithm adopted. Thereare two main approaches to gait recognition: model-based andmodel-free. Model-based approaches use the human body struc-ture [10]–[12] and model-free methods employ the whole mo-tion pattern of the human body [13], [14]. In this paper we useexisting model-free approaches to examine the effect of timeand other covariate factors.We use the Gait Energy Image (GEI)[15] since it is one of the most popular gait representations usedso far and we use the Gait Entropy Image (GEnI) [16] as it isa recent method and it is believed to be invariant to changes incovariate factors.There is a consensus in the literature that the time taken be-

tween recording the gallery and the probe affects recognitionperformance the most [9]. Time as a covariate has not been con-sidered explicitly in many studies mainly due to the lack of suit-able database. Since gait is a behavioral biometric, an impor-tant question arises: “Is it possible to recognize someone reli-ably after a certain period of time has elapsed?” Permanence isan important characteristic of any biometric. A biometric traitthat changes significantly over time is unlikely to be useful forrecognition in numerous scenarios.In previous studies considering the effect of elapsed time on

recognition by gait, in [17] a probe consisting of ten subjectsis matched to a gallery containing 115 subjects. The subjectsare filmed 6 months apart and a CCR of 37% is achieved. Astudy [9] reported a substantial drop in recognition performance(at rank 1) from 78% to 3% when shoe and time (6 months)covariates are introduced, using the Gait Challenge database.In [18], a different recognition algorithm is used on the samedataset. Seventy silhouettes are chosen and manually subtractedfrom the background. The recognition performance over timedropped to 10%. In [10], two different recognition algorithmsare employed on the MIT database. Experiments are performedover two months in an indoor environment with different back-grounds and lighting. On a dataset of 24 subjects, the recogni-tion rate varies between 30%–60%.Another study [19] using the

1556-6013/$26.00 © 2011 IEEE

544 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012

MIT database shows a performance drop to 45% from 100% forsamples taken on the same day. Experiments on the UMD data-base reveals a drop in recognition performance to 30% over 3months using 25 subjects in indoor environment [20]. Clothingwas not controlled in any of these studies.Many studies, such as [6], [7] and [21], do not consider tem-

poral data and focus solely on data acquired on the same day.Papers such as [22] treat time (over 6 months) as the most dif-

ficult covariate for gait recognition. They have proposed a newpattern classification method to solve the elapsed time problemin gait recognition.In this paper, we present empirical evidence to show that by

controlling clothing worn by the subjects and the environmentthe recognition performance is not affected drastically over 9months. This is the longest time period yet considered for gaitand we report, rather to our surprise, a much higher recognitionrate than any previous study. We employ existing gait represen-tations to examine the effects of time and other covariates.The work presented in this paper does not aim to propose

a new recognition algorithm, but builds on our previous work[23], and we have extended the database, the techniques, theanalyses and the results to show that elapsed time does notappear to affect the recognition performance considerably inthe short-medium term as long as some factors (like clothing,footwear etc.) are kept unchanged. Our work aims to direct theresearch community towards solving other gait covariates thatappear to affect performance more considerably than elapsedtime. Consequently, we provide evidence to show that clothingseems to be the major problem for model-free approaches.Studies such as [24] and [25] have recently started to approachthis problem. We have used existing gait representations toobtain our results. Lack of a suitable database has been amajor obstacle for understanding the effects of time. Our noveldatabase enables researchers to examine each covariate sepa-rately and discover new algorithms invariant to each covariate.We show that the ability to recognize someone from a largedistance, at a low resolution is not affected by elapsed timeif all other characteristics are controlled (even though this isvery unlikely to be achieved in the real world). We also con-sider the effect of walking speed and change of shoes. Theseexperiments are performed using a relatively small number ofsubjects and samples. Therefore, it is important to note that theprimary aim of these experiments is to provide an indication ofwhether change in any of these is likely to have a significantimpact on recognition and not to quantify the exact effect onthe performance.The remainder of this paper is organized as follows: Section II

describes existing temporal datasets and describes our newmultimodal temporal dataset. Section III elaborates on themethodology used to obtain the results which are presented inSection IV. Finally, the conclusions are presented in Section V.

II. DESCRIPTION OF DATASETS

A. Existing Temporal Datasets

Table I shows some of the most well-known datasets that con-tain data sampled at different times. Our new database is shown

TABLE ISUMMARY OF EXISTING TEMPORAL GAIT DATASETS

at the bottom of the table. It is the largest gait temporal datasetwith the longest time period yet considered for gait recognition.Other databases add more than one covariate at a time and

therefore it is difficult to quantify the individual effect. Our data-base allows gait analyses with covariates such as elapsed time,change of clothing, variations in walking speed and the effectof footwear by introducing only a single covariate between twodata capture sessions. Such analyses are not possible with any ofthe existing datasets. For instance in the Gait Challenge dataset,three covariates are added simultaneously: clothes, shoes andtime. The UMD database uses different viewing angle for thedata recorded in March compared to the data recorded in June.Clothing, shoes and background have not been controlled either.None of the existing databases contains samples over a time pe-riod of subjects wearing the same clothes and minimizing asmuch as possible the effect of other covariates. This is a key dif-ference between our new database and all the existing ones.As such, investigating the effect of time in a principled

manner is not possible with any previous dataset. It is impos-sible to determine whether the recognition performance variesdue to changes of gait over time or due to other factors suchas change in clothes, shoes, background or viewing angle.Another advantage of our database is the availability of morethan just two acquisition sessions. It is possible to achieveanalysis between ten different time periods.

B. Details of Our Novel Dataset

The data is captured using the University of SouthamptonMulti-Biometric Tunnel. The “tunnel” is a constrained environ-ment that contains 12 synchronized cameras to capture subject’sgait, a camera to capture face video and a camera to take earimages. The intended use of the “tunnel” is at airports and otherhigh throughput environments. It also allows a researcher to per-form analysis in a controlled environment and therefore facil-itates the deployment of gait in outdoor surveillance/forensicscenarios. The data is acquired automatically in a noninvasivemanner as the subject walks through it [26].A novel and unique database (SOTON temporal) has been

created that enables analyses to be performed over time whilecontrolling covariates that are known to affect gait recognitionperformance. Some of the factors that have been accounted forand remained unchanged over time are as follows.1) Environment: Background, lighting, walking surface, po-sition of cameras (automatically calibrated).

2) Subjects: No major changes in any subject’s life style werereported. None of the subjects reported any major injuries.

MATOVSKI et al.: EFFECT OF TIME ON GAIT RECOGNITION PERFORMANCE 545

Fig. 1. Raw data frames from SOTON multimodal temporal database. (a) Ex-amples of data taken in sessions 1, 2 and 3. (b) Examples of data taken in session4 (month 9).

Fig. 2. Number of subjects for each acquisition session (time is in months).

3) Other: Twenty samples per subject per session were col-lected to enable samples of similar speeds to be comparedwithout explicitly controlling it.

Twenty-five subjects (17 male and 8 female) took part in ourstudy. The age distribution is between 20–52 years old. Datawas acquired in five sessions spanning 12 months. The samesubjects were used in all sessions. The number of subjects avail-able for each session is shown in Fig. 2.The sessions taken in months 0, 1 and 4 consist of 20 sam-

ples per subject where subjects wear white overalls over theirnormal clothes. The session in month 9 differs from the previousthree. It consists of ten samples of each subject wearing normalclothes in addition to ten samples of each subject wearing over-alls. The last session (month 12) was collected to quantify theeffect of additional clothing types and footwear. During this ses-sion, subjects were asked to wear two different types of “ordi-nary” clothes that were different from the clothes that they worein the session recorded inmonth 9. Twenty-five samples per sub-ject were collected, ten for each type of clothes and additionalfive of the subjects wearing shoes. The only instruction to sub-jects is to “walk normally”. Examples of raw data frames areshown in Fig. 1.In hindsight, a session taken in month 7 should also have been

collected as it would provide data for differences of 2, 6 and 7months.There are 12 cameras available in the Biometric Tunnel,

hence a gait sequence is available from 12 different viewingangles.

Fig. 3. Example of feature concatenation.

Using our database, it is possible for the first time to isolatethe effect of time and other covariates on the performance ofgait recognition while keeping other things constant.

III. METHODOLOGY

The data processing steps are described below.Step 1—Data acquisition: A data sample consists of 12

video sequences from different angles sampled at 30 f/s. Ex-ample of data samples are shown in Fig. 1.Step 2—Background subtraction: The biometric tunnel

uses partially normalized color space combined with auni-modal Gaussian distribution to model the background.More complex algorithms produce similar results with nosignificant improvements.Step 3—Gait-cycle finder: There are at least two gait cycles

for each sequence as the subjects walk from the start to the endof the tunnel. By analyzing the silhouettes of a walking subjectthe most suitable gait cycle is automatically identified.Step 4—Signatures computation: Two gait signatures are

used in this study: Gait Energy Image (GEI) and Gait EntropyImage (GEnI). GEI was chosen because it is a baseline gait sig-nature and is one of the most widely used approaches in the lit-erature. Gait is represented as a single gray scale image obtainedby averaging the silhouette extracted over a complete gait cycle.It is computed over a complete gait cycle. If is the firstimage of the cycle and the next image is the averagesilhouette can be computed using [15]

(1)

where is the total number of images in the gait period.Three different views of the GEI have been used in our exper-

iments: top view, side view and front view. Two versions of eachview have been computed: normalized (Norm) and non-normal-ized (NN). Unlike the non-normalized signature, the normalizedone does not retain the subject’s height and body mass. Pro-ducing a non-normalized version is possible due to full 3-D re-construction. The size of the normalized signature is 64 64pixels and the size of the non-normalized is 50 50 pixels.Gait Entropy Image (GEnI) is a newer approach which cap-

tures mostly motion information and is suggested to be robust

546 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012

Fig. 4. Recognition performance over time for the GEnI signature, two different views of the GEI signature and fusion of three different GEI views (Fusion of Side, Front and Top view).

to covariate conditions changes. It offers a compact representa-tion and possess the strengths of SVB Frieze Pattern [16]. GEnIis computed by calculating Shannon entropy for every pixel

(2)

is the probability that a pixel takes on the th value.A binary image corresponds to . GEnI is computed usinga binary height normalized image of a walking subject. Conse-quently, using this signature it is not possible to utilize the ben-efits of using multiple cameras and producing a non-normalizedversion is not possible.Step 5—Feature concatenation: The feature concatenation

module creates a modified GEI signature by combining side,front and top view . An example of signature isshown in Fig. 3.Step 6—Feature Reduction: The feature reduction module

reduces the size of the signatures and enables us to performrecognition at different resolutions. The following reduced sizesignatures have been used: 50 50, 40 40, 30 30, 20 20,17 17, 15 15, 12 12, 10 10 and 7 7 pixels.Step 7—Classification: The gait signatures are used directly

for classification. Improvements in performance could havebeen achieved if additional features (e.g., moments) had beenextracted from the gait signatures and feature set selection hadbeen performed to understand which moments/features con-tribute most towards recognition. However, since the focus ofthis study is not improvement in performance but quantifyingthe effect of time and other time-dependent covariates theseimprovements are not deemed necessary.A distance matrix containing all Euclidian distances between

all samples is constructed. The Euclidian distance between twosamples is calculated by using the intensity values of the gaitsignatures. The -Nearest Neighbor classifier ( -NN) was ap-plied using the distance matrix. Experiments forand are performed.Step 8—Performance evaluation: The performance evalu-

ation is performed in terms of:

1) Correct Classification Rate (CCR).2) Receiver operating characteristic (ROC) curve.3) Equal error rate (EER).4) Diagram of intra- (within subject) and inter- (between sub-ject) class distributions.

5) The decidability metric [27] determines how muchoverlap there is between two distributions

(3)

where and refer to the intra-classmean and standard deviation, respectively, andand refer to the inter-class mean and standarddeviation.The problem becomes more decidable if their means arefurther apart or their variances are smaller. The metricshould be very similar if changes in experimental condi-tions do not drastically affect the separation in the featurespace.

IV. RESULTS

A. Effect of Time

The combinations of probe and gallery used in this exper-iment are shown in Table II. Each probe and gallery consistsof ten samples per subject. In this experiment, we only use thesamples of the subjects wearing overalls to provide consistentclothing over time. The non-normalized version of the signature(50 50 pixels) is used for this experiment. A 4-fold cross vali-dation is performed and the highest recognition rates are shownin Fig. 4. A 2-fold cross validation is performed for experimentsfor time differences of 5, 8 and 9months due to a smaller numberof samples with overalls filmed in month 9.The error-bars for GEI indicate the lowest results

of the 4-fold cross validation. The error bars for the other signa-tures are not shown for clarity. The standard deviation for the re-sults of the 4-fold cross validation is between 0.00–0.03. Values

MATOVSKI et al.: EFFECT OF TIME ON GAIT RECOGNITION PERFORMANCE 547

TABLE IICOMBINATION OF PROBE AND GALLERY

Fig. 5. ROC curve for all combinations of gallery and probe as shown inTable II.

for the EER and the decidability metric are also shown inTable II.The results in Fig. 4 clearly indicate that the CCR does not

fall considerably over time for any of the signatures consideredin this study. The variation of performance over time is sim-ilar for all signatures considered. The EER and decidabilityfor GEI (side) and GEI (Front) shown in Table II indicate thatthe signatures have a similar discriminatory ability, although thefront view performs slightly better in most cases. However, ourresults show that there is a benefit of combining multiple viewsto construct a single gait signature. The GEI pro-duces the lowest EER and the highest for all experiments.For instance, for a time difference of 1 month the EER is 6.39%and the is 2.24. Furthermore, using a non-normalized ver-sion of the silhouettes improves performance. The GEnI uses anormalized silhouette and the EER are always higher and isalways lower compared to all types of non-normalized GEI sig-natures. For instance, for a time difference of 1 month the EERis 14.57% and is 1.90 in the case of GEnI. Although thestatistics ignores moments higher than second order it was con-sidered adequate for all experiments in this study.Fig. 5 shows that the ROC curves for all combinations of

gallery and probe (shown in Table II) are very similar and over-lapping. The curve that significantly differs from the rest is fora gallery and probe taken on the same day.

Fig. 6. (a) Intra/inter class variation for time difference of 1 month, (b) timedifference of 9 months and (c) time difference of few minutes.

Fig. 6 shows a set of intra/inter class variation diagrams. TheEuclidian distance is plotted on the horizontal axis and the prob-ability density on the vertical. The diagrams in Fig. 6(a) and(b) refer to different time periods and (c) refers to probe andgallery acquired within fewminutes. The distributions in (a) and(b) are very similar. However, there is less overlap between theintra-class and inter-class distributions in (c).There is only a 5% drop in performance over 9 months when

the combination of side, front and top is used. Decidability de-creases and EER increases for analysis over time compared toanalysis on the same day. There is less overlap between the dis-tributions in Fig. 6(c) and more area under the ROC curve fordata captured on the same day. It is not clear whether the slightdrop in performance over time can be attributed to time (aging)itself or to covariate factor(s) that we have yet to consider. How-ever, our hypothesis is that the variation of performance overtime can be caused by change of clothing underneath the over-alls, as well as some change due to elapsed time. The best recog-nition is achieved when comparing similar temperature seasons(e.g., summer spring). Presumably, subjects wore a similarstyle of clothes underneath the overalls. Having subjects wear abody suit (without clothes underneath) was considered but dis-carded as it was seen as impractical.

548 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012

Fig. 7. Highest CCR achieved by studies over time using various datasets.

The values of CCR, d and EER for the front and side viewsare similar over time. There is more dynamic gait informationavailable when the side view is used, compared to the frontalview. Therefore, a question is raised how much of the dynamicinformation is used for recognition over time in the case ofmodel-free approaches, such as the GEI and GEnI. Neverthe-less, the results show that the top view is least suitable for recog-nition over time.Achieving higher recognition rates and lower error rates

could have been possible if more sophisticated classificationtechniques had been used. Nevertheless, the results presentedin this section show that it is possible to recognize someonereliably by their gait after a certain time period has elapsed,if we were able to control several other essential parameters(like clothing and footwear). Although it is not possible tocompare the results available in the literature due to significantdifference in the datasets used, Fig. 7 provides a clear indicationthat “elapsed time” has been confused with other covariates sofar in the literature. The conflating covariates have been themain reasons for low CCRs in the previous studies that haveemployed similar techniques to the ones used here.

B. Effect of Clothes

The main purpose of the experiments presented in this sec-tion is to provide an indication of why previous studies haveachieved significantly lower recognition rates over time, by em-ploying similar techniques to this study. The results could be im-proved if algorithms that are less sensitive to change in clothingare used. However, this paper does not focus on improving per-formance of recognition approaches but on understanding theeffect of time on the performance of the baseline algorithm anda more recent gait representation.Clothes can affect the overall body shape and certain types

clothing can affect the way a person walks. Previous studieshave indicated that change in clothing can have a negative effecton the performance of current gait matchers. A study [9] hasreported a recognition rate of just 3% for the combination of thefollowing covariate factors: clothes, shoes and time. Anotherstudy [7] reported a significant drop in performance (87% to60%) when subjects wore trench coat on top of their normalclothes.The data acquired in months 9 and 12 contain samples of

subjects wearing different types of clothes and enables us toperform analysis of different type of clothes over time and overfewminutes. It enables us to quantify the effect of clothes, whilekeeping all other covariates unchanged. There are four differenttypes of clothes, three types of “ordinary” clothes of subject’schoice and overalls provided as part of this experiment. Referback to Section II for details. In this section, “Type I” clothesrefer to the samples collected in month 9. “Type II” and “TypeIII” refer to the samples collected in month 12.

Fig. 8. The recognition performance over time. OverallOverall Normal Clothes Overall .

TABLE IIICOMBINATION OF PROBE AND GALLERY

Our clothing analysis consists of three experiments. In thefirst one we utilize the temporal data and perform matching overtime with the subjects wearing different clothes. In experiment2 we use the data acquired in month 9 to investigate the effectsof clothing irrespectively of any other covariate, and finally inexperiment 3 we use the data collected in month 12 to furtherquantify the effect as it provides additional clothing types.Experiment 1: The purpose of this experiment is to perform

matching of samples acquired at different time periods withsubjects wearing different clothes. In addition to the combina-tions of gallery and probe shown in Table II the combinationsshown in Table III are used. Two-fold cross validation is per-formed. Only the highest CCRs achieved are shown in Fig. 8.The highest value for the standard deviation of the CCR is 0.04.The results presented in Fig. 8 indicate that different types ofclothes can lead to a large drop in performance. The finding issimilar for all signatures considered, but only the GEI (side) and

are shown on the diagram for clarity. It is im-portant to note that the drop in recognition is consistent withtime.Experiment 2: The aim of this experiment is to investigate

the effect of different clothes in the case when the time be-tween recording the gallery and the probe is few minutes andno other changes in the experimental condition occur. Table IVshows the combinations of probe and gallery used. Fig. 9 sum-marizes the results. The horizontal axis shows the combinationof clothing considered. Type I clothes refer to the samples ofsubjects wearing “normal” clothes. Very high recognition re-sults for all views are achieved if the same types of clothes arematched. However, when different types of clothes are used therecognition rates can fall to 40%.One ROC curve in Fig. 10 significantly differs from the

others. Clothing change affects the curve’s shape dramatically.The remainder of the curves represent samples of subjectswearing the same clothes but recorded in different sessions.

MATOVSKI et al.: EFFECT OF TIME ON GAIT RECOGNITION PERFORMANCE 549

Fig. 9. Effect of clothes on the recognition performance. The probe and thegallery have been taken few minutes apart.

TABLE IVCOMBINATION OF PROBE AND GALLERY

Fig. 10. ROC curve quantifying the effect of clothing.

Fig. 11. Inter/intra class variations for different types of clothes. (a) GEI(side)—Same clothes %%, (b) GEI (side)—Different clothes% %.

Fig. 11 shows the class distributions for the GEI (side) signaturein the case of (a) same clothes and (b) different clothes.The results demonstrate that high recognition rates can be

achieved if clothes of the same type are matched and very lowrates if “extreme” changes in clothing occur. The experimentdescribed in part 3 investigates how the recognition rate is af-fected by change from one type of “ordinary” clothes to another,as opposed to “extreme” clothing variations such as the overallsuit.

Fig. 12. Effect of different types of clothes on the recognition performance forvarious gait matchers.

Fig. 13. EER for different types of clothes.

TABLE VCOMBINATION OF PROBE AND GALLERY

Experiment 3:We use the data captured in month 12 for thispart of the experiment. Table V shows the combination of probeand gallery used. Fig. 12 summarizes the results. The EER fordifferent gait signatures is shown in Fig. 13. It can be clearlyseen that the EER increases if the subjects wear clothes that sig-nificantly differ from each other. The decreases in this case.The results of this experiment show that the recognition ratesdecrease significantly less if different but less extreme clothingchanges occur.In conclusion, the recognition can be affected significantly

due to clothes change regardless of elapsed time or any other co-variates. Furthermore, the drop in performance is less significantif subjects are wearing similar types of clothes. Another study[7] performed on a different database came to a similar con-clusion for a model-based approach. The CCR dropped signif-icantly when subjects wore trench coats. The results presentedin this section are similar for the other studies that have consid-ered the effect of elapsed time. This is an indication that elapsedtime has been confused with other covariates (mainly clothing)so far in the literature.

C. Footwear

The combination of probe and gallery used is shown inTable VI. The data captured in month 12 is used. FromTable VI, it can be concluded that wearing footwear is unlikelyto drastically affect recognition. However, there is an indication

550 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012

TABLE VICOMBINATION OF PROBE AND GALLERY

TABLE VIICOMBINATION OF PROBE AND GALLERY

Fig. 14. EER for the footwear experiment.

that the EER can increase (Fig. 14). Further investigation usinga larger dataset containing a larger variation of shoes is needed.A study [7] employing a model-based approach reached

similar result to ours. The CCR did not decrease dramati-cally when subjects’ shoe type changed (except in the case offlip-flops). Another study [9] using the GEI representation hasalso achieved a relatively high recognition rate for a change infootwear.

D. Speed

A study [28] demonstrated the need for speed adjustmentof gait features. A different study [7] showed that variation inwalking speed can have a negative effect on the performanceof model-based approaches. The recognition performance de-creased from 87% to 60%.In this study, we use the combination of gallery and probe

as shown in Table VII. Each gallery and probe consists of 110samples and all samples are taken only few minutes apart. Othercovariates remain unchanged. The walking speed was not con-trolled; we are just utilizing the natural variation in the subject’sspeed to perform this experiment. Each acquisition session con-sists of 20 samples per subject. For each subject, ten sampleswith similar walking speed were identified and five were addedto “Gallery 1” and the other five to “Probe 1”. In addition, tensamples of different walking speed were identified for each sub-ject and five added to “Gallery 2” and the other five to “Probe2”. In this experiment we are interested in the relative changeof walking speed between different samples of the same sub-ject. The smallest variation in walking speed for a subject is5.8% and the greatest variation for a particular subject is 25%.

Fig. 15. Results of the walking speed experiment.

Fig. 16. Number of features versus Performance, over time. Time periods referto the elapsed time between capturing the gallery and the probe.

This data applies for the acquisition session taken in “month1”. However, it is anticipated that the variations are similar forthe other sessions because there were no changes in the experi-mental conditions. The results are presented in Fig. 15. There isa slight drop in performance for all signatures, except for GEI(Front). It would appear that the side view is mostly affected bychange of speed which is somewhat expected.Recently, Aqmar et al. [29] proposed a gait representation

that is robust to variations in speed.

E. Number of Features Versus Performance Over Time

A unique advantage of gait is the capability to do recognitionat a distance, at a low resolution. Therefore, it is important toshow how the performance changes with number of features.In this study, the number of features equates to the number ofpixels in the GEI or GEnI signature. If gait is captured at a dis-tance there are fewer pixels in the image that can be used forrecognition. Consequently, a decrease in the number of featuresis equivalent to an increase in the distance at which the subject isobserved. The rate at which the performance deteriorates withsmaller number of features should be consistent over time. Inother words, if the gallery and probe are acquired month apartand a recognition performance of is achieved with numberof features then a (where is very small) performance forthe same number of features should be expected if the galleryand probe are acquired months apart ( any value).

MATOVSKI et al.: EFFECT OF TIME ON GAIT RECOGNITION PERFORMANCE 551

We have performed an experiment using GEI (side view)to show whether time has any effect on the ability to do gaitrecognition at a lower resolution. The findings are presentedin Fig. 16. We have shown that irrespective of the time differ-ence (except when the time difference is few minutes) betweenthe probe and the gallery the degradation of performance as thenumber of features decrease is fairly consistent. This is evidencethat gait recognition can be used as a reliable biometrics at adistance over time. Fig. 16 shows that high performance can beachieved by using a small number of features. Using only 900features (30 30) a CCR of 80%–90% can be achieved. How-ever, a larger database is needed to correctly verify the numberof features needed to achieve a recognition . There is on-going work in developing gait recognition algorithms capableof achieving high recognition rates at low resolutions [30]. Themain aim of this experiment is to show that elapsed time doesnot play an important role to achieve recognition at various res-olutions.

V. CONCLUSION

In studies to date, there has been a lack of understanding ofelapsed time on the performance of gait recognition. The goal ofthis paper was to study the effect of time on gait recognition bycontrolling all covariate factors and removing their influence.This study analyses various covariate factors using existing ap-proaches to provide an indication of why previous studies haveachieved low recognition rates over time. Themajor obstacle forquantifying the effect of time and other factors independentlyis the lack of suitable databases and conflating more than onecovariate between different acquisition sessions. We have cre-ated a novel multimodal temporal database which could be usedfor developing and evaluating covariate invariant algorithms. Itis worth noting that the paper does not aim at proposing newrecognition algorithms.We have shown for the first time that gait per se is time-in-

variant in the short and short/medium term and, thus, can beused as a reliable biometric trait over time if influential covariantfactors (namely clothing, footwear) were controllable. A similarrecognition performance was achieved over seven different timeperiods and a CCR of 95% is achieved over period of 9 months.We hypothesize that a CCR of nearly 100% could be achievedif various covariate factors (like clothing, footwear etc.) werecontrollable. Unfortunately, this situation is unlikely to occurin the real world. We have shown that the major problem forrecognition over time can be partially attributed to the changeof clothes. We have shown a dramatic fall in performance inthe case of “extreme” changes in clothes. However, given thatthe problem of clothing can be overcome using techniques as in[25], [31], the issue of gaining/losing weight would still pose achallenge in the case of model-free approaches, such as the GEI.Further research is needed to identify an age-invariant featuresubspace for gait by performing feature set selection [25], [32].A second significant finding is that we have shown that other

authors reported low gait recognition since clothing and othercovariates have been confused with “elapsed time” previouslyin the literature. The CCR drops from 100% to 30% for nontem-poral combination of gallery and probe, which corresponds tothe results achieved by other studies over time.

We have also confirmed that recognition by gait at low res-olution (at a distance) is not affected by elapsed time. This isa very significant finding because recognition at a distance is amajor advantage of gait over all other biometrics.It would appear that shoes and speed are unlikely to affect

recognition performance significantly in the case of model-freegait representations. However, it is important to note that thedata collected to analyze the effect of these covariate factorscan merely provide indication and not quantify the exact effect.Experiments performed outdoors were not considered appro-

priate for this study because additional environment related co-variates would be introduced and it would be difficult to quan-tify how much recognition changes due to the imperfections ofthe extracted silhouettes and how much due to elapsed time orother subject specific covariates.Both approaches used in this work are model-free and there-

fore the conclusions in this study should be verified for model-based algorithms. However, our results show that recognition ispossible over a significant period of time and at a low resolu-tion. These results bring gait one step closer to being used as aforensic tool.

REFERENCES[1] Z. Xue, D. Ming, W. Song, B. Wan, and S. Jin, “Infrared gait recogni-

tion based on wavelet transform and support vector machine,” PatternRecognition, vol. 43, no. 8, pp. 2904–2910, 2010.

[2] K. Bashir, T. Xiang, and S. Gong, “Gait recognition without subject co-operation,” Pattern Recognition Lett., vol. 31, no. 13, pp. 2052–2060,2010.

[3] F. Jean, A. B. Albu, and R. Bergevin, “Towards view-invariant gaitmodeling: Computing view-normalized body part trajectories,” Pat-tern Recognition, vol. 42, no. 11, pp. 2936–2949, 2009.

[4] I. Bouchrika, M. Goffredo, J. N. Carter, and M. S. Nixon, “On usinggait in forensic biometrics,” J. Forensic Sci., vol. 56, no. 4, pp.882–889, 2011.

[5] P. K. Larsen, E. B. Simonsen, and N. Lynnerup, “Gait analysis inforensic medicine,” J. Forensic Sci., vol. 53, no. 5, pp. 1149–1153,2008.

[6] I. Bouchrika,M. Goffredo, J. Carter, andM.Nixon, “Covariate analysisfor view-point independent gait recognition,” in Proc. 3rd IAPR Int.Conf. Biometrics, Italy, 2009.

[7] I. Bouchrika and M. Nixon, “Exploratory factor analysis of gait recog-nition,” in Proc. 8th IEEE Int. Conf. Automatic Face and GestureRecognition (AFGR), Amsterdam, The Netherlands, 2008.

[8] Y. Shiqi, T. Daoliang, and T. Tan, “A framework for evaluating theeffect of view angle, clothing and carrying condition on gait recogni-tion,” in Proc. 18th Int. Conf. Pattern Recognition (ICPR), 2006, pp.441–444.

[9] S. Sarkar, P. J. Phillips, L. Zongyi, V. I. Robledo, P. Grother, and K.W. Bowyer, “The HumanID gait challenge problem: Data sets, perfor-mance, and analysis,” IEEE Trans. Pattern Analysis Machine Intell.,vol. 27, no. 2, pp. 162–177, Feb. 2005.

[10] L. Lee and W. E. L. Grimson, “Gait analysis for recognition and clas-sification,” in Proc. 5th IEEE Int. Conf. AFGR, 2002, pp. 148–155.

[11] B. Bhanur and J. Han, “Model-based human recognition—2D and 3Dgait,” in Human Recognition at a Distance in Video. London, U.K.:Springer, 2011, pp. 65–94, Advances in Pattern Recognition.

[12] G. Junxia, D. Xiaoqing, W. Shengjin, and W. Youshou, “Action andgait recognition from recovered 3-D human joints,” IEEE Trans. Syst.,Man, Cybern., Part B: Cybernetics, vol. 40, no. 4, pp. 1021–1033, Apr.2010.

[13] J. Han and B. Bhanu, “Individual recognition using gait energy image,”IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 2, pp. 316–322,Feb. 2006.

[14] C. Chen, J. Liang, H. Zhao, H. Hu, and J. Tian, “Frame difference en-ergy image for gait recognition with incomplete silhouettes,” PatternRecognition Lett., vol. 30, no. 11, pp. 977–984, 2009.

[15] Z. Liu and S. Sarkar, “Simplest representation yet for gait recognition:Averaged silhouette,” in Proc. 17th IEEE Int. Conf. Pattern Recogni-tion (ICPR), Cambridge, U.K., 2004, pp. 211–214.

552 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 7, NO. 2, APRIL 2012

[16] K. Bashir, T. Xiang, and S. Gong, “Gait recognition using gait entropyimage,” in Proc. 3rd Int. Conf. Crime Detection and Prevention (ICDP2009), 2009, pp. 1–6.

[17] G. Veres, M. Nixon, and J. Carter, “Model-based approaches for pre-dicting gait changes over time,” in Proc. Int. Conf. ISSNIP, 2005, pp.325–330.

[18] Z. Liu, L. Malave, and S. Sarkar, “Studies on silhouette quality andgait recognition,” in IEEE Int. Conf. CVPR, Washington, D.C., 2004,pp. 704–711.

[19] R. T. Collins, R. Gross, and J. Shi, “Silhouette-based human identifi-cation from body shape and gait,” in Proc. 5th IEEE Int. Conf. AFGR,2002, pp. 366–371.

[20] A. Kale, A. Rajagopalan, N. Cuntoor, and V. Kruger, “Gait-basedrecognition of humans using continuous HMMs,” in Proc. 5th IEEEInt. Conf. AFGR, 2002, pp. 336–343.

[21] M. Okumura, H. Iwama, Y.Makihara, and Y. Yagi, “Performance eval-uation of vision-based gait recognition using a very large-scale gaitdatabase,” in Proc. 4th IEEE Int. Conf. Biometrics: Theory, Applica-tions and Systems (BTAS), Washington, D.C, 2010.

[22] D. Tao, X. Li, X. Wu, and S. Maybank, “Elapsed time in human gaitrecognition: A new approach,” in Proc. IEEE Int. Conf. ICASSP, 2006.

[23] D. S. Matovski, M. S. Nixon, S. Mahmoodi, and J. N. Carter, “The ef-fect of time on the performance of gait biometrics,” in Proc. 4th IEEEInt. Conf. Biometrics: Theory, Applications and Systems (BTAS), Wash-ington, D.C., 2010.

[24] M. A. Hossain, Y. Makihara, W. Junqui, and Y. Yagi, “Clothing-in-variant gait identification using part-based clothing categorizationand adaptive weight control,” Pattern Recognition, vol. 43, no. 6, pp.2281–2291, 2010.

[25] I. Venkat and P. De Wilde, “Robust gait recognition by learning andexploiting sub-gait characteristics,” Int. J. Computer Vision, vol. 91,no. 1, pp. 7–23, 2011.

[26] S. Samangooei, J. Bustard, M. S. Nixon, and J. N. N. Carter, OnAcquisition and Analysis of a Dataset Comprising of Gait, Ear andSemantic Data. In Multibiometrics for Human Identification. Cam-bridge, U.K.: Cambridge Univ. Press, 2010.

[27] J. Daugman, Biometric Decision Landscapes Univ. Cambridge Com-puter Lab., Tech. Rep., 2000.

[28] R. Tanawongsuwan and A. Bobick, “Performance analysis of time-dis-tance gait parameters under different speeds,” Lecture Notes in Com-puter Sci., pp. 715–724, 2003.

[29] M. R. Aqmar, K. Shinoda, and S. Furui, “Robust gait recognitionagainst speed variation,” in Proc. 20th IEEE Int. Conf. Pattern Recog-nition (ICPR), Istanbul, Turkey, 2010.

[30] Z. Junping, P. Jian, C. Changyou, and R. Fleischer, “Low-resolutiongait recognition,” IEEE Trans. Syst., Man, Cybern., Part B: Cyber-netics, vol. 40, no. 4, pp. 986–996, Apr. 2010.

[31] A. Balan andM. Black, “The naked truth: Estimating body shape underclothing,” in Proc. ECCU, 2008, pp. 15–29.

[32] B. Guo and M. Nixon, “Gait feature subset selection by mutual infor-mation,” IEEE Trans. Syst., Man, Cybern., Part A: Systems and Hu-mans, vol. 39, no. 1, pp. 36–46, Jan. 2009.

Darko S. Matovski received the B.Eng. degree incomputer engineering from the School of Electronicsand Computer Science, University of Southampton,U.K. He is working towards the M.B.A. and Engi-neering Doctorate degrees from the same universityin the area of gait recognition.He has also been working as a Research Engi-

neer at the U.K, National Physical Laboratory onprojects related to translating gait recognition re-search to real-life environments. His research in-terests are: image processing, computer vision, gait

biometrics and visual surveillance.

Mark S. Nixon is a Professor in computer vision atthe University of Southampton, Southampton, U.K.His research interests are in image processing andcomputer vision. His team develops new techniquesfor static and moving shape extraction which havefound application in automatic face and automaticgait recognition and in medical image analysis. Histeam contains early workers in face recognition, andthey later came to pioneer gait recognition and morerecently joined the pioneers of ear biometrics. Hisvision textbook, with A. Aguado, Feature Extraction

and Image Processing (Academic) reached second edition in 2008. With T. Tanand R. Chellappa, the book Human ID Based on Gait is part of the SpringerSeries on Biometrics and was published in 2005.Dr. Nixon has chaired or had major involvement in many conferences

(BMVC, AVBPA, IEEE Face and Gesture, ICPR, ICB, IEEE BTAS) and hasgiven many invited talks. He is a Fellow IET and FIAPR.

Sasan Mahmoodi (M’02) received the B.Sc. degreefrom Amirkabir University of Technology in 1988,the M.Sc. degree from Sharif University of Tech-nology, Tehran, Iran, in 1991, and the Ph.D. degreefrom the University of Newcastle, Newcastle uponTyne, U.K., in 1998.In 1999, he held a Research Fellowship post in

mathematical analysis of human vision in the Univer-sity of Newcastle upon Tyne. He joined the Univer-sity of Southampton in 2007 as a Lecturer. His cur-rent research interests are in image processing, com-

puter vision and Biometrics.Dr. Mahmoodi sits in the executive committee of BMVA.

John N. Carter (M’90) received the B.A. degree inexperimental physics from Trinity College, Dublin,Ireland, and the Ph.D. degree in astrophysics from theUniversity of Southampton, Southampton, U.K.In 1985, he changed discipline and joined the

School of Electronics and Computer Science, Uni-versity of Southampton, as a Lecturer researching insignal and image processing, where he is currentlya Senior Lecturer with the Information: Signals,Images and Systems Research Group. In the past,he worked on programs as diverse as diesel engine

diagnostics and vocal tract imaging. A recent success in this field is thedevelopment of a new dynamic form of magnetic resonance imaging, whichmakes it possible to reconstruct high time resolution multiplanar views ofthe human vocal tract while a subject is repeating a short phrase. His currentresearch interest is in the general area of 4-D image processing, i.e., analyzingsequences of images to extract both 2-D and 3-D features, exploiting coherenceover the whole sequence, i.e., imposing simple smoothness and continuityconstraints. This has applications in object tracking and feature detection,where it is tolerant of high levels of noise and missing data. This has foundapplication in biometrics, particularly in automatic gait analysis.