Feature-based human motion parameter estimation with radar

Feature-based human motion parameter estimationwith radar

P. van Dorp and F.C.A. Groen

Abstract: Radar can be an extremely useful sensing technique to observe persons. It perceivespersons behind walls or at great distances and in situations where persons have no or poor visibility.Human motion modulates the radar signal which can be observed in the spectrogram of thereceived signal. Extraction of these movements enables the animation of a person in virtualreality. The authors focus on a fast feature-based approach to estimate human motion featuresfor real-time applications. The human walking model of Boulic is used, which describe thehuman motion with three parameters. Personification information is obtained by estimating theindividual leg and torso parameters. These motion parameters can be estimated from the temporalmaximum, minimum and centre velocity of the human motion distribution. Three methods are pre-sented to extract these velocities. Additionally, we extract an independent human motion repetitionfrequency estimate based on velocity slices in the spectrogram. Kalman filters smooth the par-ameters and estimate the global Boulic parameters. These estimated parameters are input to thehuman model of Boulic which forms the basis for animation. The methods are applied to realradar measurements. The animated person generated with the extracted parameters provides a rea-listic look-alike of the real motion of the person.

1 Introduction

Radar can be an extremely useful sensing technique toobserve persons at critical infrastructures [1]. Radarsbecame a common component of environment monitoringsystems. They have properties not present in othersensors, such as insensitive to dark and light, penetratethrough common materials and their transmitted signal isnot visible. A disadvantage of radar is that it transmits elec-tromagnetic energy and that the measurements cannot bedirectly visualised like an electro-optical sensor. One wayto visualise radar measurements is by showing them as a3D scene in a virtual environment. For the human observer,this requires the estimation of human motion features toanimate a person. Detailed processing of the radar measure-ments gives range and radial velocity as a function of timeof the person. These are used to visualise the scene in virtualreality. Real-time estimation of the human posture is anessential aspect in security application for propercountermeasures.Radar recognition was primarily designed for battlefield

surveillance radars applied for target classification.Hewish [2] and Nebabin [3] used the radial speed distri-bution to classify the targets in humans, wheeled vehicles,tracked vehicles, helicopters and unknowns. The correctclassification given is .80%. Another approach uses thephysical dimensions of the target. Rohling et al. [4] describe

# The Institution of Engineering and Technology 2008

doi:10.1049/iet-rsn:20070086

Paper first received 12th June 2007

P. van Dorp is with the TNO Physics and Electronics Laboratory, PO Box96864, The Hague, JG 2509, The Netherlands

F.C.A. Groen is with the Intelligent Autonomous Systems (IAS) Group,University of Amsterdam, Kruislaan 403, Amsterdam, SJ 1098, TheNetherlands

E-mail: [email protected]

IET Radar Sonar Navig., 2008, 2, (2), pp. 135–145

a radar system with the range trilateration technique thatestimates the object geometry. The classification betweencars, cyclists, pedestrians, trees, traffic signs and a groupof persons is more than 95% correct. The described recog-nition methods classify the human class but do not estimatethe human posture. Estimation of the human posture can becarried out hereafter.Recognition of human motion with radar is described in

[5, 6]. Their approach is based on radial speed and radialacceleration estimation which give the individual humancomponents. The acceleration estimation is time-consumingand has errors due to the interfering reflections of individualhuman components, which makes the individual humancomponents estimation difficult and sometimes impossible.If the object or a part of the object has rotation or trans-

lation in addition to the body motion, it might induce a fre-quency modulation and amplitude modulation on the returnsignal. These modulations give sidebands on both sides ofthe reflection of the human torso. The torso component isthe reflection of the human torso. This so-calledmicro-Doppler or micro-motion effect is observed in the2D time–speed space [7–9]. Additional range gives the3D time–speed–range space referred as the 3D cubicmicro-motion feature space [9–11]. These references giveexamples of micro-motion, time–frequency analyses,algorithms and suggestions for future research. No recog-nition or parameter estimation techniques are described.Chen et al. [7] and Mitra et al. [12] give the theoreticalmodel studies of micro-motion.Another approach is the feature-based approach that esti-

mates the parameters of a human walking model.Model-based approaches with the Boulic walking model[13, 14] to estimate human motion parameters are describedin [15, 16]. Boulic describes the human walking with threeglobal parameters, the cycle length, cycle frequency and thehuman speed. Although the Boulic model is used, the ani-mation showed motion personification originating from

135

changes in parameters during the motion. The enables toapply the Boulic model to other motion as well. Its disad-vantages are: requires different human motion models andthe computation time is not real-time. Although the model-based approach has a certain level of motion personifica-tion, not all personification of human motion is presentedin the models; for example, a different left/right step length.The feature-based approach we introduce in this paper

estimates the global Boulic parameters in a more efficientway, making real-time posture estimation possible. Themodel-based method that estimates the global Boulic par-ameters is the basis of our research. In this paper, weextract the global Boulic parameters (the basic level) andindependent torso and leg features (personification) with afeature-based approach which could be applied in real-timeapplications. Walking, running and jogging persons withdifferent aspect angles are studied. The scene of themoving person in a virtual environment is visualised withthe estimated global Boulic parameters and personificationtorso and leg parameters.The paper is organised as follows. Section 2 gives an

overview of the method. The theory with human model ispresented in Section 3. Section 4 gives the feature extractionmethods with Kalman filtering. Section 5 presents theexperiments and they are discussed in Section 6. Finally,Section 7 concludes the paper.

2 Overview of the method

A schematic overview of the method is presented in Fig. 1.The spectrogram shows dominant torso reflections charac-terised by the average torso speed, torso cycle frequencyand torso deviation. The deviation is the amplitude of thesinusoidal. It also shows the average leg speed, leg cyclefrequency and a leg amplitude. The leg amplitude isrelated to the average leg speed. Differences between the

Fig. 1 Overview of the method with example spectogram

Top: Spectogram, the horizontal axis is time the vertical axis is speedMiddle: Spectrogram with estimated sinusoidals of the torso and legBottom: Animated human posture with estimated parameters of thetorso and leg movements

136

left and right legs are also observed in the two sinusoidal-shaped lines. The results give the human posture thatcould be visualised in virtual reality.The Boulic model [13, 14] describes the global human

walking kinetics with three global parameters. The motionof all human limbs can be formulated as global equations,based on these parameters. The Boulic model is a globalmodel where the personification of the motion is removedby averaging over a large number of experimental measure-ments. The Boulic model is extended by Yam et al. [17]who present a hip and a knee rotation equation for humanmotions such as strolling, walking and running.The aim of the research is the fast estimation of the global

Boulic parameters and personification parameters from thespectrogram for real-time applications. Fig. 2 shows ourfour-component feature-based method. The features are asfollows.

1. The pre-processing performs the radar processing tasks.It tracks the person and calculates the corresponding vel-ocity–time slices or spectrogram. The spectrograms showthe views of the target for one range cell. Adding adjacentrange spectrograms removes the quantisation effects.2. The feature extraction calculates the essential featuresfrom the spectrograms and estimates the global and person-alised Boulic parameters from these. Least-squares fit of thecentre frequency gives the sinusoidal parameters of thetorso and the least-squares fit of the velocity bounds givesthe sinusoidal parameters of the hip and knee flexing.3. The walking dynamics component calculates the fullposture of the person from the parameter values requiredfor visualisation.4. The body model component displays the posture of thehuman in virtual reality [18].

3 Theory

3.1 Introduction

The Boulic model introduced here describes the globalhuman walking kinetics with three global Boulic par-ameters, cycle frequency f (Hz), cycle length L (m) andcycle phase w. The cycle frequency is the number ofcycles in 1 s. One cycle is the motion between two succes-sive contacts of one heel with the floor. The cycle length isthe distance between these two contacts. The phase indi-cates the time in the cycle.Fig. 3 gives a typical spectrogram of a walking human

with the radial velocities of the Boulic model. The Boulicmodel used has torso, upper leg, lower leg, feet, upperarm, lower arm and head as components. The humanelement heaviness points are the locations used to calculatethe radar response. Subplot a presents the spectrogram,subplot b all human components according to Boulic andthe projections of the Boulic components on the spectro-gram. The projections agree reasonably with the

Fig. 2 Feature-based approach

IET Radar Sonar Navig., Vol. 2, No. 2, April 2008

Fig. 3 Typical spectrogram of a walking person with radial velocities of the Boulic model

spectrogram. The measured spectrogram in subplot (a)shows the strong reflections caused by the motion ofhuman components and zero velocity ground clutter reflec-tions. The inbound walking human average velocity is 2 m/s. The cycle frequency is 1 Hz, half the cycle is the forwardfoot movement and half the cycle the foot is in contact withthe floor. The torso, subplot (d) is the maximum or strongresponse. The torso has average velocity with superimposedsinusoidal with double frequency. The upper arms’ subplot(e) and upper legs’ subplot (g) lie around the maximumresponse. The backward movement of the lower armssubplot (f) has zero velocity, whereas the forward move-ment is comparable with the forward upper leg movementsubplot (g). The lower leg maximum forward movementsubplot (h) is approximately two times the average speed;the contact phase with the ground has approximately zerovelocity. The feet subplot (i) gives the highest forwardmovement approximately three times the average velocity;the minimum velocity is zero.Subplot (c) with all human components shows the

minimum and maximum velocity bounds where all othervelocities lie. The lower legs and feet almost completelycontribute to the velocity envelopes. The contribution ofthe feet is less than that of the lower leg due to its smallreflected energy. The torso, upper arms, lower arms andupper legs show sinusoidal amplitudes around the averagevelocity. The torso frequency is twice the frequency ofarms and legs. The lower leg response looks like a sinusoi-dal component with additional component during theforward movement. This additional component is asecond sinusiodal with higher frequency which gives theextra movement above on top of the other sinusoidal. Allhuman components show a periodic behaviour with cyclefrequency periodicity in a velocity slice. The velocityslices with the torso component show double cyclefrequency.Deviations from the spectrogram and Boulic trajectories

arise from imperfect parameter estimates and fluctuationsof these in the registration window. Other physical imper-fections are the small reflected energy, interfering reflec-tions and multipath reflections. Small reflected energy of


human components lies below the noise level and is notvisible. Interfering components with almost equivalentspeed and range but different positions influence thesignal intensity and cause unreliable peaks which do notcorrespond with a human component. Multipath reflectionsof individual human components and multipath with theground also cause unreliable peaks.The relation between the extracted spectrogram features

and the three models is summarised as follows.

1. The centre velocity C(t) is related with the torso velocitymodel vT(T ).2. The minimum velocity L(t) and the maximum velocityU(t) are related with the lower leg model velocities vU

L(t)and vL

L(t). The other human components lie between thesetwo velocities.3. The repetition frequency extracted of velocity slices is ameasure for the cycle frequency.

3.2 Global Boulic parameters

The Boulic model [13] introduced here describes the globalhuman walking kinetics with three global Boulic par-ameters, cycle frequency f (Hz), cycle length L (m) andcycle phase w. One complete cycle is the motion betweentwo successive contacts of one heel with the floor. Thecycle length is the distance between these two contacts.The phase offset indicates the time in the cycle. Thehuman normalised velocity V (m/s) is expressed in thesize of the human and average velocity S (m/s). The sizeof the human is expressed by the constant b (m/m) thatscales the leg length of the human to the metre (V ¼ bS).The normalised velocity depends on the cycle length Land the cycle frequency f of the human, V ¼ Lf andS ¼ Lf/b.

3.3 Approximations of human body trajectories

This section gives the approximations of the torso, legs andarms trajectories in the direction of movement accordingBoulic et al. [13, 14] and Yam et al. [17].

137

The walking pattern is structured in three parts: averagehuman velocity, torso velocity and legs velocity. We willpresent the velocity equations. The range equations arethe integral of the velocity equations with additional rangeconstant provided by the human track information [19].The average human velocity is approximated with an

average speed S and acceleration S of the human body:v(t) ¼ Sþ St.The torso moves forward and backwards round about the

average human velocity. The forward and backward vel-ocity of the torso indicated with superscript T is approxi-mated with a linear average velocity of the human with asuperimposed sinusoidal function with twice the cycle fre-quency

vT(t, zT) ’ S þ _St þ ST cos(2p2ft þ 2pwT ) (1)

where ST is the sinusoidal amplitude, f the cycle frequencyand zT ¼ (ST, S, S, f, wT) the parameter vector. We use thecentre velocity of each velocity slice as an estimate of thistorso velocity.The forward and backward translations of the human

arms and legs are superimposed on the torso motion andare approximated by sinusoidal functions according toBoulic and Yam. The arms result in an approximationwith superimposed sinusoidal function with equivalentcycle frequency but other amplitude and cycle phase. Thelegs result in an approximation with a superimposed sinu-soidal function with equivalent cycle frequency and twoparts. The first part of the leg cycle that corresponds withthe swing movement has a large amplitude the other partthat corresponds with the rest phase of the leg has a smallamplitude. The strong amplitude and a small amplitudeare out of phase for each leg. The amplitude of the armsare smaller than that of the legs and the envelope belongsto the lower legs.We mentioned that the motion of the leg is superimposed

on the torso motion. The torso motion is small comparedwith the leg motion and could be ignored. Combining thetwo out-of-phase lower leg motions gives envelopes indi-cated with superscript L

vLU (t, zL) ’ S þ _St þ SLjcos(2pft þ 2pwL)j (2)

vLL(t, zL) ’ S þ _St � SLjcos(2pft þ 2pwL)j (3)

where zL ¼ (S, S, f, wL) is the lower leg parameter vector.The personification of the Boulic model with only three

parameters originates from individual frequency, phaseand amplitude parameters for the torso and leg. The perso-nification torso frequency is fT and leg frequency fL.

4 Feature extraction

Fig. 2 gives the four components of feature extraction. Thesegmentation component determines the human response inthe spectrogram. The next component extracts themaximum velocity U(t), minimum velocity L(t) andcentre velocity C(t) to estimate the leg and torso parametersof each time slice. The methods are based on detections,percentiles and correlation. In addition, the cycle frequencyon bases of velocity slices in the spectrogram is extracted.The model-fitting component calculates the model featuresof the legs and torso. The feature filtering smoothes the fea-tures and estimates the global Boulic parameters.

138

4.1 Segmentation

The segmentation starts with the calculation of an adaptivethreshold and decides on the basis of this threshold whethera spectrogram component belongs to the human response ornot. The adaptive threshold u is calculated with the trianglehistogram algorithm [20]. Velocity components smallerthan vstatic are removed because they contain reflections ofstatic objects or ground clutter and the human responsearea around zero velocity.The components with human response are clustered [19]

and selects connected spectrogram components and othertargets in the scene are removed. The velocity bounds inwhich the human response lays are vmin and vmax.The spectrogram is indicated by y(v, t) with time window

0 � t � tmax, where v is the velocity and t the time. Thedetection spectrogram yd(v, t) ¼ (y(v, t) . u) ^ (jvj .vstatic) ^ (vmin � v � vmax) has ones onto the detectedhuman and zeros otherwise.

4.2 Extraction of the velocity bounds, target centreand cycle frequency

4.2.1 Detections: This method uses the detection spectro-gram (Fig. 4b). The method first extracts the average vel-ocity. For each time slice, see Fig. 4a; the sum ofdetections above the average velocity gives the upper vel-ocity and the sum of detection below the average velocitythe lower velocity. The centre velocity is the maximum vel-ocity response of each time slice.The average linear acceleration model is v(t) ¼ v0þ at

for 0 � t � tmax in which v0 is the estimated averagehuman velocity and a the estimated human acceleration.This model is fitted on the detection spectrogram time slices

x(�v0, �a) ¼X

0�t�tmax

Xv¼vmax

v¼vmin

(vyd(v, t)� �v0 � �at)2 (4)

Fig. 4 Three methods which extract the velocity bounds

a Spectrogram with time and velocity slicesb Velocity strength and detection threshold time slice 0.5 sc Percentile time slice 0.5 sd Three examples of Mexican hat wavelet functions with varyingwidth


The summed detections above this line give themaximum velocity U(t)

U (t) ¼ �v0 þ �at þ DvXv¼vmax

v¼�v(t)

yd(v, t) (5)

the summed detection below this line gives the lower vel-ocity L(t)

L(t) ¼ �v0 þ �at � DvXv¼�v(t)

v¼vmin

yd(v, t) (6)

The centre velocity is defined as the maximum velocityresponse of each time slice

C(t) ¼ maxv[yd (v,t)

(y(v, t)) (7)

4.2.2 Percentiles: This method calculates the cumulativeamplitude distribution for each time slice (Fig. 4c). Theamplitude distribution gives the target versus backgroundbounds. The background noise lies at the extremities andthe target lies in the middle.The cumulative amplitude distribution function P(v, t) of

each time slice is

P(v, t) ¼

Pvv¼vmin

y(v, t)Pv¼vmaxv¼vmin

y(v, t)(8)

The percentile P(U(t), t) ¼ 95% gives the upper velocity,the percentile P(L(t), t) ¼ 5% the lower velocity and P(C(t), t) ¼ 50% the centre velocity.

4.2.3 Correlation: The correlation method estimates thewidth and centre of the human velocity distribution of atime slice by correlating a Mexican hat wavelet [21] withvarying width s and the best correlation is selected. TheMexican hat wavelet is given by, (Fig. 4d)

h(v, s) ¼1ffiffiffiffiffiffi2p

ps3

1�v2

s2

� �exp �

v2

2s2

� �(9)

The velocity response y(v, t) has a Poisson distribution; weuse a log transformation to obtain normal-distributed back-ground noise. Each log time slice is correlated with h(v, s)for varying widths s

yc(v, t, s) ¼ corr(h(v, s), log(y(v, t))) (10)

Each time slice t has a maximum correlation with velocityvcor and width scor. The centre velocity is defined as themaximum correlation C(t) ¼ vcor. The envelopes are twicethe sigma L(t) ¼ vcor2 2scor and U(t) ¼ vcorþ 2scor.

4.2.4 Repetition frequency: The Fourier transform toextract a repetition fails due to the short observation timeand deformed sinusoidal shape. The alternative is to fit asinusoidal with linear trend to the velocity slice.Before fitting, the log transformation is applied to obtain

the normal-distributed background noise. The sinusoidalamplitude fit of velocity slice v is

x2(zA, v) ¼X

0�t�tmax

( log (y(v, t))� A� _At

� ~A cos(4pft þ 2pwA))2 (11)

with zA ¼ (A, A, A, f, wA), the parameters to be estimated.The repetition frequency is twice the cycle frequency dueto the combined effect of two out-of-sync leg motions in a


velocity slice. The sinusiodal variables are extracted withsubtracted linear trend. We remove the velocity dependencyof A( f, v) by integrating over these

~A(f ) ¼Xv¼vmax

v¼vmin

~A(f , v) (12)

The repetition frequency belongs to the maximum sinu-soidal amplitude fA ¼ maxA(A( f )).

4.3 Model fit

4.3.1 Leg fit: The leg fit has two stages. The first stage fitsa sinusoidal curve to the difference of the upper and lowervelocities

x2(zAL) ¼X

0�t�tmax

([U (t)� L(t)]� [vLU(t, zL)

� vLL(t, zL)])2 þ 12 (13)

with zAL ¼ (2SL, fL, wL), the estimated parameters. The 12

indicates the imbalance that results from an acceleratinghuman. This fit is less sensitive to noise than the individualfits and independent of the upper and lower velocities.The second stage extracts the linear trend and two off-

phase cosine functions with fixed leg frequency fL of theprevious fit

x2(zL) ¼X

0�t�tmax

(L(t)� vLL(t, z

L))2

þX

0�t�tmax

(U (t)� vLU(t, zL))2 þ 12 (14)

4.3.2 Torso fit: The sinusoidal torso fit extracts the lineartrend and sinusoidal of the centre velocity

x2(zT) ¼X

0�t�tmax

(C(t)� vT(t, zT))2 (15)

4.4 Filtering

Feature filtering with Kalman filters is applied to smooth theestimated parameters for imperfect estimates, and the esti-mates smoothed the Boulic parameters in time. Section 9gives the essential parts. The filter input vectors used are:

† torso zT ¼ (ST, ST, ST, wT, fT)† leg zL ¼ (SL, SL, SL, wL, fL)† global Boulic parameters with leg frequency and withoutrepetition frequency ¼(V, L, fL)† global Boulic parameters with leg frequency and rep-etition frequency ¼( fA, V, L, fL).

5 Experiments

Two experiments are carried out applying the algorithmonto a test database and validation of the algorithm withthe model-based approach [15].

5.1 Database

We carried out 50 radar experiments of walking, joggingand running persons in normal terrain conditions. Theaspect angles vary between 08 and 1808. The 9.68 GHzfrequency-modulated continuous wave (FMCW) radarused has a variable bandwidth and 2 kHz sweep repetitionfrequency. Detailed description of FMCW radar processing

139

and target tracking is covered in [22, 23]. The spectrogramintegration time is 1.024 s and the time between successivespectrograms is 40 ms.

5.2 Feature extraction

The methods are applied onto a typical walking humanequivalent to the response presented in Fig. 3.Subplots (a) and (b) in Fig. 5 present the results of the

detection method. The velocity bounds in subplot (a) corre-spond with the bounds of the human response. A highsignal-to-noise ratio gives a small number of gaps. Thenumber of gaps increases with a low signal-to-noise ratioand the estimated velocity bounds lie dense at the torso vel-ocity causing a systematic error with smaller amplitude. Thetorso response fluctuates and there are cases where otherresponses have higher intensity. There are occurrences

Fig. 5 Results of the various methods

a Spectrogram with velocity bounds and centre velocity, detectionmethodb Spectrogram with detectionsc Spectrogram with velocity bounds and centre velocity, percentilemethodd Estimated sinusoidals, percentile methode Spectrogram with velocity bounds and centre frequency, correlationmethodf Example of the velocity-width space of one time slice, correlationmethod. Low correlation is black and a high correlation is whiteg Amplitude fit function hAverage repetition frequency strength

140

where other human components interfere with the torsoand reduce the torso amplitude and another element withhigher intensity is selected. Other occurrences show twointerfering components that give a high intensity greaterthan the torso response.Subplot (c) in Fig. 5 presents the percentile method with

5%, 50% and 95% percentiles. The percentile method upperand lower velocity shows equivalent behaviour with thedetection method. The torso estimate gives a good corre-lation with the expected torso in the spectrogram. The esti-mated sinusoidals in subplot (d) show the cyclicity of thetorso and legs. The torso frequency is twice the leg fre-quency. The estimated leg sinusoidal is less affected bythe forward movement of the foot near 0.5 s; it gives theaverage upper and lower velocities which correspond withthe leg.The correlation results in Fig. 5 subplot (e) presents the

estimated centre and envelopes and subplot (f) the corre-lation time slice yc(v, t ¼ 0, s). The correlation results insubplot (f) shows a smoothed function with time-varyingmaximum (vcor, scor). The velocity bounds are always sym-metric around the centre velocity vcor. The centre velocitydeviates strongly with the expected torso. The velocityenvelope and centre velocity are correlated with theMexican hat function which differs from the real relationbetween these.Fig. 5 subplots (g) and (h) present the results of the rep-

etition frequency extraction. The fit-function subplot (g)shows multiple minima on the line with 1 Hz repetition fre-quency. The torso component (2 m/s velocity slice) shows arepetition frequency of 1.3 and 2.6 Hz in accordance withthe double cycle frequency. The summed amplitudes insubplot (h) show a strong peak with 1 Hz which correspondswith the average repetition frequency.

5.3 Estimated parameters

The percentiles results of a typical experiment of aninbound walking human is presented in Fig. 6. Subplot (a)presents the spectrogram. The human has four stages: theacceleration stage ta ¼ 0–4 s, the fast walking stagetf ¼ 4–11 s with 2 m/s, the decceleration stage ta ¼ 11–13 s and the slow walking stage ts ¼ 13–15 s with0.25 m/s. Subplots (b)–(d) present the three estimatedglobal Boulic parameters zB ¼ ( f, L, w). The parametersare zB(ta) ¼ (0 Hz, 0 m, 2), zB(tf) ¼ (1 Hz, 2 m, 2),zB(td) ¼ (0 Hz, 0 m, 2) and zB(ts) ¼ (2 Hz, 0 m, 2). Thecycle phase normalised between zero and one shows theperiodic behaviour of the movement. The cycle frequencyand cycle phase show some irregularities at the start andthe end where the human has almost zero speed. The gener-ated synthetic human with these parameters shows a veryrealistic moving human in which all individual human com-ponents show smooth behaviour. The start and end also lookvery realistic due to the small cycle length which corre-sponds with a slow walking human.Subplots (e)–(i) give the leg parameters zL ¼ (SL, SL, SL,

wL, fL), and the estimated sinusoidal is presented in subplot(j). The parameters are zL(ta) ¼ (0 m/s, 1.5 m/s2, 0 m/s,2,0 Hz), zL(tf) ¼ (2 m/s, 0 m/s2, 1.5 m/s, 2, 1 Hz)zL(td) ¼ (0 m/s, 21.5 m/s2, 0 m/s, 2, 0 Hz) andzL(ts) ¼ (0.25 m/s, 0 m/s2, 0.25 m/s, 2, 1.5 Hz). Theacceleration fluctuates during walking due to sinusoidalapproximation of the leg motion. These fluctuations givethe personification of the moving human. The estimatedsinusoidal of one feet in m/s has a smoothed behaviourduring the walking part: the start shows two small stepsand the end shows four small steps. During walking, the


Fig. 6 Estimated Kalman parameters of a walking person by the percentile method

minimum velocity of the lower leg is 0.5 m/s indicating noground touch, the heel touched the ground with zerominimal velocity.Subplots (k)–(o) give the torso parameters zT ¼ (ST, ST,

ST, wT, fT), and the estimated torso sinusoidal is presented insubplot (p). The parameters are zT(ta) ¼ (0 m/s, 1.5 m/s2,0 m/s, 2, 3 Hz), zT(tf) ¼ (2 m/s, 0 m/s2, 0.5 m/s, 2,2 Hz), zT(td) ¼ (0 m/s, 21.5 m/s2, 0 m/s, 2, 3 Hz) andzT(ts) ¼ (0.25 m/s, 0 m/s2, 0.1 m/s, 2, 3 Hz). The speedshows a ripple due to the sinusoidal approximation of thetorso movement. The estimated amplitude has a parabolicshape with a maximum of ST ¼ 0.5 m/s during walking,it increases and decreases during the movement. In thelow walking stage, the human has some small steps withfast phase shift. The torso frequency fT is twice the leg fre-quency and increases at the end due to the fast small steps.The estimated sinusoidal shows the torso behaviour aroundthe average motion of the human.The global Boulic parameters, estimated leg and torso

parameters show smooth behaviour with personification.The estimated personification differs from the leg andtorso parameters calculated with the global Boulic par-ameters and results in a very realistic animated walkinghuman.

5.4 Test on database measurements

The 50 experiments give 25 000 spectrograms belonging toa tracked person and the feature extraction is applied to it.


Table 1 gives the Kalman filter error standard deviationsof the extracted features. Fig. 7 presents the average compu-tation time of the MATLAB implementation.The results show that the three-feature extraction

methods give equivalent standard deviations. The leg par-ameters have higher standard deviation than the torso par-ameters. The repetition frequency standard deviation issmaller than the leg frequency standard deviation. Fusionof the repetition frequency and the leg frequency gives a0.7 improvement in the standard deviation that agreeswith uncorrelated parameters.The detection and percentile methods are six times faster

than the correlation method. The computation time of thepre-processing is equivalent to the detections and percentilecomputation time, and the repetition frequency extractionrequires 1.2 times this time.

5.5 Comparison with the model-based approach

Table 2 gives the standard deviations of the differencebetween the model-based approach and the feature-basedapproach. The cycle frequency and cycle length estimateshave negligible zero offsets, whereas the cycle phase hasa small offset. The cycle frequency lies within 5% of thecycle frequency range, the cycle length lies within 10% ofthe cycle length range and the cycle phase lies within 5%of the cycle phase range. The cycle phase has 15% offset.Detailed investigations show that the differences dependon the speed. High velocities’ small differences and low

141

142

Table 1: Standard deviations estimated parameters based on the Kalman filter errors

Parameter, (range)

units

Detections Percentiles Correlation

torso sST

[0–3] m/s 0.14 0.12 0.18

s_ST

[0–3] m/s2 0.26 0.19 0.25

sST

[0–3] m/s 0.05 0.04 0.05

swT

[0–1] 0.17 0.17 0.17

sfT

[0–2] Hz 0.58 0.58 0.61

leg sSL

[0–3] m/s 0.12 0.09 0.12

s_SL

[0–3] m/s2 0.18 0.16 0.16

sSL

[0–3] m/s 0.13 0.10 0.06

swL

[0–1] 0.06 0.06 0.06

sfL

[0–2] Hz 0.22 0.24 0.23

velocity sV [0–6] m/s 0.10 0.10 0.10

repetition frequency sfA

[0–2] Hz 0.19 0.19 0.20

global Boulic parameters without repetition

frequency

sL [0–3] m 0.35 0.32 0.26

sf [0–2] Hz 0.22 0.24 0.23

global Boulic parameters with repetition

frequency

sL [0–3] m 0.20 0.20 0.21

sf [0–2] Hz 0.15 0.16 0.16

The torso and leg are the personification parameters. The repetition frequency and velocity are estimatedparameters. The Boulic parameters based on the leg frequency are given the fifth row. The Boulic parameters withthe leg frequency and repetition frequency are given in the last row

velocities’ greater differences. The reason is the flat envel-ope in the case of low velocities.

6 Discussion

6.1 Methods

The detection method velocity bounds are affected by thenumber of missed detections and lie dense at the centre vel-ocity in the case of a low signal-to-noise ratio. The percen-tile method accounts the cumulative sum of the signal, andthe signals outside the velocity bounds affect the amplitudedistribution. The correlation method gives an average sym-metric velocity bounds based on the signal distribution.The region in the spectrogram between the upper and

lower bounds belongs to all scatters of the human andcause interfering signals. These interfering signals causestrengthening and weakening of the received signals, and

the torso estimates deviate from the real location of thetorso. Sometimes, the maximum intensity differs from thetorso but lies always dense at the real value. The percentilestorso estimate is very stable and gives a better estimate ofthe torso. The correlation method gives the worst torso esti-mate, an average between the upper and lower bounds.The repetition frequency extracted is very stable irrespec-

tive of the shape of the velocity bounds. The estimated stan-dard deviation is ,5%.

6.2 Parameter estimation

The standard deviations presented in Table 1 show noessential difference between the extracted parameters. Therepetition frequency standard deviation sfA ¼ 0.19 issmaller than that of the leg frequency sfL ¼ 0.22; fusionof these with a Kalman filter gives 0.7 standard deviationimprovement sf ¼ 0.16. This agrees with two uncorrelated

Fig. 7 Average computation time


IET Radar So

Table 2: Comparison of the model-based and feature-based approaches

Global Boulic parameters Parameter (range) units Detections Percentiles Correlation

frequency sf [0–2] Hz 0.07 0.07 0.06

length sL [0–3] m 0.26 0.23 0.22

phase Dw [0–1] þ0.14 þ0.12 þ0.10

sw [0–1] 0.05 0.04 0.03

The cycle frequency and cycle length have negligible offset; the phase offset is presented

parameters. The Kalman feature filtering gives smoothedfeatures in time without irregularities. The repetition fre-quency is not necessary; both methods give a realisticlook-alike of the observed human motion. The generatedhuman with global Boulic parameters shows personificationand the results show that the addition of torso and leg par-ameters shows more personification.Detail investigation shows some of the following

differences.

† Leg features: The correlation method produces the bestleg features. The correlation method centre frequency liesin the middle and has less relation with the torso frequency.The percentile method gives the second best leg features.† Torso features: The percentile method produces the besttorso features, and the features are less affected by noise andinterfering caused by other limbs.† Repetition frequency: The leg cycle frequency of all themethods and the repetition frequency have equivalent stan-dard deviation but the repetition frequency produces betterfeatures in the case of flat velocity bounds. How flatterthe velocity bounds? How worse the quality of the leg fre-quency estimate.

The detection and percentile methods with and withoutrepetition frequency require one-sixth computation time ofthe correlation method. We advise the percentile methodwith leg frequency and repetition frequency fusion ifenough computation time is available; the repetition fre-quency could be ignored in a real-time application. At thecurrent state-of-the-art, a real-time solution is possible.

6.3 Difference between the model-based andfeature-based approach

The model-based parameters and feature-based parameterscorrespond with each other with an exception of the offsetof the cycle phase. The phases of the body components ofthe Boulic model in the model-based approach are allrelated to each other, whereas the feature-based approachestimates different phases of the body and the legs. Themodel-based approach estimates an average cycle phase,whereas the cycle phase of the feature-based approachbelongs to the phase of the leg.

7 Conclusions

In this paper, we present a feature-based approach to esti-mate the human motion parameters from radar spectro-grams. The walking model of Boulic is used withpersonification information of the torso and leg. We haveproposed a sinusoidal model for the torso and leg. Thetorso sinusoidal is related with the centre velocity and theleg sinusoidal is related with the lower and upper velocitybounds. Three methods are described which extract thesevelocities. Sinusoidal fits of velocity slices give the

nar Navig., Vol. 2, No. 2, April 2008

repetition frequency that originates from periodic spectro-gram components. Kalman filters smoothes the features.The animated human generated with the features providesa realistic look-alike of the real motion of human irrespec-tive of the walking model used to generate it.The methods are applied to real radar measurements with

different scenarios. The three methods give approximatelyequivalent results. The percentile method has the bestmatch with the velocity bounds and centre velocity. Thecomputation time of the correlation method is about sixtimes that of the other methods. We advise the percentilemethod with leg frequency and repetition frequencyfusion if enough computation time is available; the rep-etition frequency could be ignored in a real-timeapplication.The comparison of the feature-based approach with the

model-based approach shows that the standard deviationcycle frequency f and cycle phase L lie within 5% and thecycle length within 10% of the parameter range. Thecycle phase has a small offset due to the average cycle ofthe model-based approach and the leg phase in the feature-based approach. The accuracy of both approaches are equiv-alent but the feature-based approach is faster.The feature-based approach brings the animation in the

real-time domain and opens possibilities for person recog-nition besides animation.

8 References

1 Withington, P., Fluhler, H., and Nag, S.: ‘Enhancing homelandsecurity with advanced UWB sensors’, IEEE Microw. Mag., 2003,4, (5), pp. 51–58

2 Hewish, M.: ‘The last line of defense, reducing the vulnerability offixed installations’, Janes’s Int. Def. Rev., 1999, 10, pp. 28–35

3 Nebabin, V.G.: ‘Methods and techniques of radar recognition’ (ArtechHouse, 1995)

4 Rohling, H., Folster, F., Kruse, F., et al.: ‘Target classification basedon a GHz radar network’. Radar 2004, Int. Conf. on Radar Systems,2004

5 Woltkiewicz, A., Nalecz, M., Kulpa, W., et al.: ‘Use of polynomialphase modeling to fmcw radar. Part c: estimation of targetaccelerationin fmcw radars’. NATO Research and TechnologyAgency, Sensors and Electronics Technology Symp. Passiveand LPI Radio Frequency Sensors, 23–25 April 2001 paper number40C

6 Nalecz, M., Rytel-Andrianik, R., and Wojtkiewicz, A.:‘Micro-Doppler analysis of signals received by FMCW radar’. Int.Radar Symp., 2003, pp. 651–656

7 Chen, V.C., Li, R., Ho, S.S., et al.: ‘Micro-Doppler effect in radarphenomenon, model, and simulation study’, IEEE Trans. Aerosp.Electron. Syst., 2006, 42, (1), pp. 2–21

8 Geisheimer, J.L., Greneker, E.F., and Marshall, W.S.: ‘Ahigh-resolution Doppler model of human gait’, Proc. SPIE – RadarSensor Technol. Data Vis., 2002, 4744, pp. 8–18 (Komt de patentsearch voort van Kasper Haak uit 2006)

9 Thayaparan, T., Abrol, S., and Riseborough, E. (Defence Researchand Development Canada – Ottawa, Ottawa ‘Micro-doppler radarsignatures for intelligent target recognition’, Technical ReportDRDC Ottawa TM 2004-170 Report, September 2004)

10 Marple, L.: ‘Sharping techniques for sensor feature enhancement’. 26May 2005

143

11 Ling, H.: ‘Microdoppler exploitation: preliminary data collection andanalysis’. NICOP Meeting, London, 7 June 2004

12 Mitra, A.K., Kobold, M., Lewis, T., et al.: ‘Theoretical radar-Dopplermodels for pivoting mechanical and biological objects-of-interest’,Proc. SPIE – Algorithms Synth. Aperature Radar Imagery XIII,2006, 6237, pp. 25–32

13 Boulic, R., Thalmann, M.N., and Thalmann, D.: ‘A global humanwalking model with real time kinematic personification’, VisualComput., 1990, 6, pp. 344–356

14 Boulic, R., Ulicny, B., and Thalmann, D.: ‘Versatile walk engine’, J ofGame Develop., 2004, 1, (1), pp. 29–54

15 van Dorp, P., and Groen, F.C.A.: ‘Human walking estimation withradar’, IEE Proc., Radar Sonar Navig., 2003, 150, (5), pp. 356–365

16 van Dorp, P., and Groen, F.C.A.: ‘Real-time human walkingestimation with radar’. Proc. Int. Radar Symp. (IRS) 2003, Dresden,Germany, 2003

17 Yam, C.Y., Nixon, M.S., and Carter, J.N.: ‘Automated personrecognition by walking and running via model-based approaches’, J.Pattern Recognit. Soc., 2003, 35, (5), pp. 1057–1072

18 Ballreich, C.: ‘Vrml nancy’, Humanoid Animation Working Group,available at: http://www.h-anim.org, 1997

19 Skolnik, M.I.: ‘Radar handbook’ (McGraw Hill, 1990, 2nd edn.)20 Young, I.T., Gerbrands, J.J., and van Vliet, L.J.: ‘Fundamentals of

image processing’ (Delft University of Technology, he Netherlands,1998, version 2.2 edn.)

21 Mellinger, D.K., and Clark, C.H.: ‘Recognizing transientlow-frequency whale sounds by spectrogram correlation’, Acoust.Soc. Am., 2000, 107, (6), pp. 3518–3529

22 Skolnik, M.I.: ‘Introduction to radar systems’ (McGraw-Hill, 1980)23 Blackman, S., and Popoli, R.: ‘Design and analysis of modern tracking

systems’ (Artech House, 1999)

9 Appendix

9.1 Filtering

This appendix gives the essential parts of the extendedKalman feature filters [23].

9.1.1 Torso filtering: The torso feature vector zT is intro-duced in the main text. The torso state vector xT(Dt) is

xT(Dt) ¼

v(Dt)

_v(Dt)

~S(Dt)

w(Dt)

w(Dt)

vT(Dt)

0BBBBBBBB@

1CCCCCCCCA

¼

ST þ _STDt

_ST

~ST

f TDt þ wT

fT

STþ _S

TDt þ ~S

Tcos (2pf TDt þ wT)

0BBBBBBBBB@

1CCCCCCCCCA

(16)

The average speed v(Dt) modelled contains the torso speedST and a linear trend with the acceleration ST. The averageacceleration v(Dt) model is a constant acceleration S. Theaverage sinusoidal amplitude S(Dt) model is a constantamplitude ST. The average phase w(Dt) model containsthe torso phase wT and a linear trend with frequency fT.The average phase derivative w(Dt) model is the constanttorso frequency fT. The last equation vT(Dt) is the torsospeed equation which all features combined. The transition

144

matrix is

FT¼

1 Dt 0 0

0 1 0 0

0 0 1 0

0 0 0 1

0 0 0 0

1 Dt cos(wT(Dt)) � ~ST2p sin(wT(Dt))

0BBBBBBBB@

0 0

0 0

0 0

Dt 0

1 0

� ~ST2pDt sin(wT(Dt)) 0

1CCCCCCCCA

(17)

The measurement covariance matrix RT with average vel-ocity standard deviation sST

, acceleration standard devi-ation sST, sinusoidal amplitude standard deviation sST,phase standard deviation swT

and torso repetition frequencystandard deviation sfT is

RT¼

sST2 0 0 0 0

0 s_ST2 0 0

0 0 s~ST2 0 0

0 0 0 swT2 0

0 0 0 0 sfT2

0BBBBB@

1CCCCCA

(18)

The process noise covariance matrix with process noisestandard deviation sp is QT ¼ sp22I5,5. We assume equival-ent process noise standard deviation for each parameter.

9.2 Leg and step length filtering

Filtering of the leg parameters is equivalent with the torsoparameters. Here we combine the leg filtering and steplength filtering; the superscript Total indicates this combi-nation. We use zL, xL, FL, RL and QL in our notation.The measurement vector contains the leg parameters, rep-etition frequency and average velocity

zTotal ¼

zL

f A

V

0@

1A (19)

The state vector is

xTotal(Dt) ¼

xL(Dt)

f A(Dt)

V (Dt)

f (Dt)

L(Dt)

0BBBBB@

1CCCCCA

¼

xL(Dt)

f A

V

sfA

(Dt)2f L(Dt)þ sfL

(Dt)2f A(Dt)

sf A(Dt)2 þ sf L(Dt)2

V (Dt)

f (Dt)

0BBBBBBBBB@

1CCCCCCCCCA

(20)

The first part of the model is equivalent with the torso filter-ing. The average repetition frequency fA(Dt) model is the


constant repetition frequency fA. The average normalisedspeed V(Dt) model is the constant normalised speed V.The average cycle frequency f(Dt) model is a weightedleg frequency fL plus a weighted repetition frequency fA.The weights are calculated with their standard deviations.The average step length L(Dt) model is calculated withthe Boulic relation. Without fA and f, L ¼ V/fL remains.The transition matrix is

FTotal¼

FL

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0s f A(Dt)2

s f A(Dt)2 þ s f L(Dt)20

0 0 0 0 0 0

0BBBBBBBBB@

..

. ... ..

. ...

1 0 0 0

0 1 0 0

s fL

(Dt)2

s f A(Dt)2 þ s f L(Dt)20 0 0

01

f (Dt)�

V (Dt)

f 2(Dt)0

1CCCCCCCCCCCCA

(21)


The measurement covariance matrix RTotal with amplitudemodulation frequency standard deviation sfA and velocitystandard deviation sV is

RTotal¼

RL ..

. ...

. . . sfA2 0

. . . 0 sV2

0BB@

1CCA (22)

The process noise covariance matrix with process noisestandard deviations sp is QTotal ¼ sp2I10,10. We assumeequivalent process noise standard deviation for eachparamater.

145

Feature-based human motion parameter estimation with radar

Documents

Transcript of Feature-based human motion parameter estimation with radar