International Conference on Advances in Computing, Communication and Control (ICAC3’09)

Adaptive Feature Set Updating Algorithm for Multimodal Biometrics

H B Kekre Professor, Computer Department

MPSTM, NMIMS University, Mumbai - 400056. Tel: +91 - 9323557897

hbk@yahoo.com

V A Bharadi PhD. Research Scholar

MPSTM, NMIMS University, Mumbai - 400056. Tel: +91 - 9819125676

vinu_bharadi@rediffmail.com

ABSTRACT Multimodal Biometrics is an emerging domain in biometric technology where we combine more than one biometric trait to improve the performance. Here we propose a new concept that can be used in designing future multimodal biometrics systems which can adapt to the change in the biometrics features like face, voice, signature, and gait over the time or any other factor without compromising the security. We adapt to the change in biometric features by updating feature set without re-enrollment of entire set. This technique requires use of at least one biometric feature which has very low variance or high degree of permanence, like Fingerprint, Iris, Retina etc. This algorithm can address the problem of false rejection caused by sustained change in biometric features due to aging or any other factor without the need of re-enrollment of feature set. We suggest architecture of Multimodal biometric system implementing this algorithm.

Categories and Subject Descriptors I.4.7 Image Processing and Computer vision

General Terms Algorithms, Security, Verification.

Keywords Multimodal Biometrics, Fingerprint & Face recognition, Feature Fusion.

1. INTRODUCTION A biometric is a biological measurement of any human physiological or behavior characteristics that can be used to verify

the identity of an individual. A biometric-based authentication system operates in two modes: Enrollment and Authentication. In the enrollment mode, a user’s biometric data is acquired using a biometric read and stored in a database. The stored biometric template is labeled with a user identity to facilitate authentication. In the authentication mode, a user’s biometric data is once again acquired and the system uses this to either verify the claimed identity of the user or identify who the user is. While verification involves comparing the acquired biometric information with only those templates corresponding to the claimed identity,

identification involve comparing the acquired biometric information against templates corresponding to all users in the database [1].

Biometric authentication (or identification) systems, which use physical characteristics to check a person’s identity, ensure much greater security than password and number systems. Examples of biometrics include fingerprint, palmprint, face, iris, voice, signature, hand geometry, hand vein, retinal pattern, ear, facial thermograms, odor, gait and DNA. However, each biometric technology has its strengths and limitations, and no single biometric is expected to effectively satisfy the requirements of all verification or identification applications. A single biometric sometimes fails to be accurate enough for the identification of a large user population. Another disadvantage of using only one biometric is that the physical characteristics of a person for the selected biometric might not be always available or readable.

Biometric systems based on one (Unimodal) biometric are often not able to meet the desired performance requirements, and have to contend with a variety of problems such as noisy data, intra-class variations, restricted degree of freedom, non-university, spoof attacks and unacceptable error rates [1]. Some of these limitations can be addressed by deploying multimodal biometric systems that integrate the evidence presented by multiple sources of information.

Multimodal biometrics refers to the use of a combination of two or more biometric modalities in a verification/identification system. Identification based on multiple biometrics represents an emerging trend. The most compelling reason to combine different modalities is to improve the recognition rate. This can be done when biometric features of different biometrics are statistically independent. There are other reasons to combine two or more biometrics. One is that different biometric modalities might be more appropriate for the different applications. Another reason is simply customer preference.

1.1 Biometrics in AI & Machine Vision Person identification is one application area, another important use of biometrics is we can use it for making machines like robots that interact with human, identify a person accurately and make this identification mechanism similar to humans. Human beings can tolerate the change in external physical characteristics of a person when identifying, we can identify a person that we have not seen for years. Skilled person can identify gradual changes in handwritten signatures. These are some of the example where actually some of the biometrics features are changed slightly or at a larger extent, but still human can track that.

In biometric systems many times the feature set once enrolled is kept constant as it should because that reflects identity. Also the enrollment process is performed over a short duration of days or

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are

not made or distributed for profit or commercial advantage and that copies

bear this notice and the full citation on the first page. To copy otherwise,

or republish, to post on servers or to redistribute to lists, requires prior

specific permission and/or a fee.

ICAC3’09, January 23–24, 2009, Mumbai, Maharashtra, India.

Copyright 2009 ACM 978-1-60558-351-8…$5.00

277

International Conference on Advances in Computing, Communication and Control (ICAC3’09)

weeks. As discussed earlier the biometrics traits like face, signature, gait, ear geometry can change with time but fingerprints, iris, retina, DNA, remain constant. [1],[2]. We can use this factor for improvement in the FRR caused by change in the biometrics features.

We present here an algorithm that can be used in designing a multimodal biometric system that can adapt to the change in biometric features, like humans & without compromising accuracy. This algorithm is applicable for only multimodal biometric systems that employ more than one biometric feature set and at least one of them is having low variance over time. Like Fingerprint, Palmprint, Iris, Retina, etc.

1.2. Biometric System Performance In biometric matching studies, the performance of the system is given by the accuracy of the system. False Accept Rate (FAR) and False Reject Rate (FRR) are two widely used standard metrics of the accuracy of biometric systems [3],[4]. In practice, there are two additional important metrics: Failure to Enroll Rate (FTE) and Speed [5].

2. MULTIMODAL BIOMETRICS SYSTEMS The International Committee for Information Technology Standards (INCITS) Technical Committee M1, Biometrics, and researchers have described methods for performing multi-biometric fusion [3]. In general, the use of the terms multimodal or multi-biometric indicates the presence and use of more than one biometric aspect (modality, sensor, instance and/or algorithm) in some form of combined use for making a specific biometric verification/identification decision [3].

The goal of multi-biometrics is to reduce one or more of the following:

• False accept rate (FAR) • False reject rate (FRR) • Failure to enroll rate (FTE) • Susceptibility to artifacts or mimics

To further the understanding of the distinction among the multi-biometric categories [3],[4], they are briefly summarized in the following:

Multimodal biometric systems take input from single or multiple sensors measuring two or more different modalities of biometric characteristics. For example, a system combining face and iris characteristics for biometric recognition would be considered a “multimodal” system regardless of whether face and iris images were captured by different or same imaging devices. It is not required that the various measures be mathematically combined in anyway. For example, a system with fingerprint and face recognition would be considered “multimodal” even if the “OR” rule was being applied, allowing users to be verified using either of the modalities.

Multi-algorithmic biometric systems take a single sample from a single sensor and process that sample with two or more different algorithms. The technique could be applied to any modality. Maximum benefit would be derived from algorithms that are based on distinctly different and independent principles.

Multi-instance biometric systems use one sensor (or possibly multiple sensors) to capture samples of two or more different instances of the same biometric characteristics. For example, systems capturing images from multiple fingers are considered to be multi-instance rather than multimodal. However, systems capturing, for example, sequential frames of facial or iris images

are considered to be multi-presentation rather than multi-instance. This is whether or not the repeated captured images are combined at the image (feature) level, some other level of combination or a single image is selected as the one best used for pattern matching.

Multi-sensorial biometric systems sample the same instance of a biometric trait with two or more distinctly different sensors. Processing of the multiple samples can be done with one algorithm or some combination of multiple algorithms. For example, a face recognition application could use both a visible light camera and an infrared camera coupled with specific frequency (or several frequencies) of infrared illumination.

For a specific application in an operational environment, there are numerous system design considerations, and trade-offs that must be made among factors such as improved performance (e.g. verification or identification accuracy, system speed and throughput, robustness, and resource requirements), acceptability, circumvention, ease of use, operational cost, environment flexibility and population flexibility. Especially for a large-scale identification system, there are additional system design considerations such as operation and maintenance, reliability, system acquisition cost, life cycle cost and planned system response to identified susceptible means of attacks, all of which will affect the overall deployability of the system.

3. NEED FOR ADAPTIVE FEATURE SET Here we consider extended application of multimodal biometric systems, we can use this multimodal approach in machines having AI and interacting with humans. The foremost example is the ‘Asimov’ ROBOT developed by Honda Inc. We can implement the multimodal biometrics system for human identification. We also consider that such human machine interaction will be there for longer time (for years) over which some of the biometric features change gradually, this period and amount of variations is different for different feature and different individual. This algorithm divides the biometric feature sets in two categories as follows:

1. Features with low permanence over time: Signature, Voice, Gait, Facial features. 2. Features with high permanence over time: Fingerprints, Palmprint, Iris, Retina, (DNA). Though the biometric feature as listed above change, they are still

unique for the individual and still can be used for identification of the individual. We need to develop a mechanism that can handle

Figure 1. Effect of Aging on Facial Feature

278

International Conference on Advances in Computing, Communication and Control (ICAC3’09)

this. We must also consider the security aspects, as the forgers should no be allowed to change the feature set & accidental updating should be prohibited.

3.1 Change in Biometric feature due to aging As we grow up our physical appearance change as well change occurs in behavioral characteristics. We know that as defined previously biometrics is derived from measurable physical and behavioral characteristics, the feature set of biometrics changes with time. The features that undergo gradual changes are Facial features, Voice, Gait, and Signature. The features like fingerprint, Iris, Retina, DNA have high permanence and they do not change drastically.

We consider an example of facial feature. Fig. 1 shows example of such phenomenon. Changes in facial geometry of a person over a time span of around 20 year shown and we can see gradual changes occurring. This facial feature will be used as a measure to identify a person. Next we consider process of enrollment of facial feature, for this we consider some of standard database available over internet for research.

3.2 Enrollment of Facial Feature Set While enrolling a person to a Biometric system we capture require biometric feature set and extract desired features after processing of input data. This is applicable for all features.

3.1.1The AR Database The AR face [5], [6] contains over 4,000 color face images of 126 people (70menand 56 women), including frontal views of faces with different facial expressions, lighting conditions, and occlusions. The pictures of 120 individuals (65 men and 55 women) were taken in two sessions (separated by two Weeks) and each section contains 13 color images. Twenty face images (each session containing 10) of these 120 individuals are selected and used in our experiment. The face portion of each image is manually cropped and then normalized to 50 X 40 pixels. The sample images of one person are shown in Fig. 2. These images vary as follows:

Figure 2. Samples of the images in the AR database.

While enrolling samples are taken over a short time span of some days or weeks and they don’t consider variation in feature due to aging or any other factor. This is the case for all features. It is not feasible and impractical to take samples over years before implementation of a biometric system. Similar procedure is followed for FERT Database [7]. Fingerprint Verification

competition 2001, 2004 also used the database collected over a time span of few months (FVC 2001 & FVC 2004 fingerprint database).

Similar things can be observed for signature, voice, gait etc. As we see that these feature set is changing over years we must adapt to this gradual change in feature set if the operational time span of the system is going to be large. Otherwise re-enrollment of the feature set is required after certain time as the recent features will vary drastically from original feature set causing false rejection. Currently this aspect is not that serious as the biometrics systems are still quite new but they are surely spreading very rapidly.

4. CONSIDERATION FOR ADAPTIVE

FEATURE SET UPDATING This algorithm is mainly aimed at multimodal biometric system, based on more than one biometric feature. As the algorithm is adaptive, here we try to design a mechanism that will be able to track the change and update the feature set securely. For this we have to decide the authenticity of the feature set. We define following terms:

4.1. Independent Feature Set (SI) While developing a multimodal biometric system we have to include at least one feature which will remain constant over time. This feature set will be used to confirm the dependent feature set update procedure. Independent feature set includes Fingerprint, PalmPrint, Iris, and Retinal Scan.

4.2 Dependent Feature Set (SD) This is a feature set which changes over time [7] and need to be updated, this include voice, facial features, face, Signature (Static & Dynamic) , Gait etc.

4.3 Allowed Degree of Variance Allowed degree of variance in the biometric features for which updating is not needed, or the performance metrics are within required limits. This variance will be calculated based on a training mechanism as discussed in [8], which is based on Euclidian distance model.

4.4 Scale Invariant Feature Transform The SIFT features represent a compact representation of the local gray level structure, invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projections. SIFT has emerged as a very powerful image descriptor and its employment for face analysis and recognition was systematically investigated in [9] where the matching was performed using three techniques: (a) minimum pair distance, (b) matching eyes and mouth, and (c) matching on a regular grid. The present system considers spatial, orientation and keypoint descriptor information of each extracted SIFT point. Thus for example the input to the feature

extraction system is the face image and the output is the set of

extracted SIFT features s = (s1, S2.... . ... Sm) where each feature

point si=(x, y, 0, k) consist of the (x, y) spatial location, the local orientation 0 and k is the keydescriptor [9] of size 1x128.

Currently we use the SIFT for facial feature and use of this for other biometric features is being studied and is an open issue for research.

4.5 Consistent Fusion of Multi-biometric

Features

279

International Conference on Advances in Computing, Communication and Control (ICAC3’09)

Rather than performing fusion on sensor level or feature level we use here decision level fusion. We design independent classifier for each biometric attribute and fuse the decisions of the classifier. This allows us to update the feature set. We use the scheme as discussed in [10]. We use Fusion network in MOE (mixture-of-expert) architecture. Each vector sequence is compressed into a local score. The local scores are then fused by a gating network as shown in Fig. 3.

4.6 Interval of Update or Need for Update This is the time after which a biometric feature set is to be updated or a condition indicating need for update. The condition will be decided by classifier of the dependent feature. As shown in Fig. 3, we can see two classifiers, corresponding to an independent biometric feature and a dependent biometric feature. The update will be triggered in following conditions

Figure 3. Fusion network in a Mixture of Expert Architecture (MOE) [10].

4.6.1. Time limit is reached We set a time period for checking need for update. This time limit depends on the rate of change of dependent biometric trait. At this time we do not have any fixed number for but this can be decided by actual implementation, for a start we can set this to a time period of 1 Month, or even more than that. This is actually

depends on aging of human being and corresponding change in the biometric feature.

4.6.2. Increased FRR in dependent biometric trait

Consider case of facial features. This can be due to sudden change in facial geometry, arising from illness, facial makeup, surgery or any other factor. Voices tone my change due to illness or aging effect. This will lead to increased vector distance for feature set used while training and input vector may be rejected. This change is to be observed over time and then only update should be triggered. Intelligent module is required to detect this. Considering all these factors we now put a formal description of the steps in Adaptive Feature Set Algorithm.

5. ADAPTIVE FEATURE SET UPDATING

ALGORITHM This algorithm is working after the decision level of a biometric system; it takes input for different classifiers from the fusion network and monitors the results. The feature vectors are extracted using Scale Invariant Feature Transform (SIFT)( For facial feature) . We consider tow distinct set of feature vectors

(a) Set of dependent feature vectors SD = {sd1, sd2, sd3,….sdm}, where si is a feature point as discussed in [8]. With each feature vector we add a counter df corresponding to defaults, i.e. Number of failures while the independent feature vector gives a correct classification.

(b) Set of independent feature vectors SI = {si1, si2, si2…..sim}

For a dependent feature set we define tu as time interval for update in days.

We use a classifier based on a model as described in [8], which was initially used for classifying handwritten signatures. Here we use a variance of feature vector points and Euclidian distance of input test vector for the median feature vector point for classification. We calculate the variance (σi), threshold (thi) for distance of input feature vector SDi for the dependent biometric feature. The threshold (thi) is specific for each dependent biometric feature used. This is calculated for the feature vector set used for enrollment of a person and this indicates intra-class

variation of the feature. Any feature vector having distance from median feature point less than threshold is accepted and else rejected. This condition is monitored used for decision of update

One Classifier which belongs to independent feature plays an important role. Here we have multi-biometric system and fusion

algorithms working for combining the decisions from these classifier are used for authenticity of user and if the classifier

Figure 4. Multimodal Biometric System using Adaptive Feature Vector Update mechanism.

280

International Conference on Advances in Computing, Communication and Control (ICAC3’09)

giver input feature vector SI as correct then only the dependent feature vector SD can be updated if update condition is triggered. Here we consider a case of multimodal biometrics system having two classifiers where one belongs to Fingerprint Verification (Operating on SI) and other belongs to Facial Feature Classifier (Operating on SD).

5.1 AFVU Algorithm: 1. Reset the time interval ti=0, default counter for dependent

feature df=0; 2. Start the multimodal biometric system. Read the input test

vector ST={SDT,SIT} for user Un Where ST= Test feature vector set

SDT = Test Dependent Feature vector (Dependent Feature vector Subset) SIT = Test Independent Feature vector (Independent Feature vector Subset) 3. If Adaptive Gating Network Classifier REJECTS the test

vector ST{SIT+SDT} then go to step 4. Else go to step 2 (Next Test).

4. If Independent Feature Classifier REJECTS the test feature vector (SIT) then go to Step 2 else go to Step 5. (REJECTED TEST VECTOR). NO NEED TO UPDATE.

5. If Dependent Feature Vector Classifier REJECTS the feature vector ST{SDT} (While dependent feature vector is ACCEPTED)

6. Mark as default condition df=df+1 for user Un. 7. If default count df> permitted defaults (DP) for user Un ,

INVOKE Feature vector UPDATE MODULE. Reset Default counter df =0, for user Un. Go to step 2.

8. If time interval to update ti > permitted limit (TP) of user Un then INVOKE the Feature Vector UPDATE MODULE. Reset ti=0,df=0 for user Un. Go to step 2.

5.2 Feature Vector Update Module This is very important module, this works on the same lines as the enrollment module for each of the classifier. We operate the module on the FIFO basis, oldest feature vector is deleted and the latest Feature vector will be added. The new feature vector will be added to the set and we calculate the variance (σi), threshold (thi) for distance of input feature vector SDi for the dependent biometric feature with the included feature vector. This forms the updated feature vector set (SDi) for user Un.

The steps are as follows. 1. Prompt user Un for UPDATE of Feature Vector stating the

reason 2. If allowed by user, read again the feature vector for

corresponding dependent feature vector. Find SIFT [10] as discussed earlier. Tag the feature vector with current date & time

3. Delete the oldest feature vector from the set SD = {Sd1, sd2….Sdn}.

4. Using the training mechanism in [8] find out variance (σi), threshold (thi) for the updated set, Update the corresponding record.

5. Prompt the user of the Completion of Update Process.

5.3 Security in Updating Process As the feature vector update is triggered only when the Independent Feature Vector successfully ACCEPTED, this adds security against false updating of feature set. We take fingerprint, Palmprint, Retina, and Iris as Independent feature vector, there biometrics features have high CCR (Correct Classification Rate) and very difficult to forge as compared to the dependent features. As no other entry point for triggering the updating (INVOKE UPDATE MODULE Event) process, we can say that the update process is secure.

6. DISCUSSION & FUTURE WORK The Adaptive feature vector algorithm is proposed for development of complex multimodal biometrics systems with large operating time span. The authors are in process of development of classifier and multimodal biometric system. At this moment we present the algorithm conceptually. This algorithm is very crucial as we will have more and more multimodal biometric systems coming in human interaction. Fig.4. Shows proposed multimodal biometric system with Adaptive Feature Vector Update algorithm.

Currently unimodal biometric systems are ubiquitous and multimodal biometric systems are having low penetration owing to high cost and complex structure [11]. But where we need high accuracy and sustainability, this algorithm can surely help to improve performance of the system. The application of this algorithm is in all multimodal biometric systems, with very large time span for operation over a specific population like Schools, Offices, Hospitals, Military, Research Laboratory Access control equipments. Another application is development of Machines with AI and high degree of human interaction.

Implementation of this algorithm does not require change in any of the hardware (Sensors). Change in coding architecture is required and this can be integrated with existing work. Current multimodal biometric system designers should consider this algorithm for implementation. This algorithm is intended for improving performance of multimodal biometric system and impart adaptability bye making these systems aware of gradual changes in human biometric traits over the time. Performance of such system should be tested for longer time over a real time application scenario.

This algorithm is not final and may need revisions as many of the parameters are still unknown, like permitted number of defaults (DP) and time interval to update (TP), this need further research of variations in biometric features. Better feature vector extraction, training & enrollment mechanisms can be combined with this algorithm for improving performance.

7. REFERENCES [1] A. Ross and A.K. Jain, “Information Fusion in Biometrics”,

Pattern Recognition Letter 24, pp.2115- 2125, 2003.

[2] A. Jain, R. Bolle, and S. Pankanti, editors, Biometrics:

Personal Identification in Networked Society, Kluwer Academic Publishers, 1999.

[3] Teddy Ko , “Multimodal Biometric Identification for Large User Population Using Fingerprint, Face and Iris Recognition” , Proceedings of the 34th Applied Imagery and Pattern Recognition Workshop (AIPR05) ,2005.

[4] “Summary of NIST Standards for Biometric Accuracy, Tamper Resistance, and Interoperability,” November 13, 2002

281

International Conference on Advances in Computing, Communication and Control (ICAC3’09)

[5] A.M. Martinez and R. Benavente, “The AR Face Database,” http://rvl1.ecn.purdue.edu/aleix/~aleix_face_DB.html, 2006.

[6] A.M. Martinez and R. Benavente, “The AR Face Database,” CVC Technical Report #24, June 1998.

[7] P.J. Phillips, “The Facial Recognition Technology (FERET) Database,” http://www.itl.nist.gov/iad/humanid/feret/feret master.html, 2006.

[8] B. Majhi, Y S Reddy, D Prasanna Babu, “Novel Features for Off-line Signature Verification”, International Journal of Computers, Communications & Control Vol. I, 2006

[9] M. Bicego, A. Lagorio, E. Grosso and M. Tistarelli, "On the use of SIFT features for face authentication", Proceedings of CVPR Workshop, New York, 2006.

[10] S. Y. Kung, Man-Wai Mak, “On Consistent Fusion Of Multimodal Biometrics”, ICASSP 2006, June 2006.

[11] A. Ross, K. Nandakumar and A. Jain, “Handbook of Multibiometrics”, Springer, ISBN - 978-0- 387-22296-7, 2006.

282