Active shape model-based user identification for an intelligent wheelchair. P. Jia* and H. Hu. 1 Introduction

Similar documents
Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

Cluster Analysis of Electrical Behavior

A Binarization Algorithm specialized on Document Images and Photos

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

PCA Based Gait Segmentation

Problem Definitions and Evaluation Criteria for Computational Expensive Optimization

Detection of an Object by using Principal Component Analysis

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

Feature Reduction and Selection

An Optimal Algorithm for Prufer Codes *

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

A B-Snake Model Using Statistical and Geometric Information - Applications to Medical Images

Face Recognition Based on SVM and 2DPCA

A high precision collaborative vision measurement of gear chamfering profile

Lecture 5: Multilayer Perceptrons

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

User Authentication Based On Behavioral Mouse Dynamics Biometrics

An Image Fusion Approach Based on Segmentation Region

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1

Recognizing Faces. Outline

TN348: Openlab Module - Colocalization

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:

The Research of Ellipse Parameter Fitting Algorithm of Ultrasonic Imaging Logging in the Casing Hole

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance

FEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur

Video Object Tracking Based On Extended Active Shape Models With Color Information

PRÉSENTATIONS DE PROJETS

Fast Feature Value Searching for Face Detection

Shape Representation Robust to the Sketching Order Using Distance Map and Direction Histogram

EYE CENTER LOCALIZATION ON A FACIAL IMAGE BASED ON MULTI-BLOCK LOCAL BINARY PATTERNS

Support Vector Machines

Corner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity

On Modeling Variations For Face Authentication

CS 534: Computer Vision Model Fitting

Robust Face Alignment for Illumination and Pose Invariant Face Recognition

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach

Determining the Optimal Bandwidth Based on Multi-criterion Fusion

Load Balancing for Hex-Cell Interconnection Network

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Available online at Available online at Advanced in Control Engineering and Information Science

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics

Fitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.

Learning Ensemble of Local PDM-based Regressions. Yen Le Computational Biomedicine Lab Advisor: Prof. Ioannis A. Kakadiaris

Development of an Active Shape Model. Using the Discrete Cosine Transform

Learning a Class-Specific Dictionary for Facial Expression Recognition

R s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes

Smoothing Spline ANOVA for variable screening

TECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS. Muradaliyev A.Z.

The motion simulation of three-dof parallel manipulator based on VBAI and MATLAB Zhuo Zhen, Chaoying Liu* and Xueling Song

3D Virtual Eyeglass Frames Modeling from Multiple Camera Image Data Based on the GFFD Deformation Method

Robust visual tracking based on Informative random fern

S1 Note. Basis functions.

A New Approach For the Ranking of Fuzzy Sets With Different Heights

An Entropy-Based Approach to Integrated Information Needs Assessment

Reducing Frame Rate for Object Tracking

Classifying Acoustic Transient Signals Using Artificial Intelligence

Face Recognition by Fusing Binary Edge Feature and Second-order Mutual Information

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems

Two-Dimensional Supervised Discriminant Projection Method For Feature Extraction

The Shortest Path of Touring Lines given in the Plane

Quality Improvement Algorithm for Tetrahedral Mesh Based on Optimal Delaunay Triangulation

The Discriminate Analysis and Dimension Reduction Methods of High Dimension

IP Camera Configuration Software Instruction Manual

Development of Face Tracking and Recognition Algorithm for DVR (Digital Video Recorder)

A mathematical programming approach to the analysis, design and scheduling of offshore oilfields

Lobachevsky State University of Nizhni Novgorod. Polyhedron. Quick Start Guide

Edge Detection in Noisy Images Using the Support Vector Machines

Reading. 14. Subdivision curves. Recommended:

Modular PCA Face Recognition Based on Weighted Average

CMPS 10 Introduction to Computer Science Lecture Notes

Classifier Selection Based on Data Complexity Measures *

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters

Face Recognition Based on Neuro-Fuzzy System

Using Fuzzy Logic to Enhance the Large Size Remote Sensing Images

Robust Face Recognition through Local Graph Matching

Accounting for the Use of Different Length Scale Factors in x, y and z Directions

Collaboratively Regularized Nearest Points for Set Based Recognition

Invariant Shape Object Recognition Using B-Spline, Cardinal Spline, and Genetic Algorithm

Appearance-based Statistical Methods for Face Recognition

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices

Related-Mode Attacks on CTR Encryption Mode

3D vector computer graphics

Vectorization of Image Outlines Using Rational Spline and Genetic Algorithm

Palmprint Feature Extraction Using 2-D Gabor Filters

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Steps for Computing the Dissimilarity, Entropy, Herfindahl-Hirschman and. Accessibility (Gravity with Competition) Indices

Decision Strategies for Rating Objects in Knowledge-Shared Research Networks

Type-2 Fuzzy Non-uniform Rational B-spline Model with Type-2 Fuzzy Data

Vol. 5, No. 3 March 2014 ISSN Journal of Emerging Trends in Computing and Information Sciences CIS Journal. All rights reserved.

Weed Classification by Active Shape Models

Machine Learning 9. week

Hermite Splines in Lie Groups as Products of Geodesics

Incremental MQDF Learning for Writer Adaptive Handwriting Recognition 1

A Fast Content-Based Multimedia Retrieval Technique Using Compressed Data

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task

LOOP ANALYSIS. The second systematic technique to determine all currents and voltages in a circuit

Human Face Recognition Using Generalized. Kernel Fisher Discriminant

Transcription:

Int. J. Advanced Mechatronc Systems, Vol. X, No. Y, 200X 1 Actve shape model-based user dentfcaton for an ntellgent wheelchar P. Ja* and H. Hu School of Computer Scence and Electronc Engneerng, Unversty of Essex, Colchester, CO4 3SQ, UK E-mal: jp4work@gmal.com E-mal: hhu@essex.ac.uk *Correspondng author Abstract: Recently, some novel human-machne nterfaces (HMI) have been created for dsabled and elderly people to control ntellgent wheelchars (IW) usng facal and head gestures. o operate a wheelchar n ths new vsual-based control mode, user dentfcaton should be conducted beforehand. Rather than tradtonal user dentfcaton that requres the user to nput hs/her username and password by typng, the state-of-the-art bometrc-based user dentfcaton provdes a more convenent way for the dsabled users. hs paper frst elaborates actve shape model n detals; then, vdeo-based user dentfcaton usng Mahalanobs dstance s presented. As an extenson, an adaptve learnng module s desgned to append or update the user s face record n the constructed face database. Expermental results show that our logn subsystem s able to functon well for a comparatvely small face database. Keywords: ntellgent wheelchar; IW; user dentfcaton; human-machne nterface; HMI; actve shape model; ASM. Reference to ths paper should be made as follows: Ja, P. and Hu, H. (xxxx) Actve shape model-based user dentfcaton for an ntellgent wheelchar, Int. J. Advanced Mechatronc Systems, Vol. X, No. Y, pp.000 000. Bographcal notes: Pe Ja receved hs BEng from Wuhan echncal Unversty of Surveyng and Mappng (now part of Wuhan Unversty), Chna n 1999 and MEng n Huazhong Unversty of Scence and echnology, Chna n 2003. He s currently a PhD student n School of Computer Scence and Electronc Engneerng, Unversty of Essex, UK. Hs research nterests nclude mage processng and mage analyss, pattern recognton, computer vson, robotcs, etc. Huosheng Hu receved hs MSc from Central South Unversty n Chna and hs PhD from Oxford Unversty n the UK. He s a Professor n School of Computer Scence and Electronc Engneerng at the Unversty of Essex, UK, leadng the Human Centred Robotcs Group. Hs research nterests nclude bologcally nspred robotcs, servce robots, human-robot nteracton, evolutonary robotcs, sensor ntegraton, data fuson, artfcal ntellgence, embedded systems, mechatroncs and pervasve computng. He has publshed over 280 papers n journals, books and conference proceedngs n these areas. He s one of Foundng Members of IEEE Robotcs & Automaton Socety echncal Commttee on Networked Robots, a fellow of IE and a Senor Member of IEEE and ACM. He has been a Char or Commttee Member for many nternatonal conferences such as IEEE ICMA, IEEE ROBIO, IEEE IROS, RoboCup Symposums and IASED RA, CA and CI conferences. He currently serves as one of Edtors-n-Chef for Internatonal Journal of Automaton and Computng.. 1 Introducton Advanced mechatronc systems and ntellgent robots have played a key role to mprove the lfe qualty of the dsabled and elderly people. In many practcal applcatons of wheelchars, the severely dsabled users can not use ther hands freely and may show clear ntenton by ther heads nstead. herefore, head gestures have been used as a novel vsual human-machne nterface (HMI) n our prevous work (Ja et al., 2007), n whch four face drectons (left, rght, up and down) are recognsed to control the wheelchar n a swtchng mode. Recently, more flexble head gesture-based HMI usng 2D actve appearance model (AAM) was presented n Ja and Hu (2007), whch s able to tell face drectons by the ftted mesh detals and control the wheelchar more smoothly. Systematcally, before startng controllng the wheelchar usng hs/her head gestures, a user needs to be authentcated n advance n order to ensure the system securty. It has been notced for several decades that human bologcal nformaton can be appled to verfy dfferent users and varous knds of bometrc methods for user dentfcaton have been developed. Jan et al. (2004) summarses and compares the exstng popular bometrc Copyrght 200X Inderscence Enterprses Ltd.

2 P. Ja and H. Hu technologes, whch s modfed and publshed onlne (Wkpeda). Snce 2D statstcal face models are adopted n Essex IW to control the wheelchar, t s natural for us to adopt the same model to dentfy the users n the bometrcal manner. herefore, among the exstng bometrc technologes, face recognton s our concern n ths paper. In the last two decades, lots of face recognton methods have been proposed, among whch, egenface has been pad tremendous attenton due to ts unversalty and favourable performance. Egenface was frst proposed by urk and Pentland (1991a, 1991b), whch s based on canoncal prncpal component analyss (PCA). Derved from PCA-based egenface, ndependent component analyss (ICA)-based egenfaces (Bartlett et al., 2002) and lnear dscrmnant analyss (LDA)-based egenfaces (Etemad and Chellappa, 1997) are also proposed for face recognton. All the above mentoned methods based on egenface could be extended to non-lnear kernel methods (Yang, 2002). Moreover, elastc bunch graph matchng (EBGM) was proposed to represent faces n topologcal graphs (Wskott et al., 1997). By defnng a jet as a seres of Gabor coeffcents n dfferent scales and orentatons at one node,.e., graph ntersectons, the face s recognsed usng all jets. In addton, as one of the current hottest topcs n computer vson, actve shape model (ASM) (Cootes and aylor, 1992) successfully constructs face shape statstcs. After ASM fttng, the ftted pont coordnates can be used to recognse dfferent persons. Unlke ASM that seeks to match only the postons of the model ponts, AAM tres to match both shape and texture representatons of an object smultaneously (Edwards et al., 1998; Cootes and aylor, 2004; Matthews and Baker, 2004). As extensons, several 3D statstcal models have been developed and outperformed 2D statstcal models for face recognton. Blanz and Vetter (2003) proposed 3D morphable models (MM), n whch up to eght ponts on a 2D face are manually labelled, upon whch the correspondng 3D face can be synthessed for later recognton. Furthermore, Park and Jan (2007) concluded that vew synthess method s more appealng than the vew-based method and declares 40% mprovement n matchng performance after synthessng 3D faces by 3D AAM. In Cootes and aylor (2004), t was concluded that ASM outperforms basc AAM n terms of both accuracy and effcency. In ths paper, ASM s revsted n detal. Shape and texture model parameters from mage sequences are used to dentfy dfferent users. In addton, an adaptve learnng module s proposed to keep the user s face record n the face database up to date. Our demonstraton shows that the desgned logn subsystem works well for a comparatvely small face database. he rest of the paper s organsed as follows. Secton 2 ntroduces the system archtecture used n our ntellgent wheelchar (IW), as well as the user dentfcaton subsystem. In Secton 3, 2D statstcal models are brefly explaned, ncludng shape and texture models. Secton 4 descrbes ASMs and the canoncal fttng algorthm usng 1D profle normal of the shape. In Secton 5, vdeo-based user dentfcaton usng Mahalannobs dstance wth the adaptve learnng module s addressed. Expermental results are gven n Secton 6 to show the performance of the proposed algorthm. Fnally, a bref concluson and future work are outlned n Secton 7. 2 User dentfcaton for Essex IW Fgure 1 shows the pcture of an IW at Essex, whch has the followng major components: sx ultrasonc sensors at a heght of about 50 centmetres used for obstacle avodance (four at the front and two at the back) a joystck used to control the wheelchar manually f necessary a Logtech 4000 Pro Webcam used for recognsng the head gesture of the user an Intel Pentum-M 1.6G Centrno laptop wth Wndows XP Operatng System nstalled to analyse the head gesture a DSP MS320LF2407 mcroprocessor to control two dfferentally-drven wheels. Fgure 1 Photo of an IW at Essex (see onlne verson for colours) Instead of the tradtonal way to logn a system by usng a keypad, we are developng a vson-based user dentfcaton module to be ntegrated nto ths IW, as shown n Fgure 2. Essex IW wll be able to dentfy the users by face bometrcs. he same dea has already been appled n a very successful and popular product,.e., IBM hnkpad notebook whose logn subsystem s based on fnger prnt

Actve shape model-based user dentfcaton for an ntellgent wheelchar 3 recognton. Fgure 3 shows the framework of our face recognton-based logn subsystem. Super users can only logn IW usng the account name and correspondng password and only admnstrators may know the super users names and passwords. Ordnary users may logn IW by ther faces or by the tradtonal typng way, whch means there exsts a more convenent alternatve for them f they prefer logn wthout typng. Fgure 2 Essex IW control archtecture (see onlne verson for colours) s = [ x, y, x, y,, x, y ] (1) 1 1 2 2 v v he shape statstcal model allows lnear shape varaton. hs means an arbtrary shape s s consdered as a base shape s 0 plus a lnear combnaton of N shape vectors s : N 0 p 0 = 1 s= s + s = s + Pp (2) s where p are the shape parameters and s are orthonormal egenvectors. An arbtrary shape s mght have varous trangulaton structures due to dfferent rules. Here n our applcaton, Delaunay trangulaton (Wkpeda) s mplemented on the mean shape s 0 to defne the shape s nner trangulaton structure, whch s absolutely exstng and unque. Fgure 4 Mean shape for IMM and XM2VS (a) IMM convex hull (b) IMM trangulaton (c) XM2VS concave part (d) XM2VS concave hull (e) XM2VS trangulaton (a) (b) Fgure 3 Logn subsystem (c) (d) 3 2D statstcal models 3.1 Shape model he statstcal model for a shape s s defned by the coordnates ( xy, ) of v manually labelled ponts, whch actually compose a trangle mesh. (e) Fgures 4(a) and 4(b) generate the convex hull and Delaunay trangulaton of the mean shape computed from IMM face database (Nordstrøm et al., 2004). However, t s not guaranteed that the tranng face database wll always show a convex hull, rather than a concave hull, such as Surrey XM2VS database (Messer et al., 2004). Fgures 4(c), 4(d) and 4(e) show ts concave part, concave hull and

4 P. Ja and H. Hu Delaunay trangulaton respectvely. For dfferent face databases, the numbers of labelled ponts are dfferent. here are 58 and 68 annotated ponts for IMM and XM2VS respectvely. 3.2 exture model he statstcal model for a texture s defned n Stegmann (2000) as the pxel ntenstes across the object (here, the face) n queston (f necessary after a sutable normalsaton). hen, the texture statstcal model s a vector of coordnate-related ntenstes A ( x) defned over all u pxels nsde the base mesh s 0,.e., x s 0, where x = ( xy, ). Equaton (3) shows a texture vector when RGB channels or a sngle channel s consdered respectvely: 1 the segment connectng ths pont and the pont t connected from 2 the segment connectng ths pont and the pont t connected to. he normal drecton to the profle s then defned by normalsng the sum of the two unt normal vectors of these two lne segments. Fgure 6 shows all the profle normals at the manually labelled 87 ponts of one sample face from our self-labelled face database. Fgure 6 Profle normal vector A( x) = [ b, g, r, b, g, r,, b, g, r ] 1 1 1 2 2 2 A( x) = [ gray, gray,, gray ] 1 2 u u u u (3) he texture statstcal model allows lnear texture varaton. hs means an arbtrary texture A ( x) s looked on as a base texture A 0 ( x) plus a lnear combnaton of M textures A ( x): M A( x) = A + A ( x) = A ( x) + P, x s λ λ (4) 0 0 t 0 = 1 where λ are the texture parameters and A ( x) are orthonormal egenvectors. Fgure 5 shows the mean face textures synthessed from IMM and XM2VS face databases. For IMM, three channels are used; for XM2VS, the texture s synthessed from grey-level mages. Fgure 5 Mean texture (a) IMM (b) XM2VS (see onlne verson for colours) Along the normal drecton, at each sde of the model pont, k pxels dervatves g (rather than the orgnal grey-level values) are sampled and then normalsed as the feature set for ths specfc key pont. 1 g g k (5) = j k g j For every gven model pont of each tranng mage, a normalsed unt vector of dervatves can be extracted. Assumng a multvarate Gaussan, the qualty of fttng a new face to the model at ths specfc pont could be evaluated by Mahalanobs dstance: 1 new new g new f ( g ) = ( g g) S ( g g ) (6) 1 where g and S g are respectvely the traned mean vector and the nverse of covarance matrx n terms of ths specfc pont. (a) (b) 4 Actve shape model 4.1 Canoncal 1D profle model of ASM Canoncal ASM fttng algorthm computes the normal to the profle at each model pont. At every gven pont, the tangental lne could be defned by two lne segments: 4.2 ASM fttng algorthm An ntutve dea to fnd those key ponts on an arbtrary face based on ASM 1D profle model could be summarsed as: 1 Approxmately locate the face by Adaboost face detecton (Vola and Jones, 2001, 2004) frst.

Actve shape model-based user dentfcaton for an ntellgent wheelchar 5 2 At each sde of the model pont, along ts profle normal, m pxels dervatves m > k are sampled. We then calculate the ft qualty at each of the 2( m k ) + 1 possble slots along the normal drecton and choose the best match as the updated poston for ths model pont. hs process repeats for every model pont and gves out a modfcaton on t for the next teraton. 3 After all ponts have been updated, constran the shape by shape model parameters. Apparently, the current constructed shape s correspondng to a vector of shape model parameters p,0 < N. Wth the bult statstcal shape model, each of above model parameters s restrcted wthn 3σ around ts mean value. 4 Repeat 2 and 3 untl convergence. o speed up the search process and mprove the robustness of ths algorthm, multscale ASM s adopted n our experments. hs nvolves searchng for the face n a coarse mage frst and then refnng the face key ponts locatons n a seres of fner resoluton mages. 5 User dentfcaton ASM can ft a face accurately and generate a trangulaton mesh, through whch the face texture could also be extracted. Edwards et al. (1998) successfully carred out face recognton through the shape and texture model parameters by calculatng Mahalanobs dstance. For Essex IW, the vdeo-based user dentfcaton algorthm usng Mahalanobs dstance s revsted and an adaptve learnng module s desgned to update the user face record n the constructed face database. 5.1 Constructon of pror user face database he user face database s constructed wth the pror knowledge that we can obtan from all avalable users, ncludng the usernames, passwords and the face bometrc data here, 2D shape and texture statstcs. For each user, N frames of frontal faces are automatcally selected from an mage sequence to compute hs/her personal face statstc nformaton. For each frame n the mage sequence, ASM 1D profle model s appled to calculate the best ftted face shape frst. After two PCA transforms on the ftted shape and the correspondng extracted texture, the statstcal shape and texture parameters can be obtaned for ths sngle mage frame. Mean shape and texture parameters and the covarance matrxes can be calculated over all N mages. In summary, each user record n the face database can be constructed n the followng steps. 1 Start mage capturng and track the user s face by ASM 1D profle model. 2 If the face s well ftted (durng database constructon process, ths could be judged by the user hmself/herself), hold the same posture wth lmted movements for around one second (In our experments, N = 16. If the webcam captures 30 frames per second, the tme of capturng 16 mages wll be 16/30 = 0.53 second. Consderng the fttng tme for our algorthm, the tme of fttng 16 mages wll be at least one second). 3 Once the button add a user s actvated, N dynamc (real-tme updatng) mages are avalable to construct the user s face record. 4 Wth N ftted shapes and extracted textures, N pars of shape parameters and texture parameters can be calculated and respectvely denoted as p,0 < N and λ,0 < N. 5 hen, the mean shape parameters and texture parameters for ths person and the respectve covarance matrxes are computed as: =1 [ ] = N p E p (7) N =1 [ ] = N λ E λ (8) N Σ p = E ( [ ])( [ ]) p E p p E p Σ λ = E ( [ ])( [ ]) λ E λ λ E λ (10) 6 Store all the above statstcal data calculated n Step 5 nto the database as a record feld of the user s face. It s possble to add as many face records as possble nto the face database. However, one IW wll be normally used by a sngle person. Besdes, n order to ncrease the user dentfcaton accuracy, t s better to restrct the number of subjects n the face database. Here, three users, ncludng one female and two males partcpated n our experments. 5.2 Vdeo-based user dentfcaton usng Mahalanobs dstance After the face database s constructed, t s the tme to verfy whether the IW face dentfcaton subsystem s applcable. It s expected that the logn face could be matched to the correspondng user s record n the face database. By lookng on the user dentfcaton problem as a classfcaton one, canoncal Mahalanobs dstance s able to be used to classfy dfferent users. heoretcally, to classfy a testng sample to the class wth the mnmal Mahalanobs dstance s equvalent to selectng the class wth the hghest probablty (Wkpeda). Meanwhle, Mahalanobs dstance could favourably deal wth outlers (Wkpeda) f the testng sample does not obey the dstrbuton, whch means, t can handle the belong to none stuaton. (9)

6 P. Ja and H. Hu Algorthm 1 Votng based on Mahalanobs dstance 1 Pre-computaton durng tranng Wth the tranng N frames for each person, calculate the mean shape and texture parameters and respectve covarance matrxes for all C avalable users and store them n database: Ej[ p], E j[ λ], Σj[ p]and Σ j[ λ], where 0 j < C 2 Intalsaton WeakFttng = false PGood = 0.75 3 for j = 0 to C do 4 Votes [] j = 0 5 end for 6 for = 0 to N do 7 Obtan current ftted face shape parameters p and texture parameters λ 8 for j = 0 to C do 9 Calculate the Mahalanobs dstances sdst[ j ] from the current ftted face shape parameters p to the stored shape record of person j n the face database j j 1 j sdst[ j] = ( p E [ p]) Σ [ p] ( p E [ p]) 10 end for 11 Pck up the record ndex k wth the shortest Mahalanobs dstance n terms of shape sdst[ k ], where 0 k < C 12 f sdst[ k ] s not outler then 13 f (!WeakFttng) then 14 Calculate the Mahalanobs dstances tdst from the current ftted face texture parameters λ to the stored texture records of person k n the face database k k 1 k tdst = ( λ E [ λ]) Σ [ λ] ( λ E [ λ]) 15 f tdst s not outler then 16 Votes [ k ] ++ ; 17 end f 18 else 19 Votes [ k ] ++ ; 20 end f 21 end f 22 end for 23 Pck up the record ndex j wth the most votes, that s, the bggest value of Votes[ j ], where 0 j < C 24 f ( Votes[ j] PGood N ) then 25 return j ; 26 else 27 return 1; 28 end f Durng user logn procedure, N pars of shape and texture parameters for N qualfed mages are calculated and respectvely denoted as p,0 < N and λ,0 < N. N Mahalanobs dstances n terms of the shape and texture parameters are then calculated as: 1 M Σ D ( p ) = ( p E[ p]) [ p] ( p E [ p]) (11) 1 M Σ D ( λ ) = ( λ E[ λ]) [ λ] ( λ E [ λ]) (12) For smplcty, a votng strategy for the above mage sequence s adopted to dentfy the users as shown n Algorthm 1. 5.3 Adaptve learnng for user records updatng Every tme when a user succeeds n the system logn based on hs or her face, addtonal bometrc knowledge s deployed at the same tme. In order to keep the user s face statstcs up to date, the newly nput knowledge should be added nto the face database whenever a user succeeds n logn. Suppose that the newly collected statstcs from N qualfed frames n an mage sequence are denoted as E [ p], E [ λ], Σ [ p], Σ [ λ], then, the updated statstcs n n n n E [ p], E [ λ], Σ [ p], Σ [ λ] could be calculated as follows: u u u u E[ p] + En[ p] E u[ p] = (13) 2 E[ λ] + En[ λ] E u[ λ] = (14) 2 Σ[ p] + Σn[ p] + Σm[ p] + Σnm[ p] Σ u[ p] = (15) 2 Σ[ λ] + Σn[ λ] + Σm[ λ] + Σnm[ λ] Σu [ λ ] = (16) 2 where Σ = ( [ ] [ ])( [ ] [ ]) m p E E p Eu p E p E u p (17) Σ = ( [ ] [ ])( [ ] [ ]) nm p E En p Eu p En p E u p (18) Σ = ( [ ] [ ])( [ ] [ ]) m λ E E λ Eu λ E λ E u λ (19) Σ = ( [ ] [ ])( [ ] [ ]) nm λ E En λ Eu λ En λ E u λ (20) 6 Expermental results 6.1 Fttng results for sngle frontal mages We frst conduct the ASM fttng experments on all mages of the two mentoned face databases IMM and XM2VS. In Fgure 7, the frst row contans ASM fttng results for three mage samples from IMM database; the second row shows ASM fttng results for two mages from XM2VS database. From the samples here, we can see that ASM can

Actve shape model-based user dentfcaton for an ntellgent wheelchar 7 ft female and male faces from varous races, even can deal wth faces wth glasses or beard to some extend. o evaluate the fttng results numercally, two crtera are establshed as: obtan a real-tme trackng, more than 20 FPS usng the ASM fttng algorthm. In ths experment, a self-labelled face database wth 87 annotated ponts for each face s traned and adopted. 1 f more than 80% of the ftted ponts are wthn a coordnate tolerance t to the manually labelled ponts n one mage, ths face s well ftted 2 f crteron 1 s not satsfed, but the total devaton for all the key ponts s less than ( t + 1)/ N, ths face s well ftted. Fgure 8 Face trackng sequence ftted by ASM (see onlne verson for colours) Fgure 7 ASM fttng effects (see onlne verson for colours) Fgure 9 User face dentfcaton, (a) logn dalog (b) status bar of IW applcaton dalog wth user s account (see onlne verson for colours) We can see from able 1 that no matter whether evaluated from numercal aspect or evaluated by human eyes, ASM fttng results are reasonably well. Full fttng results can be found from the personal webste of the frst author at http://www.vsonopen.com/cv/aam.php able 1 Fttng results Database IMM XM2VS Number of mages 37* 2,360 ASM Numercal human 94.59% 85.89% Eye 100% 88.81% Note: *A subset of all 37 frontal faces from IMM s tested here, rather than the full set of 240 frontal and profle faces. (a) 6.2 Fttng results for an mage sequence Sx mages out of an mage sequence ftted by ASM are gven n Fgure 8, whch demonstrates that ASM s good at trackng faces and tellng face detals n real-tme. We dd (b)

8 P. Ja and H. Hu 6.3 GUI applcaton for user face dentfcaton he experment n Fgure 9 shows that the proposed user dentfcaton method works reasonably well by a self-desgned GUI applcaton for the IW logn subsystem at Essex. 7 Conclusons and future work radtonal electrc-powered wheelchars are normally controlled by users va joystcks, whch cannot satsfy the needs of elderly and dsabled users who have restrcted lmb movements caused by dseases such as Parkngson and quadrplegcs. herefore, we have recently created some novel human-machne nterfaces for hands-free control of our ntellgent wheelchar. In ths paper, a vdeo-based user dentfcaton algorthm usng Mahalanobs dstance s proposed to ensure the ntegrty and securty of our ntellgent wheelchar after 1D profle ASM s thoroughly revsted. hs user dentfcaton mechansm s to be used n our ntellgent wheelchar for user logn. Some expermental results show that the vdeo-based user dentfcaton algorthm performs reasonably well f the stored user face records always keep updated by the adaptve learnng algorthm that we have proposed. Our future work wll be usng ASM to accurately recognse the face drectons and test t on the real IW for the control purpose. Acknowledgements We would lke to thank Emount Ltd., Chna (http://www.emount.com.cn/) for ther generous contrbuton to the Chnese face database wth 87 labelled ponts for each face. References Bartlett, M.S., Movellan, J.R. and Sejnowsk,.J. (2002) Face recognton by ndependent component analyss, IEEE ransactons on Neural Networks, Vol. 13, No. 6, pp.1450 1464. Blanz, V. and Vetter,. (2003) Face recognton based on fttng a 3D morphable model, IEEE ranscatons on Pattern Analyss and Machne Intellgence, September, Vol. 25, No. 9, pp.1 12. Cootes,. and aylor, C. (1992) Actve shape models smart snakes, Proceedngs of Brtsh Machne Vson Conference (BMVC 1992), Leeds, UK, pp.266 275. Cootes,. and aylor, C. (2004) Statstcal models of appearance for computer vson, echncal Report on Actve Shape Models and Actve Appearance Models, Unversty of Manchester, avalable at http://www.sbe.man.ac.uk/bm/models/app models.pdf. Edwards, G., Cootes,. and aylor, C. (1998) Face recognton usng actve appearance models, Proceedngs of 5th European Conference on Computer Vson, LNCS-Seres 1406 1607, pp.581 595. Etemad, K. and Chellappa, R. (1997) Dscrmnant analyss for recognton of human face mages, Journal of the Optcal Socety of Amerca A, August, Vol. 14, No. 8, pp.1724 1733. Jan, A.K., Ross, A. and Prabhakar, S. (2004) An ntroducton to bometrc recognton, IEEE ransactons on Crcuts and Systems for Vdeo echnology, January, Vol. 14, No. 1, pp.4 20. Ja, P. and Hu, H. (2007) AAM-based HCI for an ntellgent wheelchar, MIPPR 2007: Remote Sensng and GIS Data Processng and Applcatons; and Innovatve Multspectral echnology and Applcatons, Proceedngs of he Internatonal Socety for Optcal Engneerng (SPIE), Wuhan, Hube, Chna, November, Vol. 6790, 67904F. Ja, P., Hu, H., Lu,. and Yuan, K. (2007) Head gesture recognton for hands-free control of an ntellgent wheelchar, Journal of Industral Robot, Aprl, Vol. 34, No. 1, pp.60 68. Matthews, I. and Baker, S. (2004) Actve appearance models revsted, Internatonal Journal of Computer Vson, November, Vol. 60, No. 2, pp.135 164. Messer, K., Matas, J., Kttler, J., LÄuttn, J. and Matre, G. (2004) XM2VSDB: the extended M2VS database, Audo- and Vdeo-based Bometrc Person Authentcaton, AVBPA 99, Washngton DC, USA, March, pp.72 77. Nordstrøm, M.M., Larsen, M., Serakowsk, J. and Stegmann, M.B. (2004) he IMM face database an annotated dataset of 240 face mages, echncal Report of Informatcs and Mathematcal Modelng, echncal Unversty of Denmark, May. Park, U. and Jan, A.K. (2007) 3D model-based face recognton n vdeo, he 2nd Internatonal Conference on Bometrcs, Seoul, South Korea, August, pp.1085 10941. Stegmann, M.B. (2000) Actve appearance models: theory, extensons and cases, echncal Report of Informatcs and Mathematcal Modelng, echncal Unversty of Denmark, September. urk, M. and Pentland, A. (1991a) Egenfaces for recognton, Journal of Cogntve Neuroscence, Wnter, Vol. 3, No. 1, pp.71 86. urk, M. and Pentland, A. (1991b) Face recognton usng egenfaces, Proceedngs of IEEE Conference on Computer Vson and Pattern Recognton, June, pp.586 591. Vola, P. and Jones, M.J. (2001) Rapd object detecton usng a boosted cascade of smple features, Proceedngs of IEEE Computer Socety s Computer Vson and Pattern Recognton, Vol. 1, pp.511 518. Vola, P. and Jones, M.J. (2004) Robust real-tme face detecton, Internatonal Journal of Computer Vson, Vol. 57, No. 2, pp.137 154. Wkpeda, Bometrcs, Wkpeda, he Free Encyclopeda, avalable at http://en.wkpeda.org/wk/bometrcs. Wkpeda, Delaunay rangulaton, Wkpeda, he Free Encyclopeda, avalable at http://en.wkpeda.org/wk/delaunay trangulaton. Wkpeda, Mahalanobs Dstance, Wkpeda, he Free Encyclopeda, avalable at http://en.wkpeda.org/wk/mahalanobs dstance. Wkpeda, Outler, Wkpeda, he Free Encyclopeda, avalable at http://en.wkpeda.org/wk/outler. Wskott, L., Fellous, J-M., KrÄuger, N. and von der Malsburg, C. (1997) Face recognton by elastc bunch graph matchng, IEEE ransactons on Pattern Analyss and Machne Intellgence, July, Vol. 19, No. 7, pp.775 779.

Actve shape model-based user dentfcaton for an ntellgent wheelchar 9 Yang, M-H. (2002) Kernel egenfaces vs. kernel fsherfaces: face recognton usng kernel methods, Proceedngs of the Ffth IEEE Internatonal Conference on Automatc Face and Gesture Recognton, Washngton DC, USA, May, pp.215 220.