Extraction of Human Gait Features from Enhanced Human Silhouette Images

Transcription

1 2009 IEEE International Conference on Signal and Image Processing Applications Extraction of Human Gait Features from Enhanced Human Silhouette Images Hu Ng #1, Wooi-Haw Tan *2, Hau-Lee Tong #3, Junaidi Abdullah #4, Ryoichi Komiya #5 # Faculty of Information Technology, Multimedia University Jalan Multimedia, Cyberjaya, Selangor, Malaysia 1 nghu@mmu.edu.my 3 hltong@mmu.edu.my 4 junaidi@mmu.edu.my 5 komiya@mmu.edu.my * Faculty of Engineering, Multimedia University Jalan Multimedia, Cyberjaya, Selangor, Malaysia 2 twhaw@mmu.edu.my Abstract In this paper, a new approach is proposed for extracting human gait features from a walking human based on the silhouette image. The approach consists of five stages: clearing the background noise of image by morphological opening; measuring the width and height of the human silhouette; dividing the enhanced human silhouette into six body segments based on anatomical knowledge; applying morphological skeleton to obtain the body skeleton; and applying Hough transform to obtain the joint angles from the body segment skeletons. The joint angles together with the height and width of the human silhouette are collected and used for gait analysis. From the experiment conducted, it can be observed that the proposed system is feasible as satisfactory results have been achieved. Keywords Biometric, gait features, morphological skeleton, Hough transform and K-nearest neighbours. I. INTRODUCTION Personal identification or verification schemes are widely used in systems that require determination of the identity of an individual before granting the permission to access or use the services. Human identification based on biometrics refers to the automatic recognition of the individuals based on their physical and/or behavioural characteristics such as face, fingerprint, gait and spoken voice. Biometrics are getting important and widely acceptable nowadays because they are really personal / unique that one will not lose or forget it over time. Since every individual has his/her own walking pattern, gait is unique feature. Human walking is a complex locomotion which involves synchronized movements of body parts, joints and the interaction among them [1]. Gait is a new motion based biometric technology, which offers the ability to identify people at a distance when other biometrics are obscured. Furthermore, there is no point of contact with any feature capturing device and is henceforth unobtrusive. Basically, gait analysis can be divided into two major categories, namely model-based method and model-free method [2]. Model-based method generally models the human body structure or motion and extracts features to match them to the model components. It incorporates knowledge of the human shape and dynamics of human gait into an extraction process. This implies that the gait dynamics are extracted directly by determining joint positions from model components, rather than inferring dynamics from other measures, thus, reducing the effect of background noise (such as movement of other objects). For instance, Johnson used activity-specific static body parameters for gait recognition without directly analyzing gait dynamics [3]. Cunado used thigh joint trajectories as the gait features [4]. The advantages of this method are the ability to derive gait signatures directly from model parameters and free from the effect of different clothing or viewpoint. However, it is time consuming and the computational cost is high due to the complex matching and searching process. Conversely, model-free method generally differentiates the whole motion pattern of the human body by a concise representation without considering the underlying structure. The advantages of this method are low computational cost and less time consuming. For instance, BenAbdelkader proposed an eigengait method using image self-similarity plots [5]. Collins established a method based on template matching of body silhouettes in key frames during a human s walking cycle [6]. Philips characterized the spatial-temporal distribution generated by gait motion in its continuum [7]. This paper presents the unique concept of extracting the human gait features of walking human from the consecutive silhouette images. First, enhanced human silhouette is divided into six body segments to construct the 2D skeleton of the body model. Then, Hough transform technique is applied to obtain the joint angle for each body segment. This concept of joint angle calculation is found faster in process and less complicated than the model-based method like linear /09/$

2 regression approach by J. H. Yoo [8] and temporal accumulation approach by D. K. Wagg [9]. II. OVERVIEW OF THE SYSTEM First, the morphological opening is applied to reduce background noise on the raw human silhouette images. Each of the human silhouettes is then measured for its width and height. Next, each of the enhanced human silhouettes is divided into six body segments based on the anatomical knowledge [10]. Morphological skeleton is later applied to obtain the skeleton of body segments. The joint angles are obtained after applying Hough transform on the skeleton. The dimension of the human silhouette width, height and six joint angles from body segments head and neck, torso, right hip and thigh, right lower leg, left hip and thigh, and left lower leg are then used as the gait features for classification. Fig. 1 summarises the process flow of the proposed system. A B = (AB) B) (1) where A is the image, B is the structuring element, represents morphological erosion and represents morphological dilation. The opening first performs erosion, followed by dilation. Fig. 2 shows the enhancement made to the original human silhouette image. (a) Original video image (b) Original silhouette image (c) Enhanced silhouette image Fig. 2. Original and enhanced images after morphological opening B. Measurement of width and height The width and height of the enhanced human silhouette are being measured as shown in Fig. 3. These two features will be used for gait analysis in the later stage. Height Fig. 1. Flow chart of the proposed system III. EXTRACTING THE HUMAN GAIT FEATURES A. Original image enhancement The acquired original raw human silhouette images are obtained from the small subject gait database, University of Southampton [11]. They used static cameras to capture eleven subjects walking along the indoor track in four different angles. Video data was first preprocessed using Gaussian averaging filter for noise suppression, followed by Sobel edge detection and background subtraction technique to create the human silhouette images. Due to poor lighting condition during the video shooting, shadow was found especially near to the feet. It appeared as part of the subject body in the binary human silhouette image as shown in Fig. 2. The present of the artifact affects the gait feature extraction and the measurement of joint angles. The problem can be reduced by applying Morphological opening with a 7x7 diamond shape structuring element, as denoted by Width Fig. 3. The width and height of a human silhouette C. Dividing human silhouette At this stage, the enhanced human silhouette is divided into six body segments based on the anatomical knowledge [10]. First, the centroid of the subject is determined by calculating the centre of mass of the region pixels. The area above the centroid is considered as the upper body head, neck and torso. The area below the centroid is considered as the lower body hips, legs and foots. Then, one third of the upper body is divided into head and neck. The remaining two thirds of the upper body are classified as the torso. The lower body is divided into two portions (i) hips and thighs (ii) lower legs and foots by the ratio one to two. Again, the centroid coordinate is used to divide the two portions into the final four segments (i) right hip and thigh (ii) right lower leg and foot (iii) left hip and thigh and (iv) left lower leg and foot. Fig. 4 shows the six segments of the body, where a represent head and neck, b represents torso, c represents right hip and thigh, d represents right lower leg and foot, e 426

3 represents left hip and thigh and f represent left lower leg and foot. r Detected line Fig. 4. Six body segments D. Skeletonization of body segments To enhance the segments structure, morphological skeleton is used to construct the skeleton for all the body segments. Skeletonization involves consecutive erosions and opening operations on the image until the set differences between the two operations is zero. Erosion Opening Set differences AkB (AkB)B (AkB) ((AkB)) B (2) where A is an image, B is the structuring element and k is from zero to infinity. Fig. 5. shows the skeleton of the body segments. Fig. 6. Joint angle formation The parameter r represents the perpendicular distance between the detected line and the origin, while is the angle between the closest point on the detected line and the origin, and is the joint angle calculated using 0 (3) 90 Fig. 7 shows all the gait features extracted from a human silhouette, where Angle 7 is the thigh angle, calculated as Angle 7 = Angle 6 Angle 4 (4) Width Angle 2 Height Angle 1 Fig. 5. Skeleton of the body segments E. Joint angles extraction To extract the joint angle for each body segment, Hough transform is applied on the skeleton. Hough transform maps pixels in the image space to the straight line through a parameter space. The skeleton in each body segment, which is the longest line, is indicated by the highest intensity point in the parameter space. Fig. 6 shows the joint angle formation from the most probable straight line detected via Hough transform. Angle 7 Angle 4 Angle 6 Angle 3 Angle 5 Fig 7. All the extracted gait features IV. CLASSIFICATION TECHNIQUE In this project, both k-nearest neighbours (KNN) and fuzzy KNN algorithm are employed for the classification of different subjects. KNN is a classifier to distinguish the different subjects based on the nearest training data in the 427

4 feature space. In other words, subjects are classified according to the majority of nearest neighbours as shown in Fig. 8. Step 2: Sort the subjects based on the similarity and identify the k-nearest neighbours. k-nearest neighbours, KNN ={x 1, x 2,, x k } (7) Step 3: Compute the membership value for every class based on equation (5). Step 4: Classify unlabeled subject to the class with the maximum membership value as shown in Fig. 9. Fig. 8. An example for four nearest neighbours for KNN. From Fig. 8, the values on the lines denote the similarities between unlabeled and labelled subjects. The unlabeled subject will be classified as Class 1 as the most frequently representing label is Class 1. In extension to KNN, J.M. Keller [12] has integrated the fuzzy relation with the KNN. According to the Keller s concept, the unlabeled subject s membership function of class i is given by: u ( x) i xknn 1 ui ( x) x x 1 2 m1 x x xknn 2 m1 Where x, x and U i (x) represent the unlabeled subjects, labelled subjects and x s membership of class i respectively. Equation (5) will compute the membership value of unlabeled subject by the membership value of labelled subject and distance between the unlabeled subject and KNN labelled subjects. (5) Fig. 9. An example for four nearest neighbours for Fuzzy KNN. From Fig. 9, the sum of membership values for Class 1, m1=0.7 and for Class 2, m2=0.3. Since m1 is more than m2, so the unlabeled subject is classified into Class 1. V. EXPERIMENTAL RESULTS AND DISCUSSION In our experiment, data of nine subjects at normal walking speed is considered. For each subject, there are approximately twenty sets of walking data. The experiments are carried out to study whether the collected data provides any degree of significance for the recognition of different subject via the supervised classification techniques. Fig. 10 shows the thigh angle, width and height data that has been collected from a walking subject. It shows that there is a very strong relationship between the width of the human silhouette and thigh angle of walking subject. The height of the human silhouette is rather consistent, as there is not much vertical movement of the head at normal walking speed. Through the fuzziness, the KNN will annotate the appropriate class to the unlabeled subject by sum of similarities between labelled subjects. The algorithm involved in identification of the human beings is implemented as bellows: Step 1: Compute the distance between the unlabeled subject, and all labelled or training subjects. The distance between an unlabeled subject, x i and labelled subject, x j is defined as: D (x i, x j ) = x i, x j 2 (6) Fig. 10. The relationship among height, width and thigh angle 428

5 The maximum thigh angle, max for each set of walking data is extracted. For each max, the information of width, w and height, h is identified. As a result, three features are channelled into the classification process and the distance is defined as: max max D( x, x ) =( i - j ) 2 +(w i - w j ) 2 +(h i - h j ) 2 (8) i j Since the supervised classification algorithm adopted, the classification process is divided into training and testing parts. For the training part, eight walking data for each subject are utilized. The rest of the data will be used in the testing part. As such, in total there are 72 training and 111 testing data respectively. The experiments have been conducted for the different values of k neighbors, where k = 3, 4, 5, 6, 7 and 8. The maximum of k is eight as the training data of each class is eight. The comparisons have been done between KNN and fuzzy KNN as well. The obtained results are depicted in Fig. 11 and Table 1. whereas fuzzy KNN considers the summation of the highest membership value, which is more accurate. As shown in Fig. 11, fuzzy KNN show similar results for k of 5 and above. This implies that after five; the incremental of k does not make any significance in terms of accuracy any longer. Overall, these three features not generated very promising results. However, to a certain extent it does provide some degree of significance. The results may be more satisfactory if other features are included, which will be explored in future work In terms of evaluation for the accuracy of each class for fuzzy KNN, we consider the values of k with the highest accuracy only, which are 5, 6, 7 and 8. The objective is to determine which classes are affecting the accuracy more. The results are shown in Table 2. From Table 2, classes 1 and 3 have contributed more to provide better results. Classes 1 and 3 are highly recognizable via the three adopted features. On the other hand, the accuracy of the results is mostly decreased by class 6 and 7. This is because class 6 and 7 have large variation within the same class. TABLE 2. THE ACCURACY OF RECOGNITION FOR RESPECTIVE CLASS k class Fig. 11. Graph for the percentage of classification s accuracy versus the value of k % 91.67% 91.67% 91.67% 2 75% 75% 75% 75% % 91.67% 91.67% 91.67% 4 75% 66.66% 66.66% 66.66% 5 75% 75% 66.66% 66.66% % 42.86% 50% 42.86% % 42.86% 42.86% 42.86% % 66.66% 66.66% 75% 9 75% 75% 75% 83.33% TABLE 1. THE PERCENTAGE OF CLASSIFICATION S ACCURACY FOR KNN AND FUZZY KNN k KNN Fuzzy KNN % 70.27% % 70.27% % 72.07% % 72.07% % 72.07% % 72.07% From Table 1, the classification results obtained from fuzzy KNN are outperforming the results from KNN. The highest accuracy of KNN is lower than the lowest accuracy of fuzzy KNN. Therefore, KNN with fuzzy has made some significant improvements. This is because to the KNN classify the unlabeled person to a particular class according to the highest number of the nearest neighbours belong of that class, VI. CONCLUSION We have described a new approach for extracting the gait features from enhanced human silhouette image. The gait features is extracted from human silhouette by determining the skeleton from body segment. The joint angles are obtained after applying Hough transform on the skeleton. The future plan is to apply more gait features in order to achieve higher accuracy of classification. ACKNOWLEDGMENT The authors would like to thank Prof Mark Nixon, School of Electronics and Computer Science, University of Southampton, United Kingdoms for providing the database for use in this work. 429

6 REFERENCES [1] C. BenAbdelkader, R. Culter, H. Nanda and L. Davis, EigenGait: motion-based recognition of people using image self-similarity, in Proceeding of International Conference Audio and Video-Based Person Authentication, 2001, pp [2] M. S. Nixon, T. Tan and R. Chellappa, Human Identification Based On Gait, 2nd ed., Berlin, Germany: Springer, [3] A. Bobick and A. Johnson, Gait recognition using static, activityspecific prameters, in Proceeding of IEEE Computer Vision and Pattern Recognition, I, 2001, pp [4] D. Cunado, M. S. Nixon and J. Carter, Automatic extraction and description of human gait models for recognition purposes, Computer and Vision Image Understanding, vol. 90, no 1, pp. 1 41, [5] C. BenAbdelkader, R. Cutler and L. Davis, Motion-based recognition of people in EigenGait space, in Proceeding of Fifth IEEE International Conference, May 2002, pp [6] R. Collin, R. Gross and J. Shi, Silhouette-based human identification from body shape and gait, in Proceedings of Fifth IEEE International Conference, May 2002, pp [7] P..J. Phillips, S. Sarkar, I. Robledo, P. Grother and K. Bowyer. The gait identification challenge problem: Dataset and baseline algorithm, in Proceedings of 16 th international Conference Pattern Recognition, vol.1, 2002, pp [8] Jang-Hee Yoo, M. S. Nixon and Chris. J. Harris, Extracting human gait signatures by body segment properties, in Fifth IEEE Southwest Symposium on Image Analysis and Interpretation, 2002, pp [9] D. K. Wagg and M. S. Nixon On Automated Model-Based Extraction and Analysis of Gait, in Proceedings of 6th IEEE International Conference on Automatic face and Gesture Recognition, 2004, pp [10] W. T. Dempster and G. R. L. Gaughran, Properties of body segments based on size and weight, American Journal of Anatomy, vol. 120, pp , [11] J. D. Shutler, M. G. Grant, M. S. Nixon, and J. N. Carter, On a large sequence-based human gait database, in Proceedings of 4 th International Conference on Recent Advances in Soft Computing, Nottingham (UK), 2002, pp [12] J. Keller, M. Gray and J. Givens, A fuzzy K-nearest neighbour algorithm, IEEE Trans. Systems, Man, Cybern. vol. 15, pp ,