High Precision Gait Recognition Using a Large-Scale PC Cluster

Transcription

1 High Precision Gait Recognition Using a Large-Scale PC Cluster Yuuki Horita 1 Satoshi Ito 1 Kenji Kaneda 1 Takuya Nanri 1 Yasuyuki Shimohata 1 Kenjiro Taura 1 Mihoko Otake 2 Tomomasa Sato 1 Nobuyuki Otsu Graduate School of Information Science and Technology, University of Tokyo Hongo, Bunkyo-ku, Tokyo , Japan. 2 Division of Project Coordination, University of Tokyo Kashiwa-no-ha, Kashiwa, Chiba , Japan. 3 National Institute of Advanced Industrial Science and Technology Umezono, Tsukuba, Ibaraki , Japan. Abstract. A high-precision gait recognition method is vital for good video surveillance systems. Gait recognition methods proposed so far include a method with cubic higher-order local auto-correlation (CHLAC) features. However, the recognition rate of this method is lowered if the size of objects to be recognized varies dynamically. Although one possible solution to the problem is to increase the number of CHLAC features, it is not a trivial matter to implement the method because of its large amount of computation time. To overcome this problem, we propose a method that uses a larger number of CHLAC features than a traditional CHLAC method by employing parallel computing techniques. We evaluated our method by applying it to a data set containing objects whose scale changes dynamically. We ran the parallel version of the program on a 192-node cluster. The experimental results show that our method achieved a higher recognition rate than that of the traditional CHLAC method. The results also show that the parallel version of the program achieved a better speedup than the sequential version. Keywords: Gait Recognition, Cubic Higher-order Local Auto-Correlation, Feature Extraction, Cluster Computing, Parallel and Distributed Algorithms 1 Introduction With a recent increase in crimes and terrorist attacks, video-surveillance systems (e.g., video monitoring and automatic recognition techniques) are becoming more and more essential to provide a safe society. For example, the video-surveillance systems are useful in cases where criminals and terrorists usually hide their faces. While it is difficult to recognize them with their face images obtained by cameras, the video-surveillance systems can achieve a high recognition rate since the systems identify individuals using information about their movements, which are difficult for people to hide. Since recognition errors made by video-surveillance systems cause critical damage to a safe society, features enabling gait recognition methods to achieve a high recognition rate have been proposed so far. Many methods based on silhouettes of a human figure were proposed [6, 12, 15]. However, these methods need segmentation, so it is easy to be affected by noise. The cubic higherorder local auto-correlation (CHLAC) feature [7], which has been proposed by Kobayashi and Otsu, is one the features that have shown a high recognition rate. For example, a gait recognition method with the CHLAC features achieved the highest recognition rate when it was applied to data sets of HumanID Gait Challenge Problem [12], which was designed for international competitions [8]. However, the recognition rate of the existing method {horita, tau}@logos.ic.i.u-tokyo.ac.jp {itou, nanri, simohata}@isi.imi.i.u-tokyo.ac.jp kaneda@yl.is.s.u-tokyo.ac.jp otake@cb.k.u-tokyo.ac.jp tomo@ics.t.u-tokyo.ac.jp otsu.n@aist.go.jp with CHLAC features goes down when the given data sets are more complex. For example, the method faces difficulty in recognizing data sets that contain various kinds of movements and scale variation of target objects. Although one possible solution to achieve a higher recognition rate is to increase the number of CHLAC features, it is not a trivial matter to implement it; the method requires a large amount of computation time. This paper proposes a gait recognition method that achieves a higher recognition rate than the traditional CHLAC method. More specifically, our method has two distinguishing features. First, we have improved the accuracy of the traditional CHLAC method by increasing the number of CHLAC features. Second, we employ parallel computing techniques and a large-scale PC cluster to reduce the computational time. Libraries and tools used by the current implementation include MPICH [3, 4] (a communication library for parallel programming), GXP [5, 14] (a tool for remote job submission) and scalapack [13] (a library for parallel numerical calculation). We evaluated our method by applying it to a data set containing objects whose scales change dynamically. We ran the parallel version of the program on a 192-node cluster. The experimental results show that our method achieved a higher recognition rate than that of the traditional CHLAC method. The results also show that the parallel version of the program achieved a better speedup than the sequential version. Contributions of this paper are summarized as follows: We took the new approach (parallelizing the CHLAC method) to improve a recognition rate. We applied the cluster computing technologies to

2 Learning Phase Images Recognition Phase Images Differencing/Binarization Differencing/Binarization CHLAC Feature Extraction CHLAC Feature Extraction Linear Discriminant Analysis Recognition Processing Figure 1: Extraction of a second-order CHLAC feature Discriminant space Results the gait recognition, which to the authors knowledge, the technologies had not ever been applied to. Figure 2: Flow chart of recognition method with CHLAC features We achieved the impressive results (higher recognition rate as well as better speedup) with the parallel CHLAC method. The remainder of this paper is organized as follows. Section 2 presents an overview of CHLAC features. Section 3 explains the traditional recognition method with CHLAC features. Section 4 and Section 5 describe our recognition method and how the method is executed in parallel respectively. Section 6 presents performance measurements and discusses experimental results. The final section summarizes the paper. 2 CHLAC Features Cubic Higher-order Local AutoCorrelation (CHLAC) [7] is a temporal extension of Higherorder Local AutoCorrelation (HLAC) [11]. Specifically, CHLAC feature vector x = [x 1,..., x N ] t is defined as x i = f(r)f(r + a i 1)...f(r + a i D)dr, (1) where A A is cubic data which consists of successive image frames, r R 3 is a position vector of the reference point in A, a is a displacement vector, f(r) is the value at the point r, and D is the order of CHLAC. Figure 1 briefly illustrates how a second order CHLAC feature is extracted. We also define the reference point r as r = [x, y, t] t where x and y are the horizontal and vertical position in an image respectively, and t is the time. CHLAC extracts information about the shape and the motion of objects. They have some preferable characteristics for analysis of spatio-temporal data such as human Figure 3: An example of differenced and binarized image: moving points turn white (the other points stay black) motions. One is that they are model-free. In previous research, these features were applied to a gait recognition task and a high recognition rate was achieved. Another merit of CHLAC features is that they are shift-invariant. They absorb the difference of positions or the time lag. One of its merits is that its computational cost is constant regardless of the person in the image or his motion. 3 Traditional Recognition Method with CHLAC Features In this section, we explain each step of the traditional recognition method with CHLAC features. Figure 2 illustrates the flow chart of the method. The method has two phases: the learning phase and the recognition phase. Each phase consists of three steps. In the first step of learning phase, motion images are preprocessed. This step consists of image differencing and binarization. First, it creates difference images by calculating the differences between a pair of consecutive image frames. Then, it binarizes the difference images using an automatic threshold-selection technique [10]. This technique extracts only moving points from given images. An example of an extracted image is shown in Figure 3.

3 Then, CHLAC features are extracted from binarized images obtained in the previous step. In the traditional method, independent 251 kinds of CHLAC features are extracted from each image by limiting a displacement vector a in the equation (1) to a range of After that, linear discriminant analysis is applied to the extracted CHLAC feature vectors. In the recognition phase, CHLAC features of images from a video camera are extracted in the same way as learning data. Mahalanobis distances between an input CHLAC feature vector and the class mean vectors in the discriminant space are compared, and the input vector is classified to the nearest class. 4 Our Recognition Method While our recognition method basically follows the traditional method described in Section 3, our method has distinguishing features in CHLAC feature extraction, linear discriminant analysis, and recognition processing. This section describes the details of the differences below. Note that we employ N-fold cross validation to evaluate our recognition method. 4.1 CHLAC Feature Extraction In order to extract more CHLAC features from the binarized difference images, we take two different approaches. In the first approach, we rewrite the equation (1) using new parameters r x, r y and r t as follows: x i = f(r)f(r + Ra i 1)...f(r + Ra i D)dr (2) where A R = r x r y r t These parameters represent the range of displacement in each direction. We also vary the image resolution (the vertical/horizontal length) and the range of integration period (the number of image frames). By varying those parameters, we can obtain an effect similar to using multi-resolution images. We call a parameter set (including the number of image frames) a layer, and n parameter sets n layers. Since 251 CHLAC features are extracted from 1-layer, 251 n kinds of CHLAC features are obtained from n layers. In the second approach, we extend the range of the displacement vector from to As a result, 5527 kinds of CHLAC features are extracted, which are about 22 times as many as the traditional method (251). 4.2 Linear Discriminant Analysis and Recognition Process We adopt the different recognition process for the multi-layer approach and the displacement-vector extension approach. For the former approach, we adopt a majority method and a joint method. In the majority method, the final results of the recognition is decided by the majority of the results of the individual layers. The joint method obtains the final result by concatenating the CHLAC feature vector obtained from each layer into a single vector. For the latter approach, we use the same recognition process as that of the traditional method. More specifically, the majority method and the joint method perform recognition in the following ways: Majority method In the learning phase, for each layer, linear discriminant analysis is applied to extracted CHLAC feature vectors. With this analysis, we obtain information necessary for recognition: a mean vector and an inverse covariance matrix of each class in the discriminant space and discriminant axes. In the recognition phase, first, we project an extracted CHLAC feature vector to the discriminant space using the discriminant axes calculated in the learning phase. Next, we calculate the square of the Mahalanobis distances between each pair of the projected CHLAC feature vectors and class mean vectors. Then, we obtain the results of the recognition for each layer by classifying each vector to a class whose distance is the shortest. Finally, we decide the final result by a majority of the results of the individual layers. Joint method In the learning phase, linear discriminant analysis is applied to the single vector that concatenates extracted CHLAC feature vectors of the individual layers. In the recognition phase, we project combined CHLAC feature vectors to the discriminant space, and then we calculate the square of the Mahalanobis distances between each pair of the projected vector and class mean vectors. Then, we obtain the result of the recognition by classifying the vector to a class whose distance is the shortest. 5 Parallelization This section describes how we parallelize our recognition method in order to reduce the computational time. We parallelize time-consuming operations in both the learning phase and the recognition phase. Table 1 summarizes the target processes and parallelization techniques applied to the processes. In the rest of this section, we describe details of each parallelization. First, we explain how we parallelize the extraction of CHLAC features. Since CHLAC features can be extracted independently from each image data set, we can easily parallelize this step by equally-divided time-series image data to each processor (See Figure 4). We statically determines this allocation assuming that the feature extraction of each image requires almost same amount of computation time. In the current implementation, we achieve this parallelization using GXP [5, 14], a tool for automatically logging in many nodes and for submitting commands to them in parallel. Using the job-submission mechanism of GXP, we extract CHLAC features in parallel, collect the extracted CHLAC features, and store them as files on a hard-disk. More specifically, GXP provides fast parallel (simultaneous) command submission and parallel pipes (pipes between local command and all parallel commands). Combination of these mechanisms

4 Table 1: Parallelization of the learning phase and the recognition phase Description of target process Parallelization technique CHLAC feature extraction GXP Linear discriminant Calculation of covariance matrix MPICH analysis Calculation of eigenvalue problem scalapack N-fold cross validation GXP Image Sequences CHLAC Feature Vectors CHLAC Feature Vectors Covariance Matrix PC Cluster Figure 4: Parallelization of CHLAC feature extraction Figure 5: Parallelizing computation of a covariance matrix enables us to submit the programs to a large number of nodes in a cluster while giving different input parameters to the individual nodes. Second, we parallelize calculation of a covariance matrix required by linear discriminant analysis. We calculate covariance matrix of class Σ according to the following equation: Σ = 1 N N (x i x)(x i x) T i=1 where x i is a feature vector extracted from the ith data of the class, x is the average features of all the data of the class, and N is the number of data. To parallelize the calculation of the above equation, we assign equallydivided feature vectors to each processor, and let it compute (x i x)(x i x) T (Figure 5). Because this approach requires complex communication among processes and GXP is not suitable for such parallel processing, we implemented a parallel program for calculating the covariance matrix with MPICH [3, 4], which is a communication library for parallel programming. Third, we calculate an eigenvalue and eigenvector in parallel using scalapack [13], which is a library for parallel numerical calculation. We use the PDSYEVX routine, a solver for dense symmetric eigenproblem based on the parallel Householder reduction. Finally, we parallelize N-fold cross validation. Since individual cross validation can be executed in an independent manner, we perform every cross validation in parallel using GXP in the same way as feature extraction. Figure 6: An example of images used in the experiments 6 Gait Recognition Experiments 6.1 Experimental Environments We applied the recognition method described above to movements of people in which they go up and down stairs. Figure 6 shows an example of images used in these experiments. The recognition of such movements is a tough problem since the scale of the objects change dynamically. To our knowledge, there is no previous work targeting these movements. More specifically, we used data consisting of 11 people who moved in four different ways: (going up, going down) (walking, running). The people performed each movement six times, and the total number of the images was 44,966. For each data set, we tried to recognize the person who performed the movement and the kind of his/her

5 Table 2: Parameter range used for the recognition methods Recognition method Parameter range a. single layer with 251 features f {25, 30, 35, 40}, (w, h) {(360, 240), (240, 160), (180, 120), (120, 80)}, r x {1, 2, 3}, r y {1, 2, 3}, r t {1, 2, 3} b. majority with 3 layers any combination of 3 layers from (f, w, h, r x, r y, r t ) = (35, 360, 240, 3, 2, 1), (35, 240, 160, 2, 1, 1), (40, 360, 240, 2, 2, 1), (35, 240, 160, 1, 1, 1), (35, 240, 160, 2, 2, 1), (35, 240, 160, 3, 2, 1) (20 cases) c. majority with 3 layers any combination of 5 layers from (f, w, h, r x, r y, r t ) = (35, 360, 240, 3, 2, 1), (35, 240, 160, 2, 1, 1), (40, 360, 240, 2, 2, 1), (35, 240, 160, 1, 1, 1), (35, 240, 160, 2, 2, 1), (35, 240, 160, 3, 2, 1), (40, 360, 240, 3, 1, 1) (21 cases) d. majority with 7 layers any combination of 7 layers from (f, w, h, r x, r y, r t ) = (35, 360, 240, 3, 2, 1), (35, 240, 160, 2, 1, 1), (40, 360, 240, 2, 2, 1), (35, 240, 160, 1, 1, 1), (35, 240, 160, 2, 2, 1), (35, 240, 160, 3, 2, 1), (40, 360, 240, 3, 1, 1), (40, 360, 240, 3, 3, 1), (35, 240, 160, 3, 1, 1) (36 cases) e. joint with 3 layers same as majority with 3 layers f. joint with 5 layers same as majority with 5 layers g. joint with 7 layers same as as majority with 7 layers h. single layer with 5527 layers f = 40, (w, h) = (360, 240), r x = r y = r t = 1 f: range of integration period in equation (2) w, h: horizontal/vertical length of the image r x, r y, r t : range of displacement in each direction in equation (2) movement (11people 4 movements = 44 categories) with the following eight methods: the original version (single layer with 251 features) the majority method and joint method respectively with three, five, and seven layers the mono-layered method in which the dimension of displacement vector is extended to (single layer with 5527 features) For each recognition method, we searched the most suitable parameter set for recognition as shown in Table 2, and evaluated the recognition rates using 6-fold cross validation. The parallel version of the program was performed on a 192-node cluster with Xeon 2.4/2.8 GHz dual processors, while the sequential version on a Xeon 2.8 GHz processor. All the machines are equipped with 2 GB RAM and a 1 Gigabit Ethernet NIC. 6.2 Experimental Results Figure 7 summarizes the experimental results. Figure 7-(A) shows the highest recognition that each method achieved with varying parameters. These results show that our revised methods achieved up to about 97.7% recognition rate, while the original method (single layer with 251 features) achieved at most about 85% recognition rate. Figure 7-(B), (C), (D) and (E) show the execution time and the speedup rate of both feature extraction and cross validation. We can see that the total execution time of our recognition system was greatly reduced to a practical time due to the parallelization. For both feature extraction and cross validation step, all the methods achieved a speedup with parallelization. For example, recognition method h (single layer with 5527 features) took only minutes (about 3 hours) with parallelization while it took 7,140 minutes (about 5 days) without parallelization. For the feature extraction step, h achieved the lowest speedup ratio among all the methods while it achieved the highest for the cross validation step. We also note that the speed of the recognition processing (reading images + making differential/binarizing images + CHLAC feature extraction + recognition) on a Xeon 2.8 GHz processor was about 29 frames per second (fps) in the original method (a), and about 3.4 fps in the 7-layered joint method (g). 6.3 Discussion about the Recognition Rates We discuss three remarkable points about the recognition rates of the experiments (Figure 7-(A)). First, the recognition rate was improved as the number of features used for recognition increased. This result indicates that the extracted features become more expressive as the number of the features increases. Second, the joint methods generally have advantages in a recognition rate over the majority methods if both methods use the same number of CHLAC features. We can consider several reasons for this result. One possible reason is that CHLAC features are not scale-invariant. Since the data sets used in the experiments include objects that change their scale, 251 CHLAC features of the original method (a) were not sufficient for recognition. Another reason is that the current implementation does not support a mechanism that enables individual recognition systems to compensate for one another although it is a general technique for majority voting (e.g., boosting

6 (A) (B) (C) (D) (E) Figure 7: Experimental results: (A) recognition rate on the movements of going up/down stairs, (B) computation time of the feature extraction step, (C) speedup ratio in the feature extraction step, (D) computation time of the cross validation step, and (E) speedup ratio of the cross validation step techniques such as AdaBoost [2]). Third, the 7-layered joint method (g) achieved a 97.67% recognition rate, which is as high as that of the extended mono-layered method (h) even if they use the different dimension of the feature vectors (1757 and 5527 respectively). This result indicates that 5527 features in the extended mono-layered method (h) involve many meaningless features that do not represent the substance of the target objects. Since these meaningless features turn the result of discriminant analysis unstable, simply increasing the number of features does not necessarily improve the recognition rate. 7 Conclusion and Future Work We have presented a gait recognition method with CHLAC features. Our method achieves a higher recognition rate than the traditional one by increasing the number of CHLAC features. Since using a large number of CHLAC features requires a large amount of computation time, our method parallelizes the learning and recognition phase using a large-scale PC cluster. The experimental results show that our recognition method identifies the movements of going up/down stairs more accurately than the traditional one. It should be noted that our method can be easily applied to other kinds of movements (e.g., fighting action) because our method does not assume any model for target objects in advance. This is the advantage of our method over other model-based gait-recognition methods. We also note that there is room for further study about ways to increase the number of CHLAC features. As shown in equation 2, the current algorithm uses all kinds of displacement vectors within a certain range. However, irrelevant features should be excluded from the view point of recognition. Accordingly, we need to seek an algorithm that guesses in advance how each feature can contribute to recognition and that uses only meaningful features. Acknowledgements This work was performed as part of the Hundred- Hour Workshop at the Information Science and Technology Strategic Core, Graduate School of Information Science and Technology 21st Century COE Program, University of Tokyo. Details concerning this workshop can be found at WS100H.NET [1, 9]. References [1] A Hundred-Hour Workshop@UT-I-COE,. [2] Yoav Freund and Robert E. Schapire. A Decision- Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1): , [3] William Gropp and Ewing Lusk. User s Guide for MPICH, a Portable Implementation of MPI. Technical Report ANL-96/6, Argonne National Laboratory, 1994.

7 [4] William Gropp, Ewing Lusk, Nathan Doss, and Anthony Skjellum. A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard. Parallel Computing, 22(6): , September [15] L. Wang, T. Tan, H. Ning, and W. Hu. Silhouette Analysis-Based Gait Recognition for Human Identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12): , December [5] GXP Grid/Cluster Shell Quick Manual. phoenix/gxp_quick_man_ja.shtml. [6] A. Kale, N. Cuntoor, B. Yegnanarayana, A. N. Rajagopalan, and R. Chellappa. Gait Analysis for Human Identification. In Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication, [7] Takumi Kobayashi and Nobuyuki Otsu. Action and Simultaneous Multiple-Person Identification Using Cubic Higher Order Local Auto-Correlation. In Proceedings of 17th International Conference on Pattern Recognition (ICPR 2004), volume 4, pages , August [8] Takumi Kobayashi and Nobuyuki Otsu. A Three- Way Auto-Correlation Based Approach to Human Identification by Gait. In 6th IEEE Workshop on Visual Surveillance, pages , [9] Mihoko Otake, Ryo Fukano, Shinji Sako, Masao Sugi, Kiyoshi Kotani, Junya Hayashi, Hiroshi Noguchi, Ryuichi Yoneda, Kenjiro Taura, Nobuyuki Otsu, and Tomomasa Sato. Autonomous Collaborative Environment for Project Based Learning. In T. Arai et al., editor, Intelligent Autonomous Systems 9 (IAS-9), pages IOS Press, [10] Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on System, Man and Cybernetics, 9(1):62 66, March [11] Nobuyuki Otsu and Takio Kurita. A new scheme for practical, flexible and intelligent vision systems. In Proceedings of IAPR Workshop on Computer Vision Special Hardware and Industrical Applications, pages , October [12] Sudeep Sarkar, P. Jonathon Phillips, Zongyi Liu, Isidro Robledo Vega, Patrick Grother, and Kevin W. Bowyer. The HumanID Gait Challenge Problem: Data Sets, Performance, and Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2): , February [13] scalapack. scalapack/. [14] Kenjiro Taura. GXP : An Interactive Shell for the Grid Environment. In Proceedings of International Workshop on Innovative Architecture for Future Generation High-Performance Processors and Systems (IWIA 2004), pages 59 67, January 2004.