Automated Detection of Human for Visual Surveillance System

Transcription

1 Automated Detection of Human for Visual Surveillance System Yuji KUNO, Takahiro WATANABE, Yoshinori SHIMOSAKODA, and Satoshi NAKAGAWA Kansai Laboratory, Oki Electric Industry Co., Ltd., 2-27 Shiromi 1-chome, Chuo-ku, Osaka 540, Japan Abstract This paper describes a robust and reliable method of human detection for visual surveillance systems. The merit of this method is to use simple shape parameters of silhouette patterns to classify humans from other moving objects such as butte flies and autonomous factory vehicles. An extraction function based on the brightness level transformation is used to extract the precise shape of the silhouette patterns. An approach to overcome the occlusions of humans is also proposed. We tested our method for 2,500 images (1,100 from humans and 1,400 from other moving objects). Our test system detected the humans at the rate of 98% (=1,077/1,100) and judged 92% (=1,283/1,400) of the other moving objects as non-humans. 1. Introduction Visual surveillance systems using TV cameras are very suitable for the physical security, since the video sequences from many remote areas can be presented to watchmen at a time. However, seeingmany TVmonitors for a long time is hard work for watchmen. Therefore the automatic detection of moving objects and the classification of the detected objects are required in these systems. In particular, distinguishing humans from other objects is an indispensable function for security systems. Several methods for describing features of a human body have been proposed [I]-[4]. The mockl-based methods use the geometrical models of a human [3][4]. These methods can not only detect humans but also recognize human actions. However, real images are often too noisy to permit the model fitting easily. Murase 121 andmori et al. [ 11 used the rhythm of walking for personal identification and human detection respectively. Their methods suppose that a human continues to walk constantly for a preset direction. Therefore it is difficult to detect humans with high accuracy since the humans may suddenly change the direction and the speed of walking. In addition to this, most of the previous methods have never treated the several problems inherent to the real scenes such as the occlusions of humans and the intrusions of moving objects except humans. This paper describes a robust and reliable method of human detection. We first take the precise silhouette patterns by detecting and analyzing the change in the brightness between the background image and the current image. An extraction function based on the brightness level transformation is used to extract the silhouette patterns from images. The extraction function arranges the sensitivity of the difference between the background image and the current image according to the brightness of each pixel. We use the shape features of the silhouette patterns of humans as the detection parameters. The shape features are mainly the mean and the standarddeviation of the projection histoam of the silhouette pattem, since the projection histogram decreases noise and is useful to get the macroscopic features of the silhouette pattem. The basic values of shape features are decided from about 200 typical silhouette patterns of walking. If the difference betweenthe basic values and the values calculated from a silhouette pattern is small, the silhouette pattern is judged as a human. When several humans walk together, the silhouette pattem of a human may be changed since the human is sometimes occluded by other humans. However, we overcome the problem by using a part of the silhouette pattern. 2. Extraction of silhouette patterns 2.1. Detection of brightness change We use the difference in the brightness between the background image and the current image for extracting the regions of moving objects. The method is simple and gets /96 $ IEEE Proceedings of ICPR '96 865

2 the precise shapes of the regions. In general, the brightness change is obtained by subtracting the brightness in the background image from the one in the current image for each pixel. However, the setting of the threshold value for binarization should be paid attention. Especially in the case of low contrast images, most of the regions of moving objects may be missed out, since the brightness change is too low to distinguish the regions of moving objects from the ones of noise. This can be also caused by the limit of the sensitivity of the camera. Thus it is very difficult to decide the appropriate threshold value for obtaining the precise regions of the moving objects from the noisy image Extraction function To solve the problem in previous subsection, we set a function to detect the sensitive change in the brightness near the low level or the high level. When the brightness of the pixel in the current image is equal to the one of the pixel in the background image, the function should take the value 0. Therefore we define the function f(a(x,y),b(x,y)) in (1); f(a,b)=l- 2 J m. (a + 1) + (b + 1) 2J(256-~)(256-6) (1) (256-U) + (256- b) where a(x,y) is the brightness in the current image, b(x,y) is the brightness in the background image. The variables x andy are the coordinates on the horizontal axis and the vertical axis of the image respectively. This function is conducted from the familiar equation of the arithmetical and geometrid mean ((a + b) / 2 2 G). We call f(a(x,y),b(x,y)) the extraction function. Figure 1 shows the curves of the extraction functionflu) in the cases of b=5 and b=100. When a is equal to b, the value of the extraction functionftakes the minimum, i.e., 0. As the difference between a and b increases, the extraction function f also increases. Especially in the case of b=5, f increases rapidly. This means that a small amount of change in the brightness can be detectedat the low brightness level, and ignored at the middle brightness level. In other words, the extraction function f arranges the sensitivity of the difference between the background image and the current image according to the level of the brightness of eachpixel in the background image. For example, we estimate two cases as follows. One is at a low level (a=10, b=5) in the brightness which is indicated as black triangle in figure 1. The other is at a middle level 1 I Figure 1: Examples of Extraction function f (~105, b=100) of the brightness which is inlcated as black dot in figure 1. The threshold value off to extract the regions changed between a and b is set as 0.01 in figure 1. In both cases, the result of the subtraction between a and b is equal to 5, however, the pixel at the low level can be extracted by using the extraction functionf& at the low level and ~' at the middle level). The extraction function f varies similarly near the end (b=255) of the high level of the brightness. We compared the class discrimination ability [5] of f(u(x,y), b(x,yjj with the one of sub((a(xy),b(xy)) which means the subtraction between a(x,y) and b(xy) (i.e., la(x,y)-b(x,y)l ). Figure 2 shows the results fromabout 100 images. The vertical and horizontal coordinates show the class discrimination abilities off and sub respectively. For most of images, the class discrimination ability in f was better than the one in the subtraction sub. This mean that the threshold value to obtain the precise regions of moving objects can be easily decided for f(a(x,y),b(x,y)). When an image is inputted, the distribution off over x andy is obtained from (1). Then the silhouette patterns are extracted by binarizing the distribution off. We obtain the precise shape of the moving objects by binarizing the distribution off at the threshold which is decidd by the algorithm of [5] sub Figure 2: Comparison of class discrimination. In figure, dots indicate the results in test images. 866

3 2.3. Extraction of silhouette patterns In the previous subsection, we got the regions changedin the brightness from the background image. However the regions may be caused by the light variations. Therefore the regions are checked whether the change of the brightness comes from the moving objects or not. The sizes and the locations of the regions are examined during several frames. In the case of the moving objects, the sizes and locations of the regions are varying. On the other hand. in the case of the erroneous factors such as light variations they are not varying. Therefore the only regions of the moving objects can be extracted. A moving object may be divided into several regions in the binarizing process, since the brightness of the part of the moving object may be locally equal to the one in the background image. Therefore nearby regions are merged as one region. As a result, the merged region is regarckd as a silhouette pattern of a single moving object. 3. Judgment of silhouette patterns in comparison with the one of H2. Figure 6 shows the projection histograms obtained from an image. The histogram of the left man is similar to the one in figure 5(b), and the histogram of the right man is similar to the one in figure 4(g) Silhouette patterns of walking humans The features of a silhouette pattem are calculated by the following method. Figure 3, figure 4 and figure 5 show three typical silhouette pattems of walking (Pattern A-crossing, Pattern B-approaching and Pattern C-obliquely approaching, respectively). We first divide the rectangle around the silhouette pattem into three parts with equal size; Al, and in figure 3,4 and 5, where A 1 corresponds to the head part, the trunk part, and the legs part. In order to extract the macroscopic features of the silhouette pattem in each part, weget the projection histograms (Hl, H2 andh3 in figure 3, 4 and 5) which are obtained by counting the number of black pixels on each column of the silhouette pattem. Then smoothing is performed to the histogram for correction. The distribution d of histogram depends on walking Patterns. For example, in the histogram of Pattem A in figure 3, d in H2 spreads wider than in HI, andd in H3 is wider than in H2. In the histograms of Pattern B in figure 4, ds in H1 and H2 are wide, but d in H3 is narrow. In the histograms of Pattern C in figure 5, some ((a), (f) in figure 5) are similar to Pattem A, but others ((b), (c), (4, (e) in figure 5) are similar to Pattem B. However, they slightly differ from Pattem A and Pattern B, e.g., the histogram of H3 in figure 5(a) is located to the left of the ones of H1, H2. The range of histogram of H3 in figure 5(d) is much smaller Figure 4: Pattern B - approaching Figure 6: Projection histogram for an image 867

4 3.2. Features of silhouette patterns The following features are extractedfrom the histogram. (I) The standard deviation: The differences of the standard deviation between HI--H2, H2--H3 reflect the rough form of humans. Since the widths of the rectangles are different, the standard deviation values are normalized with the widths. (11) The mean: This is also normalized with the width. Since this feature represents the centroid of the histogram, this can be used to check the centroids of the three parts. In addition, the next feature is also used for the human detection. (III) The aspect ratio of the circumscribing rectangle: This is used as a rough criterion for checking if the silhouette pattern represents the walking of humans and if the silhouette pattern indicates the occlusions of humans. We decided the basic values of the above features for three kinds of typical walking patterns from about 200 samples. patterns of each human. Then the shape features of the new silhouette patterns are calculated as described in section 3.2. Figure 7: An example of occlusion 3.3. Occlusions of humans I I In the real scenes, the silhouette pattern of a human are often mixed with the silhouette patterns of other humans, since several humans may walk side by side. Figure 7 shows the silhouette patterns of three humans extracted as a region. The aspect ratio of the circumscribing rectangle of this region is out of the basic value which is set for a human in section 3.2. Therefore according to the value of the aspect ratio, the process for the occlusion is executed as follows. Figure 8 shows the schematic diagram to detect humans from the mixed silhouette pattem. We noticed that the left end (the region L in figure 8) and the right end (the region R in figure 8) of the mixed silhouette pattem remain the original regions of the silhouette pattems of each human. Thus the half of the silhouette pattem of the human is substituted for the other half of the silhouette pattern. We detect the left (XL) and the right (XR) of the peak points of the silhouette pattem by scanning down from the top column of the circumscribing rectangle (Arrows in figure 7). Two axes (x=xl, XR) divide the mixed silhouette pattem into three regions, i.e., L, M, R in figure 8. By turning the both ends of the divided silhouette pattern in the center (i.e., L-+L, R +R ), we obtain the new silhouette patterns (i.e., L+L and R+R ). Thenew silhouette patterns show the approximate shape of the original silhouette Figure 8: Schematic diagram for judgment of mixed pattern 3.4. Judgment process When a silhouette pattern is input, the values of the shape features in section 3.2 are calculated and compared with the basic values in Pattem A, B andc, respectively. The difference between the basic and calculated values corresponds to the probability of the new silhouette pattern to judge whether the silhouette pattern comes from human or not. We ranked the volume of the difference to the probability. For example, if the volume of the differenceis large, the probability level is low. Since we have the probability level in Pattem A, B and C, respectively, the pattern with the highest level is selected At last, if the probability level of selected pattem is higher than the threshold, the silhouette pattern is judged as a human. 868

5 4. Experiments We tested our method for 2,500 images (1,100 from humans and 1,400 from other moving objects). As the moving objects of non-humans, we used the images of the imitation butterflies, AFV(Autonomous Factory Vehicle), curtains, doors, clocks, TV programs and so on. Table 1 shows the results of the tests. silhouette pattern are checkedin three parts: i.e., head, trunk andlegs. The checkis performed on the projection histogram. The process for occluded humans is also added We implemented this method and confirmed that the rate of successhl detection of humans was about 98% with 1,100 images. Table 1 : Results of tests. In brackets, the number of test images is shown. Our test system detected the moving objects from all of images. Then the system detected the humans at the rate of 98% (=1,077/1,100) andjudged92% (=1,283/1,400) of the other moving objects as non-humans. We got good results on the whole. However the accuracy in the butterflies was relatively low, since the spring to suspend the imitation butterfly was sometimes extracted with the imitation butterfly as a silhouette pattern. Figure 9 and figure 10 show the monitor images of our test system in the cases that the human and the imitation butterfly entered respectively. The monitor image is composed from four images. The above left image shows an input image, the above right image shows the moving objects which is circumscribed by rectangles, the below left image shows the silhouette pattems and the below right image shows the result of the human detection. The double circle in figure 9 indicate the probability level described at section 3.4 is very high andthe triangle in figure 10 indicates that the probability level is low. So the moving object in figure 9 is judged as human and the one in figure 10 is judged as non-human. The processing time was 0.3 seconds per image on Sun Sparc Station 2. This does not include the calculation time of the extraction functionf in section 2.2, since the value of the extraction function f can be calculated in advance and stored as the reference table. 5. Conclusion We proposed a method of human detection for visual surveillance systems. In order to extract the precise shapes of the silhouette patterns, the extraction function based on the brightness transformation is used. The shape features of the Figure 9: Result in the image of human Figure 10: Result in the image of butterfly References [l] H. Mori andn. M. Charkari, ShadowandRhythmas Sign Pattems of Obstacle Detection, Proc. IEEE Int. Symposium Industrial Electronics, pp , Budapest, Hungary, June 1993 [2] H. Murase, Recognizing Individuals from the Silhouettes oftheir Walk, IEICE Trans. D-11, J75-D-11, 6, pp , June, 1992 (in Japanese) (English Translation Version of Abstract : IEICE Trans. Inf. & SySt., E75-D, 4, p. 617, July,!992) [3] S. Kurakake and R. Nevatia, Description and Tracking of Moving Articulated Objects, 1 I th IAPR lntemational Conference on Pattern Recognition. Vol. I, pp , J. ORourke andn.1. Badler, Mode-BasedImage Analysis of Human Motion Using Constraint Propagation, IEEZ Trans. on Pattern. Anal. and Mach. Intell., PAMI-2, 6, pp , Nov., 1980 [5] N. Otsu, A Threshold Selection Method from Gray-Level Histograms, IEEE Trans. on Sys., Man &Cybem., SMC- 9, pp , Jan.,