Person Re-Identification Using Color Enhancing Feature

Transcription

1 Person Re-Identification Using Color Enhancing Feature 1 Peng Li, 2 Haiyuan Wu, 2 Qian Chen, 3 Chongke Bi 1 2 Wakayama University, 3 RIKEN Sakaedani, Wakayama City, ,Japan , Minatojima-minami-machi, Chuo-ku, Kobe, Hyogo , Japan 1 lipeng209@hotmail.com, 2 {wuhy,chen}@sys.wakayama-u.ac.jp, 3 bichongke@riken.jp Abstract In this paper, we propose a novel feature descriptor for person re-identification without personal information. It is called Water-Drop Render Box (WDRB). The WDRB method is calculated by three steps with target color and its histogram: registration of target color, transformation of distance map, and enhancement of color using target histogram. In order to calculate WDRB, person s top-view images of entering and leaving the room are captured. This is achieved by constructing a bird s-eye view camera system. The person re-identification is carried out by estimating Bhattacharyya distance between database (entering/leaving room person) image and input (entering/leaving room person) image. Finally, the effectiveness of our WDRB descriptor will be demonstrated through several person reidentification experiments. According to the experimental results, it indicates that the WDRB method can be also used for object re-identification. pretty important feature for person re-identification. Several color based methods were proposed. For example, C.Madden has proposed a method [1] that uses the major color to re-identify a person. B.Prosser [9] used the color, histogram and texture information. However, in nowadays society, the privacy protection is being increasingly important. A re-identification method which can re-identify person without personal privacy information is quite necessary, since most people do not like to provide their private information. A bird s-eye view camera system is constructed to capture top-view image of pedestrian. In this paper, we proposed a single-shot re-identification method called as Water-Drop Render Box (WDRB). The WDRB method is a descriptor that can describe a person/object through the process of color registration and color s distance map transformation for re-identify a person or an object using color of person or object. The flow chart of personal identification is shown in Figure Introduction In recent years, more and more people turned their attention to the importance of privacy protection. In the meantime, the person re-identification techniques are applied more widely. Person re-identification is one of the most important topics in computer vision. Usually, according to necessary shot number, person re-identification methods can be classified into two categories, singleshot [10, 11, 3, 6, 2] and multi-shot [7, 8, 5, 4]. Singleshot re-identification can extract features from only one image of person/object. Multi-shot re-identification occurs with multiple person/object images, which has higher reidentification accuracy. However, it needs more learning time. Conversely, single-shot re-identification can obtain feature information quickly. In addition, most person re-identification methods use personal features, such as eyes, mouth, nose or facial feature points and color information. Color information is a Figure 1. The flow chart of person re-identification. The process of the WDRB method is the same as a waterdrop drips and wets paper. In the feature registration step of person re-identification, through the bird s-eye view system, the color information of person s head and shoulders is extracted for the WDRB method. The feature of person is described and registered into database. In the reidentification step, while a person appears in the visible area, his/her WDRB is calculated and compared with the database. For object re-identification, two images from different viewpoints are taken for each object. The first im- 1

2 age is used for building a WDRB database for all objects. The second one is used to evaluate the similarity with the database. For the similarity evaluation, for both person and object, it is achieved by calculating the Bhattacharyya distance of the WDRB values between the database and the second image of current object. The largest distance is considered as the matched person or object. Another color based descriptor called Color Distinctiveness (CD) [12] is similar to the WDRB method. The similarity with target colors and dissimilarity with non-target colors can be integrated with the framework of Bayes rule. The CD method can also re-identify a person using person s color information. Unfortunately, the CD method cannot re-identify a person who wears the same colors clothes with different design. Both the experiments of the WDRB method and the CD method are carried out to describe the merits of our method. 2. WDRB:Water-Drop Render Box In this paper, we proposed a color-based feature descriptor for person re-identification.we call it water-drop render box (WDRB). Because a water-drop (the color of a pixel) drips a paper (register a color), the paper will be wet (color enhanced). When many water-drops drip the same place of the paper, the wet area will be enlarged (enhance color based on histogram). For understanding the WDRB method easily, in this section we will divide the WDRB method into three parts to describe. (1) The color register. (2) Transformation of a distance map from a color registration map. (3) Enhance the registered color Color Register First, when a target moves into the visible area, the colors of the target can be registered by cameras. A RGB color space is created as a three dimensional matrix. According to computational complexity, the side length of the matrix can be set to 2 n, where n = 8, 7, 6, 5. Each cell of the color matrix (C l ) will be initialized with FALSE. While the target area was selected, the RGB value of each pixel will be obtained one by one. The corresponding position of the matrix should be marked as TRUE. As shown in Figure 2(a), a color registration map from the RGB three dimensional space is projected to the GB-plane. The red points represent the registered color (C i ). Finally, we can get the color registration map when the registration of all pixels color is finished Transform a Color Registration Map into a Distance Map For expressing the distance relationship of all registered color, we transform the color registration map into a distance map. In the previous step, we get a color registration map that only the registered color (C i ) is marked as TRUE. (a) Color registration map (b) Color distance map Figure 2. 2D color registration map and 2D Color distance map. We calculate the color space distance between two colors as Equation (1). f( (RGB) Cl (RGB) Ci ) = (RCl R Ci ) 2 + (G Cl G Ci ) 2 + (B Cl B Ci ) 2 The nearest color space distance (CSD) is calculated using Equation (2) from each C l to the nearest C i. The corresponding position will be filled as the CSD. A two dimensional map example is shown in Figure 2(b). (1) ID = arg max (RGB) Cl (RGB) Ci (2) iɛ{1,2,...,n} 2.3. Enhance the Registered Color Until now, we already got the distance map from the registered color of the target. In other words, our water-drop just fell to the paper, but not wet the paper yet. From now, the water-drop will wet the paper. According to the histogram of the selected target area, the registration range of each registered color will be expanded. However, it is difficult to set the range that we want to expand. In this paper, the sensitivey of person to color (SPC) will be applied to this research. For investigating SPC, we implemented a preexperiment using a series of gradient color images. We prepared 27 gradient color images. Figure 3(a) shows one of them. It was divided into 9 blocks. Set the RGB value of block 0 as the referece value. The RGB values at the other 8 blocks were decreased progressively with the same value (v). In this pre-experiment, the v has been set as -5. The next step is to find the block i, who has significant difference with the block 0. In Figure 3(a), i = 2, because block 2 has significant difference with block 0. This process is used to obtain a better v value. A statistical analysis result is shown in Figure 4, from which we can know that most of participants can easily identify the difference while v was set as from 5 to 10. For accurately identify the target that has the clothing with the same color and different pattern, we need to enhance the registered color further. We use the color histogram of the target H Ci to enhance the repeated color.

3 (a) Gradient color image (b) Enhance color(2d) Figure 3. Gradient color image and enhance color. Figure 5. The bird s-eye view system for person re-identification. Figure 4. The result of feedback. The enhance range (10) we set will be used as a standard range thr. Each registered color s enhance range Ω i will be added or subtracted as a weight on the basis of histogram (Equation (3)). Therefore, the registered color map will be transformed to Enhance Color map using Equation (4). The sample of enhance color map in 2D is shown in Figure 3(b). E Cl = { T RUE, C i, Ω i = (1.0 + H Ci n i=1 H C i ) thr (3) C i (R ± Ω i, G ± Ω i, B ± Ω i ) T hr otherwise (4) 3. Person Re-Identification using the WDRB method People s private information is not necessary in our method. We constructed a bird s-eye view system on the top of passageway for person re-identification (Figure 5). Assuming that most people do not change their hair color and the clothes often in daily life. Therefore, we can assume that the color of the shoulders and the head of a person does not change in a short time The WDRB method for Person Re- Identification Without Face-Information Through the bird s-eye view system we identify persons whose images were captured. Set the shoulder and head area as target, we can determine his/her WDRB using the Equation (1) to (4). Except two persons have completely same shoulder color and hair color, we can successfully use the value of WDRB to distinguish them. Therefore, we define the value of WDRB as a feature that can be used to describe the characteristics of persons. The person identification is divided into two parts, learning stage and identification stage. In the learning stage, when a person (ID=k) enters the room and appears in the input image, we calculate the person s water-drop render box feature W DRB d(k). Save W DRB d(k) into the database as his unique WDRB feature. In the identification stage, when a person leaves the room and disappears in the input image, we calculate the query person s water-drop render box feature W DRB p Similarity Evaluation by Bhattacharyya Distance Through calculation of the amount of overlap between two distributions, we can get the Bhattacharyya distance that measures the similarity of the two distributions. A longer Bhattacharyya distance means that the distribution is much more similar to another one. We compare the W DRB p with each person s W DRB d(k) in the database. Both W DRB p and W DRB d(k) are discrete probability distributions in the same color space X. In this paper, the similarity of a query person W DRB p with all person s W DRB d(k) is calculated using Bhattacharyya distance measure (Equation (5)). After the calculation of similarity of each WDRB pair, the ID with the highest similarity will be regarded as the matched result (Equation (6)). D(W DRB d(k) (x), W DRB p (x)) = ln( W DRB d(k) (x)w DRB p (x)) xɛx ID = (5) arg max D(W DRB d(k) (x), W DRB p (x)) (6) kɛ{1,2,...,n} 4. Experiments For verifying the effectiveness of our WDRB method, we implemented several experiments of person and object

4 respectively. The CD method s experiments were implemented. We used a PC with an Intel Core i CPU and 16GB memory, running Windows 7 and the camera was a Logicool c615 web camera. For decreasing the unnecessary calculated amount, we recorded the computational cost of one person s WDRBs using Bhattacharyya distance. In the same conditions, the computational cost of re-identification between two WDRB is also recorded. The average WDRB s cost is second when the side length of RGB space matrix is set as 128. The average reidentification cost is second and second. When the RGB space matrix is 128 and 64, respectively. Other execution time was less than 1 microsecond. Take the computational cost and calculation accuracy into consideration, the side length of RGB space matrix is set as 64 in the following experiments Person Re-Identification For person identification, we had 17 people involved in the experiments. Both the WDRB and CD method s experiments are carried out at the same time. The selection operation of person s shoulder and head for the WDRB and CD method is shown in Figure 6(a) and (b). (a) The WDRB results (b) The CD results Figure 8. WDRB and CD results of person re-identification. In the figure, each row represents an individual experiment. No.1 to 17 represent the WDRB and CD of database that was calculated using corresponding person s entering room image. With the number of experiment corresponding leaving room image, the Bhattacharyya distance similarity with each person s database feature was calculated. For understanding easily, all similarities are transformed into gray values. It means that the whiter one represents the higher similarity. Because the similarity of himself or herself experiment should be the highest one, therefore the diagonal from top left to bottom right should be the whitest line. As shown in Figure 8, the WDRB method s results have high accuracy. However, most the CD method s results can not re-identify person clearly and several results were wrong Object Re-Identification (a) The WDRB color area (b) The CD color area Figure 6. The color area selection of person s WDRB and CD. The WDRB and CD of each person were registered when he or she entered the room (upside of Figure 7) and the distance of the person to each registered person is calculated when he or she left the room (downside of Figure 7). The WDRB can represent the information of target s color and color histogram. Besides of person reidentification, the WDRB method can also be used for object re-identification. We also did experiments of object reidentification using the WDRB and CD method. We used 15 objects in the experiments and took two pictures from different viewpoints for each object (upside and downside of Figure 9. For calculating object s WDRB and CD, we save the visible area of object for each object from either of the two pictures (Figure 10). Figure 7. Posture of entering and leaving a room. The WDRB method s results of person re-identification experiments using Bhattacharyya distance are shown in Figure 8(a). The CD method s results are shown in Figure 8(b). Figure 9. The picture of objects.

5 Acknowledgement This work was supported by JSPS KAKENHI Grant Number 15K01331, and References (a) The WDRB color area (b) The CD color area Figure 10. The color area selection of object s WDRB and CD. For object re-identification, we used the pictures of the objects taken from different viewpoints to compute the WDRB and CD. Then the distance between it and each database WDRB and CD were calculated. The evaluation of WDRB and CD using Bhattacharyya distance are shown in Figure 11(a) and (b). The results of the highest evaluation of the each row are marked with red color. The object re-identification experiments were carried out with the same way of the person s. No.1 to 15 represent the WDRB and CD of database that was calculated using corresponding object s picture. Each experiment used with the corresponding number s object picture that was taken from different viewpoints. As shown in Figure 11, all WDRB method s results are successful. Most of the CD method s results can re-identify object correctly. However, they do not have obvious difference. (a) The WDRB results (b) The CD results Figure 11. WDRB and CD results of object re-identification. 5. Conclusions We have proposed a novel descriptor named as WDRB. It is a person re-identification method with the assumption that people do not change their hair s and cloth s color in short time. For avoiding personal privacy problem, a bird seye view system was used in the person re-identify experiment. Several person re-identification experiments of the WDRB and CD method were implemented. All experiment s results were evaluated using Bhattacharyya distance. The WDRB method has higher accuracy than the CD method. Several experiments demonstrated that the WDRB method can be used for both person and object reidentification. For the future work, the texture of target will be considered. The real-time person re-identify experiment will also be implemented. [1] E. D. C. C. Madden and M. Piccardi. Tracking people across disjoint camera views by an illumination-tolerant appearance representation. In Machine Vision and Applications, 18((3-4)): , August [2] D. A. F. D. Ramanan and A. Zisserman. Tracking people by learning their appearance. In Pattern Analysis and Machine Intelligence., 29(1):65 81, January [3] D. Gray, S. Brennan, and H. Tao. Evaluating appearance models for recognition, reacquisition, and tracking. In proceedings of 10th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), 3(5), September [4] C. L. Z. J. Sivic and R. Szeliski. Finding people in repeated shots of the same scene. In proceedings of British Machine Vision Conference (BMVC 2006), 3: , September [5] A. P. M. F. L. Bazzani, M. Cristani and V. Murino. Multipleshot person re-identification by hpe signature. In proceedings of 20th International Conference on Pattern Recognition (ICPR), pages , August [6] Lin and L. Davis. Learning pairwise dissimilarity profiles for appearance recognition in visual surveillance. In proceedings of 4th International Symposium on Advances in Visual Computing, 5358:23 34, [7] A. P. V. M. M. Farenzena, L. Bazzani and M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In proceedings of Computer Vision and Pattern Recognition (CVPR), pages , June [8] B. S. O. Hamdoun, F. Moutarde and B. Steux. Person reidentification in multi-camera system by signature based on interest point descriptors collected on short video sequences. In proceedings of ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), pages 1 6, September [9] B. Prosser, S. Gong, and T. Xiang. Multi-camera Matching under Illumination Change Over Time. In proceedings of Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, [10] F. B. S. Bak, E. Corvee and M. Thonnat. Person reidentification using haar-based and dcd-based signature. In proceedings of 7th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1 8, August [11] F. B. S. Bak, E. Corvee and M. Thonnat. Person reidentification using spatial covariance regions of human body parts. In proceedings of 7th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages , August [12] T. Wada. Visual object tracking using positive and negative examples. STAR, 66: , November