Swift Template Matching Based on Equivalent Histogram

Transcription

1 Swift emlate Matching ased on Equivalent istogram Wangsheng Yu, Xiaohua ian, Zhiqiang ou * elecommunications Engineering Institute Air Force Engineering University Xi an, PR China *corresonding author: hou-zhq@sohu.com Chongzhao an School of Electronics and Information Engineering Xi'an Jiaotong University Xi an, PR China czhan@mail.xjtu.edu.cn Abstract istogram-based temlate matching is an imortant method to search the globe otimization exhaustively. owever, this method is commonly algorithmic comlex. In this aer, we roose to relace the traditional histogram-based method with equivalent histogram-based method, which distinctly imroves the matching efficiency. We first introduce the equivalent histogram on the basis of the relative centralization of the temlate s color information and rove the equivalence. hen, we discuss the alication of equivalent histogram in the current algorithms and analyze the algorithmic comlexity. he equivalent histogram calculates the histograms and their distances according to the relative centralization of color information, which decreases the memory and comutation sending from the calculation of redundant information. he exerimental results indicate that equivalent histogram-based method remarkably imroves the matching efficiency with no degradation of matching effect. Keywords- visual tracking, temlate matching, exhaustive search, equivalent histogram I. INRODUCION emlate matching is an imortant technique in visual tracking, which is widely used in content based image retrieval, object detection and tracking. Comared with Particle Filter [1] and Mean-Shift [] method, its rincile is very simle and it can obtain a more robust result through global otimal exhaustive search. Comared with Otical Flow [3] method, Frame Difference [4] method and ackground Subtract [5] method, it doesn t confine to the static situation and owns a better adatability. he traditional method imlements temlate matching using -D correlation directly, whose comutational comlexity is O(N r ) when matching a temlate sized r r to an image with size of N N. An effective method to reduce the comlexity is to introduce Integral image [5] during the matching course. In 004, Viola roosed an efficient method to calculate histogram based on Summed-area [6] method. On the basis of this, Li P. [8] roosed an exhaustive search method to track visual objects. he exerimental results showed that his method is faster and more recise than the Mean-Shift method. Porikli analyzed the Integral image and roosed a novel algorithm to calculate his research is suorted by National Natural Science Foundation of China (Grant No and ) and Natural Science Foundation of Shaanxi Province of China (Grant No. 011JM8015) histogram, which is named Integral istogram (I) [9]. I based method can track the moving object swiftly and robustly with a low comlexity of O(N ) to a dimension histogram calculation roblem. owever, it takes too much memory sace to calculate histogram using I algorithm. o resolve this roblem, Sizintsev roosed Distributive istogram (D) [10], which renews histogram using the changed ixels in the current window, and utilizes the renewed histogram to calculate the histogram distance rather than writes it to memory. his D based method not only imroved the matching efficiency but also reduced the memory sending remarkably. In 010, Wei did further research on reducing the comutational comlexity and roosed an efficient histogram-based sliding window (ESW) model [11]. e ointed out that only a few histogram bins which changed during the window sliding affects the final histogram distance. So he calculated the histogram distance to construct similarity ma using the differential theory and further imroved the efficiency of temlate matching. Almost all the current temlate matching algorithms are based on rectangle temlate, so the current temlate matching based methods can not track a rotational object very well. o imrove the erformance of tracking rotational object, Adam roosed a Fragment-based method [1] using I algorithm for its fast calculation of arbitrary rectangle area. e divided the rectangle window into a certain number of small rectangle areas and matched the sub- rectangles to imrove the robust of matching. his method works well when the object is artly sheltered by background. Sizintsev ut forward a Multile istogram-based method when discussing the alication of D algorithm. Similar aroach aeared in Wei s study [11] too. esides, the matching theory is also used in Scale Invariant Feature ransform-sif [13] to match the key oints. A more robust result invariant to rotation and scale can be obtained when matching the temlate combined with SIF features. owever, both I based method and D based method send much time to construct the similarity ma. For a dimension histogram, the comlexity of similarity ma construction is O(N ). ESW model utilized the changed histogram bins to renew the similarity ma and reduced the comutational comlexity to O(N min(, sr)). In fact, the dimension of histogram is still an imortant factor which affects the comlexity. 413

2 Figure. Demonstration of the object temlate based histogram (equivalent histogram). Figure 1. he difference between original image based histogram and object temlate based histogram. his aer centralizes on how to reduce the dimension of histogram and rooses an Equivalent istogram based method, which can distinctly reduce the comlexity with no change to the matching result. II. EQUIVALEN ISOGRAM his section will mainly introduce the concetion of the Equivalent istogram and the roof of its equivalence. A. Concetion In the temlate matching based tracking situation, the object temlate usually owns distinct attribute roerties such as color, shae, etc. For examle, the color of object temlate may distribute in a certain range rather than the whole color axes. Fig.1 shows the color distribution of R, G, and channels of the original image and object temlate. It is clearly that the distribution ranges of original image are much larger than those of object temlate. emlate matching algorithm usually calculates the histograms of object temlate and current window on the basis of the color distribution of the original image, and then calculates the histogram distance of object temlate and current windows to construct a similarity ma, where the maxima similarity gives the best confidence of the real object s location. In this situation, many irrelative ixels are involved in the whole course of histogram calculation and similarity ma construction. he redundant calculation severely affected the matching efficiency. Actually, if we calculate the histogram and distance according to the color distribution range of object temlate, the redundant calculation can be removed. Enlightened by Wei s study [1], we tested a great deal of video sequences and analyzed the color distribution of object temlates, and found that the color of object temlates usually assemble in a certain range. So we roose to calculate color histogram and distance according to the assembled color information. As the comletely equivalence of the matching results between temlate based method and image based method, we name the roosed method as Equivalent istogram (Fig.).. Proof he matching result of Equivalent istogram based method is exactly the same as that of the traditional histogram based method, which will be roved in the following content. Proof: Suose S is a rectangle area of original image with the same size of object temlate, and then the histogram of S and can be calculated as follows: ( S) h f( ) X, S i (1) i1 ( ) h f( ) X, j () j1 In the formula, is the number of bins of histogram (dimension), f( ) is the gray value of ixel (we take gray 414

3 value of image for examle to rove the equivalence). h f( ) X, S calculates the number of elements in i f( ) X, S, we marked it as h S. i X i is the gray value range of the i th bins of histogram which can be described as follows: III. EQUIVALEN ISOGRAM ASED EMPLAE MACING Equivalent istogram can be used in many histogram based method such as I algorithm, D algorithm and ESW model. his section will introduce the alication of R( i1.5) R( i0.5) Xi x x 1 1 (3) R in the formula is the scales of gray image, whose value is 55. Let S h, h,..., h, h, h,..., h 1 S1 S S n 1 n, and X X, X,..., X n, then ( S ) and ( ) can be exressed as ( S) X and ( ) norm based distance of S and is: S X. he l ds (, ) S (4) ecause the gray value assembled in a certain range, there must exists a lot of elements in with value of zero. So, 0,...,0, h, h,..., h, h,0,...,0 k k1 l1 l 1 3 In the formula, ,, (5), h, h,...,, k h k 1 h l1 l, If calculate the histogram using the Equivalent istogram based method, then only S in S1, S, S3 needs to be calculated. According to the triangle inequality character of l norm, the following inequality is tenable:,,,, S S1 S S3 1 3,, S S1 S3 1 3 S S1 1 S3 3 he equal mark in formula (6) comes into existence iff S1 1, S and S 3 3. ere, the temlate matching obtains the global otimal, and the otimal matching based on the original image ( 0 ) and the otimal matching based on Equivalent istogram ( S 0 ) are exactly the same. S (6) Figure 3. he alication of equivalent histogram. Equivalent istogram (Fig.3) and analyze the comlexity about both comutation and memory. A. Imlementation of matching Suose the gray range of temlate is fmin, f max, then the length of the gray range is R' fmax fmin. If is the default setting of the numbers of histogram bins based on the original image which can obtain a comaratively better result, then the figure can be reduce to the following number if carry out the matching rocess based on Equivalent istogram: '= R ' (7) R ere the gray value range described by the i th bins of histogram is as follows: R'( i0 1.5) R'( i0 0.5) Xi ' x x ' 1 ' 1 In the formula, 0 is used to correct the corresonding relation between the bins and the gray value range which can be calculated as follows: = min ' 0 Now the Equivalent istogram of area S is: ' i1 (8) f (9) R ' ( S) h f( ) Xi ', S (10) 415

4 On the basis of the formula (10), the ixels whose gray value does not belong to fmin, f max will not articiate in the histogram and similarity ma calculation. he histogram dimension becomes the ( R' R) of the original, which does not only imrove the comutation efficiency but also reduce the memory sending.. Comlexity analysis he roosed method does not only imrove the matching efficiency to reduce the circulation time but also cut down the sace comlexity to save memory. he following content will comare the algorithmic comlexity of I based method, D based method, ESW model and those who introduced the Equivalent istogram. For I based method, there are twice dimension lus (minus) oerations and one ixel s histogram renew of each ixel when constructing the integral histogram ma. he calculation of the current window s histogram needs three times dimension lus (minus) oerations. he final ste is to calculate the distance of histograms between object temlate and current window. Suose the time consumtion of lus (minus) oeration is t 1, the time consumtion of one ixel s histogram renew is t, and the time consumtion of distance calculation of each bin is t 3, then, the total time consumtion of I based method is: t N 5t t t (11) I 1 3 If calculate the I based on Equivalent istogram (I-E), the histogram s dimension becomes, and only the ixels whose gray value belong to the gray range of object temlate involve in the calculation. We mark the roortion of these ixels to the whole image is, then the total time consumtion of I-E based method is: ma. he whole course byassed the histogram calculation of each ixel. he rerocess ste is just the same as D algorithm, and during the window sliding course, only sr ixels affected the histogram distance (s is a sarseness factor which describe the ratio of the renewed ixels to the boundary). So the comlexity of ESW model based method is ON min, sr. We will not further analyze the total time consumtion of ESW for the reason that it is affected by the factor s and the distance measurement of histogram. If introduce the Equivalent istogram to ESW, the total O N min, sr. comlexity will reduce to I based method is very memory consuming and its memory comlexity is u to N. While the figures of D based method and ESW model are both N, which is less memory consuming than I based method. If introduce the Equivalent istogram, all the memory demand will reduce to the of the original ones. It is clearly that both the factor and are less than 1, so the Equivalent istogram based method effectively reduced the comlexity of both comutation and memory. IV. EXPERIMENAL RESULS o validate the efficiency of the Equivalent istogram, we design and carry out a series of tracking exeriments and ick out three tracking examles to further analyze in this section. he hardware condition during all the exeriments is.10gz basic frequency and G memory sace. he simulation software is MALA 010. Fig.4 lists the object temlates of the test video sequences. Among the temlates, (a) is car, (b) is edestrian, and (c) is face. o be oint out is that all the tracking exeriments in the aer are based on gray histogram. t N 5t t t (1) I-E 1 3 For D based method, the initialization ste needs to calculate the histogram of Nr ixels, the distribution ste needs twice histogram bin renew and twice dimension lus (minus) oerations to form the final similarity ma. he total time consumtion of D based method is: t N t NrN t N t (13) D 1 3 If introduce the Equivalent istogram when calculate the D, the total time consumtion becomes: t N t NrN t N t (14) D-E 1 3 ESW model uses the changed ixels in current window to renew the histogram bins and further renew the similarity Figure 4. Object temlates of three test video sequences. ere gives the color temlates, but actually we only used their gray information. A. racking results I, D and ESW are all imlemented on the basis of the histogram of rectangle area, so the tracking results of these three methods are accurately the same. During the exeriments, firstly, we tested the tracking erformances of I based method, D based method and ESW based method, and then tested these three methods on the basis of Equivalent istogram. We wrote down all the tracking results to do further comarison. Fig.5 shows the tracking results of a car. he to of each art are the similarity mas obtained from the two test method. 416

5 (a) left: I, right: I+E, to: similarity mas, bottom: tracking results (a) left: D, right: D+E, to: similarity mas, bottom: tracking results (b) left: I, right: I+E, to: similarity mas, bottom: tracking results Figure 5. racking results of car. he left are results of I based method (I) and the right are revised I based method (I+E). During each art of this figure, the to two images show the confidences of the temlate, which we called similarity mas, and the bottom two images show the tracking results of two test method. (b) left: D, right: D+E, to: similarity mas, bottom: tracking results Figure 6. racking results of edestrian. he left are results of D based method (D) and the right are revised D based method (D+E). During each art of this figure, the to two images show the similarity mas, and the bottom two images show the tracking results of two test method. We can see from the similarity mas that the locations of real object give the highest similarities. On each frame, the tracking result of I based method is circled by a blue rectangle, while the result of the revised I [9] based method which introduced Equivalent istogram (I+E) is circled by a green rectangle. his revised method cut down the redundant comutation when calculates the histogram of the current window and the distance between two histograms. he results of the two test methods are exactly the same which not only can be seen from the location of the circled rectangles but also roved by the recise osition recorded during the exeriments. Fig.6 shows the tracking results of the method based on distributive histogram (D) [10] and the one revised according to Equivalent istogram (D+E). he to right similarity ma in each art of Fig. 6 reveals the real object (edestrian) with a more distinguishable confidence, which makes the tracking result more reliable. Note that the similarity mas calculated from D and D+E are different. It is the reason that the cutting down of redundant calculation makes the difference. he tracking results indicate that effect of D is exactly the same as that of D+E. Fig.7 lists the results of tracking a face, among which, the left are the ones obtained using efficient histogram based sliding window (ESW) [11], and the right are the ones obtained by the revised ESW which brings in the Equivalent istogram. We can see from the similarity ma that the similarities between two mas are quite different excet the real object. he color of the object (face) assembled in a certain art of the color axes, which reduces the comutation of matching when introducing Equivalent istogram. he next section will treat with this roblem. he tracking results of two test method are exactly the same, which can be seen from the circled rectangles with different colors. 417

6 comutation efficiency. For the first sequence, =0.703, =0.3946, and the run time er frame of I based method is 753 ms, while this figure of modified I based method is 075 ms. It means that the introduction of equivalent imroves the I based method by nearly 5 ercents. More details about the amelioration of erformance are listed in ab.1. ALE I. COMPARISON OF RUN IME ON EAC FRAME EWEEN RADIIONAL MEODS AND EQUIVALEN ISOGRAM ASED MEODS Video sequences car edestrian face Size of frame (a) left: ESW, right: ESW +E, to: similarity mas, bottom: tracking results Size of temlate I 753 ms 763 ms 19 ms I+E 075 ms 504 ms 13 ms D 038 ms 538 ms 157 ms D+E 1514 ms 415 ms 1 ms ESW 1587 ms 399 ms 118 ms ESW+E 103 ms 340 ms 96 ms (b) left: ESW, right: ESW +E, to: similarity mas, bottom: tracking results Figure 7. racking results of face. he left are results of ESW based method (ESW) and the right are revised ESW based method (ESW +E). During each art of this figure, the to two images show the similarity mas, and the bottom two images show the tracking results of two test method. he exeriments in this section indicate that the bringing in of Equivalent istogram makes some changes in similarity ma, however, does no change of the tracking results of I based method, D based method and ESW based method. We will further discuss the erformance changes in resect that bringing in of Equivalent istogram.. Performance analysis here are two factors which affect the erformance of Equivalent istogram, one is the ratio of the object temlate s color range to the original image s color range ( ), and the other is the ratio of the essential ixels to the whole ixels ( ). A smaller means the color of object temlate assembles a narrower range, which may better imrove the marching efficiency. On the other hand, a larger means that the color assembling henomenon of temlate is not obvious. Comared with, has weaker effect on imroving the From the figures in ab.1, we can see that the methods on the basis of Equivalent istogram are faster than the original ones. he average saved comutation time is 31.3%, which means that the Equivalent istogram has imroved the efficiency indeed. According to the analysis in section 3, the factor will basically reflect the amelioration extent; however, the test results are not exactly as what have been discussed in former. he test run times are usually longer than the theory analyzed results. It robably influenced by the initialization ste and the otimization of rogram. All the exeriments in this aer are designed as an ideal situation and the algorithms searched the global otimization through the whole frames. It is known as exhaustive search which is usually very time-consuming. he real tracking alication usually introduces Kalman filter theory to redicate the track and the moving erformances, which can shrink the searching area to further imrove efficiency C. Discussion We demonstrate the validity of our aroach to imrove the efficiency to execute histogram based temlate matching. owever, this aroach makes little sense when the temlate has the same color distribution to the frames. We give a failure examle in Fig. 8. In this case, if we demonstrate the object with a rectangle, then the object and the whole frame share nearly the same color information distribution. Our aroach doesn t work in imroving the calculation efficiency. If we can segment the object out from the rectangle temlate using the ariority information and an effective 418

7 ACKNOWLEDGMEN he authors would like to thank the anonymous reviewers for their many valuable comments and suggestions that heled to imrove both the technical content and the resentation quality of this aer. (a) temlate (left) and its segmentation (right) (b) similarity mas of objects in (a) (c) tracking results of objects in (a) Figure 8. a failure examle and discussion. If we segment the real object from the rectangle temlate, our aroach still works and the imrovement is much more distinct. segmentation algorithm, our aroach still works and the imrovement will be more distinct. We refer to Fig. 8 to give a demonstration of this situation. V. CONCLUSIONS his aer roosed a method to imrove the matching efficiency and reduce the comlexity of histogram matching based visual tracking. It calculates the histogram of both object temlate and the current window according to the distribution of temlate s gray information. Comared with traditional tracking method based on histogram matching, it does not only imrove the matching efficiency to reduce the circulation time but also cut down the sace comlexity to save memory. he tracking exeriments based on Equivalent istogram using 64 bins gray histogram indicates that the introduction of Equivalent istogram has no imact to the tracking result but distinctly imroves the tracking efficiency by 30 to 40 ercents. he imrovement of color histogram based method which introduces the Equivalent istogram will be much more obvious. REFERENCES [1] P. Pan, D. Schonfeld, Visual racking Using igh-order Particle Filtering, IEEE Signal Processing Letters, 8(1): 51-54, 011. [] C.. Shen, J. Kim, and. Z. Wang, Generalized Kernel-ased Visual racking, IEEE ransactions on Circuits and Systems for Video echnology, 0(1): , 010. [3] S. S. eauchemin, and J. L. arron, he comutation of otical Flow, ACM Comuting Surveys, 7(3): , [4] S. Wang,. Z. Ai and K. Z. e, Difference-image based multile motion targets detection and tracking, Chinese Journal of Image and Grahics, 4A(6): , [5] Z. Q. ou, C. Z. an, A background reconstruction algorithm based on ixel intensity classification, Chinese Journal of Software, 16(9): , 005. [6] F. Crow, Summed-area tables for texture maing, ACM Comuter Grahics, 18(3): 07-1, [7] P. Viola and M. Jones, Robust real-time face detection, International Journal of Comuter Vision, 57(): , 004. [8] P. Li, A clustering-based color model and integral images for fast object tracking, Signal Processing: Image Communication, 1 (8): , 006. [9] F. Porikli, Integral histogram: A fast way to extract histograms in Cartesian saces, Proceedings of the 005 IEEE Comuter Society Conference on Comuter Vision and Pattern Recognition(CVPR'05), San Diego, CA, USA, 0-5 Jun. 005, vol.1, , 005. [10] M. Sizintsev, K. G. Deranis and A. ogue, istogram-based search: a comarative study, Proceedings of the 008 IEEE Conference on Comuter Vision and Pattern Recognition(CVPR'08), Anchorage, AK, 3-8 Jun. 008, 1-8, 008 [11] Y. Wei, L. ao, Efficient histogram-based sliding window, Proceedings of the 010 IEEE Conference on Comuter Vision and Pattern Recognition(CVPR'10), San Francisco, CA, Jun. 010, , 010. [1] A. Adam, E. Rivlin and I. Shimshoni, Robust Fragments-based racking using the Integral istogram, Proceedings of the 006 IEEE Comuter Society Conference on Comuter Vision and Pattern Recognition (CVPR'06), 17- Jun. 006, , 006. [13] D. Lowe, Distinctive Image Features from Scale-Invariant Keyoints, International Journal of Comuter Vision, (60): ,