Moving Object Tracking Optimization for High Speed Implementation on FPGA

Transcription

1 Moving Object Tracking Optimization for High Speed Implementation on FPGA Nastaran Asadi Rahebeh Niaraki Asli M.S.Student, Department of Electrical Engineering, University of Guilan, Rasht, Iran Assistant Professor, Department of Electrical Engineering, University of Guilan, Rasht, Iran Abstract: One of the broad application of scientific fields is real time moving object tracking and its implementation on a high speed reconfigarable hardware. This implementation has a positive effect on speed, accuracy, efficiency and reduces design cost. This paper presents anoptimized design of moving object tracking based on color and edge fusion by using mean shift algorithm.the partitioned synthesizable design simulate written by VHDL carry out on Xilinx ISE14.2 and implemented on XC5SX35 FPGA. Results show the implemented tracking system has a good efficiency on speed and accuracy. difficult, because these operators cover large area of chip. So an incorrect design on the ASIC may be make more unnecessary gates, which this will slow down the system. Implementation of image processing algorithm on FPGA can achieve a system with better efficiency and dissipate less power and resources [12]. Hardware implementation is suitable solution for increase of speed desired tracking. Keywords: Target, Tracking, Mean Shift Algorithm, Feature Fusion, FPGA. 1. INTRODUCTION Investigation of the moving objects in the scene is one of the most important and applicable branches of machine vision. Perception, interpretation and applying of information related to movement can be used in various fields such as supervision and tendance systems [1], military industry [2], sports [3] and other similar systems. In previous research different methods have been proposed to track moving objects in image sequences. Mean shift algorithm is one of the proposed methods [4,5]. The initial mean shift algorithm which is based on color features, is used similarity between the color histogram of the current frame with the candidate target histogram model, to find the location of object in the current frame. This algorithm in many circumstances including partial occlusions has acceptable performance [6]. Mean shift algorithm as other tracking methods which using color feature [7], has low computation load and is independent of rotation and deformation of the object. But when there are multiple objects with the same color in the scene, or if the ambient light is a lot of changes, tracking based on initial mean shift algorithm failed [8]. To overcome the above problems, the mean shift algorithm based on the combination of color and edge features has been presented [9-11]. Hierarchical design of mean shift algorithm based on color and edge feature fusion is includes histogram, Bhattacharya coefficient, center calculation and edge extraction blocks, Each of these blocks are made from small mathematical operations such as addition, multiplication, division and square root. Until tracking process is performed on a software such as MATLAB, it is relatively easy task. But when the large number of mathematical operations such as multiplication, division and square root are implemented on ASIC chips, it is 2. OPTIMIZATION MEAN SHIFT ALGORITHM In this section, the tracking method based on mean shift algorithm and the combination of color and edge features is examined. A. Color histogram model Firstly, the model of the target is constructed using weights histogram that extract color information [13]. The target model is pixel location that centered at y0 related to zero center. Color histogram q c for the target model is calculated by using equation (1) for the target model: q Cc n k x δ[bc x uc ] (1) c In this equation, { },, is normalized pixelpositions in a defined area. Function k (x) attributes an especial weight to a pixel based on its distance to the central pixel. So that if the pixel is for from the center its weight is smaller. Cc Constant is defined to provide a clause and δ is the kronecker delta function. M column histograms are used for abate the computation. b c connect any to the related histogram column. In the next frames, a target candidate model is defined to centered y, which it obtain from equation (2): p c y Cc n k y δ[bc uc ] (2) B. Edge histogram model At the first step, the RGB image is converted to grayscale according to equation (3) for edge extraction and the edge image is extracted by calculating the gradient of pixels in the horizontal and vertical direction. Then, the mean shift algorithm is performed on the edge extracted. Then, changes in the pixels values slope in the horizontal and vertical direction is calculated by the gradient at the point (x,y). Absolute value of the gradient can be calculated from equation (4): (3) (4) G x, y G x, y + G x, y In this equationg, and G, are gradients in the horizontal and vertical direction, respectively.equation (5) shows the histogram for target model: 177

2 [ ] (5) { the normalized pixel location as the target model. beconnects any to the related histogram column. To provide a clause, Ce constant is defined. In the next frames, a target candidate model is defined centered y using kernel function, by equation (6): [ ] (6) },, is C. Target tracking After calculating the histogram of the target reference and target candidate for color and edge features, the location of target in the next frame is searched around the location of target in the previous frame and the must similarity is estimates as a new location of the target. Bhattacharya coefficient is usedfor calculating similarity for color and edge features equation (7.a) and (7.b): [, ] [, ] (7.a) (7.b) Searching for the target position in a new frame starts from the target position in the previous frame (0). Equation (8) is obtained from taylor expansion of Bhattacharya coefficient around p y value: [, ] + [, ] + (8) Simple from of equation is desired by substitution of p y in the equation (8): Where: (9) [ ] (10) [ ] (11) To have the most similarity, or in other words, to get the most Bhattacharya coefficient, the second part of the equation (9) have to be maximum mean shift algorithm is used to evaluation of this equation maximum value. In this algorithm, the target center moves from the current position to the new position using equation (11): 3. DESIGN OF MEAN SHIFT ALGORITHM BY HARDWARE DESCRIPTION LANGUAGE (VHDL) After getting video from the source, since the video is as continuous frames, current frame have to be stored in a memory as informative data for next processing. The main blocks of processing unit includes: 1) Mean shiftalgorithm based on color feature block, 2) mean shift algorithm based on edge block and 3) Extraction weight coefficient unit for feature fusion. The procedure of color and edge mean shift algorithm are similar however, the edge mean shift algorithm and tracking process is carried out on extracted edge image. Figure (1) shows the mean shift algorithm block diagram. Based on target center in horizontal direction (center_x), target center in vertical direction (center_y), target window length (hx), target window width (hy), the number of histogram columns (bin) and the image frame, a rectangular area is considered around the target. Then kernel function value is calculated in this area and the reference histogram (qu) is derived for the first image. Histogram (qu) and other inputs are transferred to the mean shift block. In order object tracking in next farame, in this block, target candidate histogram pu(c0) are calculated using the values of the old center c0(center_x,center_y)for the second frame. Then, Bhattacharya coefficients and weight array are calculated using qu and pu(c0). The new center c1(center_x,center_y) is computed using the weighted array for the second frame. Since the calculating center is the returning process and it is done in a loop, histogram of new target candidate (pu(c0)) and new Bhattacharya coefficient (ρ2) should be calculated using new center value and target histogram (qu) and the histogram of new target candidate (pu(c1)). To get final target position in new frame 1values have to be combined with suitable coefficient according to equation (12): (12) Where 1cand 1e are the position of the target in the new frames based on color and edge features and αc and αeare the effectiveness coefficients of color and edge features in the final position. αc and αe are calculated by equation (13):, ] [, ] [, ] [, ] [ + + (13.a), ] [ (13.b), ] [ Fig.1. Block Diagram of The Mean Shift Algorithm A. Image Edge Extraction The block diagram of image edge extraction is shown in figure 2. The edges extraction block is formed by two blocks, one for the color space conversion and another for the edge detection operation. First, the frame of image sent to edge extraction unit:in the transforming color space block each color components of image (R, G and B) multiply to proper coefficient. The RGB image transforms to the gray scale. Then the gray scale data is delivered to edge detection block, in which the gradient pixels of image extracts in both vertical and horizontal direction. 178

3 coordinates of each pixel from the image matrix. Normalized row and column arrays are stored in the memory. Then, the distance of each pixel from the image is determined by the sum of normalized rows and columns squares. If the distance is greater than or equal to one, the kernel value is placed zero at position i and j and otherwise, the kernel is the value of one. Fig.2. Block Diagram of the Image Edges Extraction B. Histogram Model The histogram block diagram is shown in figure 3. In optimal mean shift algorithm each of color and edge features will extract based on histogram from a rectangular area separately. Histogram calculation is based on two operations, the target area determination and kernel function calculation. Fig.5. Block Diagram of the Kernel Function C. Bhattacharya Coefficient As shown in figure 6, target reference histogram (q ) and target candidate (p ) are two required inputsto calculate Bhattacharya coefficient.if the distribution of both target reference histogram and target candidate histogram resemble to each other, Bhattacharya increases. ρ1 shows two matching uniform normalized distribution. Fig.3. Histogram Block Diagram Determination of target area: In the determination of target area block, shown in figure 4, a rectangular area is drawn around the target in the image frame by considering Center_x, Center_y, hx and hy, inputs. After this step, only Pixels in the desired range are stored in the memory and the others operations take place in this area. Fig.6. Block Diagram Bhattacharya Coefficient D. Center of Target After calculation of the target reference model (q u) and the target candidate (pu(c0)), the suitable weight of each pixel (Figure 7) is calculated. The new center (x1, y1) is obtained from the weight array calculation of the new frames. Figure 8 shows the block diagram of the computing center. Fig.4. Block Diagram of the Target Area Determination Calculation kernel function: Figure 5 presents the block diagram of kernel function. To kernel evaluation, first, rows and columns of image matrix are normalized by subtraction of Center_x and Center_y from i and j values and division of their conclusion on hx and hy. i and j are the location Fig.7. Block Diagram of Weight Array 179

4 E. Extraction of weight coefficients for features fusion After calculating the new center c1 for each feature, to find target final position in new frame, C values, come from color and image feature, have to be combined with appropriate coefficients. This coefficients determine the effectiveness of each feature in the final position. Fig.8. The Block Diagram Of The Center Calculation Center calculation is a returnable process in a loop. The number of repetitions of this loop depends to Bhattacharya coefficient of the second frame for the old center (ρ1) and Bhattacharya coefficient of the second frame for the new center (ρ2). In this way, ρ1 and ρ2 values are compared with each other. If ρ1 is greater than ρ2, loop is broken and c0 is considered as the final center. If ρ1 is smaller than ρ2, sum of squares x1-x0 and y1-y0 is calculated, if the result is smaller than one, loop is broken and c0 is considered as the final center. But if the result is greater than one, loop repeats and the maximum number of loop iterations, is 20 times. Fig.9. The block diagram of the iterative loop Fig.10. Block Diagram of the Weight Coefficients for Color and Edge Features 4. EXPERIMENTAL RESULTS Block diagram structures designed in previous section are coded in VHDL language via Xilinx ISE14.2 software and synthesized on XC5SX35. Simulation results by software Isim are shown in Figure 9. Results for the reference target, the target candidate, the weight matrix and new center is shown in Fig. 11-A, 11-B, 11-C and 11D is shown. Figure 11-a and 11-b show the values of qu and pu respectively, which are in 16-bit binary format, all bits are combined to represent a fractional number. Figure 11-c shows the provided amount for the weight matrix. Here wi_sqt index of variable is composed of 8 bits. The 4 bits primary show integer part and the 4 bits remaining show fractional part. Figure 11-d shows the values of the new center, that center_new11 and center_new22 areshow integer part and the 8 bits remaining show fractional part. Which these values correspond precisely with the results obtained from Matlab. Fig.11. A. Simulation Results For The Target Reference Values 180

5 Fig.11. B. Simulation Results for the Target Candidate Values Fig.11. C. Simulation Results for the Weight Array Fig.11. D. Simulation Results for New Center Values 181

6 5. CONCLUSION In this paper, we presented a top down hierarchical design for moving object tracking based on mean shift algorithm and color and edgefeatures fusion.thesynthesizable system designed in VHDL and simulated by Xilinx ISE14.2 software, implemented on XC5SX35 FPGA. the simulation results are completely correspond with Matlab results. Which shows the correct operation of the system. The experimental result shown the implemented object tracking system has a good efficiency on speed and accuracy of tracking. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] B. Lei, LQ. Xu, Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management, Pattern Recognition Letters, vol. 27, 2006, pp K. Huang, L. Wang, T. Tan, and S. Maybank, A real-time object detecting and tracking system for outdoor night surveillance, Pattern Recognition, vol. 41, 2008, pp J. Ren, J. Orwell, G.A. Jones, and M. Xu, Tracking the soccer ball using multiple fixed cameras, Computer Vision and Image Understanding, vol.113, 2009, pp D. Comaniciu, V. Ramesh, and P. Meer, Real-time tracking of non-rigid objects using mean shift, Computer Vision and Pattern Recognition, IEEE Conference, vol. 2, 2000, pp D. Comaniciu, V. Ramesh, and P. Meer, Kernel-based object tracking, Pattern Analysis and Machine Intelligence, IEEE Transactions, vol. 25, 2003, pp D. Comaniciu, V. Ramesh, and P. Meer, Real-time tracking of non-rigid objects using mean shift, IEEE Conference of Computer Vision and Pattern Recognition, vol. 2, 2000, pp H. Wang, C. Liu, M. Tang, Multiple feature fusion for tracking of moving objects in video surveillance, International Conference Of Computational Intelligence and Security, IEEE, vol. 1, 2008, pp D. Ross, J. Lim, and MH. Yang, Adaptive probabilistic visual tracking with incremental subspace update, Computer VisionECCV, Springer, 2004, pp M.J. Deilamani, RN. Asli, Moving Object Tracking Based on Mean Shift Algorithm and Features Fusion, International symposium of Artificial Intelligence and Signal Processing (AISP), IEEE, 2011, pp W. Liu, Y.J. Zhang, Real time object tracking using fused color and edge cues, International Symposium of Signal Processing and Its Applications, 2007, pp M. Dixit, K.S. Venkatesh, Combining Edge and Color Features for Tracking Partially Occluded Humans, Computer Vision ACCV, Springer, 2009, pp L.N. Elkhatib, F.A. Hussin, L. Xia, and P. Sebastian, An optimal design of moving objects tracking algorithm on FPGA, International Conference of Intelligent and Advanced Systems (ICIAS), vol. 2, 2012, pp J. Ning, L. Zhang, D. Zhang, and C. Wu, Robuts mean-shift tracking with corrected background-weighted histogram, IET Computer Vision, vol. 6, 2012, pp P.Y. Hsiao, L.T. Li, C.H. Chen, S.W. Chen, and S.J. Chen, FPGA architecture design of parameter-adaptive real-time image processing system for edge detection, Emerging Information Technology Conference, 2005, pp