A dynamic background subtraction method for detecting walkers using mobile stereo-camera

Transcription

1 A dynamic ackground sutraction method for detecting walkers using moile stereo-camera Masaki Kasahara 1 and Hiroshi Hanaizumi 1 1 Hosei University Graduate School of Computer and Information Sciences Tokyo Japan Astract - A method Dynamic Background Sutraction (DBS) was proposed for detecting walkers running out into the street using a stereo-camera. The method was ased on the fact that front street scene extended from a point as the automoile moving. Analyzing the scene extensions current scene was precisely predicted from the previous one. The difference etween the two scenes indicated walkers movements on a street in a video frame interval. The stereocamera provided us depth information for the scene prediction and that for distance to walker. The proposed method was characterized y its simplicity in principle and high potentiality in easily realizing the system with low cost. In this paper the principle and the procedures of the method were descried. Some experimental results were also shown. Keywords: dynamic ackground sutraction; walker detection; moile stereo-camera; scene prediction; depth image; 1 Introduction Recently numer of traffic accidents etween walkers and automoiles has een increasing and main causes of the accidents were drivers looking aside and/or carelessness [1]. In order to prevent these accidents an automated recognition system has een developed for detecting walkers running out into the street and for making alarm to driver or for stopping automoile [2]. A millimeter-wave radar system has already developed [3]. The radar system however was too expensive to spread. In the scene recognition walkers had various kinds of appearances in color as their clothes skin hair and so on. Thus walkers shape information was mainly used for their detection. Histogram of Gradient (HOG) or Support Vector Machine (SVM) [4]-[5] have used with template matching [6]- [7] for the walker detection in the moile camera images. These methods however sometimes extracted ackground oects as walkers. In the point of view of accurate extraction of walkers a ackground sutraction method had excellent performance in separating moving walkers from static ackground [8]-[9]. In case of moile camera the ackground sutraction method extracts ackground as moving oects ecause the ackground is not static. Thus highly accurate methods with high cost performance have een required. Here we proposed a method for developing a one of those systems using moile stereo camera. The asis of the method is the conventional ackground sutraction ut prediction images are used for the sutraction i.e. the sutraction is performed etween the current video scene and that predicted from the previous one. We call the method as Dynamic Background Sutraction (DBS). Distances among oects and automoile which are given y a stereo camera are needed for otaining prediction image with high accuracy. 2 Principle and procedures 2.1 Infinite point Suppose that a front view camera is mounted on an automoile moving in a straight line. We see that video scenes spread radially from a point. We call the point as Infinite Point. The point is not necessarily identical with the center of the video scene. Therefore the infinite point should e determined y captured video images. The position of the point depends on camera direction setting. The Infinite Point is otained from sequential video scenes in which some groups of corresponding points make some radial lines crossing the Infinite Point. Before the processing image distortions caused y camera lenses are compensated y the camera caliration [10]-[11]. Figure 1 shows for example 4 groups of corresponding points in some sequential video scenes. As corresponding points in a group stand in a line passing the infinite point I we can find the line using a least squares method so that sum of square of length of the perpendicular from points in the group to the line takes the minimum value S as S min a x y y x M N 1 i1 y x 2 a xi yi 2 2 a x y x y x x where and are coordinates of the i-th point in the -th group and gravity center of points in the -th group x and y coordinates of the infinite point. As least squares line always passes through the gravity center of the group we put a candidate infinite point I at the center of the image with initial lines passing through oth I and gravity center of each (1)

2 Fig. 1 Determination of the Infinite Point. point group (dot lines in Fig.1). Then we scan I so that S takes the minimum. When S reaches to the minimum I is identical to the infinite point I as shown in Fig Background extension Moile camera shows us ackground of front street scene extends from the infinite point as automoile moving. In order to predict current scene from the previous one precisely we need to analyze the position change etween the current and the previous images. Figure 2 shows geometry of the camera current position of a ackground point W and its previous position W 0. We oserve W and W 0 as image points P and P 0 respectively. We assume that direction of camera movement is identical to its optical axis. So the infinite point is given as the cross point etween the optical axis and the image. In order to predict the current scene from the previous one precisely we need to know the ratio for all ackground points. In Fig.2 relation etween the ackground point W 0 and the image point P 0 is descried as IP 0 L f Z (2) IP L f Z Z where corresponds to movement of automoile Z distance etween camera and the ackground point. The extension rate k is derived from eq.(2) as IP Z Z k (3) IP Z Z Z vt 0 where v means the velocity of the automoile time interval for video frame rate. We see that the extension rate k depends on the distance Z and the velocity v. Figure 3 shows the extension rate for the cominations of distance Z and velocity v (we assume =1/30 [s]). Figure 3 indicates that the extension rate k changes slowly in the far area ut rapidly in the near area. In the point of view of rough detection of oects we can assume k is a constant for example in the Fig. 2 Movement of ackground points. Fig. 3 Extension rate with distance Z. range over 20m from the camera. Though the dynamic ackground sutraction gives us some noisy residuals we can separate a larger oect from the noisy residuals. But in the near region increase of k yields inseparale larger residuals. That is the reason why we introduce a stereo camera. 2.3 Depth map and scene prediction In order to predict the position of a near oect in the ackground from previous scene with high accuracy distance etween camera and the oect is indispensale. Figure 4 shows schematic diagram of a stereo camera system. On the assumption that optical axes of left and right cameras are perfectly parallel the distance Z etween camera and an oect in the ackground is otained from parallax d as Z fc d fc x l x r where x l and x r are the position of image point of the oect f focal length aseline and c a constant. A line-y-line lock matching algorithm is used for finding the corresponding point pair of ackground oects. Using eqs.(2) and (4) the extension rate k is calculated as a function of the velocity of (4)

3 (a) Previous scene (left) () Previous scene (right) Fig. 4 A stereo camera system. automoile v. Thus we can predict the position of the point P from that of a point P 0 in the previous image (see Fig. 2). 2.4 Procedures of DBS The procedures of our proposed method DBS are graphically shown in Figs. 5 (a)-(h). Firstly we take stereo pair images as the previous scene as shown in (a) and (). A lock matching algorithm is applied to the images and depth map (c) is otained. In the depth map miss-matching areas such as sky are suppressed as noises y filling zeros. We can extract a range elt indicating assigned depth region as shown in (d). The range elt will e used for setting a special watch area. The image (a) is extended y multiplying k calculated y using the depth map and the predicted scene (e) is otained. The image (e) is the dynamic ackground we proposed. The predicted image is sutracted from the current video scene (f) (left image of stereo-pair). Residual image (g) indicates most of predicted ackground areas are removed y the sutraction almost the perfectly. Conventional ackground sutraction i.e. sutraction image (a) from (f) gives us larger residual especially in a near field as shown in (h). 3 Experimental results In order to evaluate the performance of the proposed method DBS we applied DBS to 2 measured data sets. One was measurement using a rail camera and the other that of actual automoile camera. The rail camera was a hand-made device on which stereo camera smoothly moved with keeping the camera direction as shown Fig.6. The rail camera realized an ideal measurement without camera viration. The stereo camera was attached to a tripod and fixed etween front glass and passenger seat. In the measurement using the automoile camera viration of automoile in driving might affect the quality of video data. In oth measurements we used a stereo camera FINEPIX REAL 3D produced y FUJIFILM; circle in Fig.6 indicates the camera. The resolution of image was 640x480. The stereo camera had two lenses for stereo vision (c) Depth map (left) (d) Range elt (18~22m ) (g) Result of DBS (h) Difference etween (f) and (a) Fig. 5 Graphical procedure of DBS with ase line of 75mm. The focal length was 18.9mm and frame rate was 30 fps. All of measured images were processed y a note PC (intel Core i5-2520m CPU 2.50GHz ). The performance was evaluated y an index; remaining rate R r as M R r B where B was numer of pixels in the ackground to e removed M those of sutraction residuals. When the sutraction results fell down under a threshold we regarded that the ackground was removed y the sutraction. 3.1 Rail camera measurement Rail camera was set on a road and a man stood 20m away from the camera. The distance 20m was the safe raking distance of 40km/h automoile. The previous scene was taken. Assuming velocity of the automoile was 40km/h we moved rail camera 37cm forward along the road and the man moved across the road 3.7cm correspondingly to walking speed 4km/h. Then we took the current scene. Figure 7 (a) shows the man on the road; ellipse in the scene indicates the man standing. The prediction image was shown in Fig.7 () where we filled zeros into stereo matching failure areas. Sutraction results were shown in (c) and (d). Though conventional (5)

4 3.2 Automoile camera measurement Figures 8(a)-(d) shows results of the experiments using an actual automoile camera. In the results processed y CBS we found larger remaining especially in near region as shown in (c). The remaining was well reduced in the result processed y DBS. The remaining rates were listed in Tale 2. Processing time of CBS depended on numer of pixels to e processed ut DBS spent much time in stereo matching. Fig.6 Rail camera. ackground sutraction (CBS) gave us larger residuals (c) the proposed DBS gave fewer ones (d). The validity of DBS was confirmed. Tale 1 indicates the quantitative performances of CBS and DBS. DBS had higher performance; one third remaining rate ut DBS needed aout 3 times longer processing time than CBS did. (a)current scene (left image) ()ackground predicted (c) result processed y CBS (d) one y DBS Fig. 8 Actual automoile camera results. (a) Current scene (left image) () predicted ackground Tale 2 Remaining rate and processing time of experiment in automoile Method Remaining rate [%] Processing time [ms] CBS DBS (c)sutraction without prediction (d) with prediction Fig.7 Experimental results y using a rail camera. Tale 1 Remaining rate and processing time for rail camera data Method Remaining rate [%] Processing time [ms] CBS DBS Fig.9 shows another results processed y the proposed method. Oncoming automoile on opposite side was well detected (see white circle right). Preceding automoile is also detected (white circle left). According to those results of experiments the propose method Dynamic Background Sutraction DBS reduced misdetection of ackground oects than conventional method CBS. However the reduction of the sutraction residuals was insufficient. We considered that main cause of the insufficient performance was stereo matching prolem.

5 Fig.9 Detection of moving target instead of walkers running into the street. 4 Conclusions We proposed method DBS for dynamic ackground sutraction etween sequential moile video images to detect walkers running out to the street. We improved accuracy of the sutraction y using distance information from stereocamera. In order to remove static ackground we needed to predict a current scene from the previous one. Depth image otained y a stereo camera enaled us to predict ackground dynamically with high accuracy. To improve the performance in ackground reduction to realize faster processing to analyze system ehavior in driving a curve and/or camera viration due to umpy road and to design a practical system are suects for a future study. 5 References [1] The Cainet Office Traffic safety white paper ver.2012 vol.1 part.1 section.1 Nikkei printing pp.18 Japan [2] T. Gandhi and M. M. Trivrdi Pedestrian collision avoidance systems: A survey of computer vision ased recent studies in Proceedings of the IEEE Intelligent Transportation Systems Conference 2006 pp [3] M. Skutek M. Mekhaiel G. Wanielik A PreCrash System ased on Radar for Automotive Applicaions Proc. IEEE Intelligent Vehicles Symposium pp June [4] N. Dalal and B. Triggs Histograms of oriented gradients for human detection IEEE Computer Society Conference on Computer Vision and Pattern Recognition pp [5] A. Shashua Y. Gdalyahu and G. Hayun Pedestrian detection for driving assistance systems: Single-flame classification and system level performance Intelligent Vehicles Symposium IEEE pp [6] D.M. Gavrila Pedestrian detection from a moving vehicle Proc. European Conference on Computer Vision pp Dulin Ireland [7] A. Broggi M. Bertozzi A. Fascioli and M. Sechi Shape-ased pedestrian detection Proc. IEEE Intelligent Vehicles Symposium pp Dearon USA [8] S. Cheungm C. Kamath Roust Background Sutraction with Foreground Validation for Uran Traffic Video EURASIP Journal on Applied Signal Processing Volume January [9] Massimo Piccardi Background sutraction techniques: a review Proc. IEEE 2004 IEEE International Comference on Systems Man and Cyernetics Vol. 4 pp Octoer [10] J.g. Frayer and D.C. Brown Lens distortion for closerange photogrammetry Photogrammetric Engineering and Remote Sensing vol.52 pp [11] Z.Zhang A flexile new technique for camera caliration IEEE transactions on Pattern Analysis and machine Intelligence vol.22 pp [12] M. J. Black P. Andan A framework for the roust estimation of optical flow Fourth International Conference on Computer Vision pp May 1993