Pedestrian Detection Using Multi-layer LIDAR

1 st International Conference on Transportation Infrastructure and Materials (ICTIM 2016) ISBN: 978-1-60595-367-0 Pedestrian Detection Using Multi-layer LIDAR Mingfang Zhang 1, Yuping Lu 2 and Tong Liu 3 1 School of Automobile, Chang an University, Xi an, China, 710064; 957631035@qq.com 2 School of Automobile, Chang an University, Xi an, China, 710064; 1127409795@qq.com 3 School of Automobile, Chang an University, Xi an, China, 710064; willie1020@163.com ABSTRACT: To avoid the potential ris of collision between pedestrians and egovehicle, we propose a new pedestrian detection algorithm based on multi-layer LIDAR. First, the points are clustered based on point-distance segmentation, and non-pedestrian clusters are eliminated with physical attributes of pedestrian. Then pedestrian probability is calculated using Bayes rules, and non-parametric ernel density lielihood function estimation model is established to estimate the pose of pedestrian. Moreover, linear Kalman filter is utilized to trac the pedestrian which improves the detection accuracy at part occlusion situation. The experiment validates that our algorithm can accurately detect pedestrians in front of ego-vehicle, and the recognition results based on the fusion of detection and tracing processes is better than those from single pedestrian attribute detection. INTRODUCTION Efficient pedestrian detection method is beneficial to reduce the collision probability among the pedestrians and vehicles in urban environment, thus developing robust pedestrian protection system is an urgent tas in the active safety area. Now the main sensors applied for pedestrian detection are vision and Lidar. The image provides rich depth information, but it is susceptible to be disturbed by the light and bad weather (Cheng 2013; Garcia 2014). Lidar has the advantages of fast processing ability and high measurement level, especially insusceptible to bad environmental condition. The algorithm of pedestrian detection using Lidar includes three categories: the detection based on geometry feature (Gidel 2008; Grassi 2011), the detection based on occupancy grid (Schutz 2012), and the detection based on behavior (Fayad 2007; Gate 2009). The detection algorithm based on geometrical model uses the specific bounding box to match with the target measurement for target recognition, but the constraint is that pedestrians with part occlusion is hard to detect, and the recognition accuracy is uncertain when the point distribution of the observed target changes. Although the detection algorithm based on occupancy grid detect the moving object fast, the category of the target is hard to determine. The detection algorithm based on behavior can not recognize every object in the scene, thus this heuristic method is employed to classify the static or moving objects before the occlusion happened. To detect the pedestrian robustly, even in the occlusion situations, in this paper we utilizes multi-layer Lidar to detect the pedestrians in close range, and uses object tracing to ensure the consistency of the occluded pedestrian detection result. First, all the discrete points are clustered, pedestrians and non-pedestrians are segmented based on the physical attribute of motion parameters. The pedestrian probability is calculated using Bayes rules in combination with the width and velocity information. Second, non-parametric unsupervised estimation model is built to mae the information fusion of four scanning 575

surfaces, and linear Kalman filer is employed to trac the pedestrian and further enhance the detection accuracy. The pitching compensation of four scanning surfaces can effectively solve the bump and vibration problems. Finally, the test in real scenario is conducted to demonstrate the performance of our method with multi-layer Lidar. PEDESTRIAN DETECTION IBEO LUX multi-layer Lidar (Figure 1) is installed on the front of test car at the height of 60cm for pedestrian detection. Its scanning frequency is 12.5Hz and angle resolution is 0.1 degree. It scans the front sector region on four different surfaces. Figure 2 shows the laser points on four surfaces in Cartesian coordinate system during one scan. The general structure of pedestrian detection in this paper is shown in Figure 3. In the module of clustering and segmentation, interest region is extracted based on the distance rule and the origin scattered points are filtered to improve the acquisition of the target object. In feature extraction module, we segment pedestrian from non-pedestrian objects coarsely and estimate the pedestrian probability as well as the pose with pedestrian physical property. In the classification module, we use the tracing result to correct the detection accuracy further. Figure 1. LUX multilayer Lidar. Figure 2. Scan data from four layers. Raw data from LUX Clustering and segmentation Feature extraction Pedestrian classification Figure 3. Framewor of pedestrian detection. Data Pre-processing Multi-layer Lidar emits the laser points along four scan surfaces vertically. In order to extract the candidate pedestrian clusters, we project the raw point cloud onto 2D plane and remove noise points. The candidate pedestrians are extracted with distance-based clustering method. Then we tacle with the individual cluster which belongs to the same object. The details for clustering are as follows: Step 1: The raw point cloud set is denoted as P1 ( m, n) ( xm, n, ym, n), where m is the index of scanning surface, n is the number of laser points on each surface, m 1,2,3,4. 576

And ( xm, n, ym, n) is Cartesian coordinate of the n th point located in the m th plane. Project all the points onto 2D horizontal plane. Step 2: Segment the point cloud on horizontal plane into several cluster based on Euclidean distance D( ri, ri 1) between two consecutive points. According to the cosine law, D( ri, ri 1) is calculated as D( r, r ) r r 2r r cos (1) 2 2 i i 1 i i 1 i i 1 If D( ri, ri 1) D0, these two points belong to the same cluster, otherwise two cluster. D is set as (Mendes 2004): 0 D 2(1 cos ) min{ ri, ri 1} C cot [cos( 2) sin( 2)] 0 0 where ri 1 and r i are respectively the distance between the projective point and Lidar for two adjacent scanning points, denotes the parameter to reduce the effect of segmentation, denotes the angle resolution of Lidar, and C 0 denotes the vertical error of Lidar. Course Detection Mixed traffic exists in urban environment with multiple participants (bus, car, bicycle and pedestrian) and we obtained the clusters including different objects from the preprocessing. We need to extract the interest region containing the target objects with the feature of pedestrian. If the motion of two legs is only employed, it is difficult to detect the pedestrian from complex outdoor environment. Therefore, we combine the motion feature with the shape characteristics and mae a box model for pedestrian. Consider inherent attribute of the fitting rectangle, such as the height, width and moving range of legs, we remove the unmatched clusters and retain the cluster which has the approximate shape feature of pedestrian. Assumed pedestrian wals with the normal speed, the maximum range of legs in waling state should be that the width is shorter than 75cm and longer than 10cm. when laser points on four scanning surfaces project on the pedestrian, the top two plane mainly focus on the top of pedestrian while the lower two surfaces scan the legs of pedestrian. The largest width of two middle scanning surfaces during each frame is regarded as the width of pedestrian. For the inetic characteristics, the width of pedestrian presents normal distribution, while the speed shows discrete uniform distribution. Next, Bayesian discriminant analysis method is utilized for coarse detection of pedestrian. At time t, set the feature vector as Z (, ) T t W V. Width W and velocity V are independent variables, the prior probabilities of pedestrian and non-pedestrian are respectively P( ped ) and P( noped ), and the class-conditional probability are P( Zt ped ) and P( Zt noped ). Set P( noped Zt) P( ped Zt) 1 and P( Zt ped ) P( W ped ) P( V ped ). The preliminary pedestrian detection problem is formulated as maximizing a posterior P( ped Z t ) of an interpretation for both shape and motion features, which denotes the (2) 577

probability that the detected object possesses the feature Z t. Given the set of feature vector Zt over T frames: 1 P( ped Zt) P( Zt ped ) P( ped ) (3) where P( Z ped ) P( ped ) P( Z noped ) P( noped ). For each sample, if t t P( ped Z t ) is larger than the predefined threshold, it will be classified as pedestrian. The threshold is decided by the expectation of the classification result. Pose Estimation The main advantage of multi-layer Lidar is that the raw point cloud from three scanning surfaces help filter the wrong detection data on the other single surface, and also help improve the detection accuracy as a whole. The multi-layer pitching compensation technology provides the possibility of correct detection in the case of part occlusion. After coarse segmentation of pedestrian and non-pedestrian is conducted, we employ ernel probability density function method to estimate the pose of the detected pedestrian and calculate the pedestrian probability density on each scanning surface. The objective of pose estimation is to determine whether each laser point is the center of pose or not and prepare for the tracing tas. Set each laser point of the pedestrian cluster as discrete random variable 1, 2, where 1 and 2 are Bernoulli variables. If the laser point is the center of pose, 1 ; otherwise, 2. The lielihood function of the probability that the laser point is the center of pose pp ( m ) in mth scanning surface is determined by both the sensor characteristics and the distance between the sensor and the target objects, and the formula is as follows: N 1 pp ( m ) (, 1) (4) N m i 1 where (, 1) is ernel function of Parzen window model to estimate the affected region using neighboring points to correct the current point; N denotes the total number of laser points in the scene; N m denotes the total number of laser points located in the pedestrian cluster on the m th scanning surface. The laser point with the maximum probability pp ( m ) is selected as the center of pose on the m th scanning surface. All the points belonging to the center of pose on different scanning surface are projected to the horizontal surface and integrated to get the final fusion pose of the pedestrian. Pedestrian Tracing During the pedestrian detection process, the measurement error of Lidar sensor cannot be avoided. Note that pedestrian occlusion frequently occurs in real road environment, and the measurement error of the gravity center exists. Therefore, the pose of moving pedestrian should be filtered and traced to improve the detection accuracy. Assumed the prior nowledge of pedestrian s trajectory is unnown, the direction of motion and the velocity are also unnown. Consider the occlusion situation in the detection process, the 578

inetic state at last moment is utilized to predict the next moment, so we use linear Kalman filter to cope with pedestrian tracing problem. Define state vector X (O, x,..., x ) contains the number of pedestrians and the feature information. 1, N, O denotes the discrete random variable, xn, ( pn,, vn,, IN, ) denotes the state of pedestrian N, pn, denotes the pose of pedestrian, v N, denotes the velocity of pedestrian, and IN, denotes the classification frames as well as the number of reflected points. The state equation for pedestrian is: X AX v (5) 1 1 1 where system matrix is A 1 0, state noise v is Gaussian function sequence with mean 0, and its variance is determined by the maximum range of legs. Observation function is Z BX w, where Z denotes the observation vector at time, observation matrix is B [1 0], state noise w is Gaussian function measurement sequence with mean 0, and its variance is determined by maximum measurement error of the Lidar. Observation update equation is as follows: Xˆ ˆ ˆ 1 X H 1(Z 1 B X ) T T 1 H 1 P 1B [ BP 1B r] (6) P 1 (I H 1B) P 1 where Xˆ 1 is the estimated value for system state vector, H 1 is Kalman filter gain, P is the covariance of Kalman filter. 1 EXPERIMENT RESULT To evaluate the effectiveness of the proposed algorithm, we conduct several tests in campus of Chang an University from 9:00am to 3:00pm, including the intersection and main street scenes. More than 7200 frames of the collected data are employed to test our algorithm. Figure 4 shows one test scenario at the street in front of School of automobile building, as well as actual scan measurement from Lidar. We compare the usual pedestrian detection method in (Naatsubo 2010) with our algorithm, and the result is shown in Figure 5. The proposed pedestrian model combines the specific physical attributes with the change of the point cloud distribution, and the fusion process of coarse detection and tracing correction maes a better detection accuracy than the method using simple physical attributes in (Naatsubo 2010). The reason for false detection from the method in (Naatsubo 2010) can be interpreted as the part occlusion from the neighboring objects or the misclassification of slowly moving bicycles. In our algorithm, both the motion information and shape feature are employed for pedestrian detection, and the tracing process figures out the part occlusion to improve the detection accuracy. Furthermore, the pitching compensation technology of multi-layer Lidar overcomes the bump and vibration of moving ego-vehicle, as well as the occasional occlusion situations. 579

(a) Real scenario with camera (b) Scan point measurement, 1 roadside green belt,2 deceleration strip,3 pedestrian A and its trajectory,4 vehicle stopping at roadside Figure 4. Lidar Scan Measurement Scenario. 580

Figure 5. ROC curve for the test result comparison between the proposed method and the pedestrian detection method in (Naatsubo 2010). CONCLUSIONS In this paper, we propose a new pedestrian detection method using multi-layer Lidar. Distance-based segmentation is used for clustering process of all the raw point cloud. According to the basic attributes of pedestrian, non-pedestrian objects are removed with ernel probability density model. The laser point information from four surfaces is integrated for pose estimation and pedestrian tracing to figure out the occlusion problem. The experiment results demonstrate that pedestrian can be detected effectively with our algorithm using multi-layer Lidar. Moreover, the tracing process can improve the detection accuracy. In the future, we will use the fusion of camera and Lidar to improve the detection result further. REFERENCES Cheng W, Jhan D. A self-constructing cascade classifier with AdaBoost and SVM for pedestriandetection. Engineering Applications of Artificial Intelligence, 2013, 26(3): 1016-1028. Fayad F, Cherfaoui V. Tracing objects using a laser scanner in driving situation based on modeling target shape. Intelligent Vehicles Symposium, 2007. IEEE: 44-49. Gidel S, Checchin P, Blanc C, et al. Parzen method for fusion of laserscanner data: Application to pedestrian detection. Intelligent Vehicles Symposium, 2008. IEEE: 319-324. Gate G, Nashashibi F. Fast algorithm for pedestrian and group of pedestrians detection using a laser scanner. Intelligent Vehicles Symposium, 2009. IEEE: 1322-1327. Garcia, Fernando, et al. Context aided pedestrian detection for danger estimation based on laser scanner and computer vision. Expert Systems with Applications, 2014, 41(15): 6646-6661. Grassi A, Frolov V, León F. Information fusion to detect and classify pedestrians using invariant features. Information fusion, 2011, 12(4): 284-292. 581

Mendes A, Nunes U. Situation based multi target detection and tracing with laserscanner in outdoor semi-structured environment. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Sendai, Japan: IEEE, 2004: 88-93. Naatsubo K, Yamada K. Detecting unusual pedestrian behavior toward own vehicle for vehicle-to-pedestrian collision avoidance. Intelligent Vehicles Symposium (IV), 2010 IEEE. IEEE, 2010. Schutz, Marus, et al. Laser-based hierarchical grid mapping for detection and tracing of moving objects. Advanced Microsystems for Automotive Applications 2012. 167-176. Tian H, Duan Z, Abraham A. A novel multiplex cascade classifier for pedestrian detection. Pattern Recognition Letters, 2013, 34(14), 1687-1693. 582