차세대지능형자동차를위한신호처리기술 정호기

Transcription

1 차세대지능형자동차를위한신호처리기술 정호기 0

2 . 지능형자동차의미래 ) 단위 system functions 운전자상황인식 얼굴방향인식 시선방향인식 졸음운전인식 운전능력상실인식 차선인식, 전방장애물검출및분류 Lane Keeping System + Adaptive Cruise Control Pre-Crash System Active Pedestrian Protection System Urban Traffic Assist (Traffic Sign Recognition) CC & CI 활욤 사고차량인식 위험지역인식 제한속도강제 측후방장애물인식 Blind Spot Detection Lane Change Assist Automatic Lane Change 후방장애물인식 주차보조시스템 자동주차시스템 Rear Collision Warning System (Active Headrest) GPS & MAP 활용 실시간교통정보 최적경로안내 에너지절약주행

3 . 지능형자동차의미래 ) SSIE (Safe Stopping In Emergency) Mission Management, Co-Pilot 9 에신고하고, 비상대피위치로사고없이이동하여, 정차한다. 상황인식 (Situation Awareness) 운전능력상실인식 현위치와비상대피위치인식 3 9에신고 4 현위치로부터비상대피장소까지의이동행동계획 행동계획 (Behavior Planning) 운전능력상실인식 주행차로및전방장애물인식 종횡방향주행안전제어 현재위치및주변지형인식 측방장애물인식 5 자동차선변경제어 6 정차및안전조치 인식시스템 경로제어

4 . Automotive Vision Systems ) Vision-Based Systems Lane Departure Warning/Lane Keeping System LKS+ACC Forward Sensing Collision Warning/Collision Avoidance Active Pedestrian Protection System Side Sensing Blind Spot Detection/Lane Change Assist Backward Sensing Intelligent Parking Assist System 3

5 . Automotive Vision Systems ) Why Vision?: Visual Features Traffic Sign Significant amount of traffic information is designed only for human visual system.. Lane departure warning system, lane keeping system: Lane markings lane information Lane Markings. Adaptive cruise control: Lane markings target vehicle selection on curved road 3. Intelligent Parking Assist System: Parking slot markings target parking slot Parking Slot Markings 4

6 . Automotive Vision Systems 3) Why Vision?: High Resolution and Rich Texture Higher resolution and rich texture of vision system provides an efficient solution for object detection and classification.. Collision Warning/Collision Avoidance: Vehicle classification reducing false alarm. Active pedestrian protection system: Pedestrian classification reducing false alarm 3. Active pedestrian protection system: Object classification class specific risk assessment Pose estimation reducing false alarm 5

7 . Automotive Vision Systems 4) Sensor Fusion-Based Approaches Lane Departure Warning/Lane Keeping System Forward Sensing LKS+ACC Sensor Fusion-Based Lane Detection Collision Warning/Collision Avoidance Active Pedestrian Protection System Sensor Fusion-Based Pedestrian Recognition Side Sensing Blind Spot Detection/Lane Change Assist Backward Sensing Intelligent Parking Assist System Target Position Designation Methods 6

8 3. Sensor Fusion-Based Pedestrian Recognition ) System Configuraiton Hypothesis Generation: Scanning Laser Radar Hypothesis Verification - GFB (Gabor Filter Bank)-based Feature Extractor - SVM (Support Vector Machine) Obstacle detection + Obstacle classification Pedestrian or Vehicle or Nothing 7

9 3. Sensor Fusion-Based Pedestrian Recognition ) GFB-Based Feature Extractor Sub-window Splitting In order to extract local characteristics of pedestrian images 36x8 image 8x9 nine overlapped sub-images 4 Gabor filters (6 sections in orientation, 4 sections in scale) 3 statistical values of filtered result: mean, variance, skewness Feature vector dimension: 9x4x3=648 8

10 3. Sensor Fusion-Based Pedestrian Recognition 3) Support Vector Machine SVM is a kind of classification method with hyperplane. Margin m can be expressed by w. SVM learning is a constrained optimization problem. By introducing Lagrangian multipliers 9

11 3. Sensor Fusion-Based Pedestrian Recognition 4) Nonlinear, soft margin SVM Nonlinear problems can be solved by introduction of kernel function. Soft margin SVM is defined by introducing regulation parameter C. 0

12 3. Sensor Fusion-Based Pedestrian Recognition 5) Optimization of SVM Learning Parameters Candidate Image Sub-window Splitting Kernel parameter and regulation parameter are critical for classification Filtering Extraction with Gabor Filter Bank performance. Parameter Optimization is needed! GA is robust to non-linear and discontinuous optimization. Optimization of SVM Learning Parameters SVM Learning with Training Data n n n maximize W( α) = α αα y y e i i j i j i i j = = = subject to C α 0, α y = 0 i i i i= 0 n xi xj σ Gene is composed of two parameters. Fitness function is defined by crossvalidation. SVM Execution with Cross-Validation Data Ns f( x ) = α y e i= i i si x σ P = P( non ped) + P( ped non) error GA-based Minimization of P error With respect to C and σ

13 3. Sensor Fusion-Based Pedestrian Recognition 6) Experimental Results DCX pedestrian database - 5 datasets 3 for training ( for learning and for cross validation), for final test. - One dataset: 4,800 pedestrian images (800 persons), 5,000 non-pedestrian images. Better performance and lower complexity Detection Rate False Positive Rate Comparison of ROCs. The dotted line is ROC of LRF quadratic SVM shown in Fig. 5(d) of [] and the solid line is ROC of GFB RBF SVM. [] S. Munder and D. M. Gavrila, An Experimental Study on Pedestrian Classification, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 8, No., pp , Nov. 006.

14 4. Sensor Fusion-Based Lane Detection ) Projection of Range Data onto Input Image Image pixel coordinate is (x i, y i ) World coordinate of range data is (X w, Y w, Z w ) Relation of two coordinate X b xi Y b y = H i Z b Xw Xb / Zb Z = w Yb / Z b Y w is 0, meeting point of ground and external disturbance hc cosθ hc sinθ sinϕ f cosϕ hc sinθ H = hc cosθ hc cosθ sinϕ f cosϕ hc cosθ 0 cosϕ f sinϕ 3

15 4. Sensor Fusion-Based Lane Detection ) Projection of Range Data onto Input Image Range Data Acquired by Scanning Laser Scanner Range Data Projected onto Input Image 4

16 4. Sensor Fusion-Based Lane Detection ) Free Space-Based ROI Establishment Clustering laser scanning data Establish ROI: area below border line consisting of recognized range data clusters and the sky line ROI: Free Space Recognized Free Space 5

17 4. Sensor Fusion-Based Lane Detection 3) Experimental Results Lane Feature without Range Data ROI Lane Detection Result without Range Data ROI Input Image Lane Feature with Range Data ROI Lane Detection Result with Range Data ROI 6

18 4. Sensor Fusion-Based Lane Detection 3) Experimental Results Lane Feature without Range Data ROI Lane Detection Result without Range Data ROI Input Image 7 Lane Feature with Range Data ROI Lane Detection Result with Range Data ROI

19 4. Sensor Fusion-Based Lane Detection 3) Experimental Results Lane Feature without Range Data ROI Lane Detection Result without Range Data ROI Input Image 8 Lane Feature with Range Data ROI Lane Detection Result with Range Data ROI

20 4. Sensor Fusion-Based Lane Detection 3) Experimental Results General Case With Tree Shadow With Wall Shadow With Cutting-in Vehicle 9

21 4. Sensor Fusion-Based Lane Detection 3) Experimental Results in Day Condition in Night Condition 0

22 5. Intelligent Parking Assist System ) Target Position Designation Methods GUI-Based Markings-Based Free Space-Based Infra-structure-Based

23 5. Intelligent Parking Assist System ) Free Parking Space Detection Methods Free Space-Based Ultrasonic sensor-based Short range radar (SRR)-based Single image understanding-based Motion stereo-based Binocular stereo-based Light stripe projection-based Scanning laser radar-based

24 5. Intelligent Parking Assist System (Motion Stereo-Based) ) LKT Tracking-Based 3D Reconstruction Point correspondence the image location where the minimum eigen value of G is larger than the threshold is selected as a feature point and tracked through the image sequence. Lukas Kanatae Tomasi Tracking. Fundamental matrix estimation: RANSAC M-estimator 3D reconstruction 3 F Pre-calibration, Essential Matrix Estimation R, T estimation P, P' ( ) ( ' ) x P X = 0 x' P X = 0 ΑX= 0 A SVD 3T T xp p yp p x ' p' p' y ' p' p' 3T T = 3T T 3T T X X = Y Z

25 5. Intelligent Parking Assist System (Motion Stereo-Based) ) Degradation of 3D Data Near the Epipoles Optical flow is short near the epipoles. Adjacent vehicles are near the epipoles. 3D data of adjacent vehicle is severely degraded. Since the last column of P represents the epipole, the last column of A becomes closer to a zero vector when the feature point ([x,y ] T ) nears the epipole. This causes unreliable estimation of the 3D points. P = K[ I 0] = [ K 0] P' = K[ R t] = [ KR e] 4

26 5. Intelligent Parking Assist System (Motion Stereo-Based) 3) Feature Selection and Mosaicking Y = crx + t i i ( Rt,, ) = Y ( RX+ t) n e c i c n i = i 5 T UDV = SVD A and B are the 3 n matrices of {X, X,, Xn} and {Y, Y, Yn} T ( AB ) T R = UV, t = μy crμx, c= trace( D) σ μ Y μ X X μ n n n Y = i, X = i, σ X = i X n i= n i= n i= X

27 5. Intelligent Parking Assist System (Motion Stereo-Based) 4) Self Calibration: Metric Recovery θ = e y f x 0 arctan( ) 6

28 5. Intelligent Parking Assist System (Motion Stereo-Based) 5) Experimental Results 54 real image sequences Success rate was 90.3% 7

29 5. Intelligent Parking Assist System (Scanning Laser Radar-Based) ) Rectangular Corner Detection l : ax+ by+ c= 0 l : bx ay+ d = 0 x y 0 M a xn yn 0 b = 0 yn xn 0 c M d { yn xn 0 X n A n [ U, S, V ] = SVD( A ) n n n n error Unaffected by orientation and robust to noise n y (m) y (m) y (m) x (m) x (m) x (m)

30 5. Intelligent Parking Assist System (Scanning Laser Radar-Based) ) Round Corner Detection 4 3 l : px+ qy+ r = 0 : = 0 e ax bxy cy dx ey f e ax bxy cy dx ey f : = 0 l : px+ qy+ r = or x (m) 4 3 y (m) y (m) y (m) x (m) x (m) Rectangular corner detection Round corner detection y (m) y (m) x (m) x (m)

31 5. Intelligent Parking Assist System (Scanning Laser Radar-Based) 3) Free Space Detection Established target position Corner detection Main reference corner detection (ROI, free space constraint) Subreference corner detection y (m) y x x (m) 30

32 5. Intelligent Parking Assist System (Scanning Laser Radar-Based) 4) Experimental Results ( situations 98.%, 65.3msec) Against the sun Dark underground pakring lots With various kinds of vehicles Cloudy weather 3

33 Thank you for your attention! 차세대지능형자동차를위한신호처리정호기기술 만도정호기 3