Automatic vehicle classification based on vision

Transcription

1 Redouane Kachach Universidad Rey Juan Carlos

2 Index 1 Index Introduction Software Design Background estimation Tracking and vehicle counting Automatic vehicle classification Stats subsystem Experiments Conclusions and future guidelines

3 Introduction 2 Introduction The vehicle traffic sector uses a lot of cameras Automatic information extraction from the video is the big challenge

4 Introduction (cont.) 3 Examples of 3D Vision applications Google Car Automatic vehicle classification

5 Software Design 4 Software Design Background extraction Vehicle Tracking and counting Automatic vehicle classification Stats subsystem

6 Software Design (cont.) 5 jderobot Gazebo progeo Gtk+ mysql++

7 Software Design (cont.) 6

8 Software Design (cont.) 7 Background estimation Different algorithms: GMM, Mode based, Mean based, etc

9 Software Design (cont.) 8 Automatic road detection

10 Software Design (cont.) 9 Automatic road detection We only process the region delimited by the road

11 Software Design (cont.) 10 Automatic Vehicle tracking Algorithm: Build the foreground image Growing regions segmentation Blob matching

12 Software Design (cont.) 11 Foreground image extraction A binary mask is built Each pixel is labeled STATIC or MOVED

13 Software Design (cont.) 12 Image segmentation using the growing region technique

14 Software Design (cont.) 13 The growing region technique is used to identify the connected blobs A motorcycle 3D wire is used to estimate the minimum size of blobs All the blobs smaller than this one are discarded

15 Software Design (cont.) 14 Blob matching KLT algorithm is used to match vehicles between two frames We use a fixed number of points from the vehicle as KLT input Each vehicle is assigned to the nearest blob produced by the KLT

16 Software Design (cont.) 15 Vehicle speed estimation V = d t = Center 3D(out) Center 3D (in) t out t in

17 Software Design (cont.) 16 Automatic vehicle classification

18 Software Design (cont.) 17 Automatic vehicle classification based on 3D wire models Category Height(m) Width(m) Large(m) Motorcycles Cars Vans Trucks Road trains

19 Software Design (cont.) 18 3D vehicle center estimation First we get the 2D center of the blob: point A A is reproyected to get its position in 3D (using progeo) The 3D center is used to project the 3d model on Z=0 plane

20 Software Design (cont.) 19 Category probability estimation using the 3D wire model Calculate the Intersection of the blob with the model projection Calculate the Difference between the blob and the model projection P V Ci = I I D + I D+1 I D+1

21 Software Design (cont.) 20 Behaviour of the probability estimation function

22 Software Design (cont.) 21 PiP (Point In Polygon) Given a point and a polygon we draw a ray starting from the point: If the number of intersections is even or zero: the point is outside If the number of intersections is odd: the point is inside

23 Software Design (cont.) 22 Discrimination capacity of the probability function

24 Software Design (cont.) 23 Shadows The shadows are a problem The 3D wire model projection doesn t fit the vehicle If nothing is done this may produce bad classification results

25 Software Design (cont.) 24 Shadow integrated in the model Sun model Shadow projection

26 Software Design (cont.) 25 The shadows are represented by parallelepiped The 3D wire model projection fits the vehicle better PiP is used to decide whether a point is inside a region or not

27 Software Design (cont.) 26 3D vehicle center estimation with shadows A: 2D center of the 3D wire model including its shadow B: 2D projection of the 3D center of the 3D wire model C: 2D center of the blob

28 Experiments 27 Experiments Traffic simulated by Gazebo Real traffic Use of Gazebo to simulate the different vehicles categories Experiments with Gazebo and real traffic videos Database of more than 100 videos of real traffic

29 Experiments (cont.) 28 Counting results Video Total Shadows Tracked Lost Bad-Tracking Not-detected Hit video1 107 NO % video2 110 YES % video3 157 YES % video4 162 YES % Gazebo 560 NO %

30 Experiments (cont.) 29 Vehicle speed estimation Speed variance (Gazebo) Speed variance (measured over real traffic)

31 Experiments (cont.) 30 Classification results Video Total Hit motorcycles cars vans trucks road-train video result % video result % video result % video result % Gazebo result %

32 Experiments (cont.) 31 Conclusions and future guidelines occlusion shadows The TrafficMonitor accurracy is good in favorable conditions Resolve the occlusions and shadows issues Enhance the tracking and background estimation techniques Robust adaptation to camera position, illumination and weather changes

33 Experiments (cont.) 32 Use API Rest to allow third-party software integration

34 Redouane Kachach Universidad Rey Juan Carlos