Seminar Heidelberg University

Size: px

Start display at page:

Download "Seminar Heidelberg University"

Janel Flynn
6 years ago
Views:

1 Seminar Heidelberg University Mobile Human Detection Systems Pedestrian Detection by Stereo Vision on Mobile Robots Philip Mayer Matrikelnummer:

2 Motivation Fig.1: Pedestrians Within Bounding Box [6] Fig.2: Car Pedestrian Detection [7] 2

3 Outline 1. Problem Formulation 2. Solution Approach 3. Stereo Vision 4. Methods 5. Results 6. Summary and Conclusion 3

4 1. Problem Formulation Given: Stereo Vision Depth Image Mobile Robot Unknown Background Cluttered Environment Crowded Places Required: Pedestrian Detection Also If Partially Occluded 4

5 2. Solution Approach Fig.3: Depth Image [1] Fig.4: Segmented Regions [1] Fig.5: Candidates [1] Fig.7: Block Diagram Solution Approach Fig.6: Detected Humans[1] 5

6 3. Stereo Vision y x λ P A, A Optical Axis O, O Lense Centers B Baseline P real Point in real space P Projection of P real on Image 2 P Projection of P real on Image 1 P y x λ y real x real z real P real Fig.10: Stereo Vision Geometric Setup [3] 6

7 3. Stereo Vision Fig.8: Color Image 1 Left Lense [5] Fig.9: Color Image 2 Right Lense [5] 7

8 3. Stereo Vision Distance To Camera: 0,5 m 8 m Undefined Fig.11: Depth Image 1 Left Lense [5] Fig.12: Depth Image 2 Right Lense [5] 8

9 Graph-Based Segmentation Fig.3: Depth Image [1] Fig.4: Segmented Regions [1] 9

10 Graph-Based Segmentation 0,0 α i α i max = j max = image width w cell width α image height h cell height α j Fig.13: Depth Image With Grid [1] E imax,j max 10

11 Graph-Based Segmentation Fig.14: Random Pixel Selection Within Depth Image Grid Cell 11

12 0,0 4. Methods Graph-Based Segmentation i Point P i,j in 3D-Space E i,j P i,j P i,j = p i,j x p i,j y p i,j z j Fig.15: Depth Image With Grid Points For Depth And Normals Graph [1] 12

13 Graph-Based Segmentation P i,j 1 w = Edge Weight w Depth = z 1 z 2 z 1 = Depth of P i,j P i 1,j P i,j P i+1,j z 2 = Depth of P i+1,j P i,j+1 Fig.16: Depth Graph Weights Calculation 13

14 Graph-Based Segmentation P i 1,j 1 P i,j 1 P i+1,j 1 P i,j P i 1,j P i,j P i+1,j 8 Neighbors Of P i,j P i 1,j+1 P i,j+1 P i+1,j+1 Fig.17: Depth Graph Normals Calculation 9 Samples of P in 3D-Space Least-Square-Roots Plane Normals n i,j 14

15 Graph-Based Segmentation P i,j 1 w Normal = cos 1 (v u) w = Edge Weight u = Normal of P i,j P i 1,j P i,j P i+1,j v = Normal of P i+1,j P i,j+1 Fig.18: Normals Graph Weights Calculation 15

16 Graph-Based Segmentation Regions r i R Region points in G Normal Region r i Region points in G Depth Minimal size of a region is β Filtering noise Fig.19: Region Condition 16

17 Filtering and Merging Fig.5: Candidates [1] Fig.4: Segmented Regions [1] 17

18 Filtering and Merging y 1 x h Bounding Box r i w = x 2 x 1 x 1 w x 2 h = y 2 y 1 μ x = w 2 μ y = h 2 μ z = mean depth z (r i ) y y 2 Fig.20: Region Attributes Calculation 18

19 Filtering and Merging y π k 1. Select 3 Points Randomly n-times From r i Hypothesis Plane π k 2. Maximum Number Of Points Fitting The Plane π k n max k=1 p r i : distance of p to π k < ε r i z Fig.21: Hypothesis Plane x 3 randomly selected Points π k Points above π k Points below π k Points with distance to π k < ε 19

20 Filtering and Merging Finding a rule specifiying valid ranges for: Mean Depth Height Width Minimum Inlier Fraction Rule derived from positive examples in the training set Eliminate regions unable to be humans 20

21 Filtering and Merging Region too small but planar: Size(r i ) < β High number of fitting points on π k Mean depth rule satisfied Merging regions (merging condition) μ xz r i μ xz r j < δ xz and μ y r i μ y r j < δ y Important step due to detached parts by segmentation 21

22 Filtering and Merging Set of regions Set of (unscaled) candidates Classifcation needs scaled candidates Copy pixels of regions into candidate image with size w c h c If pixel copied raw depth pixel Undefined otherwise Candidates c i : Candidate image + bounding box Output candidate set C 22

23 Candidate Classification Fig.4: Segmented Regions [1] Fig.6: Detected Humans[1] 23

24 Candidate Classification 8x8 Pixel Cell 2x2 Cell Box Bounding Box Fig.22: Gradient Vector Calculation [2] ΔDepth x = = 167 ΔDepth y = = 202 Gradient Vector v G = ΔDepth x ΔDepth y = Fig.23: Candidate Image With Bounding Box And Fixed Size [1] 24

25 Magnitude 4. Methods Candidate Classification Angle [Deg] Fig.24: Histogram Of Oriented Depth Magnitude M = = 262,1 Gradient Angle Θ = arctan = 69,3 25

26 Candidate Classification 50% Box Overlap 2x2 Cell Box Yellow: Initial Step Green: Preceeding Step 4 Cell Histograms For Normalization Vector of Histograms Candidate Descriptor for SVM Fig.26: Candidate Image With Blocks For Normalization [1] 26

27 Candidate Classification Negative Example Positive Example A B Fig.27: Linear Support Vector Machine [4] 27

28 Candidate Classification - Set of Candidates C - Set of Humans H Fig.28: Support Vector Machine Scheme [2] Depth image frames from training set Candidates labeled as positive or negative 28

29 5. Results Two Sets Of Experiments: 1. Recall & Precision 2. Impact of varying number of training examples on Recall & Precision Hallway Café Distances 0,5 8 [m] 0,5 5 [m] Occlusion Level Varying Often Environment Not Cluttered Cluttered Ergonomic Position of People Upright Various Poses 29

30 5. Results Hallway Dataset Equal Error Rate (EER) Café Dataset Fig.29: Accuracy Results, (a) Hallway Data Set, (b) Café Data Set [1] FP = False Positives TP = True Positives FN = False Negatives Recall = TP TP + FN Precision = TP TP + FP 30

31 5. Results Hallway Dataset Café Dataset Fig.30: Impact On Accuracy By Reduction Of Positive Training Examples, (a) Hallway Data Set, (b) Café Data Set [1] 31

32 6. Summary & Conclusion Stereo Vision Segmentation Algorithm Filtering and merging HOD Descriptor SVM Precision, Recall Comparison of impact on precision and recall due to less training for SVM Missing Information: Impact Of Resolution Loss Comparison Of Datasets: Environmental Difference, Different Ergonomic Positions Presented Depth Image: No Reference About Depth Information Encoding No Measure Units in Data Sheet Table 32

33 Paper (Literatur) 1. Fast Human Detection for Indoor Mobile Robots Using Depth Images 2013 IEEE International Conference on Robotics and Automation (ICRA) Karlsruhe, Germany, May 6-10, L. Spinello and K. Arras, People Detection in RGB-D Data, in Proceedings of IROS 2011, pp Perma-Link: 3. Web-Page: 4. Web-Page: 5. Web-Page: 6. Web-Page: 33

Fast Human Detection for Indoor Mobile Robots Using Depth Images

2013 IEEE International Conference on Robotics and Automation (ICRA) Karlsruhe, Germany, May 6-10, 2013 Fast Human Detection for Indoor Mobile Robots Using Depth Images Benjamin Choi 1, Çetin Meriçli 1,