CS381V Experiment Presentation. Chun-Chen Kuo

Transcription

1 CS381V Experiment Presentation Chun-Chen Kuo

2 The Paper Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. ECCV

3 Pipeline segmentation support inference

4 Outline Run the segmentation pipeline Experiment on the segmentation pipeline Run the support inference pipeline Address strength and weakness

5 Outline Run the segmentation pipeline Experiment on the segmentation pipeline Run the support inference pipeline Address strength and weakness

6 Segmentation Pipeline Image920, RGB Depth Map

7 Compute Surface Normal y z x

8 Align to room coordinates

9 Aligned Surface Normal y z x

10 After Alignment

11 Find Major Planes by RANSAC x y z

12 Reassign Pixels to Planes

13 Watershed Segmentation Force the over-segmentation to be consistent with the previous planes 1614

14 Hierarchical Grouping Bottom-up grouping by boundary classifier (Logistic regression AdaBoost)

15 AdaBoost Decision Tree Reweigh misclassified regions merge? Optimize new tree with reweighed regions Score the tree Weighted sum over all trees optimized in each iteration

16 Final Regions Ground truth 77

17 Outline Run the segmentation pipeline Experiment on the segmentation pipeline Run the support inference pipeline Address strength and weakness

18 Experiment on Segmentation Pipeline NYU Depth Dataset V2 Images 909~1200 Assign pixels to major planes AdaBoost decision tree as boundary classifier

19 Hypothesis The trade-off between matching to 3D values, normals, and gradient smoothing If alpha is small, neighbor pixels with similar RGB tend to be assigned to a same plane If alpha is large, match pixels to planes based on 3D points and normals, regardless gradient smoothing

20 Result of Plane Labeling alpha=0

21 Result of Plane Labeling alpha=2500

22 Result of Plane Labeling alpha=0.25

23 Result of Plane Labeling alpha=0 alpha=0.25 alpha=2500

24 Segmentation Score alpha=0.25e-12 alpha=0.25 alpha=2.5

25 Hypothesis Number of iteration of an AdaBoost decision forest boundary classifier (underfit vs. overfit) At higher stage, the number of training example(boundary) decreases, causing lower accuracy and overfitting Accuracy at lower stage is more important because of error propagation

26 stage 1 stage 2 stage 3 stage 4 stage 5

27 ROC Curve at Stage 1 iteration = 30 iteration = 5 1 Train AUC: , Test AUC: Train AUC: , Test AUC: training testing 0.9 training testing true positive rate false positive rate

28 ROC Curve at Stage 2 iteration = 30 iteration = 5 1 Train AUC: , Test AUC: Train AUC: , Test AUC: training testing 0.9 training testing

29 ROC Curve at Stage 3 iteration = 30 iteration = 5 1 Train AUC: , Test AUC: Train AUC: , Test AUC: training testing 0.9 training testing

30 ROC Curve at Stage 4 iteration = 30 iteration = 5 1 Train AUC: , Test AUC: Train AUC: , Test AUC: training testing 0.9 training testing

31 ROC Curve at Stage 5 iteration = 30 iteration = 5 1 Train AUC: , Test AUC: Train AUC: , Test AUC: training testing 0.9 training testing

32 Accuracy versus Iteration at Stage training testing accuracy iteration

33 Accuracy versus Iteration at Stage training testing accuracy iteration

34 Accuracy versus Iteration at Stage training testing accuracy iteration

35 Accuracy versus Iteration at Stage training testing accuracy iteration

36 Accuracy versus Iteration at Stage training testing accuracy iteration

37 Segmentation Score iteration = [ ] iteration = [ ] iteration = [ ] Accuracy at lower stage is more important!

38 Segmentation Score iteration = [ ] training testing accuracy iteration Accuracy at lower stage is more important!

39 Outline Run the segmentation pipeline Experiment on the segmentation pipeline Run the support inference pipeline Address strength and weakness

40 Support Inference Pipeline

41 Structure Class Classifier

42 Structure Class Classifier

43 Support Classifier containment, geometry, and horz feature take ~1 day to extract features for 292 images!

44 Support Classifier

45 Infer by Linear Program 6 minutes for an image!

46 Structure and Support Inference ground furniture props structure stripe: incorrect structure prediction

47 Structure and Support Inference

48 Structure and Support Inference out of 4 classes clutter, small objects over-segmentation(color variance in an object)

49 Outline Run the segmentation pipeline Experiment on the segmentation pipeline Run the support inference pipeline Address strength and weakness

50 Strength Reason joint assignment for structure and support ~73% accuracy if ground truth segmentation is given

51 Weakness Slow in testing time -5 minutes for feature extraction -6 minutes for inference by linear programming Clutters, small(thin) objects, color variance in objects Only 4 structure classes(no human, pet, etc) ~55% accuracy if bottom up segmentation followed by support inference

52 Reference Code: indoor_scene_seg_sup.html