Contextual Classification with Functional Max-Margin Markov Networks

Transcription

1 Contextual Classification with Functional Max-Margin Markov Networks Dan Munoz Nicolas Vandapel Drew Bagnell Martial Hebert

2 Geometry Estimation (Hoiem et al.) Sky Problem 3-D Point Cloud Classification Wall Vertical Vegetation Support Ground Our classifications Tree trunk 2

3 Room For Improvement 3

4 Approach: Improving CRF Learning Gradient descent (w) Boosting (h) Friedman et al. 2001, Ratliff et al Better learn models with high-order interactions + Efficiently handle large data & feature sets + Enable non-linear clique potentials 4

5 Pairwise model Conditional Random Fields y i Lafferty et al Labels x MAP Inference 5

6 Parametric Linear Model Weights Local features that describe label 6

7 Associative/Potts Potentials Labels Disagree 7

8 Overall Score Overall Score for a labeling y to all nodes 8

9 Iterate Learning Intuition Classify with current CRF model If (misclassified) φ( ) increase score φ( ) decrease score (Same update with edges) 9

10 Max-Margin Structured Prediction Taskar et al min w Best score from all labelings (+M) Score with ground truth labeling Ground truth labels Convex 10

11 Descending Direction (Objective) Labels from MAP inference Ground truth labels 11

12 Learned Model 12

13 Unit step-size, and λ = 0 Update Rule w t+1 += Ground truth Inferred 13

14 Iterate Verify Learning Intuition w t+1 += If (misclassified) φ( ) increase score φ( ) decrease score 14

15 1. Create training set: D Alternative Update From the misclassified nodes & edges D =, +1, -1, +1, -1 15

16 1. Create training set: D 2. Train regressor: h t Alternative Update D h t ( ) 16

17 1. Create training set: D 2. Train regressor: h 3. Augment model: Alternative Update (Before) 17

18 Functional M 3 N Summary Given features and labels for T iterations Classification with current model Create training set from misclassified cliques D Train regressor/classifier h t Augment model 18

19 Create training set Illustration D

20 Illustration Train regressor h t h 1 ( ) ϕ( ) = α 1 h 1 ( ) 20

21 Illustration Classification with current CRF model ϕ( ) = α 1 h 1 ( ) 21

22 Illustration Create training set D +1-1 ϕ( ) = α 1 h 1 ( ) 22

23 Illustration Train regressor h t h 2 ( ) ϕ( ) = α 1 h 1 ( )+α 2 h 2 ( ) 23

24 Stop Illustration ϕ( ) = α 1 h 1 ( )+α 2 h 2 ( ) 24

25 Boosted CRF Related Work Gradient Tree Boosting for CRFs Dietterich et al Boosted Random Fields Torralba et al Virtual Evidence Boosting for CRFs Liao et al Benefits of Max-Margin objective Do not need marginal probabilities (Robust) High-order interactions Kohli et al. 2007,

26 Using Higher Order Information Colored by elevation 26

27 Region Based Model 27

28 Region Based Model 28

29 Region Based Model Inference: graph-cut procedure P n Potts model (Kohli et al. 2007) 29

30 How To Train The Model Learning 1 30

31 How To Train The Model Learning (ignores features from clique c) 31

32 How To Train The Model Robust P n Potts Kohli et al β Learning 1 β 32

33 Experimental Analysis 3-D Point Cloud Classification Geometry Surface Estimation 33

34 Random Field Description Nodes: 3-D points Edges: 5-Nearest Neighbors Cliques: Two K-means segmentations Features [0,0,1] θ normal Local shape Orientation Elevation 34

35 Qualitative Comparisons Parametric Functional (this work) 35

36 Qualitative Comparisons Parametric Functional (this work) 36

37 Qualitative Comparisons Parametric Functional (this work) 37

38 Quantitative Results (1.2 M pts) Macro* AP: Parametric 64.3% Precision +24% +4% % Recall -1% Functional 71.5% % 0.93 Wire Pole/Trunk Façade Vegetation 38

39 Experimental Analysis 3-D Point Cloud Classification Geometry Surface Estimation 39

40 Random Field Description Nodes: Superpixels (Hoiem et al. 2007) Edges: (none) Cliques: 15 segmentations (Hoiem et al. 2007) More Robust Features (Hoiem et al. 2007) Perspective, color, texture, etc. 1,000 dimensional space 40

41 Quantitative Comparisons Parametric (Potts) Hoiem et al Functional (Potts) Functional (Robust Potts) 41

42 Qualitative Comparisons Parametric (Potts) Functional (Potts) 42

43 Qualitative Comparisons Parametric (Potts) Functional (Potts) Functional (Robust Potts) 43

44 Qualitative Comparisons Parametric (Potts) Hoiem et al Functional (Potts) Functional (Robust Potts) 44

45 Qualitative Comparisons Parametric (Potts) Hoiem et al Functional (Potts) Functional (Robust Potts) 45

46 Qualitative Comparisons Parametric (Potts) Hoiem et al Functional (Potts) Functional (Robust Potts) 46

47 Conclusion Effective max-margin learning of high-order CRFs Especially for large dimensional spaces Robust Potts interactions Easy to implement Future work Non-linear potentials (decision tree/random forest) New inference procedures: Komodakisand Paragios 2009 Ishikawa 2009 Gould et al Rother et al

48 Acknowledgements Thank you U. S. Army Research Laboratory Siebel Scholars Foundation S. K. Divalla, N. Ratliff, B. Becker Questions? 48