Joint Shape Segmentation

Transcription

1 Joint Shape Segmentation

2 Motivations Structural similarity of segmentations Extraneous geometric clues Single shape segmentation [Chen et al. 09] Joint shape segmentation [Huang et al. 11]

3 Motivations Structural similarity of segmentations Low saliency Single shape segmentation [Chen et al. 09] Joint shape segmentation [Huang et al. 11]

4 Motivations (Rigid) invariance of segments Articulated structures Single shape segmentation [Chen et al. 09] Joint shape segmentation [Huang et al. 11]

5 Pair-wise Joint Segmentation Objective: Outline: Segmentation parameterization Segmentation score Consistency score 0-1 linear programming formulation

6 Segmentation Parameterization Shape Diameter [Shapira et al. 08] Randomized Cuts Random Walks [Lai et al. 08] Normalized Cuts [Golovinskiy and Funkhouser 08] Initial Segments

7 Segmentation Parameterization Segmentations: subsets of initial segments obtained from randomized segmentations Randomized Cuts Initial Segments

8 Segmentation Parameterization Segmentations: subsets of initial segments obtained from randomized segmentations Segmentation constraints: each point is in exactly one segment Randomized Cuts The set of initial segments that cover point p Initial Segments

9 Segmentation Parameterization Segmentations: subsets of initial segments obtained from randomized segmentations Segmentation constraints: each point is in exactly one segment Segmentation score Randomized Cuts Prevent tiny segments Repetitions Initial Segments

10 Segmentation Parameterization Randomized Cuts Patches [Golovinskiy and Funkhouser 08] Super-pixels [Ren and Malik 03] Initial Segments

11 Consistency Term Defined in terms of mappings Oriented Partial Many-to-one correspondences Partial similarity

12 Consistency Term Defined in terms of mappings Oriented Partial Mapping score [Anguelov et al.05] Correspondence weight [Osoda et al. 02] Adjacent correspondence pair weight

13 Consistency Term Defined in terms of mappings Oriented Partial Mapping score [Anguelov et al.05] Consistency score

14 Constrained Optimization

15 0-1 Linear Programming Formulation Introduce binary indicators Segments 0 1 1

16 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences 1 0

17 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences Correspondence pairs 1 0

18 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences Correspondence pairs Objective function:

19 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences Correspondence pairs Segmentation constraints: Matrix representation:

20 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences Correspondence pairs Mapping constraints:

21 0-1 Linear Programming Formulation Introduce binary indicators Segments Correspondences Correspondence pairs Compatibility constraints:

22 0-1 Linear Programming Formulation Linear programming relaxation

23 Similar Shapes As a by-product, pair-wise joint segmentation determines pairs of similar shapes Similar Less similar

24 Multi-way joint segmentation Input shapes Different objects Different categories

25 Multi-way joint segmentation Perform all pair-wise joint segmentation to determine pairs of similar shapes

26 Multi-way joint segmentation Objective function

27 Multi-way Joint Segmentation Formulation

28 Multi-way Joint Segmentation Revised Formulation Two independent constraint sets

29 Alternating Optimization Shape-wise optimizations: Pair-wise optimizations:

30 Princeton Segmentation Benchmark [Chen et al. 09] 380 shapes in 19 categories Manual segmentations for each shape (4300 in total)

31 Princeton Segmentation Benchmark [Chen et al. 09] Joint : Joint shape segmentation per each category JointAll : Joint shape segmentation over the entire database Rand index metric [Rand 1971] - the smaller, the better Significantly better than single shape segmentations Competitive against supervised segmentation JointAll is slightly better than Joint

32 Rand Index Scores on PSB [Chen et.al 09] When shape variation of the input is big Top: Joint Bottom: JointAll

33 Rand Index Scores on PSB [Chen et.al 09] When shape variation of the input is small Top: Joint Bottom: JointAll

34 Versus Supervised Method [Kalogerakis et al.10] Supervised segmentation Joint shape segmentation

35 Joint versus JointAll

36 Rand Index Scores on PSB [Chen et.al 09] Failure case

37 Other Unsupervised Methods

38 Embedded spaces [Sidi et al 11] Shape segments mapped to some feature space Segments of the same class form a (graph) cluster

39 Feature space After a spectral transform [Sidi et al 11] Two handles pulled closer

40 Algorithm [Sidi et al 11]

41 Sub-space clustering Features inspired from supervised learning [ et al 10] [Hu et al 12]

42 Comparison

43 Other Unsupervised Methods

44 Learning from labeled/unlabeled data [Wang et al 12]

45 Supervised learning [Wang et al 12]

46 Un-supervised learning [Wang et al 12]

47 Semi-supervised learning [Wang et al 12]

48 Constrained clustering [Wang et al 12] Must Link Cannot Link

49 Algorithm [Wang et al 12] Initial Co-segmentation Constrained Clustering Final result Active Learning

50 Effective on large data sets [Wang et al 12] 300 shapes