Combining Top-down and Bottom-up Segmentation

Transcription

1 Combining Top-down and Bottom-up Segmentation Authors: Eran Borenstein, Eitan Sharon, Shimon Ullman Presenter: Collin McCarthy

2 Introduction Goal Separate object from background Problems Inaccuracies Top-down lacks constraints Bottom-up lacks structure Solution Combine

3 Hierarchical Bottom-up Segmentation Why needed? Easy, low-level Effective Improved accuracy with hierarchy

4 Bottom-up: Approach Normalized cuts for clustering Weighted, undirected graph Nodes are pixels, edges are similarity Minimize cost of partitioning graph Jianbo Shi and Jitendra Malik Normalized Cuts and Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 8 (August 2000).

5 Bottom-up: Approach Normalized cuts for clustering Jianbo Shi and Jitendra Malik Normalized Cuts and Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 8 (August 2000).

6 Bottom-up: Approach Segmentation by Weighted Aggregation Same problem, but reduce nodes multiple times Sharon, E.; Brandt, A.; Basri, R., "Fast multiscale image segmentation," Computer Vision and Pattern Recognition, Proceedings. IEEE Conference on, vol.1, no., pp.70,77 vol.1, 2000.

7 Bottom-up: Approach Best Weights Average Intensity, Intensity Variance, Boundary Sharon, E.; Brandt, A.; Basri, R., "Segmentation and boundary detection using multiscale intensity measurements," Computer Vision and Pattern Recognition, CVPR Proceedings of the 2001 IEEE Computer Society Conference on, vol.1, no., pp.i-469,i-476 vol.1, 2001.

8 Background: Bottom-up Approach Result: multi-scale, hierarchical graph O(n) in number of pixels Eran Borenstein, Shimon Ullman, "Combined Top-Down/Bottom-Up Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 12

9 Class-specific Top-Down Segmentation Why needed? Solve difficult cases Capture meaningful, crucial parts Reflect human visual system Eran Borenstein, Shimon Ullman, "Combined Top-Down/Bottom-Up Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 12, pp , December 2008.

10 Top-down: Approach Find classifying fragments Frequent in class images Low correlation with non-class images Calculate optimal cover Eran Borenstein and Shimon Ullman Class-Specific, Top-Down Segmentation. In Proceedings of the 7th European Conference on Computer Vision- Part II (ECCV '02)

11 Top-down: Approach Eran Borenstein and Shimon Ullman Class-Specific, Top-Down Segmentation. In Proceedings of the 7th European Conference on Computer Vision- Part II (ECCV '02)

12 Top-down: Segmentation Segmentation = optimal cover of fragments 1. Individual match of fragments 2. Consistency 3. Reliability

13 Top-down: Optimal Cover 1. Individual match of fragments Eran Borenstein and Shimon Ullman Class-Specific, Top-Down Segmentation. In Proceedings of the 7th European Conference on Computer Vision- Part II (ECCV '02)

14 Top-down: Optimal Cover 2. Consistency Neighboring fragments Eran Borenstein and Shimon Ullman Class-Specific, Top-Down Segmentation. In Proceedings of the 7th European Conference on Computer Vision- Part II (ECCV '02)

15 Top-down: Optimal Cover 3. Reliability Detection rate / False-positive rate Reliable fragments first Eran Borenstein and Shimon Ullman Class-Specific, Top-Down Segmentation. In Proceedings of the 7th European Conference on Computer Vision- Part II (ECCV '02)

16 Top-Down: Analysis Strong Points Accurate segmentation Shape variability Reliability provides flexibility Weak Points Simple version uses manual segmentations Limited by training set Shape variability High variability regions

17 Top-down vs. Bottom-up

18 Top-down vs. Bottom-up Problem: Object vs. Background Solution: Construct a classification map C(x,y) Top-down requirement Bottom-up constraint Eran Borenstein, Shimon Ullman, "Combined Top-Down/Bottom-Up Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 12

19 Implementation: Global Cost Function Segment Label: Figure or Ground Per segment, per level Classification map = Labels at Lowest Level Cost function per level

20 Implementation: Global Cost Function Top-down cost Penalizes levels far from top-down classification Bottom-up cost Penalizes levels splitting homogenous regions into both figure and ground

21 Implementation: Global Cost Function Global minimum Total cost = Sum local cost for segment i to classify its pixels by given parent classified as s i s i Uses sum-product algorithm Two passes Min cost if node is figure or ground Compute confidence of region

22 Results Average contour distance from ground truth Bottom-up: 17 pixels Top-down: 5.21 pixels Combined: 1.3 pixels

23 Results

24 Results

25 Results

26 Conclusion Weak Points Top-down depends on training Bottom-up depends on weighting Strong Points Accuracy O(n) Multiple scales, rapid convergence General in algorithms

27 Questions

28 Implementation: Global Cost Function Details

29 Implementation: Minimizing Cost Uses sum-product algorithm

30 Implementation: Minimizing Cost Two-pass algorithm with message passing between parent and child fragments

31 Implementation: Minimizing Cost Two-pass algorithm with message passing between parent and child fragments Compute confidence of each region

32 Background: Bottom-up Approach Also provides a measure of saliency, how distinctive a segment is Reflected by an energy measure, Γ i Γ i = (Segment dissimilarity to its surrounding ) / (Internal homogeneity) Uniform segments, high contrast with background High saliency, Low Energy Uniform segments, low contrast with background Low saliency, High energy