SCALP: Superpixels with Contour Adherence using Linear Path

Transcription

1 SCALP: Superpixels with Contour Adherence using Linear Path Rémi Giraud 1,2 with Vinh-Thong Ta 1 and Nicolas Papadakis 2 1 LaBRI CNRS UMR University of Bordeaux, FRANCE PICTURA Research Group 2 IMB CNRS UMR University of Bordeaux, FRANCE

2 Superpixel Context Simple Linear Iterative Clustering The SCALP method Results Conclusion Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 2 / 16

3 Superpixel Context Low-level representation of an image into homogeneous areas. Respect of image geometry (contrary to multi-resolution approach). Properties: Adherence to the image contours Homogeneity of the color clustering Regularity/compactness of the decomposition 1500 superpixels 400 superpixels 400 superpixels Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 3 / 16

4 Simple Linear Iterative Clustering (SLIC) - (Achanta, et al. 2012) State-of-the-art results on standard superpixel metrics Produces regular superpixels in limited computational time Regular grid initialization Constrained K-means clustering Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 4 / 16

5 SLIC - Distance Distance between pixel p, and average cluster C k of S k : D(p, C k ) = d c (p, C k ) 2 + d s (p, C k ) 2 m Pixel p = [(l p, a p, b p), X p] Average feature cluster C k = [(l k, a k, b k ), X k ] d c color distance d s spatial distance Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 5 / 16

6 SLIC - Limitations Parameter m globally set for the whole image SLIC result Difficulty to maximize all aspects: Adherence to contours Color homogeneity Decompositon regularity Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 6 / 16

7 Superpixels with Contour Adherence using Linear Path (SCALP) Enforces respect of image contours, color homogeneity and compactness at the same time Definition of the linear path to superpixel barycenter Additional features within the clustering distance Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 7 / 16

8 Superpixels with Contour Adherence using Linear Path (SCALP) Enforces respect of image contours, color homogeneity and compactness at the same time Definition of the linear path to superpixel barycenter Additional features within the clustering distance SCALP result Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 7 / 16

9 SCALP - Linear Path Definition Path between pixel p at position X p and superpixel S k of barycenter X k Near real-time computation (Bresenham, et al. 1965) Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 8 / 16

10 SCALP - Features on Linear Path P k p Improvement of color distance: d c(p, C k ) = (l p l Ck ) 2 + (a p a Ck ) 2 + (b p b Ck ) 2 d c(p, C k, P k p ) = λdc(p, C 1 k) + (1 λ) P k d c(q, C k ) p q P k p Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 9 / 16

11 SCALP - Features on Linear Path P k p Improvement of color distance: d c(p, C k ) = (l p l Ck ) 2 + (a p a Ck ) 2 + (b p b Ck ) 2 d c(p, C k, P k p ) = λdc(p, C 1 k) + (1 λ) P k d c(q, C k ) p q P k p Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 9 / 16

12 Superpixel Context Simple Linear Iterative Clustering The SCALP method Results Conclusion SCALP Framework Properties: Regularity/compactness of the decomposition Homogeneity of the color clustering Adherence to the image contours R emi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 10 / 16

13 SCALP Framework Properties: Regularity/compactness of the decomposition Homogeneity of the color clustering Adherence to the image contours Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 10 / 16

14 SCALP - Features on Linear Path P k p Use of Contour prior: d C (p, C k, P k p ) = 1 + γ max C(q) q P k p Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 11 / 16

15 SCALP - Features on Linear Path P k p Use of Contour prior: d C (p, C k, P k p ) = 1 + γ max C(q) q P k p Improved clustering distance: ( ) D(p, C k ) = d c(p, C k, P k p) + d s(p, C k )m d C (p, C k, P k p) Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 11 / 16

16 Superpixel Context Simple Linear Iterative Clustering The SCALP method Results Conclusion Example Results GC SEEDS SLIC (Veksler, et al. 2010) (Van den Bergh, et al. 2012) (Achanta, et al. 2012) R emi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path SCALP 12 / 16

17 Superpixel Context Simple Linear Iterative Clustering The SCALP method Results Conclusion Example Results GC SEEDS SLIC (Veksler, et al. 2010) (Van den Bergh, et al. 2012) (Achanta, et al. 2012) R emi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path SCALP 12 / 16

18 Quantitative Results Validation on Berkeley segmentation dataset (BSD) (Martin, et al. 2001) 200 test images with human ground truth segmentations Undersegmentation Error (UE): Overlap with multiple objects Compactness: Regularity of the produced superpixels F-measure: Contour detection performances (PR) (Martin, et al. 2004) Method UE Compactness F-measure GC (Veksler, et al. 2010) SEEDS (Van den Bergh, et al. 2012) SLIC (Achanta, et al. 2012) SCALP Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 13 / 16

19 Precision Quantitative Results (PR) [F = 0.787] Human [F = 0.596] GC [F = 0.581] SEEDS [F = 0.649] SLIC [F = 0.676] SCALP Recall Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 14 / 16

20 Precision Quantitative Results (PR) - Influence of parameters [F = 0.787] Human [F = 0.649] SLIC [F = 0.660] SLIC + Color [F = 0.671] SLIC + Contour [F = 0.676] SLIC + Color + Contour = SCALP Recall Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 15 / 16

21 Conclusion SCALP method summary New general method to use color and contour features along linear path Increase of contour adherence, compactness and respect of image objects Limited computational time (< 0.4s) Work in progress Use of advanced color features (Li et al., 2015) State-of-the-art results Perspectives Adaptation to supervoxels (3D and video) Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 16 / 16

22 SCALP: Superpixels with Contour Adherence using Linear Path Rémi Giraud 1,2 with Vinh-Thong Ta 1 and Nicolas Papadakis 2 1 LaBRI CNRS UMR University of Bordeaux, FRANCE PICTURA Research Group 2 IMB CNRS UMR University of Bordeaux, FRANCE

23 References Superpixel Metrics Superpixel segmentation S, Ground-truth segmentation G Precision-Recall Framework: (Average of superpixel boundaries on several scales) R = B(S) B(G) / B(G) P = B(S) B(G) / B(S) F = (2.P.R)/(P + R) Undersegmentation Error: (Overlap with multiple objects) i UE(S, G) = 1 I k:s k G i S k\g i Achievable Segmentation Accuracy: (Respect of image objects) ASA(S, G) = 1 I k maxi S k G i Compactness: (Regularity, circularity of the superpixels) k CO(S) = 1 I 4π S k 2 B(S k ) 2 Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 1 / 5

24 References UE Quantitative Results (UE) GC SEEDS SLIC SCALP Number of Superpixels Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 2 / 5

25 References ASA Quantitative Results on BSD (ASA) GC SEEDS SLIC SCALP Number of Superpixels Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 3 / 5

26 References SLIC - Limitations R emi Giraud (PICTURA) m=5 m = 10 m = 20 m = 50 SCALP: Superpixels with Contour Adherence using Linear Path 4/5

27 References References [1, 2, 3, 4, 5, 6] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, SLIC superpixels compared to state-of-the-art superpixel methods, PAMI, vol. 34, no. 11, pp , M. Van den Bergh, X. Boix, G. Roig, B. de Capitani, and L. Van Gool, SEEDS: Superpixels extracted via energy-driven sampling, in ECCV, 2012, pp D. Martin, C. Fowlkes, D. Tal, and J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in ICCV, vol. 2, 2001, pp J. E. Bresenham, Algorithm for computer control of a digital plotter, IBM Syst. J., vol. 4, no. 1, pp , O. Veksler, Y. Boykov, and P. Mehrani, Superpixels and supervoxels in an energy optimization framework, in ECCV, 2010, pp D. R. Martin, C. C. Fowlkes, and J. Malik, Learning to detect natural image boundaries using local brightness, color, and texture cues, PAMI, vol. 26, no. 5, pp , Rémi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 5 / 5