Stable and Multiscale Topological Signatures

Transcription

1 Stable and Multiscale Topological Signatures Mathieu Carrière, Steve Oudot, Maks Ovsjanikov Inria Saclay Geometrica April 21, / 31

2 Shape = point cloud in R d (d = 3) 2 / 31

3 Signature = mathematical objects used in shape analysis Can be very different by nature: local/global intrinsic/extrinsic volumetric/defined on the surface type of information (geometry, topology...) Satisfy 3 main properties: invariant to a relevant deformation class (rotation, scaling...) stability informativeness 3 / 31

4 Most common signatures: curvature (mean, gaussian...) PCA features spin image shape context shape diameter function heat kernel signature wave kernel signature geodesic features (eccentricities...) Lack of mathematical framework to define stability Idea: use persistent homology to build topological point signatures on shapes that are: provably stable both local and global 4 / 31

5 Persistence Diagrams Mapping to Signatures Shape Matching Shape Segmentation 5 / 31

6 Persistence Diagrams (PDs) are the building blocks of the topological signature Let x shape S. We introduce a metric d g on S and we compute the persistent homology of the sub-level set filtration of the distance function f x (y) = d g (x, y) S = sampling from compact connected smooth manifold of dimension 2 without boundary no homology of dimension 3 trivial homology of dimension 0 and 2 only interesting dimension is 1 Let PD(f x ) = PD 1 (F x ) where F x = {fx 1 ([0, α[)} α R+ In practice, F x has a finite index set 6 / 31

7 d g is computed with Dijkstra s algorithm Edges come from: a triangulation of the shape a neighborhood graph if no triangulation is given 7 / 31

8 8 / 31

9 Problem: 1D persistence is costly to compute use symmetry Theorem: [Cohen-Steiner, Edelsbrunner, Harer, 2009] For a real-valued function f on a d-manifold, the ordinary dimension r persistent classes of f correspond to the ordinary dimension d r 1 persistent classes of f We focus on the ordinary dimension 0 persistent classes of f x Essential dimension 1 persistent classes are lost PDs are much easier to compute (Union-Find data structure) 9 / 31

10 Stability? Definition: Let (X, d X ) and (Y, d Y ) be two metric spaces. A correspondence between them is a subset C of X Y such that: x X, y Y s.t. (x, y) C y Y, x X s.t. (x, y) C Definition: The metric distortion ɛ m of a correspondence is: ɛ m = sup (x,y) C,(x,y ) C d X (x, x ) d Y (y, y ) x x y y Definition: Let f : X R and g : Y R. The functional distortion ɛ f of a correspondence is: ɛ f = sup (x,y) C f (x) g(y) 10 / 31

11 Theorem: Let S 1 and S 2 be two compact Riemannian manifolds. Let f : S 1 R and g : S 2 R be two c-lipschitz functions. Let x S 1, y S 2 and a correspondence C such that (x, y) C. Then, for sufficiently small ɛ m : d b (PD(f ), PD(g)) 19cɛ m + ɛ f Theorem: Let S 1 and S 2 be two compact Riemannian manifolds. Let x S 1, y S 2 and a correspondence C such that (x, y) C. Then, for sufficiently small ɛ m : d b (PD(f x), PD(f y )) 20ɛ m Nearly-isometric shapes have very similar PDs for corresponding points 11 / 31

12 d b is costly to compute in practice It is hard to define simple quantities like means or variance in the space of PDs Send the PDs to R d! How? Need to be oblivious to the points order: look at the distance distribution add extra distance-to-diagonal terms sort the final values for stability add null values to the topological signatures so they have the same dimension 12 / 31

13 (p, q) PD, we compute: m(p, q) = min( p q, d (p), d (q)) x 2 x 1 x3 x 4 x 1 x 3 x 2 x 3 x 1 x 2 d (x 4 ) d (x 4 ) d (x 4 ) 13 / 31

14 Stability is preserved. Let x S 1 and y S 2. If X and Y are the topological signatures computed from PD(f x ) and PD(f y ): C(N) X Y 2 X Y d b (PD(f x), PD(f y )) N is the dimension (can be 50...) C(N) = 2 N(N 1) (can be quite small...) Most kernels methods in ML needs 2... Stability is preserved whatever the number of components kept! Invariant to scaling via log-scale 14 / 31

15 Visualization of the signature stability: 15 / 31

16 MDS on the signatures with : 16 / 31

17 knn segmentation with the signatures and : 17 / 31

18 Application: functional maps Let S 1 and S 2 be two shapes. Functional maps are linear applications L 2 (S 1 ) L 2 (S 2 ) Finite case: functions are vectors, linear maps are matrices Possibility to derive a correspondence from the functional map: shape matching 18 / 31

19 Assume: S1 and S 2 have the same number of points n you have m functions defined on them stored in matrices G1 and G 2 of sizes n m you have a diagonal m m matrix D weighting the functions Then solve the following problem: C = argmin C (CG 1 G 2 )D F In practice, D is computed over a set of training shapes We used the topological signature in addition to the other classical signatures 19 / 31

20 Weights: 20 / 31

21 Effects on the correspondence quality: 21 / 31

22 Improvements on the shapes: 22 / 31

23 Application: shape segmentation and labeling A segmentation of a shape with n vertices is a nth dimensional vector c giving a label to every point Goal: find the segmentation of a test shape given the ones of several training shapes Supervised algorithms: map every vertex v of label l from a shape to its signature vector x R d consider the pairs (x, l) in the training set as realizations of pairs of random variables (X, L) 23 / 31

24 Assume the test shape has n vertices. The core of supervised algorithms: model (with training set): f (c) = P(L 1 = c 1... L n = c n X 1 = x 1... X n = x n ) derive c = argmax c f (c) evaluate result through comparison to ground-truth segmentation c gt (often given manually) with specific comparison functions d: ɛ = d(c, c gt ) Recognition rate : d(c, c gt ) = 1 n n i=1 1 ci =c gt i Rand Index : d(c, c gt ) = ( ) n 1 2 i<j (C ijp ij + (1 C ij )(1 P ij )) where C ij = 1 iif c i = c j and P ij = 1 iif c gt i = c gt j 24 / 31

25 Attention, there are dependencies between the labels, conditionally to the test shape! n f (c) P(L i = c i X 1 = x 1... X n = x n ) i=1 Indeed, if all the neighbors of v have same label l, it is very unlikely for v s label to be l Instead: conditional Markov property: P(L i = c i L j = c j, j i, X ) = P(L i = c i L j = c j, j N i, X ) where N i is the 1-ring neighborhood of vertex i in the mesh 25 / 31

26 Modeling the joint conditional probability distribution f with the conditional Markov property is the purpose of probabilistic graphical models Proposition: [Hammersley, Clifford, 1971] The family of possible joint probabilities F is: 1 n F = Z exp f i (c i, x i ) + g ij (c i, c j, x i, x j ) e ij E i=1 There is no requirements for the functions f i and g ij Z is the normalization factor 26 / 31

27 GraphCut algorithm (Boykov et al., 2001) is mostly used to find argmax f when f F Several other algorithms exist : belief propagation, sum-product, max-product... very common in probabilistic graphical models Very often, f i is a probability and g ij is a compatibility term In our case, f i is the output of a classifier (like SVM) trained on the training set We computed the f i s and c both with and without the topological signatures 27 / 31

28 SB5 SB5+PDs Human Cup Glasses Airplane Ant Chair Octopus Table Teddy Hand Plier Fish Bird Armadillo Bust Mech Bearing / 31

29 Some examples: 29 / 31

30 We introduced a provably stable topological multiscale signature for points in shapes that gives complementary information to the other classical signatures Directions for future work: Other distance functions (diffusion)? Other shape analysis tasks (classification, retrieval)? Other objects (images, point clouds of high dimension)? 30 / 31

31 Thank you! Questions? 31 / 31