Shape analysis through geometric distributions

Transcription

1 Shape analysis through geometric distributions Nicolas Charon (CIS, Johns Hopkins University) joint work with B. Charlier (Université de Montpellier), I. Kaltenmark (CMLA), A. Trouvé, Hsi-Wei Hsieh (JHU)... Applications-Driven Geometric Functional Data Analysis Workshop, FSU

2 1. Representation of shapes as generalized distributions 2. Metric spaces of oriented varifolds 3. Applications

3 1 Shapes as distributions: Shape similarity metrics between unparametrized submanifolds Our goal is to propose a notion of shape similarity that: Is intrinsically parametrization-invariant. Leads to closed form expressions for the metrics and can be easily differentiated. Applies consistently to continuous or discrete objects. Is robust to imperfections: noise, disconnections, segmentation issues... Can handle a variety of situations: open curves and surfaces, bundles, multi-shapes... 3/24

4 1 Shapes as distributions: 4/24 One must abandon the idea of describing all the competing surfaces by continuous maps from a single predetermined parameter space. One should rather think of surfaces as m-dimensional mass distributions, with tangent m-vectors attached. H. Federer

5 1 Shapes as distributions: 5/24 A simple case: point clouds A point cloud is by definition a set of unlabelled point positions {x k } k=1,..,p in a vector space R n

6 1 Shapes as distributions: A simple case: point clouds Alternatively, it can be viewed as a sum of Dirac distributions p k=1 δ x k acting on the set of functions f C 0 (R n, R) : p δ xk (f ) = k=1 p f (x k ) This gives an embedding of all point clouds in a functional space : the set of signed measures of R n, C 0 (R n, R), which can be then equipped with various possible metrics 1. k=1 1 Glaunès, Trouvé, Younes. Diffeomorphic matching of distributions. A new approach for unlabelled point sets and submanifolds matching /24

7 1 Shapes as distributions: An extended framework: oriented varifolds 2 We focus on the special case of curves and surfaces in R n, n = 2, 3. Definition An oriented varifold of R n is a distribution (or measure) on the space R n S n 1. Specifically, if W C 0 (R n S n 1 ) is a given Banach space of continuous test functions, an oriented varifold is an element of W. 2 Kaltenmark, Charlier, Charon. A general framework for curve and surface comparison and registration with oriented varifolds. CVPR /24

8 1 Shapes as distributions: An extended framework: oriented varifolds 2 We focus on the special case of curves and surfaces in R n, n = 2, 3. Definition An oriented varifold of R n is a distribution (or measure) on the space R n S n 1. Specifically, if W C 0 (R n S n 1 ) is a given Banach space of continuous test functions, an oriented varifold is an element of W. Example: the Dirac delta distribution δ (x,u) correspond to a singular mass located at position x R n carrying the unit oriented vector u S n 1. 2 Kaltenmark, Charlier, Charon. A general framework for curve and surface comparison and registration with oriented varifolds. CVPR /24

9 1 Shapes as distributions: The oriented varifold representation of shapes Given an embedded oriented curve or surface X, one can define the map µ : X µ X by: µ X (ω) = X ω(x, u(x))dvol(x) for all ω W where u(x) is the unit tangent or normal vector to X at x. 8/24

10 1 Shapes as distributions: The oriented varifold representation of shapes Given an embedded oriented curve or surface X, one can define the map µ : X µ X by: µ X (ω) = X ω(x, u(x))dvol(x) for all ω W where u(x) is the unit tangent or normal vector to X at x. In other words, the mapping q µ q for q Emb(M, R n ) is invariant to positive reparametrizations in Diff + (M). 8/24

11 1 Shapes as distributions: 8/24 The oriented varifold representation of shapes Given an embedded oriented curve or surface X, one can define the map µ : X µ X by: µ X (ω) = X ω(x, u(x))dvol(x) for all ω W where u(x) is the unit tangent or normal vector to X at x. In other words, the mapping q µ q for q Emb(M, R n ) is invariant to positive reparametrizations in Diff + (M). µ extends to d-dimensional rectifiable subsets of R n : ( ) µ X (ω) = ω(x, u(x))dh d (x) = δ (x,u(x)) dh d (x) (ω) X X

12 1 Shapes as distributions: 9/24 Discrete shapes and approximation Discrete shapes are usually given as X = p i=1 X i with cells X i (segments or faces). The associated µ X = p i=1 µ X i can be conveniently approximated by the discrete varifold: µ X = p i=1 r iδ (xi, t i )

13 1. Representation of shapes as generalized distributions 2. Metric spaces of oriented varifolds 3. Applications

14 2 Varifold metrics: 11/24 Dual metrics and induced distance Idea: given a norm on W and the dual norm W, define the distance induced by µ: d W (X 1, X 2 ) = µ X1 µ X2 W.

15 2 Varifold metrics: Dual metrics and induced distance Idea: given a norm on W and the dual norm W, define the distance induced by µ: d W (X 1, X 2 ) = µ X1 µ X2 W. Remaining issues: In general, d W is only a pseudo-distance: conditions on W for µ to be injective? Closed form expression of the distance? 11/24

16 2 Varifold metrics: 12/24 Separable kernel metrics on oriented varifolds Construct W and W with the following conditions: W is a Reproducing Kernel Hilbert Space (RKHS) generated by a kernel k on R n S n 1. k is separable: k(x, u, y, v) = k pos (x, y)k or (u, v). k pos (x, y). = ρ( x y 2 ) is a C 1 radial positive kernel on R n. k or (u, v). = γ(u v) is a C 1 zonal positive kernel on S n 1.

17 2 Varifold metrics: 12/24 Separable kernel metrics on oriented varifolds Construct W and W with the following conditions: W is a Reproducing Kernel Hilbert Space (RKHS) generated by a kernel k on R n S n 1. k is separable: k(x, u, y, v) = k pos (x, y)k or (u, v). k pos (x, y). = ρ( x y 2 ) is a C 1 radial positive kernel on R n. k or (u, v). = γ(u v) is a C 1 zonal positive kernel on S n 1. Under those assumptions, one has W C0 1(Rn S n 1 ) and: µ X1, µ X2 W = ρ( x 1 x 2 2 ) γ(u 1 (x 1 ) u 2 (x 2 ))dh d (x 1 )dh d (x 2 ) X 2 X 1

18 2 Varifold metrics: 13/24 Induced distance Theorem (Hsieh, C.) If, in addition, k pos is a C 0 -universal kernel on R n and for all u S n 1, k or (u, u) > 0 then d W (X 1, X 2 ) = µ X µ Y W is a distance between rectifiable subsets. We also have the following invariance: Property The action of rigid motions is isometric for d W, i.e for all (R, b) SO n (R) R n : d W (RX 1 + b, RX 2 + b) = d W (X 1, X 2 ).

19 2 Varifold metrics: 14/24 Examples of kernels Standard choices for k pos are Gaussian kernels k pos (x, y) = e x y 2 σ 2 (smooth, radial and C 0 universal) or sum of Gaussian kernels (for multiscale problems), inverse multiquadrics...

20 2 Varifold metrics: Examples of kernels For k or, multiple possibilities leading to fundamentally different properties: k or ( u, v) = u v (linear kernel): equivalent to currents. k or ( u, v) = ( u v) 2 (Binet kernel): unoriented varifold. 2( u v) σ k or ( u, v) = e s 2 non-linear. (oriented spherical Gaussian): oriented but 14/24

21 1. Representation of shapes as generalized distributions 2. Metric spaces of oriented varifolds 3. Applications

22 16/24 Shape clustering based on varifold-type metrics Figure: Kimia shapes database We compute a rigid-invariant metric by quotienting out rotations and translations: d(x, Y ) = inf µ RX +b µ Y W (R,b) SO(n) R n

23 17/24 Clustering results Method 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th Current Unoriented varifold Oriented varifold Method 9 th 10 th 11 th Total Current % Unoriented varifold % Oriented varifold % Table: Clustering scores for k nearest neighbors, k = 1,..., 11 overall percentage of correct assignment.

24 Registration of curves, surfaces and more... Those metrics can be embedded as fidelity terms in the many existing registration models. For example, in the case of Large Deformation Diffeomorphic Metric Mapping (LDDMM) 3, registration of template shape X 0 on target Y consists in the optimal control problem: min v J(v). = 1 v t 2 V dt +γ µ φ v 1 (X 0) µ Y 2 W 0 }{{ }}{{} fidelity term regularization term over time-varying velocity field v L 2 ([0, 1], V ), with φ v t the flow application at time t. 3 Beg, Miller, Trouvé, and Younes. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International journal of computer vision, /24

25 Registration of curves, surfaces and more... Those metrics can be embedded as fidelity terms in the many existing registration models. For example, in the case of Large Deformation Diffeomorphic Metric Mapping (LDDMM) 3, registration of template shape X 0 on target Y consists in the optimal control problem: min v J(v). = 1 v t 2 V dt +γ µ φ v 1 (X 0) µ Y 2 W 0 }{{ }}{{} fidelity term regularization term over time-varying velocity field v L 2 ([0, 1], V ), with φ v t the flow application at time t. Optimal solutions correspond to geodesics on a diffeomorphism group and the problem can be solved efficiently in practice with geodesic shooting algorithms. 3 Beg, Miller, Trouvé, and Younes. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International journal of computer vision, /24

26 3 Applications: Example of registration Linear Binet Oriented Gaussian Figure: Registration for the three different kernels kor 19/24

27 20/24 Registration with noise Linear Binet Oriented Gaussian Figure: Registration on noisy target for three different kernels k or

28 21/24 Shape denoising with currents Smooth surface Noisy version

29 21/24 Shape denoising with currents Smooth surface Noisy version Deformation (t=0)

30 21/24 Shape denoising with currents Smooth surface Noisy version Deformation (t=0.3)

31 21/24 Shape denoising with currents Smooth surface Noisy version Deformation (t=0.6)

32 21/24 Shape denoising with currents Smooth surface Noisy version Deformation (t=1)

33 21/24 Shape denoising with currents Smooth surface Noisy version Matched shape + smooth target

34 22/24 Partial shapes Unoriented varifolds are advantageous in the situation of incomplete shapes: Fully sampled surface 30% of faces left

35 22/24 Partial shapes Unoriented varifolds are advantageous in the situation of incomplete shapes: Fully sampled surface 30% of faces left

36 22/24 Partial shapes Unoriented varifolds are advantageous in the situation of incomplete shapes: Fully sampled surface 30% of faces left

37 22/24 Partial shapes Unoriented varifolds are advantageous in the situation of incomplete shapes: Fully sampled surface 30% of faces left

38 22/24 Partial shapes Unoriented varifolds are advantageous in the situation of incomplete shapes: Fully sampled surface 30% of faces left

39 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

40 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

41 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

42 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

43 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

44 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching

45 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

46 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

47 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

48 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

49 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

50 23/24 Multi-varifolds Multi-dimensional objects can be similarly represented and compared as direct sums of varifolds. Example: represent a surface with boundary as µ = µ X µ X. Figure: Surface matching Figure: Surface + curve matching

51 Other related directions Varifold relaxation for intrinsic metric mapping. 4 Sparse approximations of shapes. Incorporate curvature information in the representation and distance: with mutlidirectional varifolds 5 or normal cycles 6. Scale invariance/robustness: use similarity terms based on optimal transport 7? 4 Bauer,Bruveris,Charon,Moeller-Andersen. Varifold-based matching of curves via Sobolev-type Riemannian metrics Rekik,Li,Lin,Shen. Multidirectional and Topography-based Dynamic-scale Varifold Representations with Application to Matching Developing Cortical Surfaces Roussillon,Glaunès. Kernel Metrics on Normal Cycles and Application to Curve Matching Feydi,Charlier,Vialard,Peyré. Optimal Transport for Diffeomorphic Registration /24