Estimating affine-invariant structures on triangle meshes

Transcription

1 Estimating affine-invariant structures on triangle meshes Thales Vieira Mathematics, UFAL Dimas Martinez Mathematics, UFAM Maria Andrade Mathematics, UFS Thomas Lewiner École Polytechnique

2 Invariant descriptors for shape analysis Rigid surface registration Gelfand et al (2005)

3 Invariant descriptors for shape analysis Rigid surface registration Gelfand et al (2005) Rigid shape descriptors

4 Invariant descriptors for shape analysis Rigid surface registration Non-rigid shape matching & similarity Gelfand et al (2005) Gal and Cohen-Or (2006) Rigid shape descriptors

5 Invariant descriptors for shape analysis Rigid surface registration Local/rigid shape descriptors Non-rigid shape matching & similarity Gelfand et al (2005) Gal and Cohen-Or (2006) Rigid shape descriptors

6 Invariant descriptors for shape analysis Rigid surface registration Local/rigid shape descriptors Non-rigid shape matching & similarity Gelfand et al (2005) Gal and Cohen-Or (2006) Rigid shape descriptors How about non-rigid shape descriptors?

7 Space of transformations Let A 2 M 3 (R) and t 2 R 3.

8 Space of transformations Let A 2 M 3 (R) and t 2 R 3. Rigid transformations (Euclidean geometry) T (p) =Ap + t, A T = A 1.

9 Space of transformations Let A 2 M 3 (R) and t 2 R 3. Rigid transformations (Euclidean geometry) T (p) =Ap + t, A T = A 1. Non-rigid transformations T (p) =?

10 Space of transformations Let A 2 M 3 (R) and t 2 R 3. Rigid transformations (Euclidean geometry) T (p) =Ap + t, A T = A 1. Non-rigid transformations T (p) =? Equi-affine transformations (Affine geometry) T (p) =Ap + t, det(a) =1.

11 Related Euclidean Work Measures from Euclidean geometry: invariant to rigid transformations Euclidean distance Mean / Gaussian curvature Gelfand et al (2005)

12 Related Euclidean Work Measures from Euclidean geometry: invariant to rigid transformations Euclidean distance Mean / Gaussian curvature Gelfand et al (2005) How to estimate these geometric measures on triangle meshes?

13 Related Euclidean Work Estimating the curvature tensor from normal vectors Taubin (1995) Rusinkiewicz (2003)

14 Euclidean geometry background Shape operator S e : T p S! T p S S e w = D w n

15 Euclidean geometry background Shape operator S e : T p S! T p S S e w = D w n Principal directions and curvatures S e e e min = ke min ee min S e e e max = kmaxe e e max e e max,e e min 2 T ps Gaussian and mean curvatures K =det(s e ) H = 1 2 tr(se )

16 Euclidean geometry background Shape operator S e : T p S! T p S S e w = D w n Principal directions and curvatures S e e e min = ke min ee min S e e e max = kmaxe e e max e e max,e e min 2 T ps Gaussian and mean curvatures K =det(s e ) H = 1 2 tr(se ) Fixing an orthonormal frame {e 1, e 2 } of T p S: 0 S e =(D e1 n, D e2 hd 1 e 1 n, e 1 i hd e2 n, e 1 i A hd e1 n, e 2 i hd e2 n, e 2 i

17 Euclidean geometry background Fixing an orthonormal frame {e 1, e 2 } of T p S: 0 S e =(D e1 n, D e2 hd e 1 n, e 1 i hd e1 n, e 2 i Rusinkiewicz tensor hd e2 n, e 1 i hd e2 n, e 2 i 1 A

18 Euclidean geometry background Fixing an orthonormal frame {e 1, e 2 } of T p S: 0 S e =(D e1 n, D e2 hd e 1 n, e 1 i hd e1 n, e 2 i Rusinkiewicz tensor hd e2 n, e 1 i hd e2 n, e 2 i 1 A 1. Compute a normal vector field by averaging face normals

19 Euclidean geometry background Fixing an orthonormal frame {e 1, e 2 } of T p S: 0 S e =(D e1 n, D e2 hd e 1 n, e 1 i hd e1 n, e 2 i Rusinkiewicz tensor hd e2 n, e 1 i hd e2 n, e 2 i 1 A 1. Compute a normal vector field by averaging face normals 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j x hn i n j, e 2 i 3 x1 x 2

20 Euclidean geometry background Fixing an orthonormal frame {e 1, e 2 } of T p S: 0 S e =(D e1 n, D e2 hd e 1 n, e 1 i hd e1 n, e 2 i Rusinkiewicz tensor hd e2 n, e 1 i hd e2 n, e 2 i 1 A 1. Compute a normal vector field by averaging face normals 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j x hn i n j, e 2 i 3 x1 3. Apply least-squares to find shape operators per face x 2

21 Euclidean geometry background Rusinkiewicz tensor 1. Estimate a normal vector field by averaging face normals x 2 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j hn i n j, e 2 i 3. Apply least-squares to find shape operators per face x 3 x 1 4.To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system.

22 Euclidean geometry background Rusinkiewicz tensor 1. Estimate a normal vector field by averaging face normals x 2 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j hn i n j, e 2 i 3. Apply least-squares to find shape operators per face x 3 x 1 4.To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system.

23 Euclidean geometry background Rusinkiewicz tensor 1. Estimate a normal vector field by averaging face normals x 2 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j hn i n j, e 2 i 3. Apply least-squares to find shape operators per face x 3 x 1 4.To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system.

24 Euclidean geometry background Rusinkiewicz tensor 1. Estimate a normal vector field by averaging face normals x 2 2. Discretize the shape operator per face applying finite differences per edge 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j hn i n j, e 2 i 3. Apply least-squares to find shape operators per face x 3 x 1 4.To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system. only normal vectors are required to discretize the shape operator!

25

26 Related Affine Work Measures from affine geometry: invariance to equi-affine transformations (volume preserving) Affine Mean / Gaussian curvature Affine principal directions, Andrade et al (2012)

27 Related Affine Work Measures from affine geometry: invariance to equi-affine transformations (volume preserving) Affine Mean / Gaussian curvature Affine principal directions, How to estimate these affine measures on triangle meshes? Andrade et al (2012)

28 Related Affine Work

29 Related Affine Work Equi-affine metric tensor Raviv et al (2011)

30 Related Affine Work Equi-affine metric tensor Equi-affine curvature tensor on implicit surfaces Raviv et al (2011) Andrade and Lewiner (2012)

31 Contributions Estimators for equi-affine differential structures on triangle meshes: Discrete affine normal vector Affine curvature tensor Affine Gaussian and mean curvatures Affine principal directions and curvatures

32 Outline 1. Affine geometry background 2. Least-squares based affine normal estimator 3. Affine shape operator estimator 4. Experiments 5. Limitations & Future Work

33 Affine geometry background Affine co-normal vector = K e 1/4 N e Affine normal vector The a ne normal vector is implicitly defined by the equations h, i =1 h, u i = h, v i =0

34 Affine geometry background Affine co-normal vector = K e 1/4 N e Affine normal vector a ne contravariant: m(a(p)) = (A 1 ) T (m(p)) The a ne normal vector is implicitly defined by the equations h, i =1 h, u i = h, v i =0

35 Affine geometry background Affine co-normal vector = K e 1/4 N e Affine normal vector a ne contravariant: m(a(p)) = (A 1 ) T (m(p)) a ne covariant: m(a(p)) = A(m(p)) The a ne normal vector is implicitly defined by the equations h, i =1 h, u i = h, v i =0

36 Affine geometry background Affine shape operator For each p 2 S, the operator S a : T p S! T p S defined by S a w = D w is called the a ne shape operator of S at p. Affine principal directions and curvatures Affine Gaussian and mean curvatures K a =det(s a ) H a = 1 2 tr(sa ) S a e a max = kmaxe a a max S a e a min = ka min ea min e a max,e a min 2 T ps

37 Affine geometry background Affine shape operator For each p 2 S, the operator S a : T p S! T p S defined by S a w = D w is called the a ne shape operator of S at p. Affine principal directions and curvatures Affine Gaussian and mean curvatures K a =det(s a ) H a = 1 2 tr(sa ) S a e a max = kmaxe a a max S a e a min = ka min ea min e a max,e a min 2 T ps a ne invariant: m(a(p)) = m(p)

38 Outline 1. Affine geometry background 2. Least-squares based affine normal estimator 3. Affine shape operator estimator 4. Experiments 5. Limitations & Future Work

39 Least-squares based affine normal estimator Affine co-normal vector = K e 1/4 N e

40 Least-squares based affine normal estimator Affine co-normal vector Estimated by averaging per-face normals = K e 1/4 N e discretization i = K e i 1/4 n i Estimated from Rusinkiewicz tensor

41 Least-squares based affine normal estimator Affine normal vector h, i =1 h, u i = h, v i =0

42 Least-squares based affine normal estimator Affine normal vector h, i =1 h, u i = h, v i =0 Any directional derivative of the co-normal vector is orthogonal to the affine normal vector!

43 Least-squares based affine normal estimator Affine normal vector h, i =1 h, u i = h, v i =0 Any directional derivative of the co-normal vector is orthogonal to the affine normal vector! Directional derivatives of the co-normal vector D j i j i kx j x i k i j x i x j

44 Least-squares based affine normal estimator Affine normal vector h, i =1 h, u i = h, v i =0 Any directional derivative of the co-normal vector is orthogonal to the affine normal vector! Directional derivatives of the co-normal vector D j i j i kx j x i k x i

45 Least-squares based affine normal estimator Affine normal vector h, i =1 h, u i = h, v i =0 Any directional derivative of the co-normal vector is orthogonal to the affine normal vector! Directional derivatives of the co-normal vector D j i j i kx j x i k Estimated directional derivatives are prone to approximation error! x i

46 Least-squares based affine normal estimator Affine normal vector hard constraint soft constraints h, i =1 h, u i = h, v i =0

47 Least-squares based affine normal estimator Affine normal vector soft constraints Least-squares minimization hard constraint h, i =1 h, u i = h, v i =0 Let v =(x, y, z), i =(a, b, c) and D j i =(a j,b j,c j ). i v D j i

48 Least-squares based affine normal estimator Affine normal vector soft constraints Least-squares minimization hard constraint h, i =1 h, u i = h, v i =0 Let v =(x, y, z), i =(a, b, c) and D j i =(a j,b j,c j ). The hard constraint h i, vi = 1 can be written as i v D j i x = 1 by cz a

49 Least-squares based affine normal estimator Affine normal vector soft constraints Least-squares minimization hard constraint h, i =1 h, u i = h, v i =0 Let v =(x, y, z), i =(a, b, c) and D j i =(a j,b j,c j ). The hard constraint h i, vi = 1 can be written as i v D j i x = 1 by cz a minimize y,z X j2n 1 (i) w j b a a j + b j y + c a a j + c j z + a j a 2

50 Least-squares based affine normal estimator Affine normal vector soft constraints Least-squares minimization hard constraint h, i =1 h, u i = h, v i =0 i v D j i Let v =(x, y, z), i =(a, b, c) and D j i =(a j,b j,c j ). The hard constraint h i, vi = 1 can be written as x = 1 by cz a sum of areas of adjacent triangles to the edge minimize y,z X j2n 1 (i) w j b a a j + b j y + c a a j + c j z + a j a 2

51 Outline 1. Affine geometry background 2. Least-squares based affine normal estimator 3. Affine shape operator estimator 4. Experiments 5. Limitations & Future Work

52 Affine shape operator estimator Euclidean shape operator S e : T p S! T p S Affine shape operator S a : TpS! TpS S e w = D w n S a w = D w

53 Affine shape operator estimator Euclidean shape operator S e : T p S! T p S Affine shape operator S a : TpS! TpS S e w = D w n S a w = D w Derivatives of vector fields on surfaces

54 Back to the Euclidean Rusinkiewicz tensor Rusinkiewicz tensor 1. Estimate a normal vector field by averaging face normals x 2 2. Discretize the shape operator per face using finite differences 0 S e [e ij ]=@ hn 1 i n j, e 1 i A, e ij = x i x j hn i n j, e 2 i 3. Apply least-squares to find shape operators per face x 3 x 1 4.To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system. only normal vectors are required to discretize the shape operator!

55 Rusinkiewicz tensor + affine geometry Rusinkiewicz tensor 1. Estimate an affine normal vector field x Discretize the shape operator per face using finite differences 0 i j S e [e ij ]=@ hn 1 i n j, e 1 i A, eij = x i x j hn i n j, e 2 i i j 3. Apply least-squares to find shape operators per face 0 x 3 x To find the shape operator at each vertex, average adjacent face operators after changing coordinates to a common coordinate system.

56 Affine curvatures estimators Affine principal directions and curvatures S a e a max = k a maxe a max S a e a min = ka min ea min e a max,e a min 2 T ps Affine Gaussian and mean curvatures K a =det(s a ) H a = 1 2 tr(sa )

57 Affine curvatures estimators Affine principal directions and curvatures S a e a max = k a maxe a max S a e a min = ka min ea min e a max,e a min 2 T ps diagonalize estimated S a Affine Gaussian and mean curvatures K a =det(s a ) H a = 1 2 tr(sa ) straightforward from estimated S a

58 Experiments

59 Convergence and scalability Meshes uniformly tessellated from analytical models at different resolutions: Comparison with exact measures from analytical models Average relative error as a measure of accuracy

60 Sphere Paraboloid Torus Ellipsoid

61 Sphere Paraboloid Torus Ellipsoid due to sampling on low K e regions

62 K a H a e a min K a and

63 K a non-uniform color distribution H a e a min K a and

64 Mesh regularity K a, non-smoothed K a, smoothed

65 Comparison with Andrade and Lewiner (2012) Not so fair: explicit vs. implicit: mean absolute error for meshes with vertices implicit ours K a error on the sphere

66 Invariance Experiments on equi-affine transformations with increasing spectral radius Average relative covariance error: E ( ) = nx i=1 k i (A (M)) A ( i (M))k ka ( i (M))k Average relative error: E K a( ) = nx i=1 K a i (A (M)) K a i (M) K a i (M)

67 Sphere Torus Horse

68 Sphere Torus Horse

69 Invariance on real-world meshes

70 Error correlation Affine estimators error depends on Euclidian Gaussian curvature estimators accuracy! i = K e i 1/4 n i Estimated from Rusinkiewicz tensor

71 ellipsoid K e error error Enneper s surface K e error K a error

72 Limitations & Future Work Estimators accuracy depends on mesh regularity: better weighting schemes? minimize y,z X j2n 1 (i) w j b a a j + b j y + c a a j + c j z + a j a 2 Estimators accuracy depends on Euclidean estimators Next step: develop practical affine invariant descriptors for shape analysis which kind of neighborhoods are more accurate for our estimators?

73 Thank you for your attention! Questions?