Stability and Statistical properties of topological information inferred from metric data

Transcription

1 Beijing, June 25, 2014 ICML workshop: Topological Methods for Machine Learning Stability and Statistical properties of topological information inferred from metric data Frédéric Chazal INRIA Saclay

2 Introduction Data often come as (sampling of) metric spaces or sets/spaces endowed with a similarity measure with, possibly complex, topological/geometric structure. TDA: infer relevant topological and geometric features of these spaces.

3 Introduction Goals: Better understanding of the phenomena from which data have been observed. Taking advantage of the structure of data for more efficient analysis (dimensionality reduction, parametrization,...). clustering, comparison, classification, visualization, learning,...

4 Introduction Challenges: no direct access to topological/geometric informationfrom data: need of intermediate constructions (e.g. simplicial complexes); distinguish topological signal from noise: robustness, stability results; topological information may be multiscale; statistical analysis of topological information. Curve or torus?

5 Topological signatures for data Metric data set Build a geometric filtered simplicial complex on top of the data Filtered simplicial complex Compute persistent homology of the complex. Signature: persistence diagram dimensional homology generators 1-dimensional homology generators A classical approach: Build a geometric filtered simplicial complex on top of (X, ρ X ) (ρ X metric/similarity on X). being a

6 Topological signatures for data Metric data set Build a geometric filtered simplicial complex on top of the data Filtered simplicial complex Compute persistent homology of the complex. Signature: persistence diagram dimensional homology generators 1-dimensional homology generators A classical approach: Build a geometric filtered simplicial complex on top of (X, ρ X ) (ρ X metric/similarity on X). being a Compute the persistent homology of the complex persistence diagrams: multiscale topological signature.

7 Topological signatures for data camel cat elephant face head horse [C., Cohen-Steiner, Guibas, Mémoli, Oudot 09] Use the metric on the space of persistence diagrams A classical approach: 0 0 Build a geometric filtered simplicial complex on top of (X, ρ X ) (ρ X metric/similarity on X). being a Compute the persistent homology of the complex persistence diagrams: multiscale topological signature. Compare the signatures of close data sets robustness and stability results

8 Topological signatures for data A classical approach: Build a geometric filtered simplicial complex on top of (X, ρ X ) (ρ X metric/similarity on X). being a Compute the persistent homology of the complex persistence diagrams: multiscale topological signature. Compare the signatures of close data sets robustness and stability results. Statistical properties of signatures?

9 Filtered complexes and filtrations A filtered simplicial complex S built on top of a set X is a family (S a a R) of subcomplexes of some fixed simplicial complex S with vertex set X s. t. S a S b for any a b. A filtration F of a space X is a nested family (F a a R) of subspaces of X such that F a F b for any a b. Example: If f : X R is a function, then the sublevelsets of f, F a = f 1 ((, a]) define the sublevel set filtration associated to f.

10 Filtered complexes and filtrations Rips Čech Examples: Let (X, d X ) be a metric space. The Vietoris-Rips and Čech complexes Rips(X) and Čech(X) are the filtered complexes defined by: for a R, [x 0, x 1,, x k ] Rips(X, a) d X (x i, x j ) a, for all i, j [x 0, x 1,..., x k ] Čech(X, a) k where B(x, a) = {x X : d X (x, x ) a}. i=0 B(x i, a),

11 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

12 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

13 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

14 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

15 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

16 Persistent homology of filtered complexes An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties

17 Persistent homology of filtered complexes Persistence barcode Rips parameter death An efficient way to encode the evolution of the topology (homology) of families of nested spaces (filtered complex, sublevel sets,...). Multiscale topological information. Barcodes/persistence diagrams can be efficiently computed. Stability properties 0 Add the diagonal Multiplicity: 14 birth Persistence diagram

18 Distance between persistence diagrams death Add the diagonal D 1 D 2 Multiplicity: 2 0 The bottleneck distance between two diagrams D 1 and D 2 is d B (D 1, D 2 ) = inf γ Γ sup p D 1 p γ(p) where Γ is the set of all the bijections between D 1 and D 2 and p q = max( x p x q, y p y q ). birth Persistence diagrams provide easy to compare topological signatures.

19 Filt(X µ ): persistence and stability of filtrations built on top of (pre)compact metric spaces Joint works with V. de Silva (Pomona), M. Glisse (INRIA) and S. Oudot (INRIA) F. Chazal, V. de Silva, S. Oudot, Persistence Stability for Geometric complexes, Geometria Dedicata 2014 (online first Dec. 2013). F. Chazal, V. de Silva, M. Glisse, S. Oudot, The Structure and Stability of Persistence Modules, arxiv: , 2012 (submitted as a monography).

20 Topological signatures for data dgm(filt(x)) metric data: X Filt(X) Examples: 0 - Filt(X) = Rips α (X) - Filt(X) = Čech α(x) - Filt(X) = sublevelset filtration of ρ(., M). 0 Questions: Is dgm(filt(x)) well-defined? (X may not be finite) Stability properties of dgm(filt(x))?

21 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. Examples: Let S be a filtered simplicial complex. If V a = H(S a ) and v b a : H(S a ) H(S b ) is the linear map induced by the inclusion S a S b then (H(S a ) a R) is a persistence module. Given a metric space (X, ρ), H(Rips(X)) is a persistence module. Given a metric space (X, ρ), H(Čech(X)) is a persistence module.

22 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. Definition: A persistence module V is q-tame if for any a < b, v b a has a finite rank. Theorem [CCGGO 09-CdSGO 12]: q-tame persistence modules have well-defined persistence diagrams.

23 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. An idea about the definition of persistence diagrams: d c a b Measures on rectangles: Number of points in any rectangle [a, b] [c, d] above the diagonal: rk(v c b) rk(v d b ) + rk(v d a) rk(v c a) a b c d

24 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. Definition: A persistence module V is q-tame if for any a < b, v b a has a finite rank. Theorem [CCGGO 09-CdSGO 12]: q-tame persistence modules have well-defined persistence diagrams. Theorem[CdSO 12]: Let X be a precompact metric space. Then H(Rips(X)) and H(Čech(X)) are q-tame. As a consequence dgm(h(rips(x))) and dgm(h(čech(x))) are well-defined! Recall that a metric space (X, ρ) is precompact if for any ɛ > 0 there exists a finite subset Fɛ X such that d H (X, Fɛ) < ɛ (i.e. x X, p Fɛ s.t. ρ(x, p) < ɛ).

25 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. A homomorphism of degree ɛ between two persistence modules U and V is a collection Φ of linear maps (φ a : U a V a+ɛ a R) U a V a+ɛ U b V b+ɛ such that v b+ɛ a+ɛ φ a = φ b u b a for all a b. An ε-interleaving between U and V is specified by two homomorphisms of degree ɛ Φ : U V and Ψ : V U s.t. Φ Ψ and Ψ Φ are the shifts of degree 2ɛ between U and V. U a u a+2ɛ a U a+2ɛ φ a V a+ɛ ψ a+ɛ v a+3ɛ a+ɛ V a+3ɛ

26 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. Stability Theorem [CCGGO 09-CdSGO 12]: If U and V are q-tame and ɛ-interleaved for some ɛ 0 then d B (dgm(u), dgm(v)) ɛ

27 Tame persistent modules Definition: A persistence module V is an indexed family of vector spaces (V a a R) and a doubly-indexed family of linear maps (v b a : V a V b a b) which satisfy the composition law v c b v b a = v c a whenever a b c, and where v a a is the identity map on V a. Stability Theorem [CCGGO 09-CdSGO 12]: If U and V are q-tame and ɛ-interleaved for some ɛ 0 then d B (dgm(u), dgm(v)) ɛ Strategy: build filtered complexes on top of metric spaces that induce q-tame homology persistence modules and that turns out to be ɛ-interleaved when the considered spaces are O(ɛ)-close. Need to be defined.

28 Gromov-Hausdorff distance Let (X, ρ X ) and (Y, ρ Y ) be compact metric spaces. An ɛ-correspondence C between X and Y is a subset C X Y such that: (i) x X, y Y s.t. (x, y) C, (ii) y Y, x X s.t. (x, y) C, (iii) (x, y), (x, y ) C, ρ X (x, x ) ρ Y (y, y ) ε. Y y y x x X C d GH (X, Y) = 1 2 inf{ε 0 : there exists an ε-correspondence between X and Y}

29 The example of the Rips and Čech filtrations Proposition: Let (X, ρ X ), (Y, ρ Y ) be metric spaces. For any ɛ > 2d GH (X, Y) - the persistence modules H(Rips(X)) and H(Rips(Y)) are ɛ-interleaved, - the persistence modules H(Čech(X)) and H(Čech(Y)) are ɛ-interleaved. Theorem: Let X, Y be compact metric spaces. Then d b (dgm(h(čech(x))), dgm(h(čech(y)))) 2d GH(X, Y), d b (dgm(h(rips(x))), dgm(h(rips(y)))) 2d GH (X, Y). Remark: The proofs never use the triangle inequality! The previous approch and results easily extend to other settings like, e.g. spaces endowed with a similarity measure.

30 Why persistence Even when X is compact, H p (Rips(X, a)), p 1, might be infinite dimensional for some value of a: X a It is also possible to build such an example with the open Rips complex: [x 0, x 1,, x k ] Rips(X, a ) d X (x i, x j ) < a, for all i, j

31 Why persistence Even when X is compact, H p (Rips(X, a)), p 1, might be infinite dimensional for some value of a: X a It is also possible to build such an example with the open Rips complex: [x 0, x 1,, x k ] Rips(X, a ) d X (x i, x j ) < a, for all i, j For any α, β R such that 0 < α β and any integer k there exists a compact metric space X such that for any a [α, β], H k (Rips(X, a)) has a non countable infinite dimension (can be embedded in R 4 [Droz 2013]).

32 Why persistence Even when X is compact, H p (Rips(X, a)), p 1, might be infinite dimensional for some value of a: X a It is also possible to build such an example with the open Rips complex: [x 0, x 1,, x k ] Rips(X, a ) d X (x i, x j ) < a, for all i, j For any α, β R such that 0 < α β and any integer k there exists a compact metric space X such that for any a [α, β], H k (Rips(X, a)) has a non countable infinite dimension (can be embedded in R 4 [Droz 2013]). If X is compact, then dim H 1 (Čech(X, a)) < + for all a ([Smale-Smale, C.-de Silva]). If X is geodesic, then dim H 1 (Rips(X, a)) < + for all a > 0 and Dgm(H 1 (Rips(X))) is contained in the vertical line x = 0. If X is a geodesic δ-hyperbolic space then Dgm(H 2 (Rips(X))) is contained in a vertical band of width O(δ).

33 From stability to statistical properties Joint works with B.T. Fasy (CMU), M. Glisse (INRIA), C. Labruère (Univ. Bourgogne), F. Lecci (CMU), B. Michel (LSTA Paris 6), A. Rinaldo (CMU), L. Wasserman (CMU). F. Chazal, M. Glisse, C. Labruère, B. Michel, Convergence rates for persistence diagram estimation in Topological Data Analysis, to appear in Int. Conf. on Machine Learning 2014 (ICML 2014). F. Chazal, B. Fasy, F. Lecci, A. Rinaldo, L. Wasserman, Stochastic Convergence of Persistence Landscapes and Silhouettes, to appear in ACM Symposium on Computational Geometry 2014 (arxiv version ).

34 (M, ρ) metric space Statistical setting µ a probability measure with compact support X µ. Sample n points according to µ. Examples: - Filt( X n ) = Rips α ( X n ) - Filt( X n ) = Čech α( X n ) - Filt( X n ) = sublevelset filtration of ρ(., X µ ). dgm(filt( X n )) X n Filt( X n ) 0 0 Questions: Statistical properties of dgm(filt( X n ))? dgm(filt( X n ))? as n +?

35 (M, ρ) metric space Statistical setting µ a probability measure with compact support X µ. Sample n points according to µ. Examples: - Filt( X n ) = Rips α ( X n ) - Filt( X n ) = Čech α( X n ) - Filt( X n ) = sublevelset filtration of ρ(., X µ ). dgm(filt( X n )) X n Filt( X n ) 0 0 Questions: Statistical properties of dgm(filt( X n ))? dgm(filt( X n ))? as n +? Can we do more statistics with persistence diagrams?

36 (M, ρ) metric space Statistical setting µ a probability measure with compact support X µ. Sample n points according to µ. Examples: - Filt( X n ) = Rips α ( X n ) - Filt( X n ) = Čech α( X n ) - Filt( X n ) = sublevelset filtration of ρ(., X µ ). dgm(filt( X n )) X n Filt( X n ) 0 0 Stability thm: d b (dgm(filt(x µ )), dgm(filt( X n ))) 2d GH (X µ, X n ) So, for any ε > 0, ( ) P (d b dgm(filt(x µ )), dgm(filt( X n )) ) > ε P ( d GH (X µ, X n ) > ε ) 2

37 Deviation inequality (M, ρ, µ) X µ compact X 1, X 2,, X n i.i.d. sampled according to µ. X n Filt( X n ) For a, b > 0, µ satisfies the (a, b)-standard assumption if for any x X µ and any r > 0, we have µ(b(x, r)) min(ar b, 1).

38 Deviation inequality (M, ρ, µ) X µ compact X 1, X 2,, X n i.i.d. sampled according to µ. X n Filt( X n ) For a, b > 0, µ satisfies the (a, b)-standard assumption if for any x X µ and any r > 0, we have µ(b(x, r)) min(ar b, 1). Theorem: If µ satisfies the (a, b)-standard assumption, then for any ε > 0: ( ) P (d b dgm(filt(x µ )), dgm(filt( X n )) ( ) Moreover d b (dgm(filt(x µ )), dgm(filt( X n )) lim P n where C 1 is a constant only depending on a and b. ) > ε min( 8b aε b exp( naεb ), 1). C 1 ( log n n ) 1/b ) = 1. Remark: Confidence intervals only depending on a and b (see also [Balakrishnan & al 2013] when X µ is a smooth manifold and a, b unknown).

39 Minimax rate of convergence Let P(a, b, M) be the set of all the probability measures on the metric space (M, ρ) satisfying the (a, b)-standard assumption on M:

40 Minimax rate of convergence Let P(a, b, M) be the set of all the probability measures on the metric space (M, ρ) satisfying the (a, b)-standard assumption on M: Theorem: Let (M, ρ) be a metric space and let a > 0 and b > 0. Then: sup µ P(a,b,M) E [ ] d b (dgm(filt(x µ )), dgm(filt( X n ))) C ( ln n n ) 1/b where the constant C only depends on a and b (not on M!). Assume moreover that there exists a non isolated point x in M and a sequence (x n ) such that ρ(x, x n ) (1/n) 1/b. Then for any estimator dgm n of dgm(filt(x µ )): lim inf ( n n ln n ) 1/b sup µ P(a,b,M) where C is an absolute constant. E [ d b (dgm(filt(x µ )), dgm ] n ) C Remark: we can obtain slightly better bounds if X µ is a submanifold of R D - see [Genovese, Perone-Pacifico,Verdinelli, Wasserman 2011, 2012]

41 From persistence diagrams to landscapes d d+b 2 d b 2 b D = {( d i+b i, d i+b i )}i I 2 2 Persistence landscape [Bubenik 2012]: For p = ( b+d 2, d b 2 ) D, Λ p (t) = λ D (k, t) = kmax p dgm Λ p(t), t R, k N, where kmax is the kth largest value in the set. d+b 2 t b t [b, b+d ] 2 d t t ( b+d 2, d] 0 otherwise. Stability: For any t R and any k N, λ D (k, t) λ D (k, t) d B (D, D ).

42 Convergence of landscapes Let L T be the space of landscapes with support contained in [0, T ]. Let P be a probability distribution on L T, and let λ 1,..., λ n P. Let Λ P be the mean landscape: Λ P (t) = E[λ i (t)], t [0, T ]. λ n (t) = 1 n λ i (t), t [0, T ], is a point-wise unbiased estimator of Λ P. n i=1 For fixed t: pointwise convergence of λ n (t) to Λ P (t) + CLT Convergence of the process { n ( λn (t) Λ P (t) ) } t [0,T ]

43 Convergence of landscapes Let F = {f t } 0 t T where f t : L T R is defined by f t (λ) = λ(t). Empirical process indexed by f t F: G n (t) = G n (f t ) := n ( λ n (t) Λ P (t) ) = 1 n n i=1 (f t (λ i ) Λ P (t)) = n(p n P )(f t ) Theorem [Weak convergence of landscapes]. Let G be a Brownian bridge with covariance function κ(t, s) = f t (λ)f s (λ)dp (λ) f t (λ)dp (λ) f s (λ)dp (λ), for t, s [0, T ]. Then G n G. Theorem [Uniform CLT]. Suppose that σ(t) = nvar(λ n (t)) > c > 0 in an interval [t, t ] [0, T ], for some constant c. Then there exists a random variable W d = sup t [t,t ] G(f t) such that sup z R ( P sup t [t,t ] ) ( ) (log n) 7/8 G n (t) z P (W z) = O. n 1/8

44 Confidence bands for landscapes Bootstrap for landscapes confidence bands for landscapes. Theorem. Suppose that σ(t) > c > 0 in an interval [t, t ] [0, T ], for some constant c. Then, given a confidence level 1 α, one can construct confidence functions `n (t) and un (t) such that 7/8 (log n) P `n (t) ΛP (t) un (t) for all t [t, t ] 1 α O. 1/8 n q 1 Also, supt (un (t) `n (t)) = OP. n

45 To summarize (M, ρ, µ) X µ compact X 1, X 2,, X m i.i.d. sampled according to µ. X m Rips( X m ) Repeat n times: λ 1 (t),, λ n (t) λ n (t) Bootstrap λ n (t) Λ P (t) Λ P (t) = E[λ i (t)] λ Xµ (t) λ XP (t) Λ P (t) 0 as m Stability w.r.t. µ?

46 (Sub)sampling and stability of expected landscapes (M, ρ, µ) X µ compact X 1, X 2,, X m i.i.d. sampled according to µ. X m Rips( Xm ) λ Rips( Xm ) µ m Φ P = Φ (µ m ) Λ µ,m (t) = E P [λ(t)] Theorem: Let (M, ρ) be a metric space and let µ, ν be proba measures on M with compact supports. We have Λ µ,m Λ ν,m m 1 p W p (µ, ν) where W p denotes the Wasserstein distance with cost function ρ(x, y) p.

47 (Sub)sampling and stability of expected landscapes (M, ρ, µ) X µ compact X 1, X 2,, X m i.i.d. sampled according to µ. X m Rips( Xm ) λ Rips( Xm ) µ m Φ P = Φ (µ m ) Λ µ,m (t) = E P [λ(t)] Theorem: Let (M, ρ) be a metric space and let µ, ν be proba measures on M with compact supports. We have Λ µ,m Λ ν,m m 1 p W p (µ, ν) where W p denotes the Wasserstein distance with cost function ρ(x, y) p. Consequences / applications: Subsampling: efficient and easy to parallelize algorithm to estimate relevant persistence information of huge data sets. Robustness to outliers. R package to be released soon - see also the Gudhi library:

48 (Sub)sampling and stability of expected landscapes (M, ρ, µ) X µ compact X 1, X 2,, X m i.i.d. sampled according to µ. X m Rips( Xm ) λ Rips( Xm ) µ m Φ P = Φ (µ m ) Λ µ,m (t) = E P [λ(t)] Theorem: Let (M, ρ) be a metric space and let µ, ν be proba measures on M with compact supports. We have Λ µ,m Λ ν,m m 1 p W p (µ, ν) where W p denotes the Wasserstein distance with cost function ρ(x, y) p. Consequences / applications: Subsampling: efficient and easy to parallelize algorithm to estimate relevant persistence information of huge data sets. Robustness to outliers. R package to be released soon - see also the Gudhi library:

49 Thank you for your attention! References: F. Chazal, M. Glisse, C. Labruère, B. Michel, Convergence rates for persistence diagram estimation in Topological Data Analysis, in Int. Conf. on Machine Learning 2014 (ICML 2014). F. Chazal, B. Fasy, F. Lecci, A. Rinaldo, L. Wasserman, Stochastic Convergence of Persistence Landscapes and Silhouettes, in ACM Symposium on Computational Geometry (SoCG 2014). F. Chazal, B. Fasy, F. Lecci, B. Michel, A. Rinaldo, L. Wasserman, Subsampling Methods for Persistent Homology, arxiv: , June C. Li, M. Ovsjanikov, F. Chazal, Persistence-based Structural Recognition, in proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014). F. Chazal, V. de Silva, S. Oudot, Persistence Stability for Geometric complexes, Geometria Dedicata 2014 (online first Dec. 2013). F. Chazal, V. de Silva, M. Glisse, S. Oudot, The Structure and Stability of Persistence Modules, arxiv: , July 2012.