Kernel Methods for Topological Data Analysis

Transcription

1 Kernel Methods for Topological Data Analysis Toward Topological Statistics Kenji Fukumizu The Institute of Statistical Mathematics (Tokyo Japan) Joint work with Genki Kusano and Yasuaki Hiraoka (Tohoku Univ.) supported by JST CREST 2nd UCL Workshop on the Theory of Big Data. 6-8 January

2 Topological Data Analysis TDA: a new method for extracting topological or geometrical information of data. Persistence homology: a recent mathematical tool Edelsbrunner et al 2002; Carlsson 2005) Progress of computational topology: computing topological invariants in feasible computational cost. Big data? Not only the size buy complex structure matters TDA. 2

3 TDA: Various applications Computer Vision Data of highly complex geometric structure Often difficult to define good feature vectors / descriptors Biology / protein Material Science Liquid Brain Science Glass Shape signature natural image statistics (Freedman & Chen 2009) Structure change of proteins Brain artery trees e.g. age effect (Bendich et al 2014) eg. open / closed (Kovacev-Nikolic et al 2015) Non-crystal materials etc (Nakamura Hiraoka Hirata Escolar Nishiura. Nanotechnology 26 (2015)) TDA provides a compact representation for geometry of such data 3

4 Algebraic Topology Homology group Algebraic operation The generators of 1 dim. homology group 0 dim connected comp. 1 dim ring Homology group: topological invariance Independent ℓ-dim. holes Various topological invariances e.g. Euler number = 1 dim 4

5 Topology of statistical data? True structure neighborhood (e.g. manifold learning) Small disconnected object Noisy sample Stable extraction of topology is NOT easy! Large Small ring is not visible 5

6 Persistence Homology All considered = ( ) small large Two rings generators of 1 dim homology persist in a long interval. 6

7 Persistence homology (formal definition) Filtration of topological spaces X (X): ( ) [ ] The lifetime of each generator is rigorously defined and can be computed numerically. at at : field Irreducible decomposition Persistence diagram (PD) Birth and death of a generator Handy descriptor of complex geometry Birth Death PD plots the birth and death of each generator of PH in a 2D graph ( ). 7

8 Topological statistics: systematic data analysis in TDA Conventional analysis in TDA: analysis WITH PH/PD Data Computation of PH Display PD Analysis by human Our statistical approach to TDA: analysis OF random PH/PD Many data sets Many persistent diagrams Computation of PD s PD1 How? PD2 Analysis of PD s PD3 Software available CGAL / PHAT Features / Descriptors PDn Topological Statistics 8

9 I predict a new subject of statistical topology. Rather than count the number of holes Betti numbers etc. one will be more interested in the distribution of such objects on noncompact manifolds as one goes out to infinity. Isadore Singer

10 Kernel representation of PD Vectorization of PD by positive definite kernel PD = Discrete measure Embedding of PD s into RKHS ℇ : = ( ) For some kernels (e.g. Gaussian Laplace) ℇ is injective. Vectorization : positive definite kernel : corresponding RKHS By vectorization a number of methods for data analysis can be applied SVM regression PCA CCA etc. tractable computation is possible with kernel methods. 10

11 Persistence Weighted Gaussian (PWG) Kernel Generators close to the diagonal may be noise and should be discounted. = = Pers exp arctan Pers for ( { > 0) } Pers(x1) 11

12 Stability with PWG kernel embedding PWGK defines a distance measure on the persistence diagrams ℇ ℇ Statiblity Theorem (Kusano Hiraoka Fukumizu 2015+) : compact subset in. : finite sets. If > + 1 then with PWG kernel ( ) ( ) ( ). : constant depending only on ( ): dimensional persistence diagram of : Haussdorff distance This stability is not known for Gaussian kernel. A small change of a set causes only a small change in PD 12

13 2nd level kernel Data analysis method PD1 PD2 Embedding ℇ ℇ PD3 Application of pos. def. Kernel on RKHS ℇ PDm PD Data sets Vectors in RKHS 2nd-level kernel (SVM for measures Muandet Fukumizu Dinuzzo Schölkopf 2012) RKHS-Gaussian kernel = exp derives ℇ ( = exp ) ℇ ( ) : Persistence diagrams 13

14 Computational issue The number of generators in a PD may be large ( ) ℇ ( For = computation ℇ ( = ) ℇ ( () Δ ) ℇ ( ) = exp requires ) The number of exp ( ). computationally expensive for = max{ = 1 } 14

15 Approximation by random features (Rahimi & Recht 2008) By Bochner s theorem Gaussian distribution =: exp = (Fourier transform) Approximation by sampling: :... ~ exp l l l l l ( ) l ( ) = l l dim. ( ) l ( ) Computational cost ( ) 2nd level Gram matrix ( + ). c.f. ( ) Big gain if 15

16 Comparison: Persistence Scale Space Kernel (Reininghaus et al 2015) PSS Kernel 1 = 8 exp 8 = ( ) for exp = ( ). ℇ ( ) is considered. 8 Pos. def. on 0 on Δ. Comparison between PWGK and PSSK PWGK can control the discount around the diagonal independently of the bandwidth parameter. PSSK is not shift-invariant Random feature approximation is not applicable. In Reininghaus et al nd level kernel is not considered. 16

17 Synthetic example: SVM classification Classification of PD s by SVM Synthetic data One big circle (random location and sample size) 1 with or without small circle 0 = XOR((birth( 1)<1 && death( 1)>4)) 0 exists) In fact data include noise. S1 PD1 100 for training =1 and 100 for testing Result (correct classification) PWGK (proposed): 83.8% PSSK (comparison): 46.5% S1 S0 Data points 1 S1 No S0 S0 noise Data points 2 17

18 Application: Transition of Silica (SiO2 ) If cooled down rapidly from the liquid state SiO2 changes into the glass state (not Amorphous: soft structure to crystal). Goal: identify the temperature of phase transition. Data: Molecular Dynamics simulation for SiO2. 3D arrangements of the atoms are (Nakamura et al 2015; Hiraoka et al 2015) used for computing PD at 80 temperatures. Examples of PD s Liquid Glass (Amorphous) 18

19 Change point detection Data along a parameter = 1. Kernel Change Point Analysis with Fisher Discriminant score (Harchoui et al 2009): For each two classes are defined by the data before and after. Fisher score on RKHS is used. For each compute : = Φ( ) and Φ( ). : = Compute Δ : + : + ( : : ). Find max Δ. For the packing problem =ℇ ( = 1 80). 19

20 Detection of liquid-glass state transition Approach in physics Estimation using derivatives of enthalpy but not so accurate. Estimate = [2000K 3500K] Our approach: Change point detection by Kernel FDR. Number of generators in a PD is at most difficult to use PSSK directly Only PWGK (proposed) is applied with random features. KFDR Gram matrix Detected change point = 3100K 20

21 Kernel PCA with PWGK Sharp change between the two phases. (Colored by the result of change point detection. Colors are not used for KPCA). PD + PWGK successfully capture soft and complex structure of the glass state. 21

22 Conclusion Goal: establishing topological statistics Persistence homology can be used for defining useful descriptors to complex objects of geometrical structures. Systematic statistical methods for TDA: Kernel methods introduce systematic data analysis to TDA. Vectorization of persistence diagrams by kernel embedding. Persistence weighted Gaussian kernel flexible kernel for noise. A toolbox of kernel methods become available: SVM kernel ridge regression kernel PCA kernel CCA etc. 22

23 Thank you 23