Machine Learning : supervised versus unsupervised

Transcription

1 Machine Learning : supervised versus unsupervised Neural Networks: supervised learning makes use of a known property of the data: the digit as classified by a human the ground truth needs a training set, and runs through many feedback cycles has (very) many parameters that need to be optimised (danger of overfitting the data) performance is assessed with the test set In contrast, unsupervised learning: does not have free parameters that need to be optimised does not need to use or know the ground truth does not overfit the data has good performance with low numbers of data sets 1

2 Work in our lab: assessing the homo- or heterogeneity of noisy experimental data Why do an experiment? because we want to find out a property (or several) of an object Why do multiple experiments? because the experimental error (noise) may be high and the signal weak, the signal-tonoise-ratio (SNR) may be low: the property cannot be measured/detected accurately averaging of results from multiple experiments improves the SNR because random differences cancel out (following a N law if we do N measurements) What is the problem? 2 anything that systematically changes the measured value between experiments leads to different values of the property we need to cluster our experiments (i.e. find objects that belong to same population) before averaging. Within a cluster, the property of interest should be the same.

3 Nomenclature, properties, and a few examples in general: heterogeneity versus homogeneity in crystallography: non-isomorphism versus isomorphism (from the Ancient Greek: ἴσος isos "equal", and μορφή morphe "form" or "shape") homogeneity is required (assumed) to hold if data should be averaged. Data must be from the same population. violation of this assumption can lead to additional error: the averaged data no longer represent the object. 1) can/should images in Electron Microscopy be averaged? Only if the molecules have the same composition and conformation, and the projection is the same 2) can/should the weak data sets from two crystallographic experiments be averaged, to obtain a better data set? Only possible if indexed consistently, and crystals are from identical protein preparations. 3) do two amino-acid sequences belong to the same protein family? If so, they can be aligned, and identical residues may have an important function. 4) do noisy IR spectra from different samples describe the same object, or are the molecules actually different? 3

4 Crystallographic example: two forms of lysoyzme 3aw6 3aw7 RMSD = 0.18 Å Δcell = 0.7 % 4 RH: 84.2% vs 71.9% Riso = 44.5%

5 Crystallography: multiple crystals/datasets Femtosecond X-ray protein nanocrystallography Chapman et al. (2011) Nature 470, nanocrystals of photosystem I, one of the largest membrane protein complexes. More than 3,000,000 diffraction patterns were collected in this study, and a three-dimensional data set was assembled from individual photosystem I nanocrystals (~200 nm to 2 μm in size). (15445 xtals used) Data collection at XFEL (LCLS, Stanford) 5

6 Assessing heterogeneity By way of an easy-to-measure quantity ( proxy ) Crystallography: cell parameters are easy to measure (but the really interesting data are the intensities of the reflections) (Until now) we didn't understand what differences of reflection intensities are due to (are they random or systematic??), but for difference of cell parameters there is a simply theory that tells us how different they should be at most Agreement of cell parameters is only a necessary, but not a sufficient condition for isomorphism More general: data-based approach comparison of datasets based on pairwise correlation coefficients cc ij = 6 ( x k x )( y k y ) (x k x )2 ( y k y )2 ccij= hierarchical cluster analysis

7 Mathematical treatment Need to separate the random error from the systematic error 2 2 total error (difference of values that should be equal) is random + systematic pairwise CC has contributions from total error, i.e. form both sources of error separation of random and systematic errors is not generally possible A new way to analyze CCs Brehm and Diederichs (2014) minimize ϕ ({ x })= (ccij x i x j )2 i>j with {x}={x1,x2,...xn} where xi and xj are N vectors in n-dimensional space representing the datasets, and ccij is 7 (Pearson's) correlation coefficient between intensities of datasets i and j with n = 2 or 4, this solves the indexing ambiguity ( twinning) present in point groups 3, 4, 6, 312, 321 and 23, and additional cases with particular values of cell parameters. This type of analysis is called Multidimensional Scaling; the method has no name so far It turns XFEL data collection into a technique with general applicability

8 The idea (n=2) x1 x2 x3 x 1 x 2 =cc12 x 1 x 3 =cc13 x 2 x 3=cc n*n unknowns N*(N-1)/2 knowns overdetermined system of equations if N > 2*n 8

9 Least-squares iterations starting from random positions each point represents one dataset with one of two indexing modes Brehm, W. & Diederichs, K. (2014) Breaking the indexing ambiguity in serial crystallography. Acta Cryst. (2014). D70,

10 Which information can be extracted from the matrix of pairwise CCs? The analysis (Acta D73, ) shows that 2 the least-squares solution of ϕ ({ x })= (ccij x i x j ) exists and is "unique" if ccij known i>j the solution can be obtained from the n Eigenvalues/Eigenvector of the ccij matrix the x i vectors are arranged in a sphere with radius 1, in n-dimensional space vectors can be given as coordinates, or (better) length and spherical angles Amount of signal the length of a vector is CC*, the correlation with its prototype ( true ) dataset, and depends on the random error of the dataset CC* may be calculated from multiple observations in a dataset (crystallography) Relation between datasets 10 angle between x i and x j is proportional to the systematic difference between i and j ccij = CC*i CC*j cos(angle( x i, x j))

11 Example I: two kinds of noisy images noisy images (SNR=1/13 and SNR=1/9) of original and mirror picture 11 Result of averaging without knowledge whether original image, or its mirror

12 CC analysis with n=2 weakest S/N ratio 12

13 Result of averaging - after clustering 100 original 13 mirror

14 Example 2: MNIST data - classify handwritten digits Wikipedia: The MNIST database (Modified National Institute of Standards and Technology database) is a large database of handwritten digits that is commonly used for training various image processing systems

15 CC analysis in n=2 dimensions; N=6000 images 0,1 0,1,2,3 15 0,1,2 0,1,2,3,4

16 CC analysis in n=3 dimensions: 0,1,2,3,4 N=6000 The higher the possible number of systematic ways in which data sets may differ, the higher n must be otherwise no segmentation of data sets is achieved. n should correspond to the number of large Eigenvalues 16

17 CC analysis in n=3 dimensions; digits 0,1,2,3 N=6000 N= N=60

18 Example 3: PS I data: N=15445, n=3 after resolving the indexing ambiguity x-axis along strongest Eigenvector y-axis along second Eigenvector 18 second (x-axis) and third (y-axis) Eigenvectors (the view is along the first Eigenvector)

19 Example 4: attractors in 5 μs MD trajectory of a hepta aspartic acid N= conformations described by the 84 distances between Cα,γi and Cα,γj atoms (i j) ~ CCs n=5 (Collaboration with S. Hunkler, O. Kukharenko, C. Peter) 19

20 Summary Unsupervised learning A method that separates random and systematic effects "Segmentation" of objects; this enables clustering and e.g. averaging of similar ones Mathematical properties: "unique" solution; complexity ~ O(N2) Limitation: the axes of low-dimensional space are not "labelled" with specific properties Applications in Structural Biology: crystallographic datasets, or images of molecules Analogously applicable in other Sciences: spectra, expression patterns, sequences, patient data Further applications wherever correlation coefficients need to be understood