Matlab project Independent component analysis

Transcription

1 Matlab project Independent component analysis Michel Journée Dept. of Electrical Engineering and Computer Science University of Liège, Belgium September 2008

2 What is Independent Component Analysis? 2

3 The cocktail party problem 3

4 ICA performs a linear projection into independent components 4 Assumptions linearity no delay statistically independent sources

5 ICA performs a linear projection into independent components 5 X = A S p n n x p number of components number of variables Statistically independent random variables Real matrix Random vector

6 ICA performs a linear projection into independent components 6 X = A S p x N n x N n x p p number of components Samples of statistically independent random variables n N number of variables number of samples

7 ICA for blind source separation: fecg extraction 7 ICA

8 ICA for blind source separation: Analysis of EEG 8 CSF EEG

9 ICA for EEG 9

10 ICA for data analysis 10 Principal directions Independent directions

11 ICA for denoising 11 Original image Noisy image Wiener filtering ICA filtering

12 ICA has applications in many areas 12 Blind source separation (e.g., biomedical signal processing, radar and mobile communication) Data analysis Noise reduction Feature extraction (image, audio, video representation)

13 ICA is an optimization problem 13 estimator of statistical independence

14 ICA algorithms compute the unmixing model 14 Mixing model Unmixing model

15 ICA is an optimization problem Estimation of the statistical independence of the z's: 2. Minimization of the contrast:

16 The contrast presents two inherent symmetries 16 If are independent, then are independent, and as well.

17 The contrast presents two inherent symmetries 17 If are independent, then are independent, and as well. invertible diagonal matrix permutation matrix

18 Furthermore, most ICA methods use prewhitening 18 For any matrix : W= U S V T Determined by Orthogonal ICA Determined by SVD of X Principal Component Analysis

19 In dimension 2 19 x 2 x 2 a2 s 2 x 1 = a 1 x 1. s 1

20 In dimension 2 20 x 2 z 2 x 1 PCA ICA z 1

21 ICA as an optimization on the orthogonal group 21 Orthogonal ICA (also called prewhitening-based ICA): with The orthogonal group automatically gets rid of the scaling indeterminacy.

22 A whole bunch of ICA algorithms 22 Contrast Estimation of the mutual information Joint diagonalization of cumulant matrices Diagonalization of cumulant tensors Non-gaussianity Constrained covariance Manifold Orthogonal group Stiefel manifold Oblique manifold Flag manifold (independent subspace analysis) Optimization method Jacobi rotations Gradient descent Second-order approaches

23 Outline of the project 23 Manifold : Orthogonal group Contrast: Joint diagonalization of cumulant matrices

24 Joint diagonalization of a set of matrices 24 Given m cumulant matrices C i, minimize Diagonalization of one matrix: C i W T W =

25 Joint diagonalization of a set of matrices 25 Given m cumulant matrices C i, minimize Joint diagonalization of m matrices: W T W = C i

26 Outline of the project 26 Manifold : Orthogonal group Contrast: Joint diagonalization of cumulant matrices Optimization method: conjugate gradient Applications: blind source separation of images, bioinformatics

27 Separation of images 27 ICA

28 Analysis of gene expression data 28 Microarray Each spot reflects the expression of a gene Gene expression database Rows genes ( ~10 4 ) Columns experiments ( ~10 2 )

29 Analysis of gene expression data 29 DNA Microarray Each spot reflects the expression of a gene mrna Gene expression database Protein Rows genes Columns experiments

30 Such a database is a goldmine for new knowledge about the cellular machinery 30 Global picture of the transcriptome under several conditions Genes that are coexpressed across similar conditions are very informative Identification of interesting structures in the genome Some interesting questions: What does this gene do? Which genes are responsible of a phenotype? How do the genes act on a phenotype?

31 ICA in case of gene expression data 31 weigths Expression mode (statistically independent)

32 Analysis of an ovarian cancer database genes 17 tissues + some clinical data

33 ICA expression modes are highly correlated with the observed phenotypes 33 Expression modes benign mucinous cystadenoma Pre-menopause Tissues poorly differentiated serous papillary adenocarcinoma

34 ICA identifies genes likely to be coexpressed for an observed phenotype 34 E.g. poorly differentiated serous papillary adenocarcinoma (pd-spa) Expression mode 15 genes HLA CLASS I MEMBRANE GLYCOPROTEIN GP130 PLACENTAL-CADHERIN COFILIN TIE1

35 References 35 P.-A. Absil and K. A. Gallivan, Joint diagonalization on the oblique manifold for independent component analysis, ICASSP 2006, P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization algorithms on matrix manifolds, Princeton University Press, F. R. Bach and M. I. Jordan, Kernel independent component analysis, Journal of Machine Learning Research, 3,1-48, J.-F. Cardoso, High-order contrasts for independent component analysis, Neural Computation 11, no. 1, , P. Comon, Independent Component Analysis, a new concept?, Signal Processing, Elsevier 36, no. 3, , Special issue on Higher-Order Statistics, A. Hyvärinen, J. Karhunen, and E. Oja, Independent component analysis, John Wiley & Sons, E.G. Learned-Miller and J.W.Fisher III, ICA using spacings estimates of entropy, Journal of Machine Learning Research, 4, ,2003.

36 References 36 W. Liebermeister, Linear modes of gene expression determined by independent component analysis, Bioinformatics 18, 51 60, A.-M. Martoglio, J. W. Miskin, S. K. Smith, and D. J. C. MacKay, A decomposition model to track gene expression signatures: preview on observer-independent classification of ovarian cancer, Bioinformatics 18, no. 12, , A. E. Teschendorff, M Journée, P.-A. Absil, R. Sepulchre, andc. Caldas, Elucidating the altered transcriptional programs inbreast cancer using independent component analysis, PLoS Computational Biology 3, Number 8, page , 2007.

37 Schedule 37 4 Matlab session: - Wednesday 11:30 12:30 - Wednesday 16:30 17:30 - Thursday 16:30 17:30 - Friday 11:30 12:30

38 Good work! 38