Out-of-sample extension of diffusion maps in a computer-aided diagnosis system. Application to breast cancer virtual slide images.

Transcription

1 Out-of-sample extension of diffusion maps in a computer-aided diagnosis system. Application to breast cancer virtual slide images. Philippe BELHOMME Myriam OGER Jean-Jacques MICHELS Benoit PLANCOULAINE philippe.belhomme@unicaen.fr m.oger@baclesse.fr jj.michels@baclesse.fr benoit.plancoulaine@unicaen.fr Acknowledments: Paulette HERLIN BioTICLA-HIQ, EA 4656 Université de Caen Basse-Normandie, France F. Baclesse Cancer Center, F Caen France Summary Objectives of our work Short term: to develop a CADS (VSI of breast cancer) 1) Building a knowledge database from expertised VS 2) Classifying new unknown VS Mean term: to be confident in the results (quality control) to transpose the CADS to other tumoral cases Materials, Road Map of the project Results for out-of-sample Virtual Slides Conclusion / Perspectives 2

2 Computer Aided Diagnosis System To help pathologists in their daily practice (limit the work overload) To compute (reliable) numerical parameters: Why? to find objective criteria for differential diagnosis for a better discrimination of difficult/rare cases for a better characterization of histological subtypes to explore relationships between histological types NOT with the ambition to replace pathologist's expertise... 3 Computer Aided Diagnosis System How? By providing measures from Virtual Slides histological type (fibroadenoma, invasive papillary carcinoma...) heterogeneity of tumors (distribution, hot spots...) with a quality control of the results (stereology for ground truth) coming from significant areas stroma, normal tissue, inflammatory cells... By developping a supervised classification system Pathologists are needed to expertise the items placed in a knowledge database and to embed their way of thinking in the system. 4

3 Materials [ > 400 images ] Breast Cancer HES staining Aperio, ScanScope CS Benign lesions Malignant carcinoma tumors Malignant sarcoma tumors 'normal' tissue 30 histological types/subtypes [Fibroadenoma] 20X 0.5 µm / pixel x Gigabytes 5 Building the database A stereological grid of points is superimposed onto the VS Spacing: x Random starting point Pathologists indicate the histological type under each point of the grid (in corresponding ImageScope layers) Patches of size 400 x 400 are extracted around the points (16% of the slide are covered) 6

4 Road Map of the project (1) SQL DataBase pathologists -1- Obtaining a knowledge database 7 Features Extraction - histograms Fibroadenoma Invasive Papillary Carcinoma 8

5 Histogram Parameters Original histograms reduced to 64 levels 6 features Mean Mode Median = 1 n i=1 Skewness (asymmetry) Pearson Mode Skewness Kurtosis (peakedness) n X i 2 = 1 X n i 2 i=1 n 1 = = Other parameters on histograms Quantiles on cumulative normalized histograms less sensitive to color variations 0,04 0,04 0,03 0,03 0,02 0,02 0,01 0, ,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 4 features 80% 60% 40% 20% 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 10

6 Quantiles 20%, 40%, 60%, 80% Fibroadenoma Invasive Papillary Carcinoma 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0, ,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0, Dimensionality Reduction Dimension of a patch: 400 x 400 x 3 = data Dimension of a feature vector (signature) Color Deconvolution Haematoxylin & Eosin (*) 10 numerical values (mean, mode, skewness, quantiles...) 3 reduced histograms (normalized, sorted, cumulative) 11 color components (RGB - I 1 I 2 I 3 - YCh 1 Ch 2 - HE) (10 3 x 64) * 11 = data Subset used: n=50 values SQL DataBase (*) Ruifrok AC, Johnston DA. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol 23: ,

7 Diffusion Maps (1) Mathematical technique working with a normalized Laplacian graph of non-linear data Analogy: Heat diffusion or Random Walk in a graph DM = Spectral Connectivity Analysis used for: Manifold learning (to discover the embedded structure of data) Clustering, Dimensionality reduction For N samples to analyse: Matrix of dissimilarity (size N x N) n D KL (x 1, x 2 )= i=1 x 1, i log ( x 1, i x 2, i ) x log 2, i ( x 2,i x 1, i ) independant coefficients parallel processing 'metric' respecting a symmetry property Kullback-Leibler 13 Diffusion Maps (2) Let X = {x 1, x 2,..., x N } be a set of N feature vectors A kernel P for the Laplacian graph is given by: p( x1, x1) p( x1, x2)... p( x1, x N) p( x P=[ 2, x 1 ) p( x i, x j ) p( x N, x 1 ) p(x N, x N )] p x i, x j = wx i, x j d x i N d x i = w x i, x k k=1 Symetrical Metric w( x i, x j )=e ( D KL ( x i, x j ) ϵ ) D KL (x i, x i )=0 D KL (x i, x j )=D KL (x j, x i ) N (N 1) 2 coefficients to compute 0wx i, x k 1 w(x i, x i )=1 14

8 Diffusion Maps (3) Computation of eigenvectors i and eigenvalues λ i Sorted eigenvectors provide a transformation matrix for dimensionality reduction 0 : not used (associated to 0 =1 : trivial solution) D Visualization 3D Visualization 15 3D Visualization Pure Fibroadenoma (2793 patches) Invasive Ductal Carcinoma (812 patches) Mastosis (919 patches) Normal Tissue (126 patches) 16

9 Knowledge Database Real neighborhood of a patch (center) First page of patches 17 Road Map of the project (2) -2 unk - Eva now luat n im ing age s SQL transaction Areas of interest Histological types Quality Control... Feature vector? Out-of-Sample extension 18

10 Introducing a new image... New image cut in X unknown patches Database comparison M patches already expertised Theoretical Complexity N = M X O(N 3 )? 19 Out-of-sample Extension Diffusion maps = Spectral Connectivity Analysis compute the spectrum of a positive definite kernel Eigenvalue decomposition is O(n 3 ) rapidly impossible to compute this limits its application to moderately sized problems Number of patches Features extraction (in seconds) Spectral analysis (in seconds) Time in seconds Computational time Number of patches Dual-core PC Features extraction Spectral analysis 20

11 Nyström formula Given a set of M eigenvectors u i (M) and their eigenvalues λ i (M) Allows to estimate N eigenvectors with N = M X From a P N,M new kernel 1 1 Off-line procedure from a kernel (M x M) M N = û M i N 1 λ i (M ) (M ) P N, M u i M On-line procedure for X new elements N 21 Visual comparison Total number of patches: N=1857 (mastosis, IDC, normal tissue) Training set: M=1000 patches (X 857) Training set: M=500 patches (X 1357) In black: 3D projection of all patches ( true eigenvectors) In red: projection of N patches according to the Nyström formula 22

12 Numerical comparison Size of the training set (M) Mastosis Invasive Ductal Carcinoma Mastosis IDC Mastosis IDC Reference (1857) 73.9% 17.10% 14.8% 82.1% % 14.4% 14.0% 83.0% % 15.0% 13.9% 83.1% Values obtained from the k-nearest neighbors (k=5) 23 Road Map of the project (2) Off-line procedure SQL DataBase pathologists a se g in aba n ai at bt e d O g -1 led w o kn True eigenvectors -2 unk - Eva now luat n im ing age s SQL transaction Feature vector Areas of interest Histological types Quality Control? On-line procedure = Nyström formula 24

13 Conclusion Our approach for a CADS: Starts from the expertise of VSI by pathologists 400 VSI of breast cancers, 30 histological types and subtypes Yields to an unbiased database of referenced patches Stereological grid for random selection of items Ground truth reachable by a human expert Manages non linear data, in a reasonable time Diffusion maps Nyström formula Parallel processing, 3D visualization Will provide a statistical estimation of histological types 25 Perspectives All future developments will be done with the idea of a Quality Control some patches will be used as a training set computation of sensitivity/specificity Improve feature selection adding new features for texture (Fourier, LBP...) adding second order parameters being device independant for multi-lab collaboration color calibration ICC profiles 26

14 Road Map of the project (3) Device independance Color calibration New samples Refine features -3- Feedback Machine Why some results diverge from the experts? Learning Pathologist's way of thinking 27 Out-of-sample extension of diffusion maps in a computer-aided diagnosis system. Application to breast cancer virtual slide images. Philippe BELHOMME Myriam OGER Jean-Jacques MICHELS Benoit PLANCOULAINE philippe.belhomme@unicaen.fr m.oger@baclesse.fr jj.michels@baclesse.fr benoit.plancoulaine@unicaen.fr Acknowledments: Paulette HERLIN BioTICLA-HIQ, EA 4656 Université de Caen Basse-Normandie, France F. Baclesse Cancer Center, F Caen France 28

15 Road Map of the project SQL DataBase pathologists a se ng ba i n a ai at bt e d O g -1 led w o kn -2 unk - Eva now luat n im ing age s SQL transaction Areas of interest Histological types Quality Control Feature vector? Refine features -3 Device independance Color calibration New samples k ac b d ee -F Why some results diverge from the experts? Machine Learning Pathologist's way of thinking 29