Machine Learning for Medical Image Analysis. A. Criminisi

Transcription

1 Machine Learning for Medical Image Analysis A. Criminisi

2 Overview Introduction to machine learning Decision forests Applications in medical image analysis Anatomy localization in CT Scans Spine Detection in CT Scans Brain Tumour Segmentation in MR Scans

3 Machine Learning Training phase Lots of labelled data Training algorithm A predictor (e.g. a classifier) Test phase Previously unseen data Predictor Predicted label

4 Supervised Machine Learning (classification) Training phase (usually offline) Training data set Learned model Training algorithm measurements (features) & associated class labels structure & parameters (colors used to show class labels)

5 Supervised Machine Learning (classification) Test phase (run time, online) Input test data point Learned model Output measurements (features) only structure + parameters predicted class label

6 Data representation, feature vectors and data points Features in 2D space Data point = Feature vector Features in 3D space

7 Data representation, feature vectors and data points Features in 2D space

8 Application: Kinect body part recognition Task: assigning body part labels to each pixel in Kinect depth images Input test depth image Body part segmentation image measurements made relative to pixel classifier per-pixel prediction of class label e.g. depth, color, neighbors

9 Overview Introduction to machine learning Decision forests Applications in medical image analysis Anatomy localization in CT Scans Spine Detection in CT Scans Brain Tumour Segmentation in MR Scans

10 Decision trees A general (binary) tree structure A decision tree 0 root node Is top part blue? internal (split) node 1 2 Is bottom part green? Is bottom part blue? terminal (leaf) node

11 Decision forests Forest prediction is an aggregate of the predictions across all trees (e.g. average probability)

12 Decision forests: key concepts Forest is an ensemble (collection) of trees The output of a forest aggregates the outputs of multiple trees e.g. average Number of trees will depend on application with lots of data you can get away with fewer, deeper trees (e.g. Kinect) less data probably requires more trees

13 Decision trees: test time prediction test input data prediction

14 D=13 D=5 Effect of tree depth and randomness Weak learner: axis aligned Weak learner: oriented line Weak learner: conic section Parameters: T=400 predictor model = prob.

15 Overview Introduction to machine learning Decision forests Applications in medical image analysis Anatomy localization in CT Scans Spine Detection in CT Scans Brain Tumour Segmentation in MR Scans

16 Anatomy Localization in 3D Computed Tomography Scans - Direct mapping of voxels to organ bounding boxes. - No search, no sliding window. - No atlas registration. Input CT scan Output anatomy localization Key idea: each voxel votes (probabilistically) for the position of each organ s bounding box.

17 Organ labelling: why is it hard? spleen liver gall bladder left kidney High variability in appearance, shape, location, resolution, noise, pathologies

18 Organ labelling: the ground-truth database Different image cropping, noise, contrast/no-contrast, resolution, scanners, body shapes/sizes, patient position

19 Organ labelling: regression forest Each voxel in the volume votes for the position of the 6 box sides We wish to learn a set of discriminative points (landmarks, clusters) which can predict the kidney position with high confidence. Input data point Output Multiple organs Node split function Node optimization Node training Feature response (voxel position in volume) (bound. box continuous pos.) (mean over displaced 3D boxes) Error in model fit (weighted uncertainty for all organs) (relative displacement) (Gaussian repres. of distribs) Regressing an n-d piece-wise constant model

20 Organ labelling: context-rich visual features Possible visual features Computing the feature response Capturing spatial context

21 Organ labelling: automatic landmark discovery Discovery of landmark regions Here the system is trained to detect left and right kidneys. The system learns to use bottom of lung and top of pelvis to localize kidneys with highest confidence. Input CT scan and detected landmark regions

22 Overview Introduction to machine learning Decision forests Applications in medical image analysis Anatomy localization in CT Scans Spine Detection in CT Scans Brain Tumour Segmentation in MR Scans

23 Vertebrae Detection and Classification

24 Where? Which? Name of this vertebra?

25 Challenges

26 Challenges Repetitive nature of structures Variability of normal anatomy Presence of pathologies Varying image acquisition (FOV, noise level, resolution, )

27 Clinical motivation Patient-specific coordinate system Guided visualization/navigation in diagnostic tools Impact on Clinical Routine! Longitudinal assessment after surgical Intervention Shape/population analysis for disease modelling Impact on Clinical Research!

28 Some results

29 Some results

30 Some results

31 Overview Introduction to machine learning Decision forests Applications in medical image analysis Anatomy localization in CT Scans Spine Detection in CT Scans Brain Tumour Segmentation in MR Scans

32 Automatic Segmentation of Brain Tumour Segmentation of tumorous tissues: T1-gad T1 T2 DTI-p 3D MRI input data FLAIR DTI-q ---- Active cells ---- Necrotic core ---- Edema ---- Background

33 Training a Pixel-Wise Forest Classifier Tumour Tissue Classification

34 Testing the Pixel-Wise Forest Classifier New Patient, previously unseen Tumour Tissue Classification

35 Building the Training Database of Patients Images 1 st Step: Obtain Expert Segmentation

36 Building the Training Database of Patients Images 1 st Step: Obtain Expert Segmentation

37 Building the Training Database of Patients Images 1 st Step: Obtain Expert Segmentation

38 Glioblastoma Segmentation

39 Glioblastoma Segmentation

40 Machine learning can have a huge impact on medicine!