MEDICAL IMAGE COMPUTING (CAP 5937) LECTURE 19: Machine Learning in Medical Imaging (A Brief Introduction)

Transcription

1 SPRING MEDICAL IMAGE COMPUTING (CAP 5937) LECTURE 19: Machine Learning in Medical Imaging (A Brief Introduction) Dr. Ulas Bagci HEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL bagci@ucf.edu or bagci@crcv.ucf.edu

2 2 Outline Role of Machine Learning (ML) in Radiology CAD Systems (ML based detection/diagnosis) Image Segmentation using ML Image Registration using ML Deep Learning and its applications in radiology applications

3 3 Motivation Radiologists need to interpret high volume of images and as the number of images increases, radiologists workload increases as well.

4 4 Motivation Radiologists need to interpret high volume of images and as the number of images increases, radiologists workload increases as well. The increasing number and complexity of the images threatens to overwhelm radiologists capacities to interpret them.

5 5 Motivation Radiologists need to interpret high volume of images and as the number of images increases, radiologists workload increases as well. The increasing number and complexity of the images threatens to overwhelm radiologists capacities to interpret them. Automated and intelligent image analysis and understanding are becoming an essential part or procedure, such as image segmentation, registration, and computer-aided diagnosis and detection.

6 6 Motivation Radiologists need to interpret high volume of images and as the number of images increases, radiologists workload increases as well. The increasing number and complexity of the images threatens to overwhelm radiologists capacities to interpret them. Automated and intelligent image analysis and understanding are becoming an essential part or procedure, such as image segmentation, registration, and computer-aided diagnosis and detection. Machine learning algorithms underpin the algorithms and software that make computer-aided diagnosis/prognosis/treatment possible

7 11/03/15 7

8 11/03/15 8

9 11/03/15 9

10 10 11/03/15

11 11 11/03/15

12 12 11/03/15

13 13 11/03/15

14 14 11/03/15

15 15 11/03/15

16 16 11/03/15

17 17 Feature Engineering? Feature Learning? 11/03/15

18 18 Representation Learning Is there some way to extract meaningful features from data in a supervised or unsupervised manner? 11/03/15

19 19 Representation Learning Is there some way to extract meaningful features from data in a supervised or unsupervised manner? Biologically inspired systems to make the computer more robust, intelligent, and learn, 11/03/15

20 20 Representation Learning Is there some way to extract meaningful features from data in a supervised or unsupervised manner? Biologically inspired systems to make the computer more robust, intelligent, and learn, Model our systems after the brain! 11/03/15

21 21 Representation Learning Is there some way to extract meaningful features from data in a supervised or unsupervised manner? Biologically inspired systems to make the computer more robust, intelligent, and learn, Model our systems after the brain! Brain interprets imprecise information from the senses at an incredibly rapid rate! It discerns a whisper in a noisy room, a face in a dimly lit alley, and hidden agenda in a political statement. 11/03/15

22 22 Recap: Types of Learning Supervised (inductive) learning Training data includes desired outputs Unsupervised learning Training data does not include desired outputs Semi-supervised learning Training data includes a few desired outputs Reinforcement learning Rewards from sequence of actions 11/03/15

23 23

24 24 Applications of ML in Radiology Image Segmentation Image Registration CAD (computer aided detection/diagnosis) Brain function or activity analysis Neurology disease diagnosis Text analysis of radiology reports

25 25

26 Evolution of ML Algorithms 26

27 27 Supervised Learning In supervised learning the predictive model represents the assumed relationship between input variables in x and output variable y.

28 28 Linear Models and Regression Linear models assume that there is a linear relationship between the input of the model and the output of the model. Perhaps it is the simplest method for classification and regression. It has been widely used in computer-aided classification.

29 29 Ex: LDA (Linear Discriminant Analysis) J(w) = wt S B w w T S w w S B = (m 1 m 2 )(m 1 + m 2 ) T is called the between scatter matrix (mi is the mean of samples from class i, i ϵ {1, 2}), and S w = S 1 + S 2 is called the within scatter matrix (Si= x Di(x-m i )(x-m i ) T, D i is the collection of samples from class i, i ϵ {1, 2}). f(x) =w T x + w 0

30 30 Ex. Artificial Neural Networks Artificial neural networks (ANNs) are techniques that were inspired by the brain and the way it learns and processes information. Neural networks are composed of nodes and interconnections. Nodes usually have limited computation power. They simulate neurons by behaving like a switch, just as neurons will be activated only when sufficient neurotransmitter has accumulated. x1 x2 w1 w2 Dendrites Terminal Branches of Axon x3 w3 S Axon xn wn

31 31 Ex. Learning with Kernels By applying traditional supervised and unsupervised learning methods in the feature space, kernel methods provide powerful tools for data analysis and have been found to be successful in a number of real applications. SVM Support vector machines

32 32 Ex: Fisher Linear Discriminant and SVM (a) The Fisher LD fails to separate two classes because training example D adversely influences decision boundary T. (b) The SVM defines the decision boundary using only points A, B, and C, called support vectors, and is not influenced at all by point D.

33 Ex: CAD for finding micro-calcifications in mammogram region 33

34 CADx 34

35 Ex. CAD for Pulmonary Abnormalities 35

36 Ex. CAD for Pulmonary Abnormalities 36

37 37 Ex. CAD for Pulmonary Abnormalities CT or PET/CT MRI Preprocessing Feature Extraction Classification Image Processing/Computer Vision Machine Learning/PR

38 Ex.Tree-in-Bud (TIB) Detection 38

39 39 TIB Appearance Thickened bronchial structures Locally surrounded by clusters of 2-3 mm micro-nodules 1. Homogeneity (small) 2. Gradient (high) 3. A few mm in length

40 40 Scale and Rotation Invariant Features What is known about TIB? - Small (scale is known) - Gradient is high (variation is known) - Micro-nodule (dots) and thickened vessels should be observed) Design a local scale filter to select candidate regions Derive a suitable shape features rotation scaling translation

41 TIB Overview 41

42 42 At any voxel v in a scene, b-scale: largest homogeneous ball centered at v. t-scale: largest homogeneous ellipsoid centered at v. Local Scale Concept g-scale: largest connected homogeneous region containing v.

43 43 Local Scale Map brain PD slice ball scale tensor scale generalized scale

44 44 Computation of Ball-Scale A hyperball B k, of radius k B k, = e 2 C p ( nx i=1 0 with center at c 2 C 2 i (c i e i ) 2 min j [ 2 j ] ) For a hyperball B k,, we define the fraction of object FO k, as FO k, = P e2b k, (c) B k 1, (c) B k, (c) B k 1, (c)! W ( f(c) f(e) )

45 b-scale object scale estimation algorithm Input: c is pixel in a scene C, ts: homogeneity threshold, FO is objectfraction based on regional homogeneity Output: r(c): b-scale value for pixel c Begin End Set k=1 While FO(c) > ts do Set k to k+1 EndWhile Set r(c) to k-1

46 b-scale object scale estimation algorithm Input: c is pixel in a scene C, ts: homogeneity threshold, FO is objectfraction based on regional homogeneity Output: r(c): b-scale value for pixel c Begin End Set k=1 While FO(c) > ts do Set k to k+1 EndWhile Set r(c) to k-1

47 b-scale object scale estimation algorithm Input: c is pixel in a scene C, ts: homogeneity threshold, FO is objectfraction based on regional homogeneity Output: r(c): b-scale value for pixel c Begin End Set k=1 While FO(c) > ts do Set k to k+1 EndWhile Set r(c) to k-1

48 Candidate TIB Patterns 48

49 49 Willmore Energy/Flow Canham-Helfrich model: deformation of cell membranes is uniquely determined by its shape. In classical Gaussian (K) or mean curvature (H) based energy models are not taking into account topological changes!! However, in CH model, arbitrary topology changes are allowed. Mobius invariance (inversion and affine invariant)

50 50 Mean Curvature H =(apple 1 + apple 2 )/2 Gaussian Curvature K = apple 1 apple 2

51 51 Other Shape Features Let k 1 and k 2 be eigenvalues of the local Hessian matrix for any given local patch where k 1 k 2 Mean curvature (H = (k 1 + k 2 ) / 2) Gaussian curvature (K = k 1 k 2 ) Shape Index (SI): (2/π) *arctan((k 1 +k 2 )/(k 1 -k 2 )) Elongation: (k 1 /k 2 ) Shear: (k 1 -k 2 ) 2 /4 Compactness: 1/ (k 1 k 2 ) Distortion: k 1 -k 2 Overall, we have 8 features extracted from a patch (if there is at least one b-scale pattern in the patch)

52 Comparison to Other Methods 52

53 Deep Learning 53

54 Deep Learning in Medical Imaging 54

55 Ex. Automatic LV Segmentation from US 55

56 Ex. Automatic LV Segmentation from US 56 with Deep Belief Nets

57 Ex. Hippocampus Segmentation Using 7T MRI 57

58 Challenges in 7T 58

59 Hand-Crafted Features 59

60 Deep Learning Features 60

61 Hierarchical Feature Extraction 61

62 Multi-Atlas Segmentation 62

63 63 Qualitative Evaluations

64 Ex. Registration of Brain MR Images 64

65 Deep Learning for Image Registration 65

66 Deep Learning for Image Registration 66

67 Deep Learning for Image Registration 67

68 68 Deep Learning for Image Registration

69 69 Summary ML algorithms are key to detection and diagnosis problem Deep Learning is getting a lot of interests due to its strong features for CAD tasks Image analysis tasks can be enhanced via ML algorithms

70 70 Slide Credits and References Wernick et al., Signal Processing Magazine, Wang and Summers, Medical Image Analysis, Dinggang Shen, Deep Learning Talk at MLMI Workshop, MICCAI Wu, G., et al. MICCAI Kim, M., MLMI, MICCAI 2013.