The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in the Context of Medical Image Diagnos)cs

Size: px

Start display at page:

Download "The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in the Context of Medical Image Diagnos)cs"

Lora Thompson
5 years ago
Views:

1 The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in the Context of Medical Image Diagnos)cs Frans Coenen ( University of Mauri0us, June 2015

2 The University of Liverpool Located in NW of England. 17,405 UG students; 3,860 PG students. 1,300 Academic staff; 4,300 other staff. Department of Computer Science in Top Ten in the UK.

3 The University of Liverpool

4 The Challenge of Knowledge Discovery in Data We have many well established techniques for conduc0ng Knowledge Discovery Data (KDD). The issue is the end-to-end KDD process. More specifically the prepara0on of data so that KDD algorithms can be applied. This is rela0vely straight forward given (tradi0onal) tabular data. This is much more challenging in the context of complex data such as: graphs, free text, video and image data.

5 Presenta)on Focus Image data, both 2D and 3D. Specifically medical image data. Classifica0on (Predic0on) of condi0ons. Classifica0on genera0on algorithms tend to require feature vector (signature) represented training data (but not exclusively so). Three genres of representa0on considered: (i) Sta0s0cal, (ii) Point Series and (iii) Graph Based.

Mo)va)on Clinical decisions are regularly made with the support of medical imagery of some sort (MRI, OCT, etc.). Our ability to collect medical imagery has enhanced rapidly over the last decade (we can do it cheaper and faster).

6 Mo)va)on Clinical decisions are regularly made with the support of medical imagery of some sort (MRI, OCT, etc.). Our ability to collect medical imagery has enhanced rapidly over the last decade (we can do it cheaper and faster). We have seen rapid growth in the global medical imaging sensor market Analysis is s0ll mostly conducted manually, there is very li`le soaware automa0on (although some suppor0ng tools, e.g. Brain Voyager for MRI).

7 Classifica)on Process Labelled Training Data Data processing Classifier Generator Unlabelled Sample Data processing Classifier Labelled Sample

8 Example Applica)ons One (2D Re)na Image Data) (b) (a) Example of a 2-D re0na images: (a) Normal re0na, (b) re0nas featuring AMD. We are interested in detec0ng condi0ons such as Age related Macular Degenera0on (AMD) and Diabe0c Re0nopathy (DR).

9 Example Applica)on Two (2D and 3D MRI Brain Scan Data) (a) (b) (c) Example 3-D brain MRI scan: (a) SagiEal plane, (b) Coronal plane, (c) Transverse plane (each scan consists of 256 slices, 768 slices in total). Interested in detec0ng condi0ons such as epilepsy and handedness according to par0cular objects within MRI images (ventricles, corpus callosum).

10 Example Applica)on Three (OCT Re)na Volumes) (a) (b)! Example of a 3-D OCT re0na volumes: (a) Normal re0na, (b) re0na featuring AMD. About 19 slices per volume ( pixels). Again interested in detec0ng condi0ons such as AMD and DR.!

11 Image Capture (a) Re0na Fundus Camera. (b) Magne0c Resonance Imagery (MRI) Machine. (c) Op0cal Coherence Tomography (OCT) set up.

12 Image Representa0on for KDD Image Rep. for KD Global (Whole Image based) Local (Region/Object based) Sta0s0cal Graph based Individual (regions/ objects) Set of regions objects Sta0s0cal Point series/clouds Graph based Sta0s0cal

13 Image Representa0on for KDD cont. Two approaches: (i) Global (whole image based) and (ii) Local (region or object based). The la`er may require segmenta0on which is an en0re research issue in its own right (possibly outside of the scope of KDD). Three (broad) techniques: 1. Sta0s0cal. 2. Point Series. 3. Graph based.

14 Image Representa0on for KD cont. Note that our representa0on needs to be understandable to knowledge discovery algorithms, it does not necessarily have to be understandable to human observers. Although explana0on genera0on is an issue.

15 Image Representa0on for KD Techniques 1. Sta0s0cal. 2. Point Series. 3. Graph based.

16 Technique One (Sta)s)cal Techniques) Simplest approach and easy to represent in terms of a feature space directly compa0ble with classifier genera0on/applica0on. First Order Sta0s0cal func0ons such as the mean, variance and standard devia0on of the intensity or RGB colour values. Alterna0vely moments about some point or axis. Second order sta0s0cal func0ons applied to an intermediate representa0on (co-occurrence matrices, gradient analysis, Hough transforms). General applicability.

17 Technique Two (Point Series) Many second order sta0s0cal techniques lend themselves to representa0on in the form of histograms For example histogram of intensity values, LBPs or orienta0on gradients. Histograms can of course be directly translated into a feature vector representa0on. Alterna0vely, they can be viewed as point series or point curves.

18 Technique Two (Point Series) Given a collec0on of class labelled point series, represen0ng a set of images, a new image can be classified directly using (say) the well established KNN algorithm (a) Histogram of 20 bins. (b) Resul0ng point series.

19 Classifica)on Using Point Series cont. When using algorithms such as KNN we need an appropriate similarity measure (in the case of our point series Euclidean distance does not always do the trick). We can look to work on point series comparison techniques, for example the Dynamic Time Warping (DTW) technique which produces a warping path distance defining the difference between two series. DTW has the added advantage that the point series to be compared do not have to be of the same length.

20 Genera0on of Point Series The histogram based approaches to point series genera0on is well suited to the sta0s0cal based techniques (whole image or region based). It is not so well suited to object based representa0on. In this case some alterna0ve mechanism is required to generate the desired point series

21 Example Point Series Genera)on (a) Right ventricle from a 3D MRI Brain Scan Volume. (b) Disc based point series genera0on. (c) Spoke based point series genera0on (conducted in three planes).

22 Second Example Using The Corpus Callosum Alterna0ve 2D example using a segmented object (the corpus callosum) extracted from a 2D MRI brain scan image. (a) 2D Radius intersec0on based point series genera0on. (b) Resul0ng point series.

23 Technique Three (Graph Based) A popular method for represen0ng images is to apply some form of hierarchical decomposi0on and to store the result in a quad-tree (2D image data) or oct-tree (3D image data). Issues with: 1. boundary problem where objects appear in different branches of the tree. 2. When to stop the decomposi0on (cri0cal func0on to measure homogeneity or a prespecified maximum level of decomposi0on).

24 Example Decomposi0on One (a) Corpus callosum in a 2D MRI brain scan. (b) Segmented Corpus callosum. (c) Decomposed Corpus Callosum (level = 3). (d) Quad tree representa0on of the Corpus Callosum.

25 Example Decomposi0on Two (a) Whole image decomposi0on using an alterna0ve approach (level=4). (b) Resul0ng tree representa0on.

Used with respect to OCT re0na volumes where one dimension is significantly longer than

26 Example Decomposi0on three (a) Extra ``box in middle. (b) Combined oct-tree and quad tree decomposi0on. Used with respect to OCT re0na volumes where one dimension is significantly longer than the other two. Also an a`empt to get round the boundary problem. An alterna0ve approach to the boundary problem would be to use some sort of fuzzy rela0onship func0on.

27 Genera)ng a Feature Space from a Collec)on of (Transac)on) Graphs Apply a Frequent Sub-graph Mining (FSM) algorithm to the data. Frequent defined by some threshold σ. Popular FSM algorithm is gspan. Each frequently occurring sub graph is then a dimension in a feature space. This feature space can then be used define feature vectors (binary or real valued) for the ini0al image set, which can be input to any number of classifier generators (Decision Tree generators, SVMs, Naïve Bayes, Classifica0on Associa0on Rule Miners, Etc.).

28 Classifica)on Process (with frequent sub graph graph mining) Labelled Training Data Data processing (graph genera0on) Frequent Sub-graph mining Feature Space Classifier Generator Unlabelled Sample Data processing (graph genera0on) Classifier Labelled Sample

29 Compara0ve Evalua0on

30 Evalua0on (2D Re0na Images) Sensi)vity Specificity Accuracy AUC FNR Sta)s)cal Point Series Graph Based Whole image classifica0on. Two classes (AMD v. not AMD). 394 images, 165 that featured AMD, and 229 that did not. Re0na image featuring Age related Macular Degenera0on (AMD)

31 Evalua0on (2D Re0na Images cont.) Sensi)vity AMD Sensi)vity Other Whole image classifica0on. Three classes (AMD, DR and normal). 394 images, 165 that featured AMD, and 131 that featured DR and 98 normal. Specificity Accuracy AUC FNR Sta)s)cal Point Series Graph Based Re0na image featuring Age related Macular Degenera0on (AMD) Re0na image featuring Diabe0c Re0nopathy (DR)

32 Evalua0on (2D Musicians MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI scans, 53 represen0ng musicians and 53 nonmusicians (i.e. two equal classes). Study of interest because of the conjecture that size and shape of the corpus callosum reflects human characteris0cs such as musical or mathema0cal ability.

33 Evalua0on (2D Handedness MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI brain scans, 42 right handed and 40 lea handed. Study of interest because of the conjecture that size and shape of the corpus callosum reflects human characteris0cs such as handedness.

34 Evalua0on (2D Epilepsy MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI brain scans, 106 from the musicians study plus 106 epilepsy scans. The objec0ve was to seek support for the conjecture that the shape and size of the corpus callosum is influence by condi0ons such as epilepsy.

35 Evalua0on (3D Epilepsy MRI Images) 210 MRI brain scans, 106 from musicians study, plus 106 epilepsy scans. Used the lea and right ventricles as objet of interest. Accuracy Sensi)vity Specificity Point Series (Disc Based) Point Series (Spoke Based 2 Spacing) Point Series (Spoke Based 3 Spacing) Graph Based (σ=30%) Spoke Based Point Series Genera0on (in three planes).

36 Evalua0on (3D OCT Re0na Volumes) Accuracy Sensi)vity Specificity AUC Sta)s)cal (oct-tree) Sta)s)cal (oct-tree with overlap) Graph Based (octtree with overlap) D OCT volumes, 2 classes, 68 normal and 52 AMD. Level one 9- tree

37 Observa)ons For 2D re0na images graph based technique produced the best results. For 2D MRI scans point series technique produced the best results (except in case of epilepsy data where graph based technique produced the best results). For 3D MRI scans graph based technique produced the best results. For 3D OCT re0na images sta0s0cal technique produced the best results.

38 Conclusions Number of alterna0ve (whole and region/object based) image representa0ons considered, directed at capturing salient features in 2D and 3D image data for classifica0on/predic0on purposes. Three categories of representa0on considered: (i) sta0s0cal, (ii) point series and (iii) tree based. Evaluated with respect to a number of medical applica0on domains. No best representa0on!.

39 Further Work S0ll issues with explana0on genera0on. Lot of scope for alterna0ve representa0ons, especially fuzzy approaches. Lots of scope for further applica0on. (a) Mammogram (a) Hand X-ray

40 Acknowledgements Abdulrahman Albarrak [1], Ashraf Elsayed [2], Marta García-Fiñana [3], Hanafi Hijazi [4], Vanessa Sluming [5], Akadej Udomchaiporn [1] and Yalin Zheng [6]. [1] Dept. of Computer Science, University of Liverpool, Liverpool, UK. [2] Dept. of Computer Science, University of Alexandria, Egypt. [3] Dept. of Biosta0s0cs, University of Liverpool, Liverpool, UK. [4] School of Engineering and Informa0on Technology, University of Malaysia Sabah, Malaysia. [5] School of Health Science, University of Liverpool, Liverpool, UK. [6] Dept. of Eye and Vision Science, Royal Liverpool University Hospital, UK.

The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in The Context of Medical Image Mining

The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in The Context of Medical Image Mining Frans Coenen (http://cgi.csc.liv.ac.uk/~frans/) 10th Interna+onal Conference on Natural