The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in The Context of Medical Image Mining

Size: px

Start display at page:

Download "The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in The Context of Medical Image Mining"

Vanessa Wilkinson
5 years ago
Views:

1 The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in The Context of Medical Image Mining Frans Coenen ( 10th Interna+onal Conference on Natural Computa+on (ICNC 2014) 11th Interna+onal Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2014)

2 The Challenge of Knowledge Discovery We have many well established techniques for conduc+ng Knowledge Discovery (KD). The issue is the end- to- end KD process. More specifically the prepara+on of data so that KD algorithms can be applied. This is rela+vely straight forward given (tradi+onal) tabular data. This is much more challenging in the context of complex data such as: graphs, free text, video and image data.

3 Presenta)on Focus Image data, both 2D and 3D. Specifically medical image data. Classifica+on (Predic+on) of condi+ons. Classifica+on genera+on algorithms tend to require feature vector (signature) represented training data (but not exclusively so). Three genres of representa+on considered: (i) Sta+s+cal, (ii) Point Series and (iii) Graph Based.

Mo)va)on Clinical decisions are regularly made with the support of medical imagery of some sort (MRI, OCT, etc.). Our ability to collect medical imagery has enhanced rapidly over the last decade (we can do it cheaper and faster).

4 Mo)va)on Clinical decisions are regularly made with the support of medical imagery of some sort (MRI, OCT, etc.). Our ability to collect medical imagery has enhanced rapidly over the last decade (we can do it cheaper and faster). We have seen rapid growth in the global medical imaging sensor market Analysis is s+ll mostly conducted manually, there is very lixle soyware automa+on (although some suppor+ng tools, e.g. Brain Voyager).

5 Classifica)on Process Labelled Training Data Data processing Classifier Generator Unlabelled Sample Data processing Classifier Labelled Sample

6 Example Applica)ons One (2D Re)na Image Data) (b) (a) Example of a 2- D re0na images: (a) Normal re0na, (b) re0nas featuring AMD. We are interested in detec+ng condi+ons such as Age related Macular Degenera+on (AMD) and Diabe+c Re+nopathy (DR).

7 Example Applica)on Two (2D and 3D MRI Brain Scan Data) (a) (b) (c) Example 3- D brain MRI scan: (a) SagiEal plane, (b) Coronal plane, (c) Transverse plane (each scan consists of 256 slices, 768 slices in total). Interested in detec+ng condi+ons such as epilepsy and handedness according to par+cular objects within MRI images (ventricles, corpus callosum).

8 Example Applica)on Three (OCT Re)na Volumes) (a) (b)! Example of a 3- D OCT re0na volumes: (a) Normal re0na, (b) re0na featuring AMD. About 19 slices per volume ( pixels). Again interested in detec+ng condi+ons such as AMD and DR.!

9 Image Capture (a) Re0na Fundus Camera. (b) Magne0c Resonance Imagery (MRI) Machine. (c) Op0cal Coherence Tomography (OCT) set up.

10 Image Representa+on for KD Image Rep. for KD Global (Whole Image based) Sta+s+cal Local (Region/Object based) Graph based Sta+s+cal Set of regions objects Individual (regions/ objects) Point series/clouds Graph based Sta+s+cal

11 Image Representa+on for KD cont. Two approaches: (i) Global (whole image based) and (ii) Local (region or object based). The laxer may require segmenta+on which is an en+re research issue in its own right (possibly outside of the scope of KD). Three (broad) techniques: 1. Sta+s+cal. 2. Point Series. 3. Graph based.

12 Image Representa+on for KD cont. Note that our representa+on needs to be understandable to knowledge discovery algorithms, it does not necessarily have to be understandable to human observers. Although explana+on genera+on is an issue.

13 Image Representa+on for KD Techniques 1. Sta+s+cal. 2. Point Series. 3. Graph based.

14 Technique One (Sta)s)cal Techniques) Simplest approach and easy to represent in terms of a feature space directly compa+ble with classifier genera+on/applica+on. First Order Sta+s+cal func+ons such as the mean, variance and standard devia+on of the intensity or RGB colour values. Alterna+vely moments about some point or axis. Second order sta+s+cal func+ons applied to an intermediate representa+on (co- occurrence matrices, gradient analysis, Hough transforms). General applicability.

15 Technique Two (Point Series) Many second order sta+s+cal techniques lend themselves to representa+on in the form of histograms For example histogram of intensity values, LPBs or orienta+on gradients. Histograms can of course be directly translated into a feature vector representa+on. Alterna+vely, they can be viewed as point series or point curves.

16 Technique Two (Point Series) Given a collec+on of class labelled point series, represen+ng a set of images, a new image can be classified directly using (say) the well established KNN algorithm (a) Histogram of 20 bins. (b) Resul+ng point series.

17 Classifica)on Using Point Series cont. When using algorithms such as KNN we need an appropriate similarity measure (in the case of our point series Euclidean distance does not do the trick). We can look to work on point series comparison techniques, for example the Dynamic Time Warping (DTW) technique which produces a warping path distance defining the difference between two series. DTW has the added advantage that the point series to be compared do not have to be of the same length.

18 Genera+on of Point Series The histogram based approaches to point series genera+on is will suited to the sta+s+cal based techniques (whole image or region based). It is not so well suited to object based representa+on. In this case some alterna+ve mechanism is required to generate the desired point series

19 Example Point Series Genera)on (a) Right ventricle from a 3D MRI Brain Scan Volume. (b) Disc based point series genera0on. (c) Spoke based point series genera0on (conducted in three planes).

20 Second Example Using The Corpus Callosum Alterna+ve 2D example using a segmented object (the corpus callosum) extracted from a 2D MRI brain scan image. (a) 2D Radius intersec0on based point series genera0on. (b) Resul0ng point series.

21 Technique Three (Graph Based) A popular method for represen+ng images is to apply some form of hierarchical decomposi+on and to store the result in a quad- tree (2D image data) or oct- tree (3D image data). Issues with: 1. boundary problem where the object of interest is in different branches of the tree. 2. When to stop the decomposi+on (cri+cal func+on to measure homogeneity or a pre- specified maximum level of decomposi+on).

22 Example Decomposi+on One (a) Corpus callosum in a 2D MRI brain scan. (b) Segmented Corpus callosum. (c) Decomposed Corpus Callosum (level = 3). (d) Quad tree representa0on of the Corpus Callosum.

23 Example Decomposi+on Two (a) Whole image decomposi0on using an alterna0ve approach (level=4). (b) Resul0ng tree representa0on.

Example Decomposi+on three (a) Extra ``box in middle.

Used with respect to OCT re+na volumes where one dimension

Also an axempt to get round the boundary problem.

24 Example Decomposi+on three (a) Extra ``box in middle. (b) Combined oct- tree and quad tree decomposi0on. Used with respect to OCT re+na volumes where one dimension is significantly longer than the other two. Also an axempt to get round the boundary problem. An alterna+ve approach to the boundary problem would be to use some sort of fuzzy rela+onship func+on.

25 Genera)ng a Feature Space from a Collec)on of (Transac)on) Graphs Apply a Frequent Sub- graph Mining (FSM) algorithm to the data. Frequent defined by some threshold σ. Popular FSM algorithm is gspan. Each frequently occurring sub graph is then a dimension in a feature space. This feature space can then be used define feature vectors (binary or real valued) for the ini+al image set, which can be input to any number of classifier generators (Decision Tree generators, SVMs, Naïve Bayes, Classifica+on Associa+on Rule Miners, Etc.).

Frequent Sub- graph mining Feature Space Classifier Generator

26 Classifica)on Process (with frequent sub graph graph mining) Labelled Training Data Data processing (graph genera+on) Frequent Sub- graph mining Feature Space Classifier Generator Unlabelled Sample Data processing (graph genera+on) Classifier Labelled Sample

27 Evalua+on (2D Re+na Images) Sensi)vity Specificity Accuracy AUC FNR Sta)s)cal Point Series Graph Based Whole image classifica+on. Two classes (AMD v. not AMD). 394 images, 165 that featured AMD, and 229 that did not. Re0na image featuring Age related Macular Degenera0on (AMD)

28 Evalua+on (2D Re+na Images cont.) Sensi)vity AMD Sensi)vity Other Whole image classifica+on. Three classes (AMD, DR and normal). 394 images, 165 that featured AMD, and 131 that featured DR and 98 normal. Specif- icity Accu- racy AUC FNR Sta)s)cal Point Series Graph Based Re0na image featuring Age related Macular Degenera0on (AMD) Re0na image featuring Diabe0c Re0nopathy (DR)

29 Evalua+on (2D Musicians MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI scans, 53 represen+ng musicians and 53 non- musicians (i.e. two equal classes). Study of interest because of the conjecture that size and shape of the corpus callosum reflects human characteris+cs such as musical or mathema+cal ability.

30 Evalua+on (2D Handedness MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI brain scans, 42 right handed and 40 ley handed. Study of interest because of the conjecture that size and shape of the corpus callosum reflects human characteris+cs such as handedness.

31 Evalua+on (2D Epilepsy MRI Images) Sta)s)cal (Hough Transform) Sta)s)cal (Zernicke Moments) Accuracy Sensi)vity Specificity Point Series Graph Based MRI brain scans, 106 from the musicians study plus 106 epilepsy scans. The objec+ve was to seek support for the conjecture that the shape and size of the corpus callosum is influence by condi+ons such as epilepsy.

Evalua+on (3D Epilepsy MRI Images) 210 MRI brain scans, 106 from musicians study, plus 106 epilepsy scans. Used the ley and right ventricles as objet of interest.

32 Evalua+on (3D Epilepsy MRI Images) 210 MRI brain scans, 106 from musicians study, plus 106 epilepsy scans. Used the ley and right ventricles as objet of interest. Accuracy Sensi)vity Specificity Point Series (Disc Based) Point Series (Spoke Based 2 Spacing) Point Series (Spoke Based 3 Spacing) Graph Based (σ=30%) Spoke Based Point Series Genera+on (in three planes).

33 Evalua+on (3D OCT Re+na Volumes) Accuracy Sensi)vity Specificity AUC Sta)s)cal (oct- tree) Sta)s)cal (oct- tree with overlap) Graph Based (oct- tree with overlap) D OCT volumes, 2 classes, 68 normal and 52 AMD. Level one 9- tree

34 Conclusions Number of alterna+ve (whole and region/object based) image representa+ons considered, directed at capturing salient features in 2D and 3D image data for classifica+on/predic+on purposes. Three categories of representa+on considered: (i) sta+s+cal, (ii) point series and (iii) tree based. Evaluated with respect to a number of medical applica+on domains. No best representa+on L.

35 Further Work S+ll issues with explana+on genera+on. Lot of scope for alterna+ve representa+ons, especially fuzzy approaches. Lots of scope for further applica+on. (a) Mammogram (a) Hand X- ray

36 Acknowledgements Abdulrahman Albarrak [1], Ashraf Elsayed [2], Marta García- Fiñana [3], Hanafi Hijazi [4], Vanessa Sluming [5], Akadej Udomchaiporn [1] and Yalin Zheng [6]. [1] Dept. of Computer Science, University of Liverpool, Liverpool, UK. [2] Dept. of Computer Science, University of Alexandria, Egypt. [3] Dept. of Biosta+s+cs, University of Liverpool, Liverpool, UK. [4] School of Engineering and Informa+on Technology, University of Malaysia Sabah, Malaysia. [5] School of Health Science, University of Liverpool, Liverpool, UK. [6] Dept. of Eye and Vision Science, Royal Liverpool University Hospital, UK.

The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in the Context of Medical Image Diagnos)cs

The Prac)cal Applica)on of Knowledge Discovery to Image Data: A Prac))oners View in the Context of Medical Image Diagnos)cs Frans Coenen (http://cgi.csc.liv.ac.uk/~frans/) University of Mauri0us, June