PET/CT Image Segmentation using the Channeler Ant Model for Lymphoma Therapy Assessment

Transcription

1 Univeristy of Turin School of Natural Sciences Master of Science in Physics Nuclear and Biomedical Physics Academic Year PET/CT Image Segmentation using the Channeler Ant Model for Lymphoma Therapy Assessment Setareh Fatemi Supervisor: Prof. Cristiana Peroni Co-Supervisor: Dr. Piergiorgio Cerello Reviewer: Dr. Mariaelena Boglione

2

3 The fortress was entered by tunnels in the rock, and, over the entrance to each tunnel, there was a notice which said: EVERYTHING NOT FORBIDDEN IS COMPULSORY The Once and Future King, T.H. White

4

5 Contents Introduction XV 1 Medical Images Computed Tomography Positron Emission Tomography PET/CT Comparing Medical Imaging Modalities Medical Image Analysis Medical Image Registration Medical Image Segmentation Summary Lymphoma Therapy Assessment Hodgkin s and non-hodgkin s Lymphoma Therapy assessment Quantitative and Qualitative PET Analysis The Deauville Index Standardized Uptake Value (SUV) Total Glycolytic Volume (TGV) Swarm Intelligence Nature vs Artificial Life Stigmergy Pheromone V

6 VI Contents 3.4 Foraging Artificial intelligence and fields of application Virtual Ant Colony Algorithm The Channeler Ant Model Movement Pheromone Deposition Energy Colony Evolution Software Implementation CAM Validation The CAM-LTA International Validation Study PET standardization The CAM-LTA algorithm Preprocessing CAM Segmentation ROI List Analysis ROI Liver Analysis Registration on Template Identification of physiological ROIs Liver Identification Bladder Identification Brain Identification Kidneys Identification Heart Identification Filtering of Lymphoma candidates Lymphoma Identification Gaussian Fit Conclusions 81

7 List of Figures 1.1 Electromagnetic spectrum and working regions for medical imaging techniques CT scanner geometry [1] Backprojection: A) Reconstruction of a phantom containing three objects with different attenuation coefficients. B) Projection along one direction. C) Sum of four single projections. [2] Whole body CT image, coronal view Schematic PET scanner and various coincidence events. [3] Whole body PET scan Schematic of a combined PET/CT scanner Images of a patient with Hodgkin lymphoma A-B-C are three different views of a CT scan, D-E-F are three different views of a PET scan, G- H-I show the three views obtained by a combined PET/CT imaging modality [4] Edge detection in a blood pool cardiac image [5] Thresholding segmentation in brain CT scan [6] Region growing segmentation in lung CT scan [7] Atlas based segmentation. Top to bottom: a T1-weighted MR image (coronal, sagittal and transversal sections); the manual segmentation boundaries overlaid on the anatomy; the atlas based segmentation result [8] Lymphoma stages [9] VII

8 VIII List of Figures 2.2 Kinetics of tumor cell killing as a function of chemotherapy cycles. Line A shows a fast tumor response that kills all the cancerous cells in just 4 cycles. Line B shows a minimum rate tumor cell killng. Both the line A and line B provide a negative PET result after two cycles of chemotherapy in contrast with line C, which represents an ineffective therapy and after two cycle would give a still positive PET scan. [10] Deauville scale Examples of Deauville score 1 (left) and score 5 (right) in the International Validation Study Schematic representation of the Goss experiment [11] On the left: percentage of traffic on an arbitrary bridge in Deneubourg s experiment. On the right: percentage of traffic on the short bridge in Goss experiment [12] Ant fork decision [13] Ants life cycle Average pheromone over the colony evolution (in black) and average pheromone on each cycle (in red) vs colony cycle Monitoring of the number of total ants (black), new born ones (red) and dead ones (blue), cycle by cycle Scheme of the CAM structure D artificial objects used for validation [14] Sensitivity vs contamination in the 3D artificial objects used for validation [14] Thresholding vs CAM in the class C 3D artificial object used for validation [14] Region growing vs CAM in the class C 3D artificial object used for validation [14] Region growing vs CAM with 3x3x3 voxel smoothing in the class C 3D artificial object used for validation [14] The CAM-LTA flow chart

9 List of Figures IX 5.2 Bones suppression in PET image: a) CT scan, b) PET scan, c) PET scan without bones contribution CAM segmentation: for three slices (a, b, c), the CAM segmented ROIs are shown in colors on the PET images in grey levels Example of correlation distribution of SUV and HU in the last ROI segmented by the CAM Registrationon a template: a) template CT; b) example of registered CT ROIs volumes in template coordinate space. liver in blue, bladder in light blue, brain in green, kidneys in yellow and heart in red Liver average SUV (left) and volume (right) distributions Bladder average SUV vs average HU (left) and average SUV vs volume (right) distributions Brain average SUV vs average HU (left) and average SUV vs volume (right) distributions Kidneys average SUV vs average HU (left) and average SUV vs volume (right) distributions Hearts average SUV vs average HU (left) and average SUV vs volume (right) distributions Lymphoma candidates in Interim PET/CT, on the left SUV vs average HU and on the right average SUV vs volume distributions First selection of lymphomas from all the candidates Second selection of lymphomas from all the candidates Third selection of lymphomas from all the candidates Average SUV vs average HU in a lymphoma candidate

10

11 List of Tables 1.1 Isotopes of interest in PET imaging and some of their proprieties Activity and Uptake time for 18 F F DG in IVS database PET imaging Combinations of filtering features studied XI

12

13 Introduction Extensive use of medical images for diagnosis and treatment assessment has become extremely common: this generates huge datasets that must be analyzed. Therefore automated tools for medical image segmentation have assumed a crucial role in supporting the clinical evaluation. Physicians have to delineate and characterize the Regions Of Interest (ROIs) in every medical image, which is a time consuming task and may be affected by errors caused by its subjectivity. Instead, thanks to automated tools, the segmentation response can be fast, objective and more accurate. This thesis focuses on the use of virtual ant colonies for medical image segmentation, in particular on the Channeler Ant Model (CAM, Chapter 4), an algorithm developed by our group that exploits virtual ants in a 3D environment to extract maps built on selected features. In our case the CAM is applied to whole body scans obtained by combining Positron Emission Tomography (PET) and Computed Tomography (CT) for the lymphoma therapy assessment. These imaging techniques are described and analyzed in Chapter 1. As mentioned above, the main aim of the algorithm is to analyze the effectiveness of the lymphoma chemotherapy. Lymphoma is a type of tumor that involves the lymphatic system and it is discussed in Chapter 2: the way in which an ineffective therapy can be identified is explained XIII

14 XIV Introduction together with the qualitative and quantitative indices that can be used to perform this important assessment. The quantitative indices need to be calculated with the help of an algorithm, which in this thesis is based on Swarm Intelligence (Chapter 3), a peculiar system in which each individual has limited capabilities and knowledge of the state of the colony, and the collective behaviour generates patterns. The CAM algorithm is used to calculate the Total Glycolytic Volume (TGV), a quantitative index correlated with the tumor extension, which helps physicians in the assessment of the lymphoma chemotherapy effectiveness. The TGV correlation with the Deauville Index, a qualitative assessment by Nuclear Medicine Physicians, is then discussed (Chapter 5): the preliminary results on a dataset of 55 patients are promising, although the False Positive rejection should be improved.

15 Chapter 1 Medical Images Many imaging technologies are used in medicine, for many purposes such as diagnosis, treatment planning, therapy assessment, disease monitoring and image guided surgery. The specific fields of application of each imaging technique depend on its physical features and its advantages and disadvantages related to patients health, image quality and costs. Moreover by acquiring many scans of the same area using different imaging tech- Figure 1.1: techniques Electromagnetic spectrum and working regions for medical imaging niques a multi-analysis can be preformed, which is helpful to the physician because it may offer a more accurate view of the disease (both from the anatomical and the 1

16 2 Chapter 1. Medical Images functional point of view). In this thesis two medical imaging techniques are used in a multi-analysis: Computed Tomography (CT) and Positron Emission Tomography (PET). The first one is based on the interaction of the electromagnetic radiation with the human body, while the second one is based on the β decay which emits two photons back to back then reader by an array of detectors: as shown in Figure Computed Tomography In 1895 the X rays were discovered by Röntegen and were immediately used as a tool to investigate the human body and as such they were considered as a tool for medical diagnosis. [15, 16] The X rays wavelength is in the 0.01 nm to 10 nm range, corresponding to photon energies in the 100 ev to 100 kev range. X ray attenuation due to the transmission through most biological tissues is small and thus it is possible to obtain a radiographic image representing the distribution of the photons transmitted through the patient (the projection of the attenuating properties of tissues along the path of the detected X rays). The attenuation process in a homogeneous slab of material with a µ attenuation coefficient is determined by: I(x) = I 0 e µx (1.1.1) where I 0 and I(x) are the X rays fluences entering the slab of material and after an x thickness. Since the principal interactions causing attenuation are photoelectric absorption and inelastic scattering, bones are more attenuating than soft tissues. This is easily comprehensible due to the higher photoelectric cross section and electronic density (Z) of the bones. In conclusion the Computed Tomography (CT) is an exam that makes use of X rays to build a high resolution 3D image of the human body.

17 1.1. Computed Tomography 3 The CT images are reconstructed on the basis of the X-ray absorption profiles acquired of many angles by rotating the X ray tube around the patient. Figure 1.2: CT scanner geometry [1] The mathematical principles of CT image reconstruction were developed by Radon in [17] Today the most common method is called filtered backprojection a technique that consists in acquiring projections of the patient at different angles, convolving them with a filter function and reconstructing the image by back-projecting the filtered contributions and summing up the attenuation values, as shown in Figure1.3 The CT values are called Hounsfield Units (HU), defined for a voxel with average linear attenuation coefficient µ X as: HU(X) = µ X µ water µ water 1000 (1.1.2) where µ water is the linear attenuation coefficient in the water. Human body CT s have fixed Hounsfield Units [18,19] values: HU = [ 1000, 3000], where the lower limit corresponds to the air and the upper limit to dense bones.

18 4 Chapter 1. Medical Images Figure 1.3: Backprojection: A) Reconstruction of a phantom containing three objects with different attenuation coefficients. B) Projection along one direction. C) Sum of four single projections. [2] As shown in Figure 1.4, bones are white, soft tissue such as brain or liver is grey, while the air is completely black. Figure 1.4: Whole body CT image, coronal view. Because the CT provides better bone details and has high sensitivity for acute hemorrhage, its useful to diagnose traumas and emergencies; instead, its diagnostic power in soft tissue is not sufficient and sometimes requires the use of contrast media. [20] The CT is presently the gold standard in the diagnosis of many diseases and it is also used in early cancer screening.

19 1.2. Positron Emission Tomography Positron Emission Tomography Positron Emission Tomography (PET) is a nuclear medicine imaging technique based on the emission process of a radionuclide tracer, instead of the transmission process used by the CT, and as such its more suitable to study functional processes. The emitted radiation is proportional to the radio tracer concentration which can be functionally led. [5] Positron Emission Tomography is based on the β + positron decay of a radio tracer in the human body, as follows: A ZX A Z 1 Y + β + + ν + Q(e ) (1.2.3) where Q(e ) is the energy difference between the atom initial and final state. The X atom is proton rich and achieves stability by converting a proton to neutron, becoming the Y element. The positive proton charge is carried away with the β + positron, that starts to lose kinetic energy by interacting with the surrounding tissue (elastic and inelastic scattering with electrons, inelastic scattering with Bremsstrahlung radiation and elastic scattering with nuclei). [21] The β + positrons, emitted by the isotope, annihilate nearly at rest with the e electrons of the surrounding tissue and generate two photons. The photons are emitted approximately back to back, and each one carries an energy of 511 kev. The photons are then collected by a ring of detectors that surrounds the patient body. To build the final image, the coincidences are analyzed by reconstructing the Lines of Response (LOR) with an analytic algorithm (filtered back projection) or an iterative method (Maximum Likelihood Expectation Maximization MLEM). [22] In Figure 1.5 a schematic ring of PET detectors is shown: the coincidence detected by the PET scanner may be a true coincidence but also a scattered event (one or both γ undergo a Compton interaction in the patient body before reaching

20 6 Chapter 1. Medical Images the detectors), a multiple coincidence due to two positron annihilations in which three events are counted, and last but not least a random coincidence caused by two positrons annihilations with only one photon of each event counted. If the coincidence is not a true event a mis-assigned LOR is calculated, which contributes to the noise (background). Figure 1.5: Schematic PET scanner and various coincidence events. [3] In Table 1.1 the most used radionuclides in PET are reported: in clinics, the most common nuclide is 18 F, that is bounded to a glucose obtaining the 18 F F DG (18F-fluoro-2-deoxy-glucose). The 18F-FDG radio tracer is easily transported in the cells due to its glucose bound. Once in the cells, the chemical is phosphorylated within the cell and will no longer partecipate in metabolism (tumoral cells). [23] Moreover 18F-FDG is easily absorbed by macrophages, neutrophils and active muscle cells: this may be a problem since it can produce volumes that are active in the PET scan but are not disease-related. These volumes are called false positives; this

21 1.2. Positron Emission Tomography 7 Isotope t 1/2 [min] max(r β +) [nm] < r β + > [mm] Avaiability 11 C On site cyclotron 13 N On site cyclotron 15 O On site cyclotron 18 F Cyclotron, regional distribution 68 Ga Generator 68 Ge/ 68 Ga 82 Rb Generator 82 Sr/ 82 Rb Table 1.1: Isotopes of interest in PET imaging and some of their proprieties may happen particularly in areas of inflammation and infection, for example the bones when a patient is undergoing chemotherapy. Instead false negatives are volumes of the PET scan where there is a tumor which is not recognized because it has a low 18F-FDG absorbtion rate. This may happen if the tumor is slow-growing and as such it absorbs less glucose. [24] As said PET is a functional imaging technique, but is not a selective exam because all the active organs/volumes are marked by the radiotracer, even if they are healthy and only physiologically active. (Figure1.6) Figure 1.6: Whole body PET scan PET is widely used in research studies and is finding growing clinical acceptance, primarily for the diagnosis and the staging of cancer.

22 8 Chapter 1. Medical Images 1.3 PET/CT In the 90s Townsend s and his research group proposed and developed a combined imaging technique to integrate the anatomical information given by a CT scan to the functional information of a PET scan. [25, 26] Before PET and CT scans were separately acquired and then coregistered. [27, 28](Figure 1.7) Figure 1.7: Schematic of a combined PET/CT scanner The CT apparatus is positioned in front of the PET scanner and the centers of imaging fields are separated by 80cm. The typical range covered by both CT and PET is cm. In achieving an intrinsic accurate registration of the anatomical and functional data, dual imaging offers several advantages over conventional imaging techniques: the images are acquired with the patient in the same position and the CT scan can be used to generate a patient specific attenuation map coefficient that is very useful to correct the coincidence data for errors caused by photon attenuation, Compton scattering and other physical effects. Figure 1.8 shows an example of the PET-CT combined technique.

23 1.4. Comparing Medical Imaging Modalities 9 Figure 1.8: Images of a patient with Hodgkin lymphoma A-B-C are three different views of a CT scan, D-E-F are three different views of a PET scan, G-H-I show the three views obtained by a combined PET/CT imaging modality [4] 1.4 Comparing Medical Imaging Modalities Different types of medical imaging have different advantages and disadvantages, risks and benefits. For example modern multi-slice CT scanners provide a high spatial resolution (<1-2 mm); the CT acquisition time is short, the technology is cost effective and widely available but the main risk of this imaging technique is represented by the exposure to ionizing radiation. The PET spatial resolution is poor and the acquisition time is relatively long. The major problem with PET is its cost. The short half-life of most positron emitting isotopes often requires an on-site cyclotron. 1.5 Medical Image Analysis In the clinical practice, computerized medical image processing provides a powerful tool to help physicians in diagnosis, treatment planning and therapy assessment.

24 10 Chapter 1. Medical Images In the last decades, the quantitative image analysis has been explored to improve the sensitivity and specificity of radiological tests based on medical images. Medical image processing is aimed at the enhancement of the clinical information such as diagnostic and therapeutic information and it is comprehensive of image registration, segmentation and visualization methods. Registration involves finding the transformation to bring different images of the same object to a unique spatial and/or temporal congruence. Segmentation consists in accurately recognizing and delineating the relevant structures in the image: it remains the central aim for image processing. Visualization is about the display, manipulation and measurement of image data. Today the major effort in medical image processing is spent in the development of quantitative analysis tools: in fact, visual analysis by human observers suffers limitations due to interobserver variation and errors caused by fatigue, distraction and inexperience. Computer analysis can potentially contribute with an objective and expert point of view, improving the diagnostic accuracy and confidence. [29] Medical Image Registration Image registration is the first step in medical image analysis and is used to estimate the geometric transformation to precisely align two images. It consists in determining the mapping between two images in which the same object or region is investigated. [30] In the clinical practice, the integration of data obtained from separate images is often desired, the geometrical alignment of the human body structures allows making better segmentation and quantification. [31] The image registration process essentially consist in three steps: [32] 1. Feature Detection: the detection of distinctive regions in the two images that have to be registered. 2. Features Matching: a correspondence between the chosen features in the two

25 1.5. Medical Image Analysis 11 images is established. 3. Image Transformation: once the mapping function (the transformation) is found, the image to be registered onto the other undergoes resampling and transformation. Medical image registration has been intensively investigated for almost three decades and several algorithms have been proposed, which can be classified considering similar algorithms features: [33] Dimensionality: 2D-2D, 2D-3D, 3D-3D image registrations are possible. Nature of registration basis: the registration can be performed on raw data, features can be extracted from images or markers introduced in the scans. Features explicitly present in the data and introduced from outside are called intrinsic and extrinsic, respectively. Using the extrinsic method, the registration is easy, fast and there is no need for complex optimization algorithms: this technique is widely used in image guided surgery. The intrinsic method can be based on a limited set of identified salient points (landmarks), on binary segmentations of given structures (feature based) or directly onto variables computed from the image grey values (intensity based). Nature of transformation: the transformation could be rigid, affine, projective, curved or non rigid. Domain of transformation: it could be global or local depending on whether the whole image or a part of it has to be registered. Interaction: the registration can be automatic, semiautomatic or interactive depending on the user intervention level. Optimization Procedure: various measures of the registration quality are applied depending on the data features or on the data itself; the parameters can be computed from the images.

26 12 Chapter 1. Medical Images Modalities involved: the registration can be mono-modal or multi-modal, if the images have been obtained by the same imaging technique or by different modalities, respectively. Subject: If the two images contain the same subject, it is a intra-subject registration. If the subjects in the two images differ, it is a inter-subject registration. A registration on an atlas image is sometimes used. Object: the object of registration is the part of the human body to be registered. Registering two images permits to use the a priori knowledge during the segmentation step, because the images are resampled in the same space. In the therapy assessment and monitoring of disease progress, the registration step is very important because images from the same patient are acquired at different time points or with different imaging methods thus creating possible misalignments between the images, and this reduces the accuracy of further quantitative analysis. For this reason the images must be coregistered to each other. Any registration process is iteratively performed between two images, the fixed one and the moving one, and can be described by four components: a transformation T which is the link between the points of the fixed image p and of the moving image q, a metric M which measures the similarity of the two images, an optimization method O which evaluates the best transformation parameters θ as a function of the similarity metric M and an interpolator I which resamples the pixel/voxels of the moving image in the fixed image space. The complex optimization problems related to the image registration consist in the determination of T, the unknown set of parameters θ and the unknown correspondences (p, q). As a result of the registration process, any point q of the moving image is mapped to q : q q = T (θ)q (1.5.4)

27 1.5. Medical Image Analysis 13 obtaining the resamples moving image q, coregistered on the fixed image p. As previously remarked, the first step in the image registration process is the feature detection. In this phase the set of features to be extracted from the image is chosen and the transformation T and the metric M are set. In this thesis the T transformation used is an Affine Transformation, which uses rotation, scaling (each dimension separately), translations and shears. In 3D the transformation free parameters are 12: 3 parameters come from translation; 3 parameters come from rotation; 3 parameters come from scaling; 3 parameters come from shear; Characteristic of this registration technique is that parallel lines are kept parallel and the ratio of lengths of segments along a line remains fixed; even if the angles and the lengths are not preserved. The affine registration is commonly applied in neuroscience research to align images from different subjects. Even in the affine transformation is widely used in medical imaging analysis, additional degrees of freedom are needed for more accurate registration because the differences between patients and the possibilities of human body motion are more complex. The metric M is another crucial element of the registration technique: it quantitatively measures how well the transformed moving image fits the fixed image and depends on the feature detection modality. Second step of the registration is the Features Matching, which consists in the calculation of the optimal transformation to be applied to the moving image to register it on the fixed one. In this process the crucial role is played by the optimizer

28 14 Chapter 1. Medical Images O and the interpolator I. The basic input of an optimizer is a cost function that depends on the chosen metric: it determines the similarity between the two images given a certain transformation. The most commonly used types of optimizers are the Nelder-Mead downhill simplex [34], the first derivative-based methods (conjugate gradient and gradient descent [35]) and the Levenberg-Marquard method [36]. Often, the solution of the registration problem does not correspond to a metric global minimum: due to the local minima, there is a sizeable chance of choosing a set of parameters which results in a good image match even if it is not the best one. Hence global optimization techniques (evolutionary algorithms and simulated annealing) were developed, but they have a slow convergence rate and therefore are rarely applied in medical imaging registration. On the other hand, multi resolution and multi scale optimization frameworks have shown to be effective in obtaining a fast and robust convergence towards the optimal solution Medical Image Segmentation Segmenting an image means identifying subregions or objects with a similar set of characteristics defined by some chosen specifications: it consists in the process of dividing an image into regions with similar properties such as gray level, color, texture, brightness, contrast, etc. [37] In medical imaging the segmentation is important because it allows the extraction of clinical features from the data, for both diagnosis and treatment assessment. It can also help physicians to perform several tasks such as the quantification of tissue volumes, the localization of the pathology, the study of anatomical structures and the treatment planning. The segmentation addresses many image-related problems: the poor contrast between regions to segment and their surroundings, the noise, due to detectors and the related electronics, that modifies the intensity values introducing an uncertainty contribution, the image non-uniformity, where the intensity level of a single tissue

29 1.5. Medical Image Analysis 15 varies gradually, the partial volume effect due to the discrete nature of the digital images and the presence of artifacts. A manual medical image segmentation can always be performed by an expert and it is normally considered as the only available true segmentation for in vivo imaging, even though it is time consuming and operator dependent. Since there is an intrinsic variability between human tissues among the individuals, the segmentation is always affected by a degree of uncertainty (inter-observer and intra-observer). The inter-observer and intra-observer variabilities should be taken into account in the evaluation of the performance of the automated segmentation methods. To save time and minimize intra-observer variability there is an effort to make the segmentation more automated. The segmentation can be performed by detecting either similarities or discontinuities in the image: the most important approaches belonging to the first category are based on partitioning an image into regions that are similar according to a set of predefined criteria; in the second category, the basic approach is to partition an image based on abrupt changes in intensity called edges. The choice of which approach has to be followed is given by the intrinsic features of the image and of its subregions, such as intensity, contrast and signal-to-noise ratio. [38] These automated methods can also be divided in supervised and unsupervised. [39, 40] In the first case the process needs an operator interaction during the segmentation process. In the second case the operator is needed only at the end of the segmentation: this method is preferred since it ensures a reproducible and objective result. The segmentation algorithms can be classified in three generations. The first generation is constituted by the earliest and low level techniques in which little or any a priori knowledge is necessary. Examples of first generation models include: The Edge Detection method, based on the intensity discontinuity, mimics the way in which humans perceive the different objects in a scene. It can only be applied to images that have a strong contrast on the boundaries of the

30 16 Chapter 1. Medical Images objects that have to be detected. This method id shown in Figure 1.9 Figure 1.9: Edge detection in a blood pool cardiac image [5] Another first generation method used to segment scalar images is the Thresholding, in which different objects can be separated by partitioning in intensity. It is one of the simplest and most popular techniques, based on a global information typically obtained from the grey level histogram of the entire image (global thresholding) or of a subregion of the image (local thresholding).with a simple thresholding algorithm, two classes are found in the image; multithresholding is more general and permits to separate more than two classes in the image. In thresholding algorithms, only the best threshold value of intensity can single out the objects to be detected in the image: the optimal procedure consists in setting an initial estimate threshold and making a preliminary segmentation; after that the best threshold is calculated by means of the two average intensities in the two segmented regions and the final segmentation is then performed. In multithresholding methods the procedure is more complex, depending on the number of regions to be segmented: it consists in determining the best intensity values for the separation of the different intensity ranges of the desired objects or classes. After that the pixels or voxels belonging to the same range are grouped into the same class, forming the same object. Thresholding and multi thresholding are easy to implement and simple to apply, but they are very sensitive to noise, require images having a good object-

31 1.5. Medical Image Analysis 17 to- background or object-to-object contrast and no spatial information is taken into account during segmentation. Figure 1.10 shows the thresholding algorithm applied to a brain CT scan. Figure 1.10: Thresholding segmentation in brain CT scan [6] Last but not least the Region Growing technique is also a first generation algorithm based on the intensity of the image but, in contrast with the thresholding method, it also makes use of spatial information. Region growing is a bottom-up process which starts with the selection of a set of seed pixels/voxels and the definition of a similarity criterion to plan the deterministic stopping rule. Then the objects are segmented growing the regions surrounding the seeds following specific rules: a pixel/voxel is added to the segmented region if it is not assigned to another region yet, has a neighbour that belongs to the region and the new region created by adding the considered pixel/voxel is still compliant with the similarity criterion. The iterative growing stops when no more neighbouring pixels/voxels meet the similarity criterion. Region growing is more flexible than thresholding methods; on the other hand, due to the sequential procedure, this approach is expensive in computational time and memory. Furthermore, the result depends on the seed position and on the order in which pixels/voxels are analyzed. Figure 1.11 show the region growing algorithm applied to a lung CT scan.

32 18 Chapter 1. Medical Images Figure 1.11: Region growing segmentation in lung CT scan [7] Second generation uncertainty models and optimization methods include: Clustering, an unsupervised texture based approach in which the segmentation is carried out under the assumption that each object in the image forms a separate cluster in the feature space. Clustering means to find the natural grouping clusters in the multidimensional feature space. In order to compensate for the lack of training data, clustering methods iterate between the image segmentation and the characterization of the features of each class, and therefore this approach trains itself with the available data. The three most common algorithms are the k-means [41], the fuzzy c-means [42] and the expectation-maximization [43]. There are also several artificial intelligence techniques, developed for optimization, inspired by the behaviour of natural systems, that can be applied as unsupervised methods to the image segmentation. These approaches are based on the Swarm Intelligence and make use of several agents with a limited knowledge of the global system and a local range of action: they interact with each other and modify the environment, bringing out from the image its characteristic patterns (regions to be segmented). The most popular natural-inspired techniques are Ant Colony Optimization (ACO) [44]: this is the technique used in this thesis to perform an assessment of the lymphoma therapy. Starting from this model, several methods to exploit the swarm intelligence in image segmentation were developed.

33 1.5. Medical Image Analysis 19 In the following chapters, swarm intelligence and the ant colony model, on which this work is focused, will be extensively discussed. The main advantage of these methods is the ability to address complex problems with a relatively simple architecture and with a parallel approach. During segmentation, the spatial information is taken into account and the result is less noise dependent than for example, Region Growing. On the other hand, the computational time is not negligible. The third generation is based on the extensive use of a high level knowledge such as a priori information, templates of desired objects, etc. For example the Atlas Based Approaches are powerfull methods for medical image segmentation whenever a standard atlas or template (a composite image formed from segmented and coregistrered images of several subjects) is available. Atlases represent a reference frame for segmenting new images: during the segmentation process, they supply a priori probabilities for the pattern recognition. Conceptually, atlas-based method are similar to classifiers except that they are implemented in the image space and not in the feature space. Moreover, the segmentation is addressed as a registration problem: the the atlases registered into the image to be segmented by finding the optimal spatial transformation and then mapping the anatomical information given by the atlas onto the image being analyzed. Atlas based approached are complex and require distinct steps for the registration and the segmentation, but the spatial information is involved in the segmentation process. There are indications that certain atlas based models can compete with the expert segmentation although the selection, registration and manual segmentation of the images to be considered in the atlas formation can affect the final performance of the algorithm. On the other hand, these methods are limited in segmenting complex struc-

34 20 Chapter 1. Medical Images tures with variable shape, size and features and an expert knowledge is required to build the database. Since atlases are available for brain MRI, atlas based approaches are mostly applied to the brain segmentation as shown in Figure Figure 1.12: Atlas based segmentation. Top to bottom: a T1-weighted MR image (coronal, sagittal and transversal sections); the manual segmentation boundaries overlaid on the anatomy; the atlas based segmentation result [8] 1.6 Summary This thesis is focused on the Channeler Ant Model (CAM), a nature inspired algorithm, used for the segmentation of lymphomas in PET/CT images for the assessment of chemotherapy effectiveness. To this goal the images are coregistered using the described Affine Trasformation to filter physiological from pathological volumes and then region growing is used to study connection between ROIs, but the segmentation is performed with the Channeler Ant Model.

35 Chapter 2 Lymphoma Therapy Assessment Lymphoma is a type of cancer that involves the lymphatic system [45, 46], which consists of several lymph nodes, in clusters or alone, and the lymph, which is transported in a network through the entire body. Also the spleen, the bone marrow, the tonsils and the thymus gland are lymphatic tissues. In the lymph node, there are lymphocytes, a type of white blood cells that filter lymph, searching for pathological material such as bacteria or viruses to destroy. The lymphocytes are divided in two types: B-lymphocytes, whose task is to produce antibodies, and T-lymphocytes, whose task is to kill the pathological material found in the body. Lymphoma type cancer is due to lymphocytes that are transformed to malignant cells; once the tumor is in the body, it can travel from one lymph node to another via the lymph, and it can also reach other lymphatic tissues such as those previously mentioned. If the disease involves also non-lymphatic organs, it is named extra nodal disease. Causes of lymphoma are not known, but risk factors include: age, presence of infections and diseases of the immune system such as HIV, hepatitis B and C, exposure to toxic material (pesticides, herbicides and solvents), and also the presence of relatives with lymphoma. 21

36 22 Chapter 2. Lymphoma Therapy Assessment 2.1 Hodgkin s and non-hodgkin s Lymphoma Lymphomas are categorized in two groups: Hodgkin s lymphoma (HL) and non- Hodgkin s Lymphoma (NHL); divided in 5 and 30 subtypes respectively. The differences between HL and NHL are only seen at a microscopic level and it is important to make a biopsy to distinguish among different types because they react differently to treatments. HL only involves B-lymphocytes and there are two major age groups, years and 55 years and older. Non Hodgkin s lymphoma is the most common type and can involve both B and T lymphocytes; it is also most common in the elderly. The malignancy of lymphoma can be clarified in four stages, as shown in Figure 2.1: 1. Stage I: one group of lymph nodes is involved; 2. Stage II: more than one group of lymph nodes on the same side of the diaphragm are involved and also the non lymphatic organs next to the lymph nodes may be involved; 3. Stage III: groups of lymph nodes on both sides of the diaphragm are involved, alternatively the spleen is involved; 4. Stage IV: non lymphatic organs, that are far away from the lymphatic sites of the disease, are involved. This stage is also reached when the liver, bone marrow or lung is involved.

37 2.1. Hodgkin s and non-hodgkin s Lymphoma 23 Figure 2.1: Lymphoma stages [9] Depending on the number of risk factors the survival rates differ [47, 48]. For NHL patients younger than 60, the 5 years survival rate varies from 83% (with one risk factor) to 32% (in presence of four or five risk factors); instead, for patients older than 60 the survival rate varies between 56% and 21%. Patients with Hodgkin s lymphoma have a better survival rate, which varies from 56% to 90% when five or more risk factors are present.

38 24 Chapter 2. Lymphoma Therapy Assessment The staging of lymphoma used to be performed by CT combined with biopsy, but, after the commercial release of PET/CT scanners, the use of this combined modality became the standard staging technique [49]. Hodgkin s and non-hodgkin s lymphomas represent a major indication for FDG- PET imaging, and the ability of whole body FDG-PET to accurately stage this kind of tumors plays an important role in the patient management. FDG-PET can help better estimate the extent of disease in lymphoma patients, also by identifying extra nodal diseases in soft tissue, spleen, bone and bone marrow and consequently leading to better management and outcomes. Treatments for lymphoma are primarily chemotherapy and radiotherapy. Also, for some types of NHL patients, monoclonal antibody therapy can be used and, for HL patients, surgery is sometimes possible. 2.2 Therapy assessment As previously said, chemotherapy is the primarily treatment for lymphoma, but it is not always effective, so it is very important to make a therapy assessment in the early stages and check whether the patient is responding well to the treatment or not. The assessment was usually performed at the end of the treatment and only in the last decade it was proposed after just a couple cycles of chemotherapy. Therefore only recently the interim chemosensitivity was studied with a FDG-PET scan acquisition during the treatment of HL patients, and used to predict the chemotherapy outcome.

39 2.2. Therapy assessment 25 Figure 2.2: Kinetics of tumor cell killing as a function of chemotherapy cycles. Line A shows a fast tumor response that kills all the cancerous cells in just 4 cycles. Line B shows a minimum rate tumor cell killng. Both the line A and line B provide a negative PET result after two cycles of chemotherapy in contrast with line C, which represents an ineffective therapy and after two cycle would give a still positive PET scan. [10] As Figure 2.2 shows, lymphomas are usually diagnosed when they reach about cells. In the best case scenario chemotherapy kills cancer cell by first order kinetics (linear reduction of tumore size in logarithmic scale) and therefore a cycle of chemotherapy reduced the cancer cell number of 1 log, and as such must be repeated time to kill all the cancerous cells. Figure 2.2 also shows that the lower FDG-PET detection limit (resolution) is , which corresponds to lymphomas of cm diameter, thus the Interim PET can measure only the two first log of the kinetic curve, depending on the initial size of the tumor. Thus a negative PET after two cycles does not mean that the cancer has completely been removed, but that its spatial dimension are reduced to the equivalent of 10 7 cells. Choosing the timing to perform the Interim PET is not an easy task. The goal is

40 26 Chapter 2. Lymphoma Therapy Assessment to distinguish the patients reactive to the chemotherapy from those that are refractory and to do it early enough to avoid the complications of a prolonged ineffective therapy. Thus the Interim PET is acquired after two cycles of chemotherapy: this choice permits to change the therapy is the patient is not responding and also to offer the possibility of shortening the duration or intensity of treatment. 2.3 Quantitative and Qualitative PET Analysis To assess the therapy effectiveness both qualitative and quantitative methods can be used. Qualitative interpretation includes many informations, such as clinical experience, presence of specific disease patterns and knowledge about the artifacts: this is the most common method to assess the therapy effectiveness at the end of all the chemotherapy cycles. However, the reproducibility of a qualitative reading of a PET scan, even if of great importance, is not at all assured. Instead, the use of a quantitative method assures the reproducibility of the index, but still the error due to biologic variability, both in normal and pathological tissues, must be evaluated through repeated measurements of radio tracer uptake The Deauville Index In 2009 the First International Workshop on interim PET scan in lymphoma was held in Deauville, France. The Workshop had two main objectives: to reach a consensus on simple and reproducible criteria for Interim PET interpretation in HL and diffuse large B-cell lymphoma (DLBCL) and to launch international validation studies in order to evaluate the effectiveness of these rules [50]. A consensus among experts was reached on qualitative determination of residual FDG uptake by visual assessment, summarize with the so-called Deauville 5-Point Scale (5-PS). Using the Deauville 5-PS, the physician, expert in Nuclear Medicine, considers the

41 2.3. Quantitative and Qualitative PET Analysis 27 sites of initial disease in Interim PET and then compares the corresponding residual uptakes to the mediastinal blood pool structure (MBPS) and to the liver [51]. The 5 scores are determined with the following rules and the discrimination method is shown in Figure 2.3: Figure 2.3: Deauville scale Score 1: no uptake; Score 2: uptake < mediastinum; Score 3: mediastinum < uptake < liver; Score 4: uptake moderately increased above the liver at any site; Score 5: markedly increased uptake above the liver and/or new sites of Lymphoma. These criteria were used for nodal and extra nodal disease, considering scores 1 to 3 as negative and scores 4 and 5 as positive. Figure 2.4 shows examples of Deauville score 1 (Effective Therapy) and score 5 (Uneffective therapy): the initial FDG-PET, labelled with Baseline PET/CT and the Interim PET/CT are compared. The images were provided by the International Validation Study (IVS), that involved HL patients and started in 2009 to validate the Deauville consensus as described in the following chapters.

42 28 Chapter 2. Lymphoma Therapy Assessment Figure 2.4: Examples of Deauville score 1 (left) and score 5 (right) in the International Validation Study. On the left of Figure 2.4 the patient has Deauville score 1: the Baseline PET/CT shows a group of lymphomas in the upper left side of the neck and also in the axillary lymph nodes, recognizable for their high uptake. In the Interim PET/CT the same regions are not anymore recognizable, this is an indication of the effectiveness of the therapy. Otherwise on the right of Figure 2.4 the patient has Deauville score 5: the Baseline PET/CT shows a single lymphoma on the right side of the neck, just above the sternum. In the Interim PET/CT this lymphoma is still recognizable, thus indicating a non-effective therapy. Moreover, a more detailed set of instructions was also drawn up to deal with other variables such as the interpretation of marrow uptake. Nodal and extranodal focal FDG uptake in Interim PET represents residual

43 2.3. Quantitative and Qualitative PET Analysis 29 lymphoma if the uptake is larger than the liver s uptake at sites involved on Baseline PET, and as such the core must be 4 or 5. One or more new lesions, not shown in the Baseline PET, in a patient who is responding to treatment at other sites are unlikely to be lymphomas: the patient should be scored 1, 2 or 3 depending on the residual uptake in initial disease sites. One or more new lesions, not present in Baseline PET, in a patient with residual lymphoma are likely to represent a new lymphoma: the patient should be scored as 5, unless there is a clear alternative explanation. Diffusely increased uptake in the bone marrow is usually due to marrow stimulation after chemotherapy and should not be misinterpreted as marrow involvement even if focal uptake was present in the marrow in the Baseline PET. Diffusely increased uptake un the spleen associated with diffuse bone marrow uptake is usually due to chemotherapy effects and should not be misinterpreted as splenic involvement even in the presence of focal splenic uptake in the Baseline PET. Focal reduction of uptake in sites of marrow involvement in the Baseline PET occurs due to marrow ablation with successful treatment; focal increased uptake may occur at sites where there was non disease in the Baseline PET due to chemotherapy stimulation: focal uptake in the marrow with this pattern should not be misinterpreted as disease. Symmetrical tonsillar uptake is most likely to represent non-specific inflammation uptake in HL and should not be misinterpreted as lymphoma. Asymmetric uptake in the Interim PET should be regarded as disease only in the presence of clear evidence of tonsillar involvement in the Baseline PET Standardized Uptake Value (SUV) The Standardized Uptake Value is a quantitative method to measure the radio tracer uptake, which is the relevant information in a FDG-PET scans.

44 30 Chapter 2. Lymphoma Therapy Assessment To perform any assessment analysis it is necessary to convert the actual activity, calculated in each voxel in MBq/g, into a quantity that represents this clinical feature. The Standardize Uptake Value is used to normalize the dependence on the injected radio tracer dose and activity, on the elapsed time between injection and scan, and on patient body attenuation, which is approximately proportional to the patient weight. The SUV (v i ) in a voxel v i is calculated as in equation 2.3.1: SUV (v i ) = A(v i) w p A tot (2.3.1) where A is the activity concentration of the given voxel v i, w p is the patient weight and A tot is the radio tracer total activity corrected from the time of measurement to the time of scan. The SUV precision depends on many factors such as the accurate measurement of the injected dose, the complete injection of the dose, the accurate decay correction of all measurements, the correct entry of all relevant values into the reconstruction program, for example the patient weight; it also depends on the similar blood time activity curves of all subjects (blood perfusion, vascular volume in tissue, plasma glucose level, etc.) and on the similar systemic condition of the body (uptake of tracer in other tissues and organs, excretion rate of the radio tracer, body fat percentage, biochemical metabolism of the radio tracer in the body, etc.). Any inaccuracy in one of these parameters can translate in a significant error in the SUV calculation. In order to minimize the error and improve the reproducibility of the SUV calculation a normalization can be considered. A ratio between lesions SUV and normal tissue SUV may be used, based on the assumption that a normal tissue can be identified and its SUV is a reproducible value. Many tissues have been proposed as normal reference, among them there are thighs, back muscle, liver and mediastinum; but in this thesis the liver has been selected because of its importance for the Deauville score assignement.

45 2.3. Quantitative and Qualitative PET Analysis Total Glycolytic Volume (TGV) Another quantitative analysis index is the Total Glycolytic Volume, which is a global quantitative variable that can show the overall beahaviour of the tumor [52]. It is correlated to the tumor mass and to the average SUV as shown in equation 2.3.2: T GV = MT V < SUV > (2.3.2) where MT V is the Metabolic Tumor Volume which is defined as the part of the tumor volume that has increased FDG uptake and as such is an index of tumor extension. Even if intuitively the tumor volume seems important and desirable to determine, stem cell biology suggests that the most critical parts of the tumor are only the most aggressive portions, those that are more active in FDG-PET scans because of their high glucose use. MT V and < SUV > are usally very difficult to determine, but they can be automatically detected by using a segmentation algorithm such as the one proposed in this thesis and described in the following chapters.

46

47 Chapter 3 Swarm Intelligence Swarm Intelligence is an artificial intelligent paradigm based on the studies of systems in which the collective behaviour of several individuals generates characteristic patterns (ants, bees, termites). It is also an inspiration for a group of algorithms based on the observation of the behaviour of swarms: they are implemented by using simple individual agents that interact with each other and cooperate without any external form of central control. [53] These systems are decentralized and self organized; their evolution is characterized by non-linearity and chaos and it depends both on external and internal factors. Each agent has limited knowledge and capabilities and does not know the global state of the system. The agents act just following the local stimuli present in the environment and communicate in an indirect way through the deposition of chemical substances, called pheromones. Nevertheless, the collective behaviour gives origin to the solution that optimizes the colony survival. In fact swarm intelligence is widely used to solve hard computational problems by using a simple design characterized by flexibility and robustness; due to the fact that social insects belong to a super organism composed by many agents, as such the perception of the colony is the complex sum of perceptions of all its members. 3.1 Nature vs Artificial Life These studies of colonies social behaviour were conducted on many species of insects. From this studies three main characteristics were found and are considered 33

48 34 Chapter 3. Swarm Intelligence the reason for the almost universal presence of insects in the echospere: flexibility, robustness and self organization. 1. Flexibility: the colony adapts itself to changes in the environment and in the population size. This happens thanks to the modulation factors produced by the environment and the activity of the colony, without any changes to the behavioural rules at the individual level. 2. Robustness: the group behaviour is not influenced by single individual failures. In fact this is due to the multiple interactions between agents and the ability to compensate the failure of one individual by other interactions. Moreover colonies can survive a large variety of external perturbations and even partial destruction. A robust system must not be static or rigid: if the environment changes, the colony should be able to adapt itself to the new situation and the new fit is propagated inwards until all individuals adapt themselves to the new colony state. 3. Self Organization: in a swarm there is no central control nor local supervision. Self organization is defined as a global order that emerges from local interactions. The self organizing structure is maintained by means of four crucial methods: positive feedback, negative feedback, amplification of fluctuations and multiple interactions between individuals. The positive feedback promotes emerging of a pattern that is useful for the colony and as such helps the system stability. Negative feedback balances the positive one by taking into account the differences that can arise from the environment or from inside the colony thus allowing the colony to change its state.

49 3.2. Stigmergy 35 In the swarm behaviour the stochastic component is not negligible: it gives each individual the possibility to explore the environment finding new solutions for the colony. Multiple interactions among several individuals generate patterns seemingly in a deterministic way. In a numerical simulation, generally, an organized system is characterized by a structure with functions set by the designer: the system components are arranged in a certain order, the definition of the connections and the separations of different subsystems or components is required, and the functions aim to fulfill a given purpose. In nature instead, something completely different takes place: systems with a swarm intelligent behaviour appear to have emerged and evolved without outside intervention or programming, without a given structure with the corresponding functions. Moreover, they can tackle more complex problems than any computer system. Thanks to swarm intelligence, the functional structure appears and is spontaneously maintained: the control on the colony evolution is distributed over all the individuals. [54] Artificial life tries to mimic the natural swarm behaviour to exploit the potential advantages of this approach. In the next sections the behaviours of real insects will be explained. 3.2 Stigmergy Stigmergy is a term introduced by P.P. Grasseè, one of the first researchers to study the social behaviour of insects, that refers to the indirect communication in which each insect is stimulated by its own performance. [55] Stigmergy is based on a physical process of environment modifications and it is characterized by a local action that generates the information that can only be accessed by the insects that visit the place where the environmental changes took place.

50 36 Chapter 3. Swarm Intelligence Stigmergy can be quantitative or qualitative (discrete). The quantitative one is a threshold based process, that determines whether there is a reaction or not based on the perceived concentration of substances released in the environment. So the probability that a stimulus can be detected grows with its strength and as such plays a crucial role in the insects decision making process. The qualitative stigmergy consists in receiving different stimuli to trigger different reactions. It refers to conditions and actions that differ in kind rather than in intensity. In practice it is not possibile to clearly separate the qualitative and quantitative stigmergy because all complex actions require a collection of different stimuli that can be given by both the topologies of stigmergy. 3.3 Pheromone The indirect communication through environmental changes takes place via pheromone deposition. The term pheromone, which literally means carrier of excitement, was introduced by P. Karlson and M. Luscher [56]. A pheromone is a chemical substance that possess certain meanings that are understood by all the individuals of the same species. Pheromones have different functions in communication, for example ants use a pheromone trace as a path to food (foraging). Studies on real ants life show that pheromone power isn t linear and is a function of : The sensory capability, that reaches a saturation point, after which the perception of pheromone would not change with the released quantity. the ant s osmotropotaxis sensitivity, that is a parameter linked to the way ants smell pheromone, i.e. to the degree of randomness with which the pheromone trails are followed.

51 3.4. Foraging 37 In nature there are other factors that influence the intensity of the perceived pheromone: some examples are the ant speed, its antennas size, the angle under which the trail is perceived. Furthermore, the pheromone trails can evaporate and diffuse. Using stigmergy and pheromone deposition it is (also) possible to create communication and collaboration between the agents of a virtual colony, and in this way achieve a goal or adapt the colony behaviour to the changing environment. In computing, the update of released pheromone would increase the values associated with good and promising solutions and decrease the values associated to the bad ones. It can be implemented both through evaporation (similar to what happens in nature), and through saturation. The pheromone degradation (evaporation or saturation) is needed to avoid a too rapid convergence of the algorithm, permitting the exploration of new volumes to achieve the best solution. 3.4 Foraging Foraging is one of the best examples of how important stigmergy and self organization are for the insects survival. In fact, while at the beginning the ants explore the environment randomly searching for food, after some time the random motion component decreases and the majority of ants starts to walk the shortest way to reach the food source. This happens because the deposited pheromone evaporates and therefore, ant after ant, the shortest paths linking the anthill to the food source are better marked than longer ones. Proof of this assumption is given by an experiment conducted by S. Goss [11], the aim of which is to prove the ability of ant colonies to adapt their foraging efforts to the nearest and most promising food source. The Goss experiment consists in linking an anthill and a food source using two bridges of different lengths and studying the traffic percentage on the two paths. At the beginning the first ant has equal probability to choose one of the paths but the ants that chooses the shortest path is the first to come back to the nest and as

52 38 Chapter 3. Swarm Intelligence such it deposits two traces of pheromone on the shortest bridge while on the longest path there will be just one trace. The next ant that has to choose a path will follow the strongest pheromone trace thus strengthening it. Therefore the converging processes is not only modulated by the random fluctuation in pheromone deposition but also by the difference in length. The Goss experiment showed that, thanks to stigmergy, the ant colony converges to reach the food through the shortest path with a percentage of the traffic on it of 80% in about 80% of the experimental trials. Figure 3.1: Schematic representation of the Goss experiment [11] A similar experiment was performed by Deneubourg on ants colonies [57]: this time the two bridges linking the anthill to the food source were equally long. As in the previous experiment the initial colony behaviour is random and the ants start to deposit pheromone on both bridges, but this time, due to random fluctuations, one of the two bridges gains a stronger pheromone trace and it becomes ever stronger until the colony converges. In about 70% of the performed experimental trials, the percentage of ant traffic converges to only one bridge (> 80% in the most visited bridge); only in a few percentage of experiments (< 5%) the colony behaviour does not converge at all. We can see the percentage of traffic in the two experiments in Figure 3.2.

53 3.5. Artificial intelligence and fields of application 39 Figure 3.2: On the left: percentage of traffic on an arbitrary bridge in Deneubourg s experiment. On the right: percentage of traffic on the short bridge in Goss experiment [12] 3.5 Artificial intelligence and fields of application Over the past decades, many mathematical models were developed to describe the behaviour of social insects: these techniques are now applied to fields such as business, medical image segmentation and optimization problems in general. Artificial intelligence algorithms make use of several branches of computer science and mathematics, including: pattern recognition, predictive modeling, text mining and search, genetic programming, heuristics, inference, ontology and data analytics. Today, the artificial intelligence technology is based on a number of advanced mathematical methods for optimization, regression and classification. It finds applications in a wide variety of fields, among them: speech recognition, computer vision, expert systems, heuristic classification, medical diagnostics and credit card fraud.

54 40 Chapter 3. Swarm Intelligence Virtual Ant Colony Algorithm Virtual ant colonies were used to solve many optimization and segmentation problems. In 1995 Chialvo and Millonas [13] introduced one of the simplest and most efficient models of trail forming where the ants move in a 2D environment searching for food; in their studies the pheromone trails created by ants were compared to the cognitive brain map patterns. Ramos and Almeida [58] developed a virtual ant model where the colonies are constituted by a constant number of ants that live in an image. They proved the ants capability to reprocess different types of images reaching in the end a global perception of the image. In this model a mechanism that self-regulates the population by using the concepts of aging, death and reproduction in the ant colony is introduced. A virtual ant based approach was also applied to solve other open questions like the problem of load balancing and message routing in telecommunications, in the traffic congestion prediction and the Travelling Salesman Problem (TSP) [44]. Virtual ant algorithms have also been proposed for both the 2D and 3D HP proteins folding problem, which consider a 2D and 3D lattice respectively. [59] In economy, the ways ants use to communicate the presence of a source of food were investigated to model the raiding new markets process: the size of the organizations, the characteristics of the market place and the competitive environment present the same implications found for ant colonies. Furthermore the work allocation principles were taken into account to optimize retail distribution chains and markets management. A field in which the application of ant colony algorithms gives promising solutions is digital image processing. Ant algorithms have been used for basic low level image segmentation via boundary detection methods and via clustering methods. In 2005 Bocchi proposed an image segmentation method that makes use of an evolutionary swarm-based algorithm in which different populations of individuals compete to occupy the 2-D image to be analyzed [60]. The comparison to other techniques

55 3.5. Artificial intelligence and fields of application 41 showed an improvement in the segmentation of noisy images. Furthermore virtual ants are successfully applied to the medical images segmentation in Computed Tomography, in Magnetic Resonance Imaging for the White Matter (WM), the Grey Matter (GM) and the CerebroSpinal Fluid (CSF) segmentation [61], in Positron Emission Tomography for the medical volume delineation [62].

56

57 Chapter 4 The Channeler Ant Model The Channeler Ant Model (CAM) is a segmentation method based on swarm intelligence which implements ant colony rules to perform a 3D segmentation of complex structures, in particular in medical images [14]. The model is called Channeler Ant Model because the ants are released from an anthill and then start channeling through the object they have to reconstruct. Thanks to stigmergy, the ant colony lives in a 3D digital environment constituted by the image voxels exploiting the pheromone power to optimize the colony behaviour and creates some characteristic patterns of the system. The virtual ants move and explore the habitat releasing pheromone with patterns that depend on the image features. The number of individuals of the colony varies according to the properties of the digital environment in which the ants live: like in nature, the ants can die and breed, depending on the living resources the colony can find in the surroundings. When the colony evolution ends and all the ants in the colony are dead, the characteristic pattern emerged from the pheromone map is analyzed, providing the image processing result, that is the segmentation of a peculiar Region of Interest (ROI). In the next sections the model rules are described. 4.1 Movement One of the rules that are implemented in a Swarm Intelligence algorithm is the function that controls the ant movement. Assuming that an ant is in the voxel v i, its probability to choose one of the neigh- 43

58 44 Chapter 4. The Channeler Ant Model boring voxels v j is given by: P ij (v i vj ) = W (σ j ) 26 n=1 W (σ n) (4.1.1) where W is a function of the pheromone quantity σ j released in the voxel v j : W (σ j ) = ( 1 + ) σ j β <σ> 1 + δ σ (4.1.2) j <σ> In this equation β and δ are parameters determined in studies on real ants [13, 63]. The β parameter is related to the osmotropotaxis sensitivity: it can be considered as a physiological inverse-noise parameter of gain that controls the degree of randomness with which the ants follow the pheromone gradients. Low values of β mean that the pheromone trails do not greatly affect the ants response; high values determine a strong pheromone power in the colony evolution. In order to correctly mimic the real ants, the sensory capacity has to be considered, since the ability to smell pheromone decreases with its concentration. The sensory capacity is related to the average time an ant is able to follow a given trail. A limited sensory capacity helps the colony to explore the habitat and find new trails, this characteristic is modelled by the parameter 1 /δ. In Figure 4.1 a fork decision simulation is shown; the ant is constrained to walk until it reaches a fork and then to choose a path considering the pheromone trails σ 1 and σ 2, and σ 2 is set to one. The graphs show that the probability to move from voxel zero to voxel one P 01 (v 0 v1 ) is non-linear and changes as a function of σ and δ. If σ 1 = σ 2 = 1 they have the same probability to be walked. In principle each ant can move in all neighboring voxels (26 positions), but in reality there are forbidden voxels such as positions already occupied by an ant or voxels outside the volume of interest (called habitat). Furthermore, in the Channeler Ant Model the pheromone evaporation is not directly implemented: to mimic its effects a saturation phenomenon has been implemented.

59 4.2. Pheromone Deposition 45 Figure 4.1: Ant fork decision [13] In fact, there is a fixed maximum pheromone amount, called pheromone threshold. This parameter is set at the beginning of the colony evolution and when a voxel reaches the threshold its position becomes unavailable for future ant visits. The most important aspect of this forbidden destinations is that if an ant has not any possible future destinations it dies, but is not the only reason for an ant s death. 4.2 Pheromone Deposition When an ant moves it releases in the starting voxel v i a quantity of pheromone T (v i ) given by: T (v i = η + H fac ph(v i )) (4.2.3) where η and H fac are constants, η is used to mark the voxel visited from those that has not been used (η = 0.01); H fac is related to the algorithm speed. The most interesting term of the equation is ph(v i ), which has a crucial role in the image segmentation. ph(v i ) depends on the 3D digital habitat features and must be adapted to the specific goal of each CAM application. To determine ph(v i ) the high level fea-

60 46 Chapter 4. The Channeler Ant Model tures that lead the segmentation must be known, thus the segmentation must be performed through image intensity. ph(v i ) is the only link between the real and the virtual world: ants are able to process and elaborate it in a non linear way, with the aim of generating characteristic segmentation patterns. The definition of ph(v i ) will be given in relation to the particular algorithm used in this thesis (CAM-Lymphoma Therapy Assestement) and will be described in detail with the algorithm, in the next chapter. Depending on the requirement, the pheromone deposition can be carried out considering only some features obtained simply from the image, applying or not some enhancement filters, (intensity histogram, edges) or evaluating also a priori information, not directly related to the image variables, (shape templates, atlas). Even a segmentation based on a combination of these two approaches can be performed. 4.3 Energy Real ant colonies evolve in time changing the population due to external environmental changes and due to the presence of advantageous or adverse factors. In fact, also in the algorithm for image segmentation the number of individuals in a colony is a central point of the model. In the CAM, a life cycle is implemented with the aim of self optimizing the colony population thus converging to a solution (segmentation) without any external intervention. During the cycles the ants are born, their life cycle continues through their motion in the image and if the conditions are favorable they breed, generating new ants. When the habitat becomes hostile or the ants are not able to move anymore, they die. This life cycle is implemented in the algorithm by means of the E parameter, called the energy. Every ant has its own energy that can change during its life cycle. The energy parameter allows the ants to have a limited perception about the global

61 4.3. Energy 47 behaviour of the colony but this information is enough to perform the colony evolution and stop the segmentation process when required. Every ant is born with an E 0 energy. E 0 = 1 + α (4.3.4) The ants energy can range between E d, where the ant dies, and E r, where the ant breeds. (Figure 4.2) Figure 4.2: Ants life cycle The true free parameter that manages the colony evolution is then: E r E d α (4.3.5) The energy of each ant must be updated at every cycle and depends on both the behaviour of the single ant and of the entire colony. If we consider the cycle i + 1 the energy of the ant k is given by: ( ) k Ei+1 k = Ei k ph (i + 1) + α 1 (4.3.6) < ph > tot where k ph (i + 1) is the pheromone released by ant k in cycle i + 1 while the parameter < ph > tot is the average of all pheromone released in the colony life. The variability of the pheromone released by each ant during a cycle could be mistaken as a cause of instability of the algorithm, but the parameter that gives us the stability of the system is the average deposited pheromone over the colony evolution. In Figure 4.3 an example of the average pheromone quantity trend is shown: the average deposited pheromone over the colony evolution is more stable than the av-

62 48 Chapter 4. The Channeler Ant Model erage on each cycle. Figure 4.3: Average pheromone over the colony evolution (in black) and average pheromone on each cycle (in red) vs colony cycle When the energy of an ant reaches the value considered for breeding, the maximum number of ants that can be generated is calculated. The number of ants that can be generated is proportional to the average pheromone that can be deposited in the neighbor voxels, the higher is the average pheromone that can be deposited the more ants are generated, with an upper limit set to 26, which is the the total of the neighboring voxels. To procede with the ants generation a check over the number of free neighbor voxels is made and the number of generated ants is equal to the number of free voxels in which they can be positioned. Equation shows that if the pheromone released by an ant is lower than the average deposition over the total colony evolution, the energy decreases. Therefore the colony population is led by the relation between the pheromone depositions and < ph > tot. Another factor to consider is the death of each ant, which occurs when the energy is lower than E d or there is no possible destination.

63 4.4. Colony Evolution 49 Figure 4.4 shows, cycle by cycle, the total number of ants, the new born ones and the dead ones. In this case, the average life of an ant is about 5 cycles. Figure 4.4: Monitoring of the number of total ants (black), new born ones (red) and dead ones (blue), cycle by cycle 4.4 Colony Evolution The evolution of the colony starts from the anthill which is set in a voxel that belongs to the object to be segmented. To start the algorithm 26 ants are generated and they start moving randomly in the environment releasing pheromone as previously described. At every cycle the pheromone strength becomes more and more important in the decision making of the remaining ants. When in a voxel the pheromone threshold is reached, that destination becomes forbidden. This constraints makes the ant colony a self normalizing system. When all the voxels, belonging to the object to be segmented, are saturated the ants move outside it until they reach regions where they lose energy until they reach E d and die.

64 50 Chapter 4. The Channeler Ant Model The colony evolution pattern is strongly dependent on the object complexity and the anthill location. Figure 4.5 shows the software workflow and the tools that have been used for its implementation. Figure 4.5: Scheme of the CAM structure 4.5 Software Implementation The first version of the CAM code was developed by the MAGIC5 (Medical Applications on a Grid Infrastructure Connection) project and its implementation was based on the ROOT framework and used the DCMTK libraries for the decoding of medical images (Figure 4.5). ROOT is an object-oriented framework developed by CERN for data analysis.

65 4.6. CAM Validation 51 DCMTK is a collection of libraries and applications for the management of images stored in DICOM (Digital Imaging and COmmunications in Medicine) standard format. Both of them are available as open source software. The MAGIC5 framework was then linked to the Insight Toolkit (ITK) to simplify the management of the images and their filtering. ITK is an open-source, cross-platform system that provides developers with an extensive suite of software tools for image analysis. Depending on the user request, the CAM output could be a binary/non-binary image of the segmented region, a ROOT file containing the collection of the detected Region Of Interest (ROIs) or a text or an xml file where the ROI features are listed. 4.6 CAM Validation The CAM performance was validated using a set of 3D artificial objects, which was formed by different 3D complex connected structures, corresponding to patterns that can be found in medical images. The advantages of validating the CAM in an artificial environment is that the truth is a priori known without any uncertainty and the complexity of the segmentation challenge can be easily varied (shape, intensity uniformity, noise, background level). Three classes of artificial objects were defined: Class A is given by artificial objects with homogeneous intensity and zero noise; Class B is formed by objects with heterogeneous intensity extracted from a Gaussian distribution and with zero noise; Class C is made by objects with heterogeneous intensity and noise extracted from Gaussian distributions. (Figure 4.6)

66 52 Chapter 4. The Channeler Ant Model Figure 4.6: 3D artificial objects used for validation [14] In the CAM validation, the pheromone deposition ph is defined by: ph = I(v j ) I min (4.6.7) where I(v j ) is the image intensity of the voxel v j and I min is the minimum intensity of the digital environment. By using this definition for ph the ant colonies segment 3D connected voxels with high intensity values. Thanks to the different shapes of the artificial objects, the robustness of the model with respect to the anthill positioning is also tested. The CAM performance results independent of the anthill position, more specifically the trend of the population size and the segmentation time depend on the anthill position, but the segmentation result is not affected. Because the pheromone map is not a binary segmentation response, the pheromone value above which the voxels are considered belonging to the ant segmentation (called pheromone saturation threshold) must be set and, in general, it could be different from the pheromone saturation threshold fixed in the model rules. To perform this analysis, the sensitivity and the contamination must be studied by varying the pheromone segmentation threshold. The sensitivity is defined as the ratio between the voxels that are correctly segmented as part of the 3D artificial object and all the voxels that truly belong to it;

67 4.6. CAM Validation 53 the contamination is the ratio between the voxels that are wrongly segmented by ants and the voxels that are really part of the object. In Figure 4.7 the correlation between sensitivity and contamination, for the 3D artificial objects belonging to the Class C, is shown: for all the shapes, the CAM reaches a very high sensitivity (of about 0.997) with a negligible contamination (of about 0.05). Figure 4.7: Sensitivity vs contamination in the 3D artificial objects used for validation [14] Furthermore the CAM segmentations are compared with those obtained by applying thresholding and region growing approaches. (Figure 4.8) Figure 4.8: Thresholding vs CAM in the class C 3D artificial object used for validation [14]

68 54 Chapter 4. The Channeler Ant Model In particular, Figure 4.8 shows the comparison between the CAM and the thresholding algorithm on Class C Highway. Since the channelling features (3D connections are taken into account during segmentation) allow the rejection of high intensity voxels that are far from the structure to be segmented, the CAM performs better than the thresholding and the higher the noise level is, the larger the performance gain is. Figure 4.9: Region growing vs CAM in the class C 3D artificial object used for validation [14] In Figure 4.9 the CAM is compared to the region growing algorithm. The results are quite different: both methods take into account 3D connections and, therefore, reject noise far from the object, reaching significantly better performances with respect to thresholding. However, while the CAM contamination increases with the noise, the region growing result is very stable. Such a behaviour is strictly related to the fixed threshold in intensity applied in region growing approach while the CAM progressively stores information about intensity during colony evolution.

69 4.6. CAM Validation 55 Figure 4.10: Region growing vs CAM with 3x3x3 voxel smoothing in the class C 3D artificial object used for validation [14] Overall, the CAM performance is better when the noise is lower, while the region growing is better at higher noise values. Such differences can be overcome by smoothing the pheromone map with a 3x3x3 voxels filter: the results are shown in Figure 4.10.

70

71 Chapter 5 The CAM-LTA The Channeler Ant Model-Lymphoma Therapy Assesment (CAM-LTA) is a tool developed to compare the PET/CT images of a patient, acquired at the moment of the diagnosis (Baseline) and after two cycles of chemotherapy (Interim), as previously discussed, so as to assess the chemotherapy effectiveness in reducing the tumor or not, thus permitting to change the therapy if the current one is not effective enough. 5.1 International Validation Study To analyze the effectiveness of the algorithm a dataset of patients with lymphomas provided by the International Validation Study (IVS), a retrospective study on Hodgkin s Lymphoma, was used. The IVS study includes 17 participating centers worldwide and was launched in 2009 to validate the Deauville 5 Point Score (defined in Chapter 2) and assess the interobserver variability of this qualitative evaluation of the lymphoma progression. [64] The IVS is composed of a homogeneous cohort of 440 advanced-stage HL patients (IIB-IVB) or poor prognosis stage (IIA), diagnosed between January 2002 and December 2009 and treated with ABVD chemotherapy. The 18 F F DG activity administered and the uptake time (interval between FDG injection and PET acquisition) shown in Table 5.1, are standardized. Image 18 F F DG Activity [MBq] Uptake time [min] BASELINE 362 ± ± 43 INTERIM 355 ± ± 24 Table 5.1: Activity and Uptake time for 18 F F DG in IVS database PET imaging 57

72 58 Chapter 5. The CAM-LTA Only the patients with acquiring time in the (60±10) minutes range (standard in oncologic imaging) can be considered as possible candidates for the algorithm study. Of the 440 patients in the IVS database only 30% of them could be considered as possible candidates for the CAM-LTA study. The preliminary set of patients used to test the CAM-LTA was composed of 33 patients from the IVS study, selected by the S. Croce and Carle Hospital of Cuneo.The dataset was then widened to 55 patients. 5.2 PET standardization Since many factors affect SUV values in FDG-PET and the images of the database were acquired in different centers, the first step needed to make the images useful for the study is the standardization of the FDG-PET procedure. The first procedure guidelines for PET/CT tumor imaging have been proposed by the European Association and the American Society of Nuclear Medicine. A protocol for quantitative FDG PET/CT studies should be based on standardization of the following parameters [65]: 1. patient preparation; 2. scan statistics, by prescribing dosage as a function of patient weight, scan time per bed position, percentage of bed overlap and image acquisition mode; 3. image resolution, by prescribing reconstruction settings for each type of scanner; 4. data analysis procedure, by defining volume of interest methods and SUV calculations; 5. quality control procedure, by using uniform phantom for verification of scanner calibration and the NEMA NU Image Quality phantom for verification of activity concentration recoveries.

73 5.3. The CAM-LTA algorithm The CAM-LTA algorithm The main purpose of the algorithm is to evaluate in an automated way a physical observable (TGV related) to be correlated with the Deauville Index and to use this variable for a quantitative and reliable automated therapy assessment. In Figure 5.1 the algorithm flow chart is shown. Figure 5.1: The CAM-LTA flow chart

74 60 Chapter 5. The CAM-LTA In order to perform the therapy assessment, for each patient, the Baseline and Interim PET/CT scans have to be simultaneously analyzed. First they undergo a preprocessing, then they are segmented by the CAM-LTA and in the end some clinical observable are computed to obtain an index correlated to the Deauville score. In the following sections these steps are thoroughly described and commented. 5.4 Preprocessing The first step in the algorithm is the image preprocessing. The PET and CT scans are given in DICOM format (Digital Imaging and Communications in Medicine) which is very useful because it contains all the information of the patient and her/his spatial position during the exam, etc. but is not so easy to handle in computation because it uses a very large memory space. Therefore all DICOM series are converted to the NIfTI format. More specifically the PET images are converted to SUV units (equation 2.3.1) thanks to the metadata saved in the DICOM header. The CT scans, that usually have a better resolution than PET scans, are rescaled using the PET voxel dimensions and are used to suppress the SUV contribution given by bone uptake, which is implemented by thresholding the CT at HU> 100. The obtained binary mask is directly used to suppress the SUV in the PET image. The preprocessing step reduces the segmentation time and, using the CT s anatomical informations, suppresses the chemotherapy related uptakes that do not influence the Deauville index assignment and as such the effectiveness of the therapy itself. In Figure 5.2 the bones suppression in the PET scan obtained thanks to the CT anatomical informations is shown: it can be seen how relevant it is to suppress the bones in a patient with high bones uptake.

75 5.5. CAM Segmentation 61 Figure 5.2: Bones suppression in PET image: a) CT scan, b) PET scan, c) PET scan without bones contribution. 5.5 CAM Segmentation After the preprocessing step, the CAM segmentation starts. As previously described the segmentation is governed by the pheromone deposition rule: T (v i ) = η + H fac ph (v i ) (5.5.1) where ph (v i ) in this application given by I, which is the intensity in the voxel v i in SUV units: ph (v i ) = I(v i ) (5.5.2) The anthill is set in the voxel with the highest intensity in the PET image after a 5x5x5 smoothing filter is applied, so as to avoid anthill mispositionings due to high intensity single voxels. After the anthill positioning the ants are deployed as previously said and after each deployment the pheromone map is analyzed: the saturated voxels are saved as a segmented ROI and the SUV of all the visited voxels is set to 0 in the initial PET scan, creating a new image to be segmented in the following iteration. The process is performed until a ROI bigger than 800cm 3 and with average HU<100 and average SUV<3 is found: this ROI is expected to include the liver volume.

76 Chapter 5. The CAM-LTA 62 Figure 5.3 shows an example of the result after all the ants deployments. The colored regions are the segmentation given by the algorithm, the liver can be recognized in red, the brain in blue, the bladder in purple, the kidneys in green and last but not least the remaining coloured regions are considered possible lymphomas. Figure 5.3: CAM segmentation: for three slices (a, b, c), the CAM segmented ROIs are shown in colors on the PET images in grey levels 5.6 ROI List Analysis After the segmentation step the obtained ROIs must be analyzed for the correct identification of each physiological and/or pathological region. This classification requires some a priori information such as the spatial information or other characteristics (for example the average SUV or average HU).

77 5.6. ROI List Analysis ROI Liver Analysis As said earlier, the first segmentation stops when the algorithm finds a region in which the liver is included, but usually this region is neither anatomically nor functionally correct, thus a more accurate analysis is needed to correctly recognize the liver ROI, and then look for some possible physiological/pathological additional ROIs included in the first region. Figure 5.4: Example of correlation distribution of SUV and HU in the last ROI segmented by the CAM. The liver extraction follows the operations described next: 1. Building SUV vs HU histogram: first of all, as Figure 5.4 shows, a PET SUV vs CT Hounsfield Unit histogram is plotted for each case. In the histogram it is clear that the ROI considered is non-homogeneous in either variable thus preventing a correct liver segmentation. 2. Fitting the histogram with a 2D Gaussian: To find the correct liver volume a 2D Gaussian fit is used, thus isolating the

78 64 Chapter 5. The CAM-LTA largest contribution and rejecting the highest SUV and low HU regions. 3. recalculating the SUV of each voxel v i : every voxel v i belonging to the original ROI has its SUV recalculated with the weight function obtained by the 2D Gaussian fit, the equation used are: and SUV SUV = SUV W (v i ) (5.6.3) W (v i ) = e (SUV (v i) <SUV >) 2 2 σ SUV 2 e (HU(v i ) <HU>)2 2 σ HU 2 (5.6.4) where SUV (v i ) and HU(v i ) are the values calculated in the voxel v i, < SUV > and < HU > are mean values obtained from the Gaussian fit and σ SUV and σ HU are the standard deviations also obtained from the fit. 4. New segmentation: the filtered ROI is used as a habitat for a new CAM segmentation so as to obtain the final liver segmentation; 5. Filtering of the obtained ROI : to provide a better estimate of liver volume the obtained ROI is filtered once again with a morphological closing. 6. Checking for interesting ROIs in the rejected volume: since part of the original liver ROI ha been rejected due to the filtering step could be possible to find lymphoma candidates in this rejected volume. This step is performed thanks to a region growing module that isolates ROIs with an average SUV higher than 1.2 < SUV > liver and with a volume larger than 1cm 3. The objective is to extract all the possible relevant regions that have average SUV higher than the liver; otherwise these ROIs would be wrongly rejected.

79 5.6. ROI List Analysis Registration on Template This is a preparatory step for the ROI identification. It is crucial to use a single reference frame to exploit the spatial information of the ROIs such as the center of mass coordinates. Registering an image on a template permits to equalize correctly for all the ROIs their relative position. The other ROI features are always calculated in the original space (ROI volume). Registration is performed using masked CT images to avoid errors introduced by the random arms positioning. As discussed before the registration process is driven by a rigid registration to roughly align the images and an affine transformation to reach the final result. In order to improve the registration performance, the CT scans are thresholded masking the voxels with HU < 0 and HU > 1200, therby enhancing the contribution of bones and soft tissues to be segmented in the PET scan. Figure 5.5 a) shows the the template used for the registration, which was selected among the IVS interim PET/CT, and represents a patient with almost all the brain scanned and well centered in the Field Of View (FOV), with his/her arm over the head and no cushion. Figure 5.5 b) shows an example of an image coregistration. Figure 5.5: Registrationon a template: a) template CT; b) example of registered CT.

80 66 Chapter 5. The CAM-LTA Identification of physiological ROIs The CAM in PET/CT scans segments all the ROIs with a high uptake, not distinguishing the physiological signals from the pathological ones. During the postporcessing phase, the ROIs are analyzed to separate the physiological contributions from the lymphoma candidates, taking into account some features extracted from the images. These features are related to: ROI position: center of mass calculated in the registration template space; ROI geometry: volume, radius and sphericity in the original coordinate space; SUV: average SUV rescaled to liver and the corresponding standard deviation; HU: average HU and the corresponding standard deviation. Usually, the physiological ROIs that can be found in the CAM segmentation response include the liver, bladder, kidneys, brain and heart. Not all of them are found in each patient, for example the brain is not always included in the Field of View and the heart and kidneys might not be segmented by the ants since their SUV might be lower than the average liver SUV used to choose which regions should be segmented. Figure 5.6 shows the volumes in the template coordinates space in which the center of mass of each physiological ROI must be located.

81 5.6. ROI List Analysis 67 Figure 5.6: ROIs volumes in template coordinate space. liver in blue, bladder in light blue, brain in green, kidneys in yellow and heart in red Liver Identification For the liver identification the ROI position and volume are taken into account. The spatial requirements are shown in Figure 5.6. Another requirement is that the liver should be the largest ROI for each exam in the selected region. The liver identification is the first step in the physiological ROI analysis because its average SUV is used to normalize the other ROI uptakes, as shown in equation Figure 5.7 shows on the left the distribution of the < SUV > liver for baseline and interim PET/CT scans: the average value is (1.89 ± 0.53), which is compatible with values in literature. In the following analyses, to obtain a more reliable comparison between uptake values of different patients, the average liver SUV (< SUV > liver ) is set to 1.8 and the other < SUV > values are rescaled as follows: < SUV > = < SUV > 1.8 < SUV > liver (5.6.5) Figure 5.7 shows on the right the distribution of the liver volume V liver : the

82 68 Chapter 5. The CAM-LTA average value is (447 ± 215) cm 3. counts avsuv { physlabel==1} Entries 110 Mean RMS counts 7 6 vol { physlabel==1} Entries 110 Mean RMS <SUV> volume [cm^3] Figure 5.7: Liver average SUV (left) and volume (right) distributions Bladder Identification The bladder is a physiologically active regine because it s involved in the drug wash out, and as such it has a very high SUV and a good contrast with respect to the surroundings. Due to its uptake features (high SUV, high signal to noise ratio and homogeneity), the bladder is normally segmented in only one ROI by the CAM and this is a condition taken into account during the ROI identification process. Other relevant features are: the center of mass position (Figure 5.6) and the average HU. 1. The center of mass position is shown in Figure The average SUV minimum value is set to 5, that is about 2.5 times the liver average SUV, for most of the patients the average SUV value is remarkably higher than this limit. For bladders with a volume in the (10 V 40) cm 3 range the SUV value are in the 10 < SUV > 50 range. 3. The average HU should be close to zero and compatible to the Hounsfield Unit of water, so the accepted range is 30 < HU > 40 and, as Figure 5.8 right shows 90% of the identified bladders have a CT intensity in the 10 < HU > 30 range.

83 5.6. ROI List Analysis 69 The bladders with a lower HU value are also larger, as shown in Figure 5.8 left, and with a corresponding lower average SUV, therefore it is possible that their volume includes also the surrounding voxels. The CAM bladder segmentation is satisfactory in all the analyzed PET/CT: the correct bladder identification is very important because otherwise its contribution to the TGV could be very relevant, compromising the final CAM-LTA response. It is important to remark that for the correct spatial identification of the bladder, the co-registration step is crucial. <SUV> Bladder Analysis 3 <SUV> Bladder Analysis <HU> volume [cm^3] 0 Figure 5.8: Bladder average SUV vs average HU (left) and average SUV vs volume (right) distributions Brain Identification The brain is also a physiological volume with a high average SUV and well defined spatial position. It is not always inside the Field Of View of the PET/CT scans and, therefore, it can be split in more than a singular ROI and the corresponding volume can vary on a wide range as shown in Figure 5.9 (right). During the brain identification the imposed conditions include: 1. The maximum number of brain-labelled ROIs is set to 3.

84 70 Chapter 5. The CAM-LTA 2. The Hounsfield Units range includes only the positive values, because the brain is a soft tissue and in the head volumes of air, such as those in the paranasal sinuses, can be affected by misregistration of FDG putake or partial volume effect. 3. The volume must be bigger than 20 cm 3 or the SUV value must be higher than 5. This condition should help avoiding the possibility of wrongly classifing as brain lymphomas in the neck. In Figure 5.9 some feature of brain-labelled ROIs are shown. Most of the brainlabelled ROIs have < SUV > brain bigger than 5 and the < HU > brain distribution is restricted to the [30, 50] range. The CAM brain segmentation has good performance and the identification step works properly: all the images that include the brain in the FOV have one or more ROIs labelled as brain, which can be removed in the final response calculation. <SUV> 12 Brain Analysis <SUV> 12 Brain Analysis <HU> volume [cm^3] 0 Figure 5.9: Brain average SUV vs average HU (left) and average SUV vs volume (right) distributions Kidneys Identification The kidneys are involved in the drug wash out, just like the bladder. Their pattern in FDG-PET can be different because their volume can be homogeneous or spotted, thus making the CAM kidneys volume either compact or formed by a cluster of little ROIs. Some kidneys have a low average SUV, thus making them difficult to segment and sometimes not segmented at all.

85 5.6. ROI List Analysis 71 To correctly label the kidneys the following identification rules are applied: 1. The spatial information is crucial. ( Figure 5.6) 2. The volume should be smaller than 50 cm 3, although there are some outliers that might have a bigger volume because part of the spleen could be included in the kidney segmentation. 3. The average Hounsfield Unit should be in the [10, 40] range. Kidneys identification is a complex task because of the different ways in which the FDG uptake can be shown: any condition can be set on the maximum number of kidney-labelled ROIs and, therefore, the registration on template is crucial in giving a reliable spatial information. Figure 5.10 shows the < SUV > as a function of the < HU > on the left and the < SUV > as a function of the kidneys volume on the right. In the Baseline PET the kidneys were found in 36 out of 55 scans, approximately corresponding to 71% of the patients; in Interim PET the kidneys were found in 39 out of 55 scans, corresponding to 66% of the cases. Kidneys Analysis Kidneys Analysis <SUV> <SUV> <HU> volume [cm^3] 0 Figure 5.10: Kidneys average SUV vs average HU (left) and average SUV vs volume (right) distributions.

86 72 Chapter 5. The CAM-LTA Heart Identification The heart variable SUV value and homogeneity and its position, which can be very close to possible lymphomas, make its identification a difficult task. The identification parameters for the heart are: 1. The limit on heart-labelled ROIs is 2. The heart has a spotted SUV pattern thus it can be segmented in more than one ROI. 2. The heart average SUV can be very low and as such it may not be segmented by the ants, or it can be very high thus using the [15, 30] range to label the heart. 3. The spatial information is taken into account. 4. Since the heart is close to the lungs, their partial volume effect could make some heart ROIs have a negative average HU: during the identification step, only the ROIs with < SUV > heart > 5 are labelled heart, even if they have < HU > heart < 0. <SUV> Heart Analysis <SUV> Heart Analysis <HU> volume [cm^3] 0 Figure 5.11: Hearts average SUV vs average HU (left) and average SUV vs volume (right) distributions. Figure 5.11 shows on the left the average SUV as a function of the average HU, and on the right the average SUV correlated to the heart volume. Most of the hearts have a volume smaller than 200 cm 3, but there are outliers with

87 5.7. Filtering of Lymphoma candidates 73 bigger ROIs where other structures are partially involved, usually the spleen. The average HU in most of the patients is in the [20, 40] range. 5.7 Filtering of Lymphoma candidates After the identification of the most important physiological ROIs, the remaining regions are considered lymphoma candidates. In Figure 5.12 lymphoma candidates are shown, on the left the correlation between average SUV and average HU is studied; instead the figure on the right shows the average SUV related to the volume. Some of the remaining ROIs could be contributions given by residual diffuse bone uptake or intestinal activity. Usually these regions have a large volume and can be partially discriminated from real lymphomas with this feature; this also means that some of the lymphomas that could have a large volume because of the additional contribution will be excluded from the results becoming False Negatives. Therefore in the next sections various types of filtering will be analyzed. <SUV> Not Physiological ROIs <SUV> Not Physiological ROIs <HU> volume [cm^3] 0 Figure 5.12: Lymphoma candidates in Interim PET/CT, on the left SUV vs average HU and on the right average SUV vs volume distributions Lymphoma Identification Lymphoma candidates are filtered using different characteristics, among them the spatial position, the volume range, the average SUV and the average HU. Therefore

88 74 Chapter 5. The CAM-LTA many filtering methods were tested and are shown in Table 5.2 with corresponding sensitivity and False Positive percentage. Features I II III IV V VI VII VIII IX X XI < HU > > 10 σ HU < 40 < SUV > < 7 V > 7 cm 3 V < 35 cm 3 < Z > > 0.2 < Z > < 0.9 True Positives 87% 53% 60% 73% 73% 60% 53% 80% 73% 80% 80% False Positives 43% 33% 38% 40% 38% 33% 30% 40% 35% 35% 40% Table 5.2: Combinations of filtering features studied. Most of these methods have poor results either in True Positive recognition or False Positive discrimination and as such were not considered. Combinations I and XI were taken into account for further studies and the combinations with < Z > limits were discarded even if equally good performing as method XI because the spatial position is very much coregistration-dependent, thus being less reliable than a volumetric discrimination. Those combinations considered well performing were further analyzed, and also the filtering step previously used by the algorithm was taken into account. The first filtering considered was the one previously used on the 33 patients database to see how it performed on a widened database. The filtering used is shown in Figure 5.13, and it included: 1. average Hounsfield Unit in the [ 60, 60] range, since lymphoma HU range should be positive. The possibility to have slightly negative average HU is due to the partial volume effect experienced in chest lymphoma, where some voxels belonging to the lungs, which have < HU >= 1000, are segmented in the Lymphoma ROI;

89 5.7. Filtering of Lymphoma candidates average SUV smaller than 7, because in Interim PET/CTs the lymphoma SUV is not very high (i.e. smaller than Baseline PET/CT); 3. the volume should be smaller than 25 cm 3, since lymphomas do not have large volumes. Figure 5.13: First selection of lymphomas from all the candidates. This filtering of all the candidates gave a result that was quite good because 12/15 True Positives were recognized corresponding to 80% rate, and there were 16/40 False Positives. However this filtering method used an average SUV that was very small and not very useful to generalize the data so other filtering steps were considered to see if the performance could be improved.

90 76 Chapter 5. The CAM-LTA Studying the ROIs that were classified ad False Negatives with the previous filtering step some characteristic emerged, such as a volume larger than the one first considered, and an average HU with higher minimum value. Therefore some new filtering methods were studied, of which the best two are discussed. Figure 5.14 shows the new filtering based on just two limits: Figure 5.14: Second selection of lymphomas from all the candidates. 1. average HU larger than 10, because as previously said the lymphomas HU should be positive but some voxels belonging to the lung could be attached to the lymphoma, but the limit is now 10 instead of 60 to better filter the False Positives;