Fusion of PET and MRI images using adaptive neuro-fuzzy inference system

Transcription

1 Journal KAVITHA of Scientific & & CHELLAMUTHU: Industrial ResearchPET AND MRI IMAGES USING ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM Vol. 71, October 2012, pp Fusion of PET and MRI images using adaptive neuro-fuzzy inference system * C.T.Kavitha a and C.Chellamuthu b a ECE Deparment,Jerusalem College of Engineering, affiliated to Anna university, Pallikaranai, TamilNadu, India. b EEE Deaprtment,R.M.K. Engineering College, affiliated to Anna university, Kavaraipettai,TamilNadu, India. Received 28 May; 2012; revised 15 July 2012; accepted 03 September 2012 Fusion of the PET and MRI images provides complete information, better visualization and higher diagnostic accuracy. This paper proposes the fusion of PET and MRI images using adaptive neuro-fuzzy inference system. The PET and MRI images are initially decomposed using the second generation wavelets. The second generation wavelet transform is shift invariant. The second generation wavelet coefficients are then fused using the adaptive neuro-fuzzy inference system. Then the inverse transform is applied to the fused coefficients and the fused image is obtained. The performance of this algorithm is validated both qualitatively and quantitatively. The metrics used for the analysis are entropy, average gradient, average, standard deviation, mean square error and peak signal to noise ratio. It is also compared with the existing methods of image fusion. The proposed algorithm extracts more information from the source images, provides better contrast and brightness as compared to the existing fusion techniques. Keywords: Medical image fusion, Adaptive neuro-fuzzy inference system, Second generation wavelet transform, PET, MRI. Introduction Image fusion is the process of combining multiple input images into a single composite image. Medical image fusion helps the radiologist for accurate diagnosis and for radiation treatment of the patients. The PET image gives the functional information. It has low spatial resolution 1. The MRI image gives the anatomical or structural information with high spatial resolution. Radiologist need to interpret the PET and MRI images during a clinical round for accurate diagnosis. As these images are interpreted separately by different clinical expertise, it may lead to misinterpretations, especially in the case of small tissue structures. Thus, the fusion of the PET and MRI images are necessary and it offers higher diagnostic accuracy. The fusion process occurs at different levels of information representation 2. If the raw data obtained from different sensors are fused, it is called as Pixel level fusion. If some features on the source images are extracted and then fused, it is called as Feature level fusion. Symbol level fusion or decision level fusion allows the information from the source images to be combined at the highest level of abstraction. Averaging, principal component analysis, colour mapping, estimation theory, artificial neural network were some techniques *Author for correspondence ranjanakannan@yahoo.co.in based on which fusion rules were framed These fusion techniques can be carried out directly to the source images or can be applied in transform domain. Fusion techniques based on the pyramid transform was introduced in the mid 80 s. The contrast pyramid 12, Ratioof-low pass pyramid 13, Morphological pyramid transform 14 were some of the pyramidal transform based on which the fusion was carried out. These techniques failed when the images to be fused were significantly different 15. Many fusion techniques were introduced in the wavelet domain The disadvantage of the wavelet transform based fusion techniques is that the transform was shift variant 19. This paper proposes an image fusion technique in the transform domain. The second generation wavelet transform (SGWT is used to obtain the multi-resolution images of PET and MRI. The SGWT is shift invariant and hence overcomes the disadvantage of the wavelet transform. The proposed fusion rule is based on adaptive neuro-fuzzy inference system (ANFIS. The fused image is analysed both subjectively and objectively. Materials and methods Proposed Fusion Scheme The proposed fusion scheme used for fusing the PET and MRI images is shown in Fig. 1. The various stages of the proposed fusion scheme are:

2 652 J SCI IND RES VOL 71 OCTOBER 2012 Fig. 1 Proposed fusion scheme using ANFIS Fig. 2 Input images (a PET image (b MRI image To input the PET and MRI image. Decompose the source images using SGWT. To apply the ANFIS fusion rule to the SGWT coefficients, to get the fused coefficients. To apply inverse transform to the fused coefficients and to get the fused image. To validate the fused image both subjectively and objectively. Input Image The brain image with lesion is used for fusion. The PET image and MRI T1-W sequence image containing abnormality is shown in Fig.2 (a and Fig. 2 (b respectively. The PET image is a colour image whose size is The MRI T1-W sequence image is image with intensity values ranging from 0 to 255. Image Transformation The input images are transformed using SGWT. It requires fewer computations compared to the traditional wavelet method. The first generation wavelet works well for infinite or periodic signals. It is not suitable for bounded domain. The SGWT based on the lifting scheme is used for decomposing the input images. The basic idea is to first split a signal into its even and odd samples. Predict the odd signals from the even signals. This gives the detail coefficients. Update the even signals using the detail coefficients. This gives the coarse coefficients. The prediction operator and the update operator are both linear. For more in depth knowledge of the second generation wavelet, the readers are referred to the papers ANFIS Based Fusion Rule The fused coefficients are obtained by applying ANFIS fusion rule to the SGWT coefficients. It is functionally equivalent to fuzzy inference system with the difference that, ANFIS also uses neural networks for minimizing the error. It is a combination of artificial neural network and fuzzy logic. Sugeno type fuzzy inference system is used here. Sugeno output

3 KAVITHA & CHELLAMUTHU: PET AND MRI IMAGES USING ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM 653 Fig. 3 Input membership function plot membership function is either linear or constant. The adaptive technique of the Sugeno type fuzzy inference system customizes the membership function and models the data. It is computationally efficient and also compact. A combination of least squares method and back propagation gradient descent method is applied for training FIS membership function. Three membership functions L i, M i and H i are selected to represent low values, medium values and high values of gray level respectively for the i th input image. The computation quantity of triangular membership function is less, so the triangular membership function was selected. Membership function plot is shown in Fig. 3. The fuzzy inference system of the proposed algorithm consists of two inputs and one output. The neuro-fuzzy inference system uses the following nine fuzzy rules. 1. If (input1 is L 1 and (input2 is L 2 then (output is Z 1 2. If (input1 is L 1 and (input2 is M 2 then (output is Z 2 3. If (input1 is L 1 and (input2 is H 2 then (output is Z 3 4. If (input1 is M 1 and (input2 is L 2 then (output is Z 4 5. If (input1 is M 1 and (input2 is M 2 then (output is Z 5 6. If (input1 is M 1 and (input2 is H 2 then (output is Z 6 7. If (input1 is H 1 and (input2 is L 2 then (output is Z 7 8. If (input1 is H 1 and (input2 is M 2 then (output is Z 8 9. If (input1 is H 1 and (input2 is H 2 then (output is Z 9 Sugeno output membership function is taken to be linear and therefore the output of the Sugeno fuzzy model is of the form ax + by + c = 0, where x = input1, y = input2 and a, b, c are linear output parameters. Fig. 4 Surface viewer of the proposed neuro-fuzzy algorithm The output level Z i of each rule is weighted by the firing strength w i of the rule. For an AND rule, the firing strength is given by (1. { ( ( } = min F1 x, F y...(1 wi 2 where F 1 (x is the membership function for the input 1 and F 2 (y is the membership function for the input 2. The final output of the system is the weighted average of the rule outputs given by (2. Z = N w Z N i i i= 1 i= 1 w i...(2 where N is the number of rules used. The proposed Neuro-fuzzy fusion rule surface viewer is shown in Fig. 4. The entire output surface of the fuzzy system can be seen using the surface viewer. It is a 3- D curve. It represents the mapping from input1 and input2 to the output. The entire mapping between input and output can be seen with the help of the surface viewer. The structure of ANFIS is shown in Fig. 5. The left most node represents the input. Three membership functions are used for each input. The blue circles represent the usage of and rule. The rightmost node represents the output. Using ANFIS algorithm, the fused coefficients are obtained. The fused image is obtained by applying inverse transform to the fused coefficients. Results and Discussions Simulation was carried out using MATLAB 7.6. The proposed method was compared with the existing methods

4 654 J SCI IND RES VOL 71 OCTOBER 2012 Fig. 5 ANFIS structure of the proposed algorithm Fig. 6 Fused image obtained by (a Average fusion rule. (b PCA fusion rule. (c Proposed fusion rule like image fusion using average fusion rule and fusion rule using Principal Component Analysis (PCA. The fused images of all the three methods were shown in Fig. 6. Validation The performance of the fused image was analyzed both qualitatively and quantitatively. Qualitative Analysis Subjective evaluation is important in medical diagnosis or treatment planning. The resultant images and the original images were shown to a radiologist to visualise and to interpret all the fused images. The radiologist interpreted that the lesion is seen in the right parietal lobe and is clearly visible in the proposed fused image. Quantitative Analysis Qualitative analysis is subject to visual interpretation of the experts and no mathematical analysis is involved. Therefore, quantitative analysis is necessary for analyzing the quality of the fused image. In order to prove the effectiveness of the fusion rule, the parameters like entropy, average gradient, average and standard deviation were evaluated. Entropy is a factor which reflects the information abundance of image content. The fused image contains abundant information if the entropy value is large. Average gradient reflects the contrast of images. Average value gives the brightness of an image. Visual

5 KAVITHA & CHELLAMUTHU: PET AND MRI IMAGES USING ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM 655 Table 1 Results of different fusion methods. Fusion Entropy Average Average Standard Mean Error Peak Signal to Method gradient deviation Square Noise Ratio Average PCA Proposed method effect is better when the average value is moderate. Standard deviation reflects the deviating degree of gray value from its average. The gray distribution is more dispersive if the standard deviation is bigger. If the probability distributions of all the gray-levels tend to be equal, the maximum is the information content of the image. The mean square error (MSE of the proposed algorithm is lesser compared to the existing methods and hence the peak signal to noise ratio (PSNR of the proposed method is more. The results are shown in Table 1. Conclusion A new fusion rule for fusing PET and MRI images is proposed using ANFIS fusion rule in the present study. Input images are initially transformed using SGWT. As lifting scheme is used, there is no loss of information through forward and inverse transform. It is also shift invariant and hence edges are clearly visible. ANFIS is a combination of artificial neural network and fuzzy logic. Membership function used for the fusion is triangular. The proposed algorithm was analysed subjectively by showing it to a radiologist. Subjective evaluation by radiologist is a clear indication that the proposed algorithm produced better fused image compared with the existing algorithms. The abnormality is clearly visible in the fused image. The metrics like entropy, average gradient, average, standard deviation, MSE and PSNR were used for the quantitative analysis. Entropy value is more for the proposed method. The tabulated result also indicates that the fused image of the proposed algorithm provides better contrast and brightness as compared with the existing methods. The MSE is also less for the proposed algorithm. Acknowledgement The authors would like to thank Dr Flora Nelson for her valuable suggestions and for subjective evaluation of the images. The authors also thank the anonymous referees for their valuable suggestions. References 1 Gonzalo Pajares & Jesus Manuel de la Cruz, A wavelet-based image fusion tutorial, Pattern Recogn, 37 ( Maruthi R & Sankarasubramanian K, Multi focus image fusion based on the information level in the regions of the images, J Theor App Inform Tech, 3 ( Lallier E & Farooq M, A real time pixel-level based image fusion via adaptive weight averaging, Proc Third Int Conf on Inform Fusion (Paris, France, 2, Jul 2000, WeC3_3 WeC3_13. 4 Jinzhu Yang, Fangfang Han & Dazhe Zhao, A Block Advanced PCA Algorithm Based on PET/CT, Proc Int Conf on Intelligent Comput Technol and Automation, 2, Mar 2011, Mario Aguilar & Aaron L. Garrett, Biologically based sensor fusion for medical imaging, Proc SPIE, 4385, 2001, Rick.S.Blum, On multisensor image fusion performance limits from an estimation theory perspective, Inform Fusion, 7 Sep Dawoud A & Netchaev A, Fusing of Edge Information in Markov Random Fields Region Adjacency Graph Based Segmentation, Proc of the 7th Int Conf on Image Analysis and Recogn, 1, 2010, Chenglin Wen & Jingli Gao, Multiresolution Image Fusion Algorithm Based on Block Modelling and Probabilistic model, Lecture Notes in Comput Sci, 5263, 2008, Agrawal M, Tsakalides P& Achim A, Medical Image Fusion Using the Convolution of Meridian Distribution, Proc 32 nd Annual Int conf of the IEEE Engineering in Medicine and Biology Soceity (Buenos Aires, Argentina, Aug31-Sep4 2010, Shuo-Li Hsu, Peng-Wei Gau, I-Lin Wu, & Jyh-Horng Jeng, Region-Based Image Fusion with Artificial Neural Network, World Academy of Sci, Engg and Technol, 53, 2009, Zhaobin Wang & Yide Ma, Medical Image Fusion Using m-pcnn, Inform fusion, 9, Apr 2008, Toet A, Van Ruyven J J and Valeton J M, Merging thermal and visual images by a contrast pyramid, Optical Engg, 28, 1989, Toet A, Image fusion by a ratio of low-pass pyramid, Pattern Recogn Lett, 9, 1989, Matsopoulos G K & Marshall S, Application of morphological pyramids: fusion of MR and CT phantoms, J Vis Commun Image R, 6, 1995, Hui Li, Manjunath B S & Sanjit K Mitra, Multisensor image fusion using the wavelet transform, Graphical Models and Image Processing, 57, 1995, Susmitha Vekkot & Pancham Shukla, A Novel Architecture for Wavelet based Image Fusion, World Academy of Sci, Engg and Technol, 57, 2009,

6 656 J SCI IND RES VOL 71 OCTOBER Qu Guihong, Zhang Dali & Yan Pingfan, Medical image fusion by wavelet transform modulus maxima, Opt Express, 9, (2001, Yong Yang, Dong Sun Park, Shuying Huang & Jucheng Yang, Fusion of CT and MR images using an improved wavelet based method, J of X-ray Sci and Technol, 18 ( Ravichandran C G, Rubesh selvakumar R & Goutham S, Analysis and comparision of medical image fusion techniques, Int J of Comput Sci and Inform security, 9 ( Wim Sweldens, The lifting scheme: A construction of second generation wavelets, SIAM J Math Anal, 29 ( Feng Hui, Guo Lanying & Xiao Jinsheng, The lifting scheme based on the second generation wavelets, Wuhan Univ J of Nat Sci, 11 ( Jinsheng Xiao, Benshun Yi & Yuan Yao, Wavelets based on lifting scheme and its application on image processing, Int symp on Knowl acquisition and modeling, Dec 2008,