Feature classification for multi-focus image fusion

Transcription

1 Iteratioal Joural of the Physical Scieces Vol. 6(0), pp , 3 September, 0 Available olie at DOI: /IJPS.73 ISSN Academic Jourals Full Legth Research Paper Feature classificatio for multi-focus image fusio Abdul Basit Siddiqui, Muhammad Rashid *, M. Arfa Jaffar, Ayyaz Hussai 3 ad Awar M. Mirza Foudatio Uiversity Istitute of Egieerig ad Maagemet Sciece, New Lalazar, Rawalpidi, Pakista. Natioal Uiversity of Computer ad Emergig Scieces, Islamabad, Pakista. 3 Iteratioal Islamic Uiversity, Islamabad, Pakista. Accepted 9 July, 0 Image processig techiques have witessed icreased usage i various real world applicatios. For ay image processig techique, such as image segmetatio, restoratio, edge detectio, stereo matchig etc., to be applied successfully, the image uder cosideratio must cotai all of the scee objects i focus. Usually, due to iadequate depth of field of optical leses, especially with larger focal legth, it becomes impossible to obtai a image i which all of the objects are i focus. Image fusio deals with creatig a image by combiig portios from other images to obtai a image i which all of the objects are i focus. I this paper, a ovel feature-level multi-focus image fusio techique has bee proposed which fuses multi-focus images usig classificatio. Te pairs of multi-focus images are first divided ito blocks. The optimal block size for every image is foud adaptively. The block feature vectors are fed to feed forward eural etwork. The traied eural etwork is the used to fuse ay pair of multi-focus images. The results of extesive experimetatio performed are preseted to highlight the efficiecy ad usefuless of the proposed techique. Key words: Multi-focus image fusio, feed forward eural etwork, feature classificatio, geetic algorithm. INTRODUCTION Image fusio is a sub-field of image processig i which portios of more tha oe images are fused together creatig a image where all the objects are i focus. Image fusio is of sigificat importace due to its applicatio i medical sciece, foresic ad defese departmets. The process of image fusio is performed for multi-sesor ad multi-focus images of the same scee. Multi-sesor images of the same scee are captured by differet sesors whereas multi-focus images are captured by the same sesor. I multi-focus images, the objects i the scee which are closer to the camera are i focus ad the farther objects get blurred. Cotrary to it, whe the farther objects are focused the closer objects get blurred i the image. To achieve a image *Correspodig author. rashid.uces@gmail.com. Tel: Abbreviatios: DWT, Discrete wavelet; GA, geetic algorithm; EOG, Eergy of gradiet; SF, spatial frequecy; SD, stadard deviatio; MI, mutual iformatio; PSNR, peak sigal to oise ratio; RMSE, root mea square error. where all the objects are i focus, the process of images fusio is performed either i spatial domai or i trasformed domai. Spatial domai icludes the techiques which directly icorporate the pixel values. I trasformed domai, the images are first trasformed ito multiple levels of resolutios. A image ofte cotais physically relevat features at may differet scales or resolutios. Multi-scale or multi-resolutio approaches provide a meas to exploit this fact (De ad Chada, 006). After applyig certai operatios o the trasformed images, the fused image is created by takig the iverse trasform. Image fusio is geerally performed at three differet levels of iformatio represetatio icludig pixel level, feature level ad decisio level (Pajares ad de la Cruz, 004). I pixel-level image fusio, simple mathematical operatios such as max (maximum) or mea (average) are applied o the pixel values of the source images to geerate fused image. However, these techiques usually smooth the sharp edges or leave the blurrig effects i the fused image. I feature level multi-focus image fusio, the source images are first segmeted ito differet regios ad the the feature values of these regios are calculated. Usig some fusio rule, the regios

2 Siddiqui et al are selected to geerate the fused image. I decisio level image fusio, the objects i the source images are first detected ad the by usig some suitable fusio algorithm, the fused image is geerated. A umber of image fusio techiques have bee preseted i literature. I additio to the simple pixel level image fusio techiques, we fid the complex techiques such as Laplacia Pyramid (Toet, 989), fusio based o PCA (Naidu ad Raol, 008), discrete wavelet (DWT) based image fusio (Li et al., 995), Neural Network based image fusio (Li et al., 00) ad advace DWTbased image fusio (Zheg et al., 004). These techiques have differet merits ad demerits such as liear wavelets like Haar wavelet durig the image decompositio does ot preserve the origial data (Heijmas ad Goutsias, 000). Similarly, due to lowpass filterig process of wavelets, the edges i the image become smooth ad hece, the cotrast i the fused image is decreased. I this paper, we have proposed a ew method for multi-focus image fusio which icorporates Geetic algorithm (GA) to fid optimal block size. GENETIC ALGORITHM (GA) GA is based o Darwi s theory of evolutio. The idea of evolutioary computig was itroduced by Recheberg i 960s. I GA, iitially a populatio of chromosomes (solutios to the problem) is iitialized withi the search space. Usig the previous populatio, ew populatio is created which is comprised of better solutios. The selectio of the solutios for ew populatio is made o the basis of their fitess values. Solutios of greater fitess have more chaces to reproduce the ew solutios. Geetic operator crossover ad mutatio with a certai probability are used o the paret solutios to geerate off-sprigs. This iterative procedure is carried out util some stoppig criterio is reached. The stoppig criteria iclude the umber of populatios or maximum fitess achieved. A itroductio about the workig of GA ca be foud i Goldberg (989). Creatig image dataset I the proposed method, we first created a image-set of 0 grayscale images. This image-set is show i the Figure. These images are take from differet image processig websites. The images are of differet scees ad backgrouds. For every image i the set, two versios of the same size were created. I the first versio, some of the regios are radomly selected i the left half of the image ad are blurred. A similar process is performed i the right half of the image i the secod versio. The blurred versios are geerated by Gaussia blurrig of radius.5. For the image set of 0 grayscale images, we created 0 versios of blurred images. For proposed method experimetatio, we resized all the images ito 480x640 resolutios. Features selectio I feature-level image fusio, the selectio of differet features is a importat task. I multi-focus images, some of the objects are clear (i focus) ad some objects are blurred (out of focus). The blurred objects i a image reduce its clearess. We have used five differet features to characterize the iformatio level cotaied i a specific portio of the image. This features set icludes variace, eergy of gradiet, cotrast visibility, spatial frequecy ad cay edge iformatio. Figure shows how the blurriess of icreasig Gaussia radius affects the clearess of the image. The values of the features i the image agaist blurriess of differet degrees are give i Table. We observe from Figure that with the icreasig blurriess, the clearess of the image is reduced ad the idetificatio of differet objects i the image becomes difficult. Features values give i Table showed the degradatio of the origial image. The values of eergy of gradiet, spatial frequecy ad edge iformatio are sigificatly reduced. Cotrast visibility It calculates the deviatio of a block of pixels from the block s mea value. Therefore, it relates to the clearess level of the block. The visibility of the image block is obtaied usig Equatio. VI Where k respectively. m µ ad ( i, j ) B k I ( i, j ) µ k µ k () m are the mea ad size of the block B k, PROPOSED METHOD We used a image set of 0 differet images to trai our eural etwork. Every image is first divided ito umber of blocks. The block size plays a importat role i distiguishig the blurred ad u-blurred regios from each other. To accomplish this task, we ru the algorithm for fidig adaptive block size usig GA itroduced i Zhag et al. (005) for every image i the image set. Zhag et al. (005) used a populatio of chromosomes where every chromosome represets the width ad legth of the block. After dividig the images ito blocks, the feature values of every block of all the images are calculated ad a features file is created. A sufficiet umber of feature vectors are used to trai the eural etwork. The traied eural etwork is the used to fuse ay set of multi-focus images. Spatial frequecy Spatial frequecy measures the activity level i a image. It is used to calculate the frequecy chages alog rows ad colums of the image. Spatial frequecy is measured usig equatio (). SF + Where, RF ( RF ) ( CF ) () m m [ I ( i, j ) I ( i, j ) ] i j

3 4840 It. J. Phys. Sci. Figure. Image set used to trai eural etwork. Table. Feature values with icreasig blurriess i BARM image show i Figure. Figure Variace EOG SF VI Edge a b c D CF m m [ I ( i, I ( i, ] j i where f i f ( i +, j ) f ( i, j ) ad f j f ( i, j + ) f ( i, j ) m is the image size. A large value Here, I is the image ad of spatial frequecy describes the large iformatio level i the image ad therefore it measures the clearess of the image. Here, m ad represet the dimesios of the image block. A high value of eergy of gradiet shows greater amout of focus i the image block. Variace Variace is used to measure the extet of focus i a image block. It is calculated usig Equatio 3: Variace m m i j ( I ( i, µ ) 3 Where, µ is the mea value of the block image ad mx is the image size. A high value of variace shows the greater extet of focus i the image block. Eergy of gradiet (EOG) It is also used to measure the amout of focus i a image block. It is calculated usig Equatio 4. EOG m i j ( f i + f j ) 4 Edge iformatio The edge pixels ca be foud i the image block by usig cay edge detector. It returs if the curret pixel belogs to some edge i the image otherwise it returs 0. The edge feature is just the umber of edge pixels cotaied withi the image block. Proposed algorithm Stepwise workig of the proposed method is as follows: Fid the optimal block size for each set of LF i ad RF i usig (Zhag et al., 005). LFi is the left-focused ad RFi is the right- focused versios of the ith image i the dataset discussed i sectio (.). where i,, 3,..., 0. Divide the versios LF i ad RF i of every image i the dataset ito K umber of blocks of size MxN. Create the features file for all LF i j ad RF i j accordig to the features discussed i sectio (.). Here j,, 3,..., K. For

4 Siddiqui et al. 484 a b c d Figure. BARM image (a) Origial Image (b) blurred with radius 0.5 (c) blurred with radius.0 ad (d) blurred with radius.5. Figure 3. Block diagram of the proposed method. all i, there are two sets of features values for every block j amed as FSLF i j ad FSRF i j each of which cotais five feature values. Subtract the features values of block j of LF i from the correspodig feature values of block j of RFi ad iclude this patter i features file. Normalize the feature values betwee [0 ]. Assig the class value to every block j of ith image. If block j is visible i LF i the assig it class value otherwise give it a RF. class value -. I case of class value -, block j is visible i i Create a eural etwork with adequate umber of layers ad euros. Trai the ewly created eural etwork with adequate umber of patters selected from features file created i step. By usig the traied eural etwork, idetify the clearess of all the blocks of ay pair of multi-focus images to be fused. Fuse the give pair of multi-focus images block by block accordig to the classificatio results of the eural etwork such that; Output if of > 0, if < 0, NN Select Select for j j block from from j left right focused focused image image The block diagram of the proposed method is show i Figure 3. Quatitative measures There are differet quatitative measures which are used to evaluate the performace of the fusio techiques. We used five differet measures icludig Root Mea Square Error (RMSE), Peak Sigal to Noise Ratio (PSNR), Mutual Iformatio (MI), Correlatio ad Mea Absolute Error whe the referece image is available. For blid image fusio (referece image is ot available), we used Etropy, Mutual iformatio (MI), Stadard deviatio (SD),

5 484 It. J. Phys. Sci. Spatial frequecy (SF). These measures are defied i Table. I additio of the performace metrics listed i Table, some other performace measures usually used to check performace of ay fused algorithm ca be foud (Naidu et al., 003; Wag ad Bovik, 00; Blum ad Zheg, 006; Cvejic et al., 005). EXPERIMENTS AND RESULTS Image fusio is performed i two differet situatios. I first situatio, the referece image is available ad i secod situatio, the referece image is ot available (blid image fusio). We exercised the feed forward eural etwork with differet umber of hidde layers ad with differet umber of euros o each layer. We foud the best results with oe hidde layer of 30 euros. The learig rate α ad the threshold for mea square error are kept as 0.0. The results of the proposed techique are compared with differet existig methods icludig DWT, adwt, PCA ad Laplacia Pyramid based image fusio techiques. The performace of a existig Probabilistic Neural Network based techique is also compared with the results of the proposed techique. The experimetatio results are obtaied whe the referece image is available ad whe it is ot available (blid image fusio). I order to evaluate the performace of the proposed techique, the results for four differet pairs of multi-focus images are obtaied icludig balloo, lab, clock ad Pepsi ca images. PNN-based image fusio ad the proposed techique variatio Li et al. (00) proposed a probabilistic eural etwork based techique to perform multi-focus image fusio. They traied a eural etwork to classify the selectio of blocks of the two source images to geerate the fused image. I their techique, the source images were divided ito the blocks of fixed size, 3x3. The block size is a importat factor to achieve good fusio results. Fixed size blocks may separate the blurred ad ublurred regios withi oe pair of multi-focus images but for some other pair of multi-focus images, the cotets of the image block are partially blurred. The size of the block varies from image to image because differet images have differet blurred regios. I the proposed method, for every pair of multi-focus images, a optimal block size is foud usig the techique give i Zhag et al. (005). We have used five differet features to calculate the clearess of a block more accurately as compared to three features used by Li et al. (00) A major differece betwee the proposed method ad the existig PNN-based image fusio techique is the traiig of the eural etwork. Li et al. (00) i their techique create ad trai a ew eural etwork for every pair of multi-focus images which is time cosumig. I the proposed method, we traied the eural etwork usig the block features of te differet pair of multi-focus images. Oce the classifier is obtaied the it ca be used to fuse ay pair of multi-focus images. Quatitative assessmets ad the visual compariso Whe the referece image is available For balloo ad lab images, the referece images are available. A visual compariso is show i Figures 4 ad 5 for balloo ad lab images. Both the balloo ad lab images are of size, 480x640. We used three differet quatitative measures to evaluate the performace of fusio classifier proposed i this paper. The experimetatio results obtaied for balloo ad lab images are summarized i Table 3. Visual examiatio of Figures 4 ad 5 shows that, the fused images obtaied by the proposed techique are clearer tha the fused images obtaied by usig the existig techiques. The performace of the proposed techique ca also be verified from the results of differet quatitative measures give i Table 3. Whe the referece Image is ot available (Blid Image Fusio) We have used clock ad Pepsi ca images i this category. The clock ad Pepsi ca image sizes are 56x56 ad 5x5, respectively. Whe the referece image is ot available the the performace of the fusio process is evaluated o the basis of differet set of quatitative measures. Figures 6 ad 7, provides a visual compariso of the proposed techique with the existig techiques. I case of Pepsi ca image, block effects are visible at the table edge i the fused image geerated by PNNbased techique. I PNN-based techique, the source images are divided ito parts usig a block of fixed size ad hece it leaves the block effects i the fused image. Table 4 provides some statistics to evaluate the performace of the proposed techique whe the referece image is ot available. These statistics iclude the results of differet quatitative measures. I case of fused image geerated by Laplacia Pyramid-based techique, the value of spatial frequecy is greater tha the proposed method. However, i geeral, the proposed techique performs better tha existig techiques. CONCLUSION I this paper, a feature-level block-based multi-focus image fusio techique is proposed. A feed forward eural etwork is first traied with the block features of te pairs of multi-focus images. A feature set icludig spatial frequecy, cotrast visibility, edges, variace ad eergy of gradiet is used to defie the clarity of the

6 Siddiqui et al Table. Performace metrics used for image fusio. Metric Formula Descriptio RMSE m [ ] RMSE R( i, F( i, m i j Where, R ad F are the referece ad fused images, respectively. mx is the image size. Calculates the deviatio betwee the pixel values of referece image ad fused image. A lesser value shows good fusio results. PSNR PSNR 0 log 0 m m [ R( i, F( i, ] i j L deotes to umber of gray level i the image. L Determies the degree of resemblace betwee referece ad fused image. A bigger value shows good fusio results. MI MI m h hr, F( i, ( i, log hr ( i, hf ( i, j R, F i j ) Here h,, h R, R F h F are the joit, referece ad fused images histograms respectively. Mutual Iformatio ca also be used whe referece image is ot available. I that case, it is the sum of the mutual iformatio retrieved by fused image from image A ad image B. Determies how much iformatio the fused image retrieved from the iput source images. A bigger value shows good fusio results. CORR CORR C r ad C rf m C i j m i j C r + rf C f where R ( i, j ), C f F ( i, R( i, F ( i, m i j It is used to determie how referece ad fused images are idetical to each other. Its value varies betwee 0 ad. If it returs the it meas both referece ad fused images are absolutely similar. MAE where M, N are the umber of rows ad colums of the image respectively. It is used to calculate the mea absolute error betwee referece ad fused image. Etropy H L hf i 0 ( i) log hf i ( ) where h F is the ormalized histogram of fused image ad L is the umber of gray levels. Quatifies the quatity of iformatio cotaied i the fused image. A bigger value shows good fusio results.

7 4844 It. J. Phys. Sci. Table. Cotd. SD SD L ( i i ) h ( ) F i where i i 0 L i 0 i h Where, h F is the ormalized histogram of fused image ad L is the umber of gray levels. F Measures the cotrast i the fused image. A well cotrast image has high stadard deviatio. SF Spatial Frequecy is described i sectio Figure 4. Balloo images fused by differet image fusio techiques ad the proposed method. Figure 5. Lab images fused by differet image fusio techiques ad the proposed method.

8 Siddiqui et al Table 3. Experimetatio results of differet quatitative measures for fused balloo ad lab images. Image Balloo Lab Method Quality measures RMSE PSNR MAE CORR MI DWT adwt PCA Laplacia PNN Proposed DWT adwt PCA Laplacia PNN Proposed Figure 6. Clock images fused by differet image fusio techiques ad the proposed method. image block. Block size is determied adaptively for each image. The traied eural etwork is the used to fuse ay pair of multi-focus images. Experimetatio results show that the proposed techique performs better tha the existig techiques. By fidig the block size adaptively, the blurred ad u- blurred regios withi the source images are optimally idetified. As a result of it, the proposed techique performs better. I the proposed techique, oly oe eural etwork is created whereas i PNN-based image fusio (Li, 00), eural etwork for every pair of multifocus images is created which is really time cosumig.

9 4846 It. J. Phys. Sci. Figure 7. Pepsi ca images fused by differet image fusio techiques ad the proposed method. Table 4. Experimetatio results of differet quatitative measures for clock ad Pepsi ca images. Image Pepsi ca Clock Method Quality Measures Etropy SD SF MI DWT adwt PCA Laplacia PNN Proposed DWT adwt PCA Laplacia PNN Proposed REFERENCES Blum RS, Zheg L (006). Multi-sesor image fusio ad its applicatios.crc Press, Taylor ad Fracis Group, Boca Rato. Cvejic N, Loza A, Bull D, Cagarajah N (005). A similarity metric for assessmet of image fusio algorithms. It. J. Sigal Proc., (3): De I, Chada B (006). A simple ad efficiet algorithm for multifocus image fusio usig morphological wavelets. Sigal Processig, pp

10 Siddiqui et al Goldberg DE (989). Geetic Algorithm i Search Optimizatio ad Machie Learig. Addiso Wesley Publishig Compay, Redwood City, Cam., -3. Heijmas HJ, Goutsias J (000). Multiresolutio sigal decompositio schemes Part : morphological wavelets. IEEE Tras. Image Process., 9: Li H, Majuath S, Mitra SK (995). Multi-sesor image fusio usig the wavelet trasform. Graphical Models ad Image Processig, 57(3): Li S, Kwok JT, Wag Y (00). Multifocus image fusio usig artificial eural etworks. Patter Recog. Letters, 3(8): Naidu VPS, Girija G, Raol JR (003). Evaluatio of data associatio ad fusio algorithms for trackig i the presece of measuremet loss. I: AIAA Coferece o Navigatio, Guidace ad Cotrol, Austi, USA, pp -4. Naidu VPS, Raol JR (008). Pixel-level Image Fusio usig Wavelets ad Pricipal Compoet Aalysis. Defece. Sci. J., 58(3): Pajares G, de la Cruz JM (004). A wavelet-based Image Fusio Tutorial. Patter Recogitio, 37(9): Toet A (989). Image fusio by a ratio of low pass pyramid. Patter Recogitio Letters, 9(4): Wag Z, Bovik AC (00). A uiversal image quality idex. IEEE Sigal Proc. Lett,, 9(9): Zhag X, Ha J, Liu P (005). Restoratio ad fusio optimizatio scheme of multifocus image usig geetic search strategies.optica Applicata, 35: 4. Zheg Y, Essock EA, Hase BC (004). A Advaced Image Fusio Algorithm Based o Wavelet Trasform Icorporatio with PCA ad morphological Processig. I: Proceedigs of the SPIE, 598: