Exposure Fusion Based on Shift-Invariant Discrete Wavelet Transform *

Transcription

1 JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 27, (2011) Exposure Fusion Based on Shift-Invariant Discrete Wavelet Transform * JINHUA WANG 1,2, DE XU 1, CONGYAN LANG 1 AND BING LI 1 1 Institute of Computer Science and Engineering Beijing Jiaotong University Beijing, P.R. China 2 Institute of Information Technology Beijing Union University Beijing, P.R. China Until now, most exposure fusion methods are easy to be influenced by the location of object in the image. However, when capturing the source images, slight shift in the camera s position will yield blurry or double images. In order to solve the problem, a method called SIDWTBEF is proposed, which is based on shift-invariant discrete wavelet transform (SIDWT). It is more robust to images those have slight shift. On the other hand, in this paper, we present a novel way to get the chrominance information of the scene, and the saturation of the fused image can be adjusted using one user-controlled parameter. The luminance images sequence of the source images are decomposed by SIDWT into subimages with a certain level scale. In the transform domain, different fusion rules are used for the high-pass sub-images and the low-pass sub-images combination respectively. In the end, in order to reduce the inconsistencies induced by the fusion rule after applying the inverse transform of SIDWT, an enhancement operator is proposed. Experiments show that SIDWTBEF can give comparative results compared to other shift dependent exposure fusion methods. Keywords: exposure fusion, high dynamic range imaging, tone mapping, SIDWT, shiftinvariant 1. INTRODUCTION When irradiance across a scene changes greatly, there will be over- or under-exposed regions in an image of the scene no matter what exposure time is adopted for commonly used cameras. An over- or under-exposed region contains less detail information than the same area when it is well-exposed. The scene is usually called a high dynamic range (HDR) scene. For visualizing the HDR scene, the camera-specific response curve is recovered by computing from multiple exposure sequences and their exposure times in paper [1]. The obtained response curve can linearize the intensities of an image. However, in most situations the exposure setting is usually unknown. Thus, using the multiple exposure images directly to fuse one image with high quality is another way to display HDR scenes. We know that in order to acquire multiple exposures source images, people always employ a standard digital or analog camera. However, the chief limitations of this is that the camera must be absolutely still between exposures. Even tripod mounting will some- Received February 12, 2009; revised November 2, 2009 & March 19, 2010; accepted July 9, Communicated by Jiebo Luo. * This work was supported by National Nature Science Foundation of China (No and No ), Fundamental Research Funds for the Central Universities (No. 2009JBM024), and China Postdoctor Research Foundation ( ). 197

2 198 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI times allow slight shifts in the camera s position that yield blurry or double images. So, misaligned source images must be considered when fusion methods are designed. In our paper, we adopt the shift-invariant discrete wavelet transform (SIDWT) as a decomposition model, which is more robust to images those have slight shift. The method is called shift-invariant discrete wavelet transform based exposure fusion (SIDWTBEF), which can preserve all relevant information contained in the input images. We know that tone mapping methods are proposed for displaying HDR images well on common display devices those have much lower dynamic range. They can compress the large range of luminance range that exists in HDR images. Notice that the aim for the proposed method SIDWTBEF and tone mapping methods is both to acquire high quality image for displaying on ordinary devices. Notice that the input for the tone mapping methods just is the in-between radiance map usually obtained by [1], while the input for the SIDWTBEF is set of multiple exposure images. In the image fusion research field, the discrete wavelet transform (DWT) based fusion method is proposed to decompose an image into multi-scale edge representation, which is based on the fact that the human vision system (HVS) is primarily sensitive to local contrast change [2]. However, DWT is a shift variant signal representation, which results in a shift dependent fusion method. In addition, most image fusion methods usually focus on details preservation, and chrominance information is usually disregarded. We can observe that in these methods, the treated source images are usually grey images such as medical images or remote sensing images. Moreover, these methods only describe the fusion procedure when the inputs are only two images. How to extend the procedure when the inputs are multiple images is not represented in detail, especially when the multi-resolution fusion strategy is adopted. We know that in HDR imaging research field, the chrominance of an image is importance information related to HVS. Maintaining more chrominance information and edge details are the main objective of the proposed method SIDWTBEF. At the same time, reducing the time consuming is another point that we consider. At present, there are some exposure fusion methods such as papers [3, 4], they treat the R, G and B channel separately to preserve the chrominance information of the scene. The processing strategy is very time consuming, especially for the multi-resolution fusion model. In order to solve the problem, we propose a novel way to get the chrominance information of the scene. The rest of the paper is organized as follows. In section 2, the current studies on exposure fusion methods are briefly reviewed. The characteristics of SIDWT and SIDWTBEF are described in detail in section 3. In section 4, the experimental results are shown. Finally, a conclusion is given in section RELATED WORKS We know that HDR images have a major inconvenience in application, that is, they can not be displayed correctly on ordinary display devices such as printers or monitors. That is why tone mapping methods are proposed, which is related to exposure fusion methods mentioned in our paper. Tone mapping methods can be broadly classified by spatial processing techniques into two categories: global and local methods [5]. For global methods, they usually compress the dynamic range using gamma function, sigmoid func-

3 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 199 tion or histogram equalization. Each pixel is mapped based on global image characteristics regardless of its spatial location in a bright or dark area. The global methods are easier to implement and faster to perform. However, when the dynamic range of the scene is particularly high, these methods tend to result in either graying out or losing visible details. For local method, different operations are applied to different pixels. In this case, one input value can produce more than one output values, which depends on the pixel value and the surrounding pixel s values. The local methods can scale the image s dynamic range to the output device s dynamic range while increasing the local contrast. However, they tend to produce ringing artifacts. Since the local methods are capable of compressing quite large dynamic range and also modeling the local adaptation of the human visual system, more emphasis is recently put on developing this type of method. A brief of the local methods is given below. Reinhard et al. [6] propose a tone mapping method for HDR rendering by modeling dodging and burning in traditional photography. icam06 [7] is an image appearance model that has been extended to render HDR images for displaying on common devices. Meylan et al. [8] propose a center/surround retinex model for displaying HDR images. In that paper, the weights of surrounding pixels are computed with an adaptive filter, which can adjust the shape of the filter to the high contrast edges in images. In 2007 years, Meylan presents another tone mapping method [9] that is derived from a model of retinal processing. A method in gradient domain [10] is proposed by Fattal et al., by attenuating the magnitudes of large gradients, they operate on the gradient field of an image. Durand et al. [11] use a bilateral filter to reduce the overall contrast while preserving local details in an image. For image fusion, Li et al. [12] propose to apply orthogonal wavelet to fuse multisensor images. The wavelet transform has capability of compactness, orthogonality and the availability of directional information in the fusion process. However, the main drawbacks of the DWT are the existence of shift variance and the directional constraint in diagonal feature extraction. The basic fusion rule in is the absolute value maximum selection, i.e., the larger absolute value in the sub-images is maintained for reconstruction. There is the fact that the larger values correspond to the features in the image such as edges, lines, and region boundaries. A consistency verification method is proposed in [12] to improve the fusion result. This is an attempt to fuse the image based on the region, i.e., the subimages were combined on the basis of both the pixel itself and the region to which the pixel belongs. The paper [13] introduces the match and salience measures where the combination operations, selection or average, are carried out based on the match measure. As to exposure fusion methods, Mertens et al. [4] present a technique for fusing multiple exposure images to an image, which is processed based on laplacian pyramid in R, G, and B channels separately. The problem of it is that the results may contain brightness change phenomena, which is not consistent with original source images. It is caused by a high change in brightness among the images with different exposure times. Goshtasby [3] proposes a method to combine multiple exposures. In that paper, the optimal exposure on a per-block basis is selected, and smoothly combination between blocks is used. Since block may span across different objects, it can not deal well with object boundaries, and spill information may be caused across object edges.

4 200 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI 3. SIDWTBEF In this section, we firstly describe the characteristic of SIDWT. Secondly, the proposed method SIDWTBEF and the fusion rule are demonstrated in detail. 3.1 The Shift-Invariant Discrete Wavelet Transform (SIDWT) We know that discrete wavelet transform (DWT) can be implemented with filterbanks [14]. For image processing application, using a set of 1D low-pass and high-pass filter coefficients, and filters are applied separately on rows and columns can obtain a 2D transformation. Details can be described as follow. Considering an image denoted as I of size N N, firstly, a low-pass filter H 0 (z) and a high-pass H 1 (z) are applied to the rows of I. It creates two images which respectively contain low and high frequencies of I. Secondly, the rows of the two images are subsampled by a factor of 2. It creates two (N/2) N images. Then the filters are reapplied along the columns, followed by decimation by a factor of 2. Finally, at the output there is four subband images of size (N/2) (N/2) labeled LL, LH, HL and HH. The operation is recursively repeated on the LL band for more decomposition levels. The implementation method is shown in Fig. 1. Fig. 1. Processing of 2D discrete wavelet transform. Different from the standard DWT decomposition scheme, the subsampling is dropped in SIDWT [14]. There are lots of methods to implement SIDWT. A good review could be found in [15]. In our paper, the method proposed by Beykin [16] is adopted, which is also used as decomposition model in paper [17]. The idea of method [16] is to observe decompositions of all the translations of the input image, and to find the decomposition which is containing the most information. This particular decomposition will be considered as the SIDWT of the image. A translation along any vector could be considered as the combination of elementary shifts (0, 0), (0, 1), (1, 0) and (1, 1) [16], where the indices respectively denote the row and column shifts. In fact, all possible shifts could be browsed by applying the four elementary shifts in the input, which means that the LL band is shifted by the four translates (0, 0), (0, 1), (1, 0) and (1, 1) and it creates a tree with all possible shifts. For the input shift (a, b), the four subbands at the jth decomposition level can be defined by:

5 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 201 j ( ab, ) j ( ab, ) j ( ab, ) j ( ab, ) j 1 LL = h ( m 2 x) h ( n 2 y) LL ( m a, n b) m n j 1 HL = h ( m 2 x) h ( n 2 y) LL ( m a, n b) m n j 1 LH = h ( m 2 x) h ( n 2 y) LL ( m a, n b) m n j 1 HH = h ( m 2 x) h ( n 2 y) LL ( m a, n b) m n (1) where the superscript j 1 corresponds to the previous decomposition level, m and n are the coordinates in j 1 level, and x and y are the coordinates in the bands of level j. (a, b) is defined as (0, 0), (0, 1), (1, 0) or (1, 1). When the full decomposition is performed, containing all the DWT coefficients in a tree is got. If the size of image is N N, the tree contains N 2 circular translates. The final step is to find the best basis for decomposition, which is the particular path in the tree which minimizes a cost. The cost is calculated by computing the entropy of each subbands and the path with the minimal entropy is preserved. This best basis corresponds to a translation of vector t 1 of the image I. If the input image is a translation of I by a vector t 2, the best basis will correspond to a translation of vector t, with t = t 1 t 2. The best basis is the same for every shift of the input image [17]. In our paper, Haar wavelet is adopted, because it is easy to comprehend and fast to compute. Haar transform can be viewed as a series of averaging and differentiating operations on a discrete function. The impulse response for low-pass filter is given by [1/ 2, 1/ 2] and high-pass filter is [ 1/ 2, 1/ 2]. 3.2 The Proposed Method SIDWTBEF The general steps of the proposed method SIDWTBEF can be listed as follows. Step 1: make the multiple exposure source images to be same size for each other. Step 2: chose the median two images to preserve the chrominance information of the real scene. If the number of input images n is even, the images are denotes as {I 1, I 2,, I n }, the median two images I n/2 and I n/2+1 are chosen. Else, if the n is odd, the images I n/2 and I n/2 +1 are chosen, where n/2 denotes the nearest integer greater than n/2. After the two images are chosen, we apply the average scheme, i.e., the mean value for corresponding pixels in the two images is kept for final reconstruction. The scheme is applied to R, G and B channels separately to get the chrominance values R s, G s, and B s of the scene. Step 3: using the R s, G s, and B s obtained in the step 2, we compute the luminance value I s as: I s = R s G s B s. Then the R s = (R s /I s ) λ, G s = (G s /I s ) λ, and B s = (B s /I s ) λ are calculated, where λ is used to adjust the color saturation of the final image as paper [18]. In our paper, we limit λ [ ], and find that, for most images, pleasing results can be obtained when λ = 0.8. Step 4: convert the multiple exposure source images into luminance images that are taken

6 202 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI as input for the SIDWT. Then, the input images are decomposed into sets of sub-images with decomposition level to be three. Step 5: fuse the sub-images obtained by the step 4. In each level, any two sub-images that relate to adjacent exposure time are fused into a single one. It is an iterative process until only one image is generated. The process can be represented in Fig. 2, where the number of input images is five, and we use A, B, C, D and E to represent the input images. Notice that for high-pass sub-images and low-pass sub-images, we adopt different fuse rules. The sub-images of high-pass are fused by the absolute value maximum selection with consistency verification, while the low-pass sub-images are acquired by average measure. They are described as follows. Fig. 2. The procedure of fusion rule when the number of input images is five. To simplify the description of the absolute value maximum selection with consistency verification fusion rule [12], we make an assumption that A and B are the luminance image of two exposure source images and F is the fused luminance image. The symbol C A j(x, y) and C B j(x, y) denote the SIDWT transform coefficients of the source images A and B respectively, and C F j(x, y) represents the coefficient of the fused image F, where j is the decomposition level and (x, y) is the location of the current coefficient. The maximum absolute value within 5 5 window is served as an activity measure associated with the center pixel. In this way a high activity value indicates the presence of a dominant feature in the local area. A binary decision map of the same size as the wavelet transform is then got to record the selection results. If the center pixel value comes from image A while the majority of the surrounding pixel values come from image B, the center pixel value is switched to that of image B. This selection scheme helps to ensure that most of the dominant features are incorporated into the fused image. The fuse rule of the low frequency sub-image is different. We know that the low frequency sub-images are the coarse representation of the original image and may have inherited some of its properties such as the mean intensity or texture information. In our paper, we use the mean values of images A and B to serve as the low coefficient in the fused image. The formula is defined as, C F j(x, y) = 0.5 * C A j(x, y) * C B j(x, y). (2)

7 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 203 Step 6: by applying the inverse transformation, the fused luminance image I f is constructed. Step 7: an enhancement operator is used to preserve more details in the scene. The formula is defined as I f = (I f ) 0.7. Its shape is demonstrated in Fig. 3, where the red line denotes the used function and the blue line denotes the linear function. From the figure, we can see that using the function, when the treated pixel locates in darker area (low input value), the output luminance is increased, while when the treated pixel locates in highlight area (high input value), the output luminance is compressed. Thus, more details in the image can be preserved in the resulting image. Fig. 3. The curves of the enhancement operator. Step 8: the new luminance image I f and the chrominance information R s, G s and B s are combined to generate the final fused color image. The combination formula are defined as R out = R s I f, G out = G s I f and B out = B s I f. 4. EXPERIMENTAL RESULTS To verify the proposed exposure fusion method SIDWTBEF, we test it using several sets of multiple exposure source images. All the experiments are implemented in Matlab 7.0 and run on a Pentium (R) 4 CPU 3.0-GHz machine with 1-G RAM. 4.1 Evaluation Criterion Evaluation of exposure fusion methods is not a trivial task, because it is often difficult to say which of fused images is better without reference of real scene except the visual perception of result image is quite different. Currently, the existing evaluation techniques can be generally grouped into two categories: subjective measure and objective measure. In our paper, the visual (subjective) and quantitative (objective) tests are both used to verify the effectiveness of the proposed method SIDWTBEF. Two objective criteria are used to quantitatively evaluate the performance of exposure fusion methods in our experiments. The first criterion is mutual information (MI) [19]. It is a metric defined as the sum of mutual information between each input image and the fused

8 204 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI image. Considering two input images A and B, and a resulting fused image F, the formula is defined as, pfa( f, a) IFA( f, a) = pfa( f, a)log, (3) p ( f) p ( a) f, a f, b F A pfb ( f, b) IFB ( f, b) = pfb ( f, b)log. (4) p ( f) p ( b) F B Considering images F and A, p FA (f, a) denotes joint probability distribution, p F (f) and p A (a) are marginal probability distributions, which can be obtained by simple normalization of the joint and marginal histograms of both images. p FA (f, a) can be defined as, 1 pfa( f, a) = hfa ( f, a) (5) 2NM where h FA (f, a) is the joint histogram of images F and A. Its definition is defined as, h ( f, a) = h ( L( f), L( a)) (6) FA f, a FA where L(f) and L(a) are intensity values located same region in images F and A. The image fusion performance measure can be defined as, MI F AB =I FA (f, a) + I FB (f, b). (7) In our paper, we calculate the mutual information between each input image and fused image, and the sum of their values is served as the overall mutual information, which can reflect the total amount of information that the fused image F contains that of input source images. The second criterion is Q AB/F metric proposed by Xydeas and Petrovic, more details can be seen in paper [20]. It can measure the amount of edge information transferred from the source images to the fused image. In our paper, we extend it for a set of color multiple exposure source color images. The average value of R, G and B channels metric Q AB Z/F among the input source images and {A, B, C,, Z} fused image F is computed to evaluate the performance of the proposed method. The formula for one channel is defined as follows, Q AB... Z / F = xy, AF A BF B ZF Z x, y x, y + x, y x, y + + x, y x, y Q w Q w Q w xy, A B Z xy, + xy, + + xy, w w w. (8) Notice that for both criteria, the larger the value, the better is the fusion result. 4.2 Visual Inspection and Quantitative Comparison In this section, we firstly use the two groups of multiple exposure images to demonstrate the comparison result with typical exposure fusion methods. Secondly, we provide

9 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 205 the comparison results related to some tone mapping methods. Lastly, the effectiveness of SIDWTBEF on the performance of shift invariance compared to DWT-based method is shown. We can see from Fig. 4 that the image (e) obtained by [3] looks slight red and more details of green lawn are lost. The common problem caused by most of exposure fusion methods is the brightness change. Although the details of lawn can be preserved, the result (f) obtained by [4] contains the brightness change phenomenon that is not consistent with the original source images. For example, we can see that the background is brighter than the foreground in the input images. However, the background in the image obtained by [4] is darker than the foreground. The image (g) obtained by the proposed method SIDWTBEF can avoid the shortcoming, as well as the details of lawn and overall contrast of original images can be well preserved. (a) (b) (c) (d) (e) (f) (g) Fig. 4. (a)-(d) show four images obtained at different exposures; (e) shows the result obtained by the method of Goshtasby [3]; (f) shows the result obtained by the method of Mertens [4]; (g) shows the result obtained by the proposed method SIDWTBEF. Images courtesy of [21]. (a) (b) (c) (d) (e) (f) Fig. 5. (a)-(c) show three images obtained at different exposures; (d) shows the result obtained by the method of Goshtasby [3]; (e) shows the result obtained by the method of Mertens [4]; (f) shows the result obtained by the proposed method SIDWTBEF. Images courtesy of HDRsoft.

10 206 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI Fig. 5 shows the other comparison results. The source images are also tested in paper [4]. We can see that the brightness change is still existed in images (d) and (e), obtained by [3] and [4] respectively, even if more details of cloud can be seen. More chrominance information is also lost. As a whole, the images (d) and (e) look a little unnatural for human eyes. The image (f) treated by SIDWTBEF can provide a warm perception, although some of details of cloud in the sky are lost. In Fig. 6, it demonstrates the comparison results with typical tone mapping methods. The results obtained by Ashikhmin [22], Drago [23], Krawczyk [24], Fattal [10], Reinhard [6] are offered by Martin cadik, which are used to evaluate HDR tone mapping methods in paper [25]. We can see from Fig. 6 that more local contrast of image (b) is lost and the image looks unnatural for human eyes. The image (c) is washed out and more chrominance information is lost. In image (d), more details of the books behind lamp in background are not visible. Reinhard et al. s method [6] generates quite similar result (f) to image (e) obtained by [22] in terms of contrast and detail preservation. We can see that some details in images (e) and (f) are also lost compared to the image (g) obtained by SIDWTBEF. We can conclude that the proposed method can get comparative results with other tone mapping methods. (a) Sequences of multiple exposure source images. (b) Fattal [10]. (c) Drago [23]. (d) Krawczyk [24]. (e) Ashikhmin [22]. (f) Reinhard [6]. (g) SIDWTBEF. Fig. 6. Comparison results related to several typical tone mapping operators. Images courtesy of Martin cadik [25]. Next, in order to show the effectiveness of shift invariance for SIDWTBEF, we provide the comparison results obtained by multi-resolution model SIDWT and DWT respectively. Notice that the proposed method can also be used for fusing multi-focus images. In our experiment, the tested multi-focus images are provided by Lehigh University. There is a slight movement of the student s head. The visual comparison is demonstrated

11 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 207 (a) Focus on the right. (b) Focus on the left. (c) DWT-based. (d) Partly region of (c). (e) SIDWTBEF. (f) Partly region of (e). Fig. 7. The Lab source images and fusion results. Table 1. The Q AB Z/F and MI values between SIDWTBEF and DWT-based method. MI DWTBEF SIDWTBEF Increasing DWTBEF SIDWTBEF Increasing Percentage Percentage Fig % % Q AB Z/F in Fig. 7. We can see that the head of student in image (d) is glossier than the image (f) obtained by SIDWT, which can further validate the shift invariance of our proposed method. For the quantitative comparison, the source images in Fig. 7, Q AB Z/F and MI are calculated. The values are listed in Table 1. We can observe that the Q AB Z/F values of the SIDWTBEF are greater than those of DWT-based method. That is, the Q AB Z/F value of SIDWTBEF is , which is larger than generated by the DWT-based method. The increase percentage is 4.79%. The MI value of SIDWTBEF is , which is also larger than generated by the DWT-based method. The increase percentage is 6.04%. From this experimental result, we can conclude that the DWT-based method suffer greatly when the misaligned source images are adopted. In practice, the condition of misaligned source images is usually confronted for exposure fusion methods. Using the SIDWT can effectively overcome the shortcoming. Because the aim for SIDWTBEF and tone mapping is both to acquire high quality image for displaying on ordinary devices, we provide the quantitative comparison for the resulting images in Fig. 6 to show the performance of SIDWTBEF compared to typical tone mapping methods. However, the criteria Q AB Z/F and MI defined above are inputrelated, and the inputs for exposure fusion and tone mapping are different. So, in this part, we adopt another three no input-related criteria to validate the effectiveness of our method. Note that in our paper, we calculate the average value of these criteria for R, G, and B channels to measure the overall activity level in the resulting image. The first criterion is entropy, which is used to measure the overall information in the fused image. The larger the value is, the better the image is. The formula is

12 208 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI L Entropy = h( k)log h( k) (9) k = 0 2 where h is the normalized histogram of the image to be evaluated, L is the maximum value for a pixel in the image, in our paper, L = 255. The second criterion is the spatial frequency [26], which can be used to measure the activity levels of an image. Considering an image of size M * N, where M is the number of rows and N is the number of columns, the row and column frequencies of the image are defined respectively as M 1, N RF = [ F( x, y) F( x, y 1)], MN x= 0, y= 1 N 1, M CF = [ F( x, y) F( x 1, y)], MN y= 0, x= 1 (10) where F(x, y) represents the intensity of the pixel located (x, y). The total spatial frequency of the image is then defined by 2 2 = ( ) + ( ). (11) SF RF CF The third criterion is standard deviation, which can measure the contrast information of an image. The image with high contrast has high value of standard deviation, and low for low contrast images. The formula is defined as, M N SD 1 2 = ( F ( x, y ) F ), (12) M N i= 1, j= 1 where F is the mean value for the whole image. The resulting values for these three criteria are shown in Table 2. From this table, we can conclude that from Entropy value comparison, that the proposed method is competitive with other typical tone-mapping methods in terms of detail preservation. However, the proposed method can produce an image with higher activity levels and contrast compared other typical tone mapping methods. Table 2. The Entropy, SD and SF comparison between SIDWTBEF and tone mapping methods. Image (b) Image (c) Image (d) Image (e) Image (f) Image (g) Entropy SD SF

13 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM CONCLUSION In this paper, we propose a SIDWT based exposure fusion method for displaying high dynamic range scenes on ordinary devices. Due to SIDWT s shift invariant property; it is more suitable for exposure fusion. A novel way to fuse multiple exposure color images is proposed. For high-pass sub-image and low-pass sub-image generated by the SIDWT decomposition, we adopt different fuse rule, which can extract more information that contains in the original source images. We can draw a conclusion from the experimental results that the proposed method SIDWTBEF can greatly reduce the influence caused by the misaligned source images than DWT-based method. Furthermore, the brightness change phenomena can be avoided, which usually appears in typical exposure fusion methods. REFERENCES 1. P. E. Debevec and J. Malik, Recovering high dynamic range radiance maps from photographs, in Proceedings of SIGGRAPH, 1997, pp G. Piella, A general framework for multi-resolution image fusion: from pixels to regions, Information Fusion, Vol. 4, 2003, pp A. Goshtasby, Fusion of multi-exposure images, Image and Vision Computing, Vol. 23, 2005, pp I. Mertens, J. Kautz, and F. V. Reeth, Exposure fusion, in Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, 2007, pp E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, High Dynamic Range Imaging Acquisition, Display and Image-based Lighting, Morgan Kaufman Publishers, San Francisco, E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, Photographic tone reproduction for digital images, ACM Transactions on Graphics, Vol. 21, 2002, pp J. Kuang, G. M. Johnson, and M. D. Fairchild, icam06: A refined image appearance model for HDR image rendering, Journal of Visual Communication and Image Representation, Vol. 18, 2007, pp L. Meylan and S. Süsstrunk, High dynamic range image rendering using a Retinexbased adaptive filter, IEEE Transactions on Image Processing, Vol. 15, 2006, pp L. Meylan, D. Alleysson, and S. Süsstrunk, A model of retinal local adaptation for the tone mapping of color filter array images, Journal of Optical Society of America, Vol. 24, 2007, pp R. Fattal, D. Lischinski, and M. Werman, Gradient domain high dynamic range compression, ACM Transactions on Graphics, Vol. 21, 2002, pp F. Durand and J. Dorsey, Fast bilateral filtering for the display of high-dynamicrange image, ACM Transactions on Graphics, Vol. 21, 2002, pp H. Li, B. S. Manjunath, and S. K. Mitra, Multisensor image fusion using the wavelet transform, Graphical Models and Image Processing, Vol. 57, 1995, pp P. J. Burt and R. J. Kolczynski, Enhanced image capture through fusion, in Proceedings of the 4th International Conference on Computer Vision, 1993, pp O. Rockinger, Image sequence fusion using a shift-invariant wavelet transform, in

14 210 JINHUA WANG, DE XU, CONGYAN LANG AND BING LI Proceedings of IEEE International Conference on Image Processing, Vol. 3, 1997, pp A. P. Bradley, Shift-invariance in the discrete wavelet transform, in Proceedings of the 7th International Conference on Digital Image Computing: Techniques and Applications, 2003, pp G. Beylkin, On the representation of operators in bases of compactly supported wavelets, SIAM Journal of Numerical Analysis, 1992, pp J. Bonnel, A. Khademi, S. Krishnan, and C. Ioana, Small bowel image classification using cross-co-occurrence matrices onwavelet domain, Biomedical Signal Processing and Control, Vol. 4, 2009, pp X. G. Li, K. M. Lam, and L. Shen, An adaptive algorithm for the display of highdynamic range images, Journal of Visual Communication and Image Represent, Vol. 18, 2007, pp G. Qu, D. Zhang, and P. Yan, Information measure for performance of image fusion, Electronics Letters, Vol. 38, 2001, pp C. Xydeas and V. Petrovic, Objective image fusion performance measure, Electronics Letters, Vol. 36, 2000, pp M. Ashikhmin, A tone mapping algorithm for high contrast images, in Proceedings of the 13th Eurographics Workshop on Rendering, Vol. 28, 2002, pp F. Drago, K. Myszkowski, T. Annen, and N. Chiba, Adaptive logarithmic mapping for displaying high contrast scenes, in Proceedings of Eurographics, Vol. 22, 2003, pp G. Krawczyk, K. Myszkowski, and H. P. Seidel, Computational model of lightness perception in high dynamic range imaging, in Proceedings of Conference on Human Vision and Electronic Imaging XI, 2006, pp M. Cadik, M. Wimmer, L. Neumann, and A. Artusi, Evaluation of HDR tone mapping methods using essential perceptual attributes, Computers and Graphics, Vol. 32, 2008, pp S. T. Li and B. Yang, Multifocus image fusion using region segmentation and spatial frequency, Image Visual Computing, Vol. 26, 2008, pp Jinhua Wang ( ) received the Ph.D. degree in Institute of Computer Science and Engineering, Beijing Jiaotong University in She is working in Institute of Information Technology, Beijing Union University, Beijing, China. Her current research interests include high dynamic range images and visual perception theory.

15 EXPOSURE FUSION BASED ON SHIFT-INVARIANT DISCRETE WAVELET TRANSFORM 211 De Xu ( ) received the M.E. in Computer Science from Beijing Jiaotong University in He is now a Professor in Institute of Computer Science and Engineering, Beijing Jiaotong University, Beijing, China. He has published more than 100 papers in international conferences and journals. His research interest includes database system, computer vision and multimedia processing. Congyan Lang ( 丛 ) received the Ph.D. degree in Institute of Computer Science and Engineering, Beijing Jiaotong University in 2006, Beijing, China. Her current research interests include image processing, visual perception. Bing Li ( ) received the B.E. in Computer Science from Beijing Jiaotong University in He is pursuing the Ph.D. degree in Institute of Computer Science and Engineering, Beijing Jiaotong University, Beijing, China. His current research interests include color constancy, visual perception and computer vision.