Research on Image Splicing Based on Weighted POISSON Fusion

Transcription

1 Research on Image Splicing Based on Weighted POISSO Fusion Dan Li, Ling Yuan*, Song Hu, Zeqi Wang School o Computer Science & Technology HuaZhong University o Science & Technology Wuhan, , China {lidanhust, cherryyuanling, M067934, M047898}@hust.edu.cn Abstract Image splicing aims to process the images with similar scenes and overlapping parts through a series o matching and usion algorithm, eventually produce a large image with wide ield o vision. For the seamlessness o splicing, this paper presents an algorithm about image matching which combined method o weighted average and Poisson usion algorithm. Firstly, process images by weighted average smoothing. Then adjust the brightness o images uniormly in order to reduce the dierence o exposure between images. Finally, embed the images processed into the scene by Poisson usion. Experiments show that the proposed algorithm can not only realize the image splicing, but also improve the marks o splicing. Keywords-Image splicing; Image usion; Weighted average; POISSO usion I. ITRODUCTIO In the ield o digital image processing, wide Angle image has been more and more widely used. But because o the limitations on camera view, it is diicult to get wide perspective o high resolution image. In order to solve this problem, the image splicing algorithm arises at the historic moment. The processing ideas are as ollows: irstly, operate the input images with preprocessing, then extract eature inormation o the image and match and use them, inally generate high resolution ull image through image matching algorithm. Among them, the eature point extraction, image matching, image usion are the key o image splicing problem, they directly determines the eect o the image splicing algorithm. Currently, algorithms about eature point extraction and image matching are relatively mature, but the consistency o image usion remains to be improved. II. RELATED WORK Image usion reers to splicing a collection o original image into a high resolution and ull image with wide angle o view, its unction is to eliminate image splicing trace o the original image which overlaps, and realize smooth transition without shadow, at the same time ensure that the details o the source image have no loss. Over the nearly 0 years, scholars at home and abroad made a deep research on image usion ield. Image usion technology is limited to simple and direct integration, the eect is poor. Ater wavelet theory put orward[], the image usion technology has a big leap. At present the image usion technology[] can be divided into three levels: pixel level, eature level and decision level. Pixel level usion technology is the oundation o high-rise usion processing, the purpose is to keep the details o original image as much as possible, preparing or subsequent analysis. This level can be subdivided into spatial domain and transorm domain algorithms, spatial domain usually uses usion algorithm such as logic ilter, weighted average, compared to adjust and so on, transorm domain usually uses multiresolution pyramid usion[3] and wavelet transorm usion[4]. Feature level usion is the middle level o inormation usion, the main unction is to summary the inormation extracted ater preprocessing the source image and extracting the eature. Based on pixel level, using the model, statistical analysis[5] and other analysis, provide the required inormation or decision-making analysis. The main usion method o this level are clustering analysis and bayesian estimation method. Decision level is the top level o usion technology, which makes the optimal decision according to some usion rules and the corresponding credibility. At this point, the usion algorithm has good real-time perormance, and has a certain tolerance. In addition to the above mentioned method: the weighted average usion method, multiresolution spline usion method, the image usion method based on principal component[6] and usion algorithm based on wavelet transorm, Poisson equation has also been used to image usion splicing processing in recent years[7]. Ater that, many kinds o optimization o the usion algorithm was proposed. In order to improve the inal eect o Poisson image editing, Jiaya proposed optimal usion boundary[8] on its basis. In order to enhance the processing speed and simpliy the solution o the poisson equation, Zeev proposed using Mean value coordinates based on gradient domain[9] to calculate, this method is simple, and the eiciency has a good improvement. Image usion technology is developing continuously, there are still many problems to be solved, image usion eect remains to * Corresponding author, address:cherryyuanling@hust.edu.cn(ling Yuan) ISS : Vol. 6 o. 06 June 07 58

2 be urther promoted, and image usion ield has not been uniied by a theoretical ramework and mathematical model, where a mature and stable system has not yet ormed. III. THE BASIC FLOW OF WEIGHTED POISSO FUSIO According to the weighted average usion and Poisson usion, this paper proposed a new image usion algorithm, which can better match images whose brightness have a big dierence. Its basic idea is to combine the method o weighted average and Poisson usion algorithm. Firstly, process images by weighted average smoothing. Then adjust the brightness o images uniormly in order to reduce the dierence o exposure between images. Finally, embed the images processed into the scene by Poisson usion. Algorithm o overall process is shown in igure. Start input the image weighted average usion pretreatment bightness processing eature point extraction through SIFT Poisson usion coarse match and exact match output the image exception handling successul match Y End Figure. Flow chart o weighted poisson usion splicing algorithm IV. WEIGHTED POISSO FUSIO ALGORITHM A. The weighted average usion The thought o weighted average usion algorithm is making weighted operation on the pixel value and then cumulate or average in the treatment o the overlapping scene area. As shown in ormula (), w and w are respectively weighted value o pixels which match the image overlap scene, and satisied w +w = 0<w, w <. Select the optimal weight can achieve coincidence scene smooth transition, and eliminate the obvious splicing trace. + w = w () This paper adopts gradually into the method or choosing weights, in order to smooth stitching line. Assuming that, are splicing source images, superimpose the two images on the space, the pixel values o the used images are shown in ormula () : ISS : Vol. 6 o. 06 June 07 59

3 + d = d () Among them, d and d are weight, they meet d +d =, 0<d, d <, and the value o weight relate to the overlapping area about the size o the scene. In the overlapping scenes, d gradient rom to 0, while d gradient rom 0 to, which can realize a smooth transition rom to in the overlapping scene. B. Poisson usion Poisson usion is originally used in the usion o scene, as on the premise o keep the gradient inormation o the image, it can well eliminate splicing trace. The basic thought o the usion method is to use guide model or interpolation processing, and reconstruct the area o the pixel value, so as to realize seamless integration between scenario, the basic principle is shown in igure : Figure. The principle o Poisson usion Among them, g is the scene o the original image, v is a gradient ield o g, S is image domain ater usion, Ω is the scene covered in S, Ω is its border, * is the scalar unction which said the pixel values outside Ω, is unknown scalar unction which said the pixel values outside Ω. To realize smooth transition without aperture, Ω the gradient value should be as small as possible, so as to convert image seamless usion processing into gradient minimization problem. Variable can be solved by ormula (3): min, Ω == Ω Ω (3) Among them, = x, is the gradient operator, unction F = = x + y, the minimum value o y F meets eulerian - Lagrangian equation, so its solution can be expressed by Laplace equation: Δ = + is the Laplace operator. x y Ω, Ω = Ω Δ = 0, (4) The solution procedure in practice can appear too smooth, which will lead to the gradient allo, thus the usion eect is not ideal. Poisson equation using the gradient ield v o g as the guide ield to solve above problems. The purpose o the guide ield is to get as close as possible to the gradient o and the gradient o g, not only keep the maximum detail inormation o image, but also ensure there are not obvious transition boundary traces. The optimized using ormula (5) : min v = min g, Ω = Ω Ω Generate again into the euler - Lagrange equation, the results are as ollows: Ω () v = div( g), Ω = Ω Δ = div (6) Among them, div(v) is the divergence o gradient ield v. The mathematical is the oundation or the construction o poisson usion technology, in image splicing processing o the overlap o scene, using the image o the gradient ield as the guidance, can realize image usion. (5) ISS : Vol. 6 o. 06 June 07 60

4 C. Weighted Poisson usion algorithm Poisson usion can maintain the detail inormation o image while solving the problem o splicing trace, but when the dierence o exposure is very big, poisson usion may lead to uneven image color, and the eect is not ideal. This paper applies Poisson usion principle in the boundary transition area o overlap scene to optimizing the eect o splicing. Ater balancing the global brightness, apply the weighted usion and Poisson usion, mainly to improving the usion eect o splicing algorithm. The speciic steps are as ollows: ) The weighted average usion processing. Use the method gradually in and out to determine the weights, which is not only simple to deal with, but also can achieve good coincidence scene matching usion, and the method has an eect in smooth transition or splicing line, the usion eect is better. ) Brightness uniormity. I the dierence o exposure or two images is big, the usion eect will hava very obvious traces splicing, and it will also impact the subsequent operations o Poisson usion. This paper proposes a brightness uniormity, which can better abate the inluence o the exposure dierence, speciic process is as ollows: a) Calculate the average brightness o global image. b) Segment the usion image into small pieces and calculate the brightness matrix or each area. c) Use the brightness matrix minus the global average brightness, get the brightness dierence matrix, at this time the high brightness and low brightness area can be judged by dierential value. d) Operate the brightness dierence matrix by cubic convolution interpolation algorithm, extend to the whole image operation, which will get global brightness dierence matrix. e) Use the source image brightness value minus the corresponding values in global brightness dierence matrix, which can achieve strengthening the low brightness and weakening the high brightness. ) According to the original usion image brightness's extremum, process smoothing operation on each area, which makes the intensity distribution o area is reasonable, to realize smooth transition. The above steps process luminance equalization operation or each subdomain through regional segmentation, inally reaching the purpose o weakening exposure dierence. 3) Poisson usion solves the splicing trace. Ater the irst two steps o processing, it can not completely eliminate the splicing marks, as there still appear obvious splicing trace in the border o image. So in this paper, on the basis o poisson usion, propose a overlap transition poisson usion thought: the original image as a reerence datum, use the space transormation model to merge the overlapping scenario to image, so as to achieve the aim o seamless splicing. The main process is as ollows: a) Determine the usion zone. Ater dealing with the weighted usion, there will be a scene splicing trace where is the overlap o the border, so expand outward rom the boundaries o overlapping scene x M, x respectively, get the area Ω M and Ω waiting or usion. The size or expanding decide on the size o overlap scene, generally take /50 o the overlapping scene area. Eect is shown in igure 3: Figure 3. Determine the usion zone b) Initialize the parameters: take corresponding regions o the gradient ield in the original images as the guidance o the usion area Ω M and Ω. Get pixels corresponding to the original image on both sides o the ISS : Vol. 6 o. 06 June 07 6

5 border, when the size o two images is same and transorm is scale-ree, the upper and lower boundary can set no value, take or 0. c) Poisson usion processing. Take the results o weighted usion as the target image, usion ormula is used to use the area corresponding to the original image to image seamlessly V. ALGORITHM SIMULATIO AD RESULTS AALYSIS A. The simulation environment and the input image Carry on the experimental simulation and the analysis o experimental results according to the usion algorithm o weighted Poisson, use C++ as the development language, and use the open-source library OpenCV.4, device or image acquisition are cameras and mobile phones, acquisition way is handheld and rotary translation. This paper selects two groups o images or experimenting, there is no exposure dierence, scale change and rotation actors between the two pictures in igure 4, their size is In igure 5, the let image adjust the brightness and strengthen the exposure dierence when shooting, as both image size are (a)the let image or splicing (b) The right image or splicing Figure 4. The irst group o images (a)the let image or splicing (b) The right image or splicing Figure 5. The second group o images B. Image usion experiment Finally the usion o two experiments results as shown in the igure below. The igure 6 is direct weighted usion, it can be seen that the usion eect is poor. Traditional SIFT splicing algorithm also apply the integration strategy, but the eect dierence can not be identiied through naked eye. Here does not repeat. Figure 7 is the ISS : Vol. 6 o. 06 June 07 6

6 result o processing the brightness uniormity, as the previous group o experimental exposure dierence is not obvious, so the results do not have too big dierence, only list the brightness o the second set o experiments uniorm processing's result. Figure 8 is the result applying Poisson usion based on the treatment eect o the two, it can be seen that splicing trace has been smooth processing. The eect o weighted average usion : (a) The irst experimental group o usion (b) The second experimental group o usion Figure 6. Two experiments using the weighted average usion processing Because the irst group o experiments through brightness uniormity processing can not be seen signiicant dierence by naked eye, here no longer list. Only list the results o the second experiments through brightness uniormity processing. It is worth noting that the brightness uniormity processing is to adjust the global brightness, will not eliminate the splicing. ISS : Vol. 6 o. 06 June 07 63

7 The eect o Poisson usion : Figure 7. The second group o experiment with brightness uniormity processing (a) The irst experimental group o usion (b) The second experimental group o usion Figure 8. Two groups o experiment using the Poisson usion Take the second group o experiments as an example, use dierent usion algorithm to run the program or ten times and record the time needed or usion (based on the algorithm contains the brightness uniormity processing time), clarity, and the relevance with the original image. The relevance is to evaluate the similarity with the original image, the ideal state is. Parameter standard is similar with the mean square error, which ISS : Vol. 6 o. 06 June 07 64

8 mainly used to judge the eect o usion with ideal eect. This paper takes the usion image and the source image as the object or comparing, calculation ormula is shown in ormula (7) : R = M m= n= m= n= M (, n) B( m, n) M (, n) B( m, n) A m A m m= n= (7) Deinition: the average gradient, can be used to evaluate the image o dierences in minute details, the greater the value, the better the usion eect. Calculation ormula is shown in ormula (8) : ΔG = M M m= n= ΔF x ( m, n) + ΔF ( m, n) y (8) ( m n) Δ F i, is the dierence o the image whose size is M on the direction i. The inal usion results are shown in table (in seconds) : TABLE I. THE FUSIO RESULTS DATA TABLE usion algorithm relevance clarity running time weighted average Poisson usion weighted average Poisson usion VI. COCLUSIO For the problem o sensitive exposure dierence and image splicing traces, this paper proposed a weighted usion poisson thought. Firstly, apply the weighted average usion strategy or smooth operation, through the control o parameter, eventually splicing traces will only appear in the border, to prepare or subsequent Poisson usion processing. Then carry out brightness uniormity processing on the whole image, in order to reduce the exposure dierence, improve the robustness o ollow-up Poisson usion algorithm. Finally take the original image as the reerence images, apply the Poisson usion method with overlapping transition to embedding its corresponding area to the eect o weighted usion, so as to achieve the eect o eliminating the splicing traces. ACKOWLEDGMET This work was supported by ational atural Science Fund o China under grants and the Fundamental Research Funds or the Central Universities(o: 07KFYXJJ07). REFERECES [] Guoxin Xue,Changzhou.A Method to Improve the Retinex Image Enhancement Algorithm Based on Wavelet Theory. International Symposium on Computational Intelligence and Design, 00, ():8-85. [] Wang Yang,Jin-Ren Liu. Research and development o medical image usion. IEEE International Conerence on Medical Imaging Physics and Engineering, [3] Deepali Sale,Varsha Patil. Eective image enhancement using hybrid multi resolution image usion. IEEE Global Conerence on Wireless Computing and etworking, [4] Huaxun,ZhangXu Cao. A Way o Image Fusion Based on Wavelet Transorm. IEEE inth International Conerence on Mobile Adhoc and Sensor etworks, [5] Zhanlong Yang.Based on the eature points o image registration and stitching technology research: [Ph.D. Thesis]. Xian university o electronic science and technology,008. [6] U.Patil,U.Mudengudi. Image usion using hierarchical PCA. ACM Transactions on Graphics,0, 3(5):-6. [7] H.C.Wang,R.Raskar. Seamless video editing. Proceedings o the 7th International Conerence on Pattern Recognition [8] J.Jia. Drag-and-drop pasting. ACM SIGGRAPH Conerence Proceedings, [9] Z Farbman. Coordinates or instant image cloning. ACM Transactions on Graphics, 009, 8(3): ISS : Vol. 6 o. 06 June 07 65