International Journal of Computer Engineering and Applications, Volume XII, Issue I, Jan. 18, ISSN

Transcription

1 STUDY ON REFLECTION SEPARATION METHODS Suresha M, Madhusudhan S Post Graduate Department of Studies & Research in Computer Science Kuvempu University, Shankaraghatta Shivamogga, India srit_suresh@yahoo.com, madhu.hsd@gmail.com ABSTRACT: Light reflection effect on images is common challenge in photography and it reduces by photographer s skill. In digital photography, this can be reduced by applying reflection removal algorithms which improves the computer vision and detection. Many researchers are contributed to segment reflection effects in images. The detection and removal of specular reflections from images have been an area of interest to the Computer Vision community for many years and existing techniques for this task find applications in various areas such as medical science, video surveillance, image refinement and image reconstruction. This paper conducts survey on reflection segmentation methods which is helpful for researchers to study reflection segmentation methods and selection of algorithm. Keywords: Computer Vision, Reflection, Segmentation [1] INTRODUCTION The surface typically glossy to be very shiny, fairly matte, or anywhere in between, but virtually all the surfaces around us exhibit highlights to varying degrees. These highlights are most pronounced when the surface normal bisects the angle between the direction of illumination and the direction of view, making the position and intensity of highlights very sensitive to viewing geometry. This causes problems with many computer vision methods such as segmentation which typically assumes uniform or smoothly varying intensity across a surface or stereo and motion analysis which Suresha M and Madhusudhan S 1

2 STUDY ON REFLECTION SEPARATION METHODS attempt to match images taken from different viewpoints. According to Shafer s [1] dichromatic reflection model, every value of brightness is combination of diffuse and specular parts. The diffuse part in image is viewable from all directions and specular part is only visible when viewed from particular direction, it is based on Snells law (Morel et al. [2]). Nayar et al. [3], specular reflection is looking as compact lobe on the object surface around the specular direction and also for rough surfaces. The diffuse components represent the actual appearance of an object surface but specularity reflection is an unwanted artifact that can deteriorates computational tasks such as visual recognition, tracking, stereo reconstruction, object re-illumination. Wang et al. [4], separation of specular or diffuse components is preprocessing step in image processing. The purpose of this paper is to help the researchers who want to segment reflections from an image for further processing. Specular glare causes a number of problems both to human vision and to image understanding by computer vision methods. The albedo variations intrinsic to an object surface and intensity edges caused by specular highlights which are artifacts of illumination and are not part of object surface detail. Once a specularity has been identified, there still many times exists the problem of obscuration of visual detail by the specularity, which is caused by brightness of specular reflection. The presence of specularity in images can cause many computer traditional computer vision processes to produce misleading results. [2] METHODS OF REFLECTION SEPERATION Images are formed based on light source. To produce an image, the scene must be illuminated with one or more light sources. When light hits an object s surface, it is scattered and reflected. Different models have been developed to describe this interaction. The most general model of light scattering is the Bidirectional Reflectance Distribution Function (BRDF). The diffuse component (also known as Lambertian or matte reflection) scatters light uniformly in all directions. Diffuse reflection imparts an object surface color to the light since it is caused by selective absorption and re-emission of light inside the object s material. Another major component of a typical BRDF is specular (gloss or highlight) reflection, which depends strongly on the direction of the outgoing light. Consider light reflecting off a mirrored surface. Incident light rays are reflected in a direction that is rotated by 180 degrees around the surface normal. The detection and removal of specular reflections from images have been an area of interest to the Computer Vision community for many years and existing techniques for this task find applications in medical science, video surveillance, image refinement and image reconstruction. In this section different methods to segment reflections in images have been discussed. [2.1] Based On Polarization Method Light is fundamentally a transverse electromagnetic wave possessing a state of polarization characterizing the vibrational orientations of the electric field. For a formal description of polarization of light Fig. 1, shows a sheet polarizer transmitting the component of the polarization state projected onto the transmission axis of the polarizer. Suresha M and Madhusudhan S 2

3 Fig.1: Transmission of light wave through a linear polarizer (Image Courtesy: Wolff and Boult [73]) Polarizing filters reduces specular reflections significantly more than they reduce diffuse reflection. The principle behind this is a simple instance of the Fresnel reflectance model. Light that specularly reflects into the eye from a dielectric surface, such as from ocean water, beach sand, or road asphalt is mostly polarized horizontal with respect to the viewer. Placing a linear polarizer in front of the eye with transmission axis oriented vertically will block significantly more than half of specular reflection for most angles of incidence while blocking one half of the typically unpolarized diffuse reflection coefficient. For a specular component of reflection occurring at exactly the Brewster angle for dielectric, sunglasses using polarizing filter can remove the specular component completely. [2.2] Flash / No Flash Image Pairs Agrawal et al. [25] describes a method of removal of reflection and highlights from images with a pair of flash and non-flash images. The Researchers have presented a method to limit the effect of specularities in the images. They used modified image capturing system i.e., multiflash camera is used to capture pictures of the scene and each one in different positioned illumination source. By using a Poisson equation on a gradient field obtained from the input images reduce the specularity effect. Reflections are unwanted components in images and they are sometimes treated as disturbances in photography and effects on vision algorithms for segmentation and shading analysis to produce error output. By varying the sensor direction i.e., highlight shift, diminish rapidly, or suddenly appear in other parts of the scene. This results a problem for vision methods that rely on image correspondences such as stereo or motion algorithms. Instead of taking one single picture of the scene use a multi-flash camera with n flashes to acquire n images from the same viewpoint each one with a differently positioned flash. Fig.2 shows our different multi-flash setups for static and dynamic scenes. Note that the position of each flash in these prototypes was chosen to better detect depth edges. Note that the position of each flash in these prototypes was chosen to better detect depth edges. Fig. 2: Multi-flash camera setups used for depth edge detection. From left to right: 4-flash setup, 8-flash setup and dynamic scenes setup. Our goal is to exploit multi-flash imaging for specular reflection reduction. (Image Courtesy: Feris et al. [20]) This method is based on that specular spots shift according to the shifting of light sources that created them. Consider three cases of how specular spots in different light positions appear in each image: (i) shiny spots remain distinct on a specular surface. (ii) some spots overlap. (iii) spots overlap completely (no shift). Suresha M and Madhusudhan S 3

4 STUDY ON REFLECTION SEPARATION METHODS Agrawal et al. [25] shows that for cases (i) and (ii), which often occur in practice, method successfully removes specular highlights. Note that although specularities overlap in the input images, the boundaries around specularities in general do not overlap. The main idea is to exploit the gradient variation in n images, taken under n different lighting conditions at a given pixel location (x,y). If (x,y) is in a specular region in cases (i) and (ii) the gradient due to the specularity boundary will be high in only one or a minority of the n images. Taking the median of n gradients at that pixel will remove this outlier(s). In this method, removes shadows in outdoor scenes by noting that shadow boundaries are not static. Let I k, 1 k n be an input image taken with light source k. Agrawal et al. [26] reconstructed the specular-reduced image by using median of gradients of input images as follows: Compute intensity gradient, Gk(x, y) = Ik (x, y) Find median of gradients, G(x,y) = median k(gk(x, y)) Reconstruct image Iˆ which minimizes Iˆ G. [2.3] Based On Video Sequence Reflections are removed from images based on video sequences, Xue et al. [51] presented a unified computational method by taking several photos through reflecting or occluding elements to recover the obstruction-free image or video. This method solves the reflection separating through a computational optimization process to acquire the reflection layer and reflection free background layer. This method can retrieve perfect decomposition result but it is a computation heavy method which need to solve a large linear system included multiple frames of pixel values in one time. This computational intensive method is difficult to port to small smart devices and to integrate with other intelligent application. In [51] Xue et al. used a modified matting equation to model the obstruction image or reflected image on the acquired image. Using a short video sequence, according the motion differences between background scene and obstruction layer, Xue et al. [51] employs edge flow to guide the process of decomposing the obstruction and background layer. Gu et al. [74] described an artificial removal method that focuses on dust or artificial obstruction calibration and an automatic dust removal method that assumes that accumulated average frame of a short video sequence should be smooth. In [74] Gu et al. model the effect of dust as a composition of dust attenuation and scattering of light then after extracted the attenuation and scattering component, it can eliminate dust effect on the sensors. This method can give good result when video scene is fast changed in a short video sequence. But for long video sequence and light condition changing abruptly the coefficients of attenuation and scattering component will not valid and it leads to an error prone result. [2.4] Changing Viewpoints Li and Brown approach [45] uses the subtle changes in the reflection with respect to the background in a small set of images taken at slightly different viewpoints to tackle the reflection removal problem. The key idea of this approach is the use of SIFT-flow to register images such that a pixel wise comparison can be made across the input set. On the consumption of variant gradient of image belonging to the reflection, this approach optimizes a minimum energy function to acquire the reflection-free image and reflection layer of the image. There are two other approaches: removing specular reflection by moving the position of a light source or camera. [2.5] Based On Focus The approach of depth from focus (DFF) consists of obtaining image slices of the scene (imaging with different focus settings) from which depth is extracted by a search for the slice maximizing a Suresha M and Madhusudhan S 4

5 focus criterion. DFF methods concentrated on cases in which the depth at each point of the image is single valued. For example, looking out of a room window, see both the outside world termed real object and a semi-reflection of objects inside the room termed virtual objects. The treatment of such cases is important. Since, the combination of several unrelated images may greatly degrade the ability to understand them and also confuses autofocusing devices. The detection of the phenomenon also indicates the presence of a transparent surface in front of the camera at a distance closer than the imaged objects. The image is decomposed into layers each with an associated depth and intensity distribution. Schechner et al. [76] adopted common layer representation, in which within each layer the relative depth variations are small compared to the inter-layer difference. [2.6] Using Relative Smoothness Li and Brown [50] addressed extraction of two layers from an image where one layer is smoother than the other. This problem arises in intrinsic image decomposition and reflection interference removal. Michael and Brown introduced a method that regularizes the gradients of the two layers such that one has a long tail distribution and the other a short tail distribution. The problem of layer separation from a single-image with application to 1) intrinsic image decomposition and 2) single image reflection interference removal using focus. Both of these problems take the form: I L1 L2 (1) where I is the observed image and L 1 and L 2 are the combined layers. The intrinsic image model explained by Barrow and Tenenbaum [75] assumes that an image scene is the product of a scene s reflectance and illumination at each pixel, expressed as I = RL, where R is the reflective property or albedo at each pixel and L is the illumination falling on this pixel. Intrinsic image decomposition s aim is to estimate R and L given an input I. This can be reformulated into the form in (1) by taking log, i.e. log(i) = log(r)+log(l). Reflection interference arises when a photo of a scene is taken behind a glass window. This can be expressed as a linear combination of a reflection layer LR and the desired background scene LB, as I = LB + LR. Li and Brown [50] used a slightly modified version based on Schechner et al. [76] proposition of using focus such that the desired layer is more in focus while the reflection is blurred. [2.7] Visual Depth Guided Method Wan et al. [67] idea is to use Depth of Field (DoF) to label the background and reflection edges and proposed a DoF confidence map where pixels with higher DoF values are assumed to belong to the desired background components. Moreover Wan et al. [67] observed that images with different resolutions show different properties in the DoF map. Thus introduced a multiscale DoF computing strategy to classify edge pixels more efficiently. Based on the results of edge classification the background and reflection layers can be separated. The key assumption in this approach is that the photographers always focus on the background in a particular depth when they take photos. Reflections in different depth layers would be blurred in the images. Thus, DoF, the distance between the nearest and farthest objects in a scene that appears reasonably sharp, can be used as an important feature to distinguish background and reflection edges. The existing method can compute a DoF value for a whole image to evaluate its blurred degrees. Wan et al. [67] have proposed a DoF confidence map computing strategy to evaluate the blurred degrees for all pixels and also observed that images with different resolution can exhibit different levels of details in the DoF map. Combining the assumption and observation, developed a multi-scale inference scheme to select background and reflection edges to guide the reflection removal process. With the multi-scale inference scheme can generate the background edge map. Using this map as a mask, introduce a Suresha M and Madhusudhan S 5

6 STUDY ON REFLECTION SEPARATION METHODS framework to refine the initial reflection edges. This framework can well classify the background and reflection edges. [2.8] Using Ghosting Cues Y. C. Shih et al. [61] addressed the reflection removal problem using ghosting effects i.e., multiple reflections on glasses in the captured image. The glass pane at the inner side which is near to observer generates the first reflection, and the other side generates second reflection which is a shifted and attenuated version of the first reflection. The distance between the two reflections depends on the space between the two panes. Ghosting provides a critical cue to separate the reflection and transmission layer, since it breaks the symmetry between the two layers. The model ghosting as convolution of reflection layer R with a kernel k. Then the observed image I is mixture of ghosted reflection layer R and transmission layer T i.e., I T R k. In this method patch based Gaussian Mixture Model priors are used to achieve high quality separation of reflection and transmission images. [2.9] Mean-Shift Decomposition Technique Lahlou and Adjouadi [44] proposed a method to achieve object surface reflectance separation by studying the dissimilarities between the reflectance components distribution in scene images in a delineated on a normalized color space. Eigen decomposition transform (Mean-Shift Decomposition method) provides a direct access to surface shape information through diffuse shading pixels isolation and does not require any local color segmentation process as it differentiates between both reflectance components. This approach relies mainly on image reflectance correlations in defining a suitable linear expression that separates a scene image into two distinct sets of image reflectance. The approach described relies solely on image color information of dichromatic surfaces without requiring segmentation procedures. The reflection separation process has been achieved on uniform surfaces by shifting all specular reflectance image pixels toward the diffuse reflectance distribution using a single constant value for each of the image color channels. For multicolor surfaces, the shifting distance is processed independently for each specular reflectance pixel and correlation of each specular reflectance component. [3] DISCUSSION It is a common assumption in the computer vision community that scene surfaces are of pure diffuse reflection. However, for a wide variety of inhomogeneous materials in the real world, the reflection includes both diffuse and specular components. Hence it is quite possible that the algorithms based on an ideal Lambertian assumption will produce erroneous outputs. Because of its important role in low-level computer vision, in recent years specular reflection has attracted much attention in fields such as edge detection, color constancy, stereo correspondence and photometric stereo. The detection and removal of specular reflections from images have been an area of interest to the Computer Vision community for many years and existing techniques for this task find applications in medical science, video surveillance, image refinement and image reconstruction. Suresha et al. [79] have discussed methods of reflection segmentation, approach and their contribution to enhancement of the reflection separation methods. Table I, Table II gives reflection separation methods and results. Suresha M and Madhusudhan S 6

7 Table 1: Consolidation of Reflection Removal Methods References Approach Contribution S. A. Shafer [1] S. K. Nayar et al. [3] F. Wang et al. [4] G. J. Klinker et al. [5] G. J. Klinker et al. [6] N. Ohnishi et al. [7] 1. Algorithm for determining intrinsic images of reflection from a color image. 1. It gives general reflection model. 2. Mathematical statement for scene phenomenon. 1. Color and Polarization method 1. Separates the specular and diffuse components 1. Based on polarization imaging through 1. Separation of specular reflections. global energy minimization. 1. Color Image analysis and color variation 1. It uses the color difference between object with respect to highlight and shading color and highlight color. 2. Used Color Reflection Model 2. It gives two intrinsic images: one with highlights and other without sed Dichromatic Reflection model. nalyzed and segment real color images by u sing a physics-based color reflection model. 1. Image Segmentation with descriptions of object and highlight colors and intrinsic reflection images. 2. It shows how shading and highlight reflection vary in the image. 1. Polarization method. 1. Separation of reflections and background scenes. 1. Based on dichromatic model and color 1. Separation of specular and diffuse R. Bajcsy et al. [8] space. reflections. H. Farid et al. [9] 1. Optical Techniques 1. Reflections Removal R. O. Dror et al. [10] S. Lin et al. [11] R. Swaminathan et al. [12] P. Tan et al.[13] R. T. Tan et al. [14,15,16] B. Sarel and M. Irani [17] A. Levin et al. [18] S. Umeyama and G. Godin [19] 1. Based on statistical regularities. 1. Reflectance classification under unknown illumination 2. Automatic feature selection 1. Color analysis 1. Separates the specular and diffuse reflection 2. Multibaseline stereo components. 1. Based on behavior of specularity 1. Extract specularity 1. Based on illumination and image 1. Single image highlight removing method. inpainting 1. Inverse Intensity Chromaticity space 1. Estimation of illumination chromaticity 1. Layer Information Exchange 1. Separating two transparent layers. 1. Decomposed image into two that minimize the total amount of edges and corners. 1. Applying ICA to the images observed through a polarizer 2. Statistical analysis 1. Separation of reflection layer 1. Separation of diffuse and specular reflections. R. Feris et al. [20] 1. Multi Flash Camera is used 1. Reduction of reflections in images F. Ortiz et al. [21] 1. Image inpainting method 1. Highlight removal. S. P. Mallick et al. [22] S. P. Mallick et al. [23] R. T. Tan and K. Ikeuchi [24] A. Agrawal et al. [25] F. Ortiz et al. [26] K. J. Yoon et al. [27] 1. Data dependent rotation of RGB Color space 1. Separates diffuse and specular reflection effects 1. Differential morphology. 1. Separating specular and diffuse reflection components. 1. It utilizes the coefficient of reflectance 1. Decomposition of reflection components. basis functions. 1. Gradient Coherence scheme based on 1. This produces better flash images. Gradient Coherence Model 2. Brightness ratio based algorithm 1. Based on Intensity and Saturation of color 1. Detection and elimination of specular Image. reflections. 1. By comparison of local ratios at each 1. Separation of specular-diffuse reflection pixels. components. Suresha M and Madhusudhan S 7

8 STUDY ON REFLECTION SEPARATION METHODS References Approach Contribution Y. Li and L. Ma [28] 1. Total Variation inpainting model. 1. Highlight spot removing P. Tan et al. [29] S. H. Lee et al. [30] E. Angelopoulou [31] A. Levin and Y. Weiss [32] K. Gai et al. [33] K. Gai et al. [34] H. L. Shen et al. [35] B. Lamond et al. [36] H. L. Shen et al. [37] T. Tsuji [38, 39, 40] Q. Yang et al. [41] K. Gai et al. [42] W.K.M. Badawi et al. [43] M. Lahlou et al. [44] Y. Li and M.S. Brown [45] J. Yang et al. [46] 1. Based on diffused texture information. 1. Separating highlight reflections on textured surfaces. 1. Stochastic Approach. 1. Separating specular and diffuse 2. Dichromatic reflectance model. reflection components. 3. Markov random field models. 1. Based on Fresnel Reflection Coefficient. 1. Specularity detection and separation. 1. User labelled gradient points on each layer. 1. Decomposition of layers in 2. Iterative Reweighted Least Square Approach. images. 1. Based on spatial shifts and mixing 1. Separation of mixture of multiple coefficients layer images. 1. Based on movements, mixing coefficient and 1. Separation of mixture of multiple statistical properties. layer images. 1. Based on error analysis of chromaticity. 1. Separation of diffuse and specular reflection components. 1. Based on environmental structured 1. Separation of specular and diffuse illumination. reflections. 1. Based on smooth color transition on along the 1. Separation of specular and diffuse boundary of diffused and specular regions. components in a single image. 1. Based on luminance variation. 1. Separation of specular and diffuse reflection components. 1. Based on estimation of the maximum diffuse 1. Removal of specular reflections. chromaticity values of the specular pixels. 1. Based on statistical properties. 1. Decomposition of layer mixture in images. 1. Blind signal separation. 1. Separation of specular reflection component. 1. Eigen-decomposition transform ( Mean-Shift 1. Separation of Specular and Decomposition MSD method) Diffuse reflectance components. 1. Based on small changes in reflection with 1. Removing of reflection respect to different viewpoints. interference. 1. Based on chromaticity and hue. 1. Separation of specular and diffuse reflection components. H. Kim et al. [47] 1. Maximum a Posteriori Approach 1. Separation of specular reflection. X. Guo et al. [48] N. Kong et al. [49] Y. Li and M. S. Brown [50] 1. Based on correlation, sparsity and independent 1. Reflection separation from priors of the images. superimposed images. 1. Based on polarization. 1. Separation of reflection layer and background. 1. Based on relative smoothness. 1. Separation of image layers. 1. By using visual information. 1. Removal of common visual T. Xue et al. [51] obstructions. C. Xu et al. [52] 1. Based on light field imaging 1. Separation of specular reflection. 1. Based on reflection layer information. 1. Separates reflection layer from D. Prakash et al. [53] image mixtures. M. W. Tao et al. [54] 1. Based on light field data. 1. Separation of specular regions. Q. Yang et al. [55] D. An et al. [56, 57] C. Simon and I. K. Park [58] 1. By using maximum fraction of the diffuse 1. Separation of specular reflections. colour component 1. Based on L2 normalized dichromatic model. 1. Separation of diffuse and specular components. 1. Exploits the spatio-temporal coherence of 1. Separation of reflection layer. reflection. Suresha M and Madhusudhan S 8

9 J. Wang and J. Yang [59] 1. By using sparsity-induced signal decomposition. 1. Removal of reflections from stele images. 1. Utilizes an optimization approach based on the 1. Removal of reflections. T. Sirinukulwattana et probabilistic model of relative smoothness al. [60] algorithm Y. C. Shih et al. [61] 1. Use of ghosting cues that exploit asymmetry 1. Separation of reflection layer. References Approach Contribution between the layers. Y. Liu et al. [62] 1. By preserving the saturation of surface colors. 1. Separation of specular reflections. Y. Akashi and T. Okatani [63] 1. Based on a modified version of sparse nonnegative matrix factorization (NMF). 1. Separation of specular reflections. 1. By using structural similarity both in diffuse and 1. Removal of specular reflections. Y. Zhao et al. [64] specular components with original highlight Image. 1. This method utilizes a principle of estimation 1. Separation of specular reflections. S. Iwata et al. [65] using images with luminances varied by the flickering of a strobe. C. Sun et al. [66] 1. Based on Intensity Score and Aggregate Motion 1. Separation of the background image Score. and reflection. R. Wan et al. [67] 1. Visual depth guided method. 1. A method to remove reflections with a single image is proposed. Z. Y. Huang et al. [68] 1. Based on variance between foreground layer and 1. Dust and reflection removal from background layer. the corrupted video streams. M. Liao et al. [69] 1. GPU-based implementation. 1. Separation of reflections. A. Sulc et al. [70] H. Akbar et al. [71] W. Ren et al. [72] R. T. Tan and K. Ikeuchi [77] H. L. Shen and Z. H. Zheng [78] 1. Based on observation that light field as a sparse linear combination of a constant-color specular term and a small finite set of albedos. 1. Separates a dichromatic reflection component from diffuse object colors for the set of rays in a 4D light field. 1. Thresholding technique. 1. Separation of diffuse and specular reflection in dichromatic reflection objects. 1. Global color-lines constraint from dichromatic 1. Separation of specular and diffuse reflection model. reflections. 1. Based on Chromaticity 1. Separation of diffuse and specular reflection components. 1. Based on pixel intensity ratio. 1. Separation of specular reflection in single image. Table 2: Consolidation of Reflection Removal Methods Method and Example Input Image Result Polarization method e.g. F. Wang et al. [4] Flash / No Flash image pairs e.g. R. Feris et al. [20] Suresha M and Madhusudhan S 9

10 STUDY ON REFLECTION SEPARATION METHODS Video sequence e.g. T. Xue et al. [51] Changing Viewpoints e.g. Y. Li and M.S. Brown [45] Method and Example Input Image Result Based on user markup e.g. A. Levin and Y. Weiss [32] Based on focus e.g. Y. Y. Schechner et al. [76] Using Relative smoothness e.g. Y. Li and M. S. Brown [50] Visual depth guided method e.g. R. Wan et al. [67] Using ghosting cues e.g., Y. C. Shih et al. [61] Mean-Shift Decomposition Technique e.g., Lahlou and Adjouadi [44] [6] CONCLUSION Specular reflection often has some negative effects on visual quality and degrades the performance of various computer vision algorithms, such as image segmentation, color constancy, object detection, and visual tracking. Segmentation of reflection part is challenging task and it is a preprocessing step in the image processing. In this work we have focused on different methods to Suresha M and Madhusudhan S 10

11 segment reflections in images and tabulated many computer vision scientists contributions and also their approach. ACKNOWLEDGEMENTS This work was supported in part by Kuvempu University, Shankaraghatta, Karnataka, India. REFERENCES [1] S. A. Shafer, Using color to separate reflection components, University of Rochester, (1984). [2] O. Morel, F. Meriaudeau, C. Stolz, and P. Gorria, Polarization imaging applied to 3d reconstruction of specular metallic surfaces, International society for optics and photonics, In Electronic Imaging, pages , (2005). [3] S. K. Nayar, X. Fang, T. Boult, Separation of reflection components using color and polarization, International journal of computer vision 21, pp , (1997). [4] F. Wang, S. Ainouz, C. Petitjean, and A. Bensrhair, Specularity removal: A global energy minimization approach based on polarization imaging, Computer vision and image understanding, (2017). [5] G. J. Klinker, S. A. Shafer, and T. Kanade, Using a color reflection model to separate highlights from object color, IEEE London, Proc. 1st Intern. Conf. Comp. Vision, pp , (1987). [6] G. J. Klinker, S. A. Shafer, and T. Kanade, A physical approach to color image understanding, Int. J. Computer Vision 4, pp. 7-38, (1993). [7] N. Ohnishi, K. Kumaki, T. Yamamura, and T. Tanaka, Separating real and virtual objects from their overlapping images, In ECCV, (1996). [8] R. Bajcsy, W. L. Sang, and A. Leonardis, Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation, International Journal of Computer Vision, vol. 17, no. 3, pp , (1996). [9] H. Farid and E. H. Adelson, Separating reflections from images using independent component analysis, In journal of the optical society of america, (1999). [10] R. O. Dror, E. H. Adelson and A. S. Willsky, Surface reflectance estimation and natural illumination statistics, ICCV, (2001). [11] S. Lin, Y. Li, S. B. Kang, X. Tong and H. Y. Shum, Diffuse-specular separation and depth recovery from image sequences. In proceedings of european conference on computer vision (ECCV), pp , (2003). [12] R. Swaminathan, S. B. Kang, R. Szeliski, A. Criminisi, and S. K. Nayar, On the motion and appearance of specularities in image sequences, ECCV 2002, pp , (2002). [13] P. Tan, S. Lin, L. Quan and H. Y. Shum, Highlight removal by illumination-constrained inpainting, In proceeding IEEE International conference on computer vision (ICCV), vol. 1, pp , (2003). [14] R. T. Tan, K. Nishino and K. Ikeuchi, Illuminant chromaticity estimation using inverse-intensity chromaticity space, In proceedings of the IEEE computer vision and pattern recognition conference, pp , (2003). [15] R. T. Tan, K. Nishino and K. Ikeuchi, Color constancy through inverse-intensity chromaticity space, J. Opt. Soc. Am Vol. 21, No. 3, (2004). [16] R. T. Tan, K. Nishino and K. Ikeuchi, Separating reflection components based on chromaticity and noise analysis, IEEE Transaction on pattern analysis and machine intelligence PAMI 10, pp , (2004). Suresha M and Madhusudhan S 11

12 STUDY ON REFLECTION SEPARATION METHODS [17] B. Sarel and M. Irani, Separating transparent layers through layer information exchange, ECCV volume 4, pp , (2004). [18] A. Levin, A. Zomet and Y. Weiss, Separating reflections from a single image using local features, Proceedings of the IEEE computer society conference on computer vision and pattern recognition CVPR, (2004). [19] S. Umeyama and G. Godin, Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images, IEEE Transaction on pattern analysis and machine intelligence, pp , (2004). [20] R. Feris, R. Raskar, K. H. Tan and M. Turk, Specular reflection reduction with multi-flash imaging, Proceedings of the XVII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 04), (2004). [21] F. Ortiz and F. Torres, A new inpainting method for highlights elimination by colour morphology, ICAPR, (2005). [22] S. P. Mallick, T. E. Zickler, D. J. Kriegman and P. N. Belhumeur, Beyond lambert: reconstructing specular surfaces using color, IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, (2005). [23] S. P. Mallick, T. Zickler, P. N. Belhumeur and D. J. Kriegman, Specularity Removal in Images and Videos: A PDE Approach, Computer Vision ECCV [24] R. T. Tan and K. Ikeuchi, Reflection components decomposition of textured surfaces using linear basis functions, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2005). [25] A. Agrawal, R. Raskar, S. K. Nayar, and Y. Li, Removing photography artifacts using gradient projection and flash-exposure sampling, ACM Transactions on Graphics (TOG), vol. 24, no. 3, pp , (2005). [26] F. Ortiz and F. Torres, Automatic detection and elimination of specular reflectance in color images by means of ms diagram and vector connected filters, IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, Vol. 36, no. 5, (2006). [27] K. Yoon, Y. Choi and I. S. Kweon, Fast separation of reflection components using a specularityinvariant image representation, IEEE ICIP, (2006). [28] Y. Li and L. Ma, Metal highlight spots removal based on multi-light-sources and total variation inpainting, Conference: Proceedings VRCIA 2006 ACM International Conference on Virtual Reality Continuum and its Applications, Chinese University of Hong Kong, Hong Kong, China, (2006). [29] P. Tan, S. Lin and L. Quan, Separation of highlight reflections on textured surfaces, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2006). [30] S. H. Lee, H. Koo, N. I. Cho and J. Park, Stochastic approach to separate diffuse and specular reflections, IEEE ICIP, (2006). [31] E. Angelopoulou, Specular highlight detection based on the fresnel reflection coefficient, IEEE, (2007). [32] A. Levin and Y. Weiss, User assisted separation of reflections from a single image using a sparsity prior, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 9, (2007). [33] K. Gai, Z. Shi and C. Zhang, Blindly separating mixtures of multiple layers with spatial shifts, IEEE, (2008). [34] K. Gai, Z. Shi and C. Zhang, Blind separation of superimposed images with unknown motions, IEEE, (2009). [35] H. L. Shen, H. G. Zhang, S. J. Shao and J. H. Xin, Chromaticity-based separation of reflection components in a single image, The Journal of Pattern Recognition, (2008). Suresha M and Madhusudhan S 12

13 [36] B. Lamond, P. Peers, A. Ghosh and P. Debevec, Image-based separation of diffuse and specular reflections using environmental structured illumination, IEEE, (2009). [37] H. L. Shen and Q. Y. Cai, Simple and efficient method for specularity removal in an image, Optical Society of America, (2009). [38] T. Tsuji, High-speed stroboscope for specular reflection removal of dc illumination, The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems October 18-22, Taipei, Taiwan, (2010a). [39] T. Tsuji, Specular reflection removal on high-speed camera for robot vision, IEEE International Conference on Robotics and Automation Anchorage Convention District May 3-8, 2010, Anchorage, Alaska, USA, (2010b). [40] T. Tsuji, An Image-Correction Method for Specular Reflection Removal Using a High-speed Stroboscope, IEEE, (2011). [41] Q. Yang, S. Wang, and N. Ahuja, Real-time specular highlight removal using bilateral filtering, Computer Vision ECCV, (2010). [42] K. Gai, Z. Shi, and C. Zhang, Blind separation of superimposed moving images using image statistics, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 1, (2012). [43] W.K.M. Badawi, C.C. Chibelushi, M.N. Patwary and M. Moniri, Specular-based illumination estimation using blind signal separation techniques, IET Image Process., Vol. 6, Iss. 8, pp , (2012). [44] M. Lahlou and M. Adjouade, Surface reflectance components separation from single color images using the mean-shift decomposition technique, International Journal of Innovative Computing, Information and Control Vol 8, Number 7(B), (2012). [45] Y. Li and M. Brown, Exploiting reflection change for automatic reflection removal, Proceedings of the 2013 IEEE International Conference on Computer Vision ICCV, (2013). [46] J. Yang, L. Liu and S. Z. Li, Separating specular and diffuse reflection components in the HSI color space, IEEE ICCV, (2013). [47] H. Kim, H. Jin, S. Hadap and I. Kweon, Specular reflection separation using dark channel prior, IEEE CVPR, (2013). [48] X. Guo, X. Cao and Y. Ma, Robust separation of reflection from multiple images, IEEE CVPR (2014). [49] N. Kong, Y. W. Tai and J. S. Shin, A physically-based approach to reflection separation: from physical modeling to constrained optimization, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 36, Issue: 2, (2014). [50] Y. Li and M. S. Brown, Single image layer separation using relative smoothness, IEEE CVPR, (2014). [51] T. Xue, M. Rubinstein, C. Liu and W. T. Freeman, A computational approach for obstruction-free photography, ACM transactions on graphics, Vol. 34, No. 4, Article 79, (2015). [52] C. Xu, X. Wang, H. Wang and Y. Zhang, Accurate image specular highlight removal based on light field imaging, IEEE VCIP, (2015). [53] D. Prakash, P. Kalwad, V. Peddigari and P. Srinivasa, Automatic reflection removal using reflective layer image information, IEEE, (2015). [54] M. W. Tao, J. C. Su, T. C. Wang, J. Malik and R. Ramamoorthy, Depth estimation and specular removal for glossy surfaces using point and line consistency with light field cameras, IEEE Pattern Analysis and Machine Intelligence, (2015). [55] Q. Yang, J. Tang and N. Ahuja, Efficient and robust specular highlight removal, IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: 37, Issue: 6, (2015). Suresha M and Madhusudhan S 13

14 STUDY ON REFLECTION SEPARATION METHODS [56] D. An, J. Suo, X. Ji, H. Wang and Q. Dai, Fast and high quality highlight removal from a single image, Computer Vision and Pattern Recognition, (2015). [57] J. Suo, D. An, X. Ji, H. Wang and Q. Dai, Fast and high quality highlight removal from a single image, IEEE Transactions on Image Processing, Vol. 25, No. 11, (2016). [58] C. Simon and I. K. Park, Reflection removal for in-vehicle black box videos, IEEE CVPR, (2015). [59] J. Wang and J. Yang, Reflection removal for stele images via sparse signal decomposition, IEEE VCIP, (2015). [60] T. Sirinukulwattana, G. Choe, and I. S. Kweon, Reflection removal using disparity and gradientsparsity via smoothing algorithm, IEEE ICIP, (2015). [61] Y. C. Shih, D. Krishnan, F. Durand and W. T. Freeman, Reflection removal using ghosting cues, IEEE CVPR, (2015). [62] Y. Liu, Z. Yuan, N. Zheng and Y. Wu, Saturation-preserving specular reflection separation, IEEE CVPR, (2015). [63] Y. Akashi and T. Okatani, Separation of reflection components by sparse non-negative matrix factorization, Computer Vision and Image Understanding, Elsevier, (2015). [64] Y. Zhao, Q. Peng, J. Xue and S. G. Kong, Specular reflection removal using local structural similarity and chromaticity consistency, IEEE ICIP, (2015). [65] S. Iwata, K. Ogata, S. Sakaino and T. Tsuji, Specular reflection removal with high speed camera for video imaging, IEEE IECON, (2015). [66] C. Sun, S. Liu, T. Yang, B. Zeng, Z. Wang and G. Liu, Automatic reflection removal using gradient intensity and motion cues, Proceedings of the ACM on Multimedia Conference, pp , (2016). [67] R. Wan, B. Shi, T. A. Hwee and A. C. Kot, Depth of field guided reflection removal, IEEE ICIP, (2016). [68] Z. Y. Huang, B. Xiong, C. Tian, J. Zhan, X. Fei and N. Shah, Dust and reflection removal from videos captured in moving car, IEEE International Conference on e-business Engineering, (2016). [69] M. Liao, C. Lv, G. Li, J. Lin and X. Gao, Reflection removal using ghosting cues based on GPU via CUDA, The 11th International Conference on Computer Science & Education (ICCSE 2016) August 23-25, Nagoya University, Japan. IEEE, (2016). [70] A. Sulc, A. Alperovich, N. Marniok and B. Goldluecke, Reflection separation in light fields based on sparse coding and specular flow, Eurographics Proceedings, (2016). [71] H. Akbar and N. S. Herman, Removal of highlights in dichromatic reflection objects using segmentation and inpainting, IEEE International Conference on Robotics, Automation and Sciences (ICORAS), (2016). [72] W. Ren, J. Tian and Y. Tang, Specular reflection separation with color-lines constraint, IEEE Transactions on Image Processing, (2017). [73] L. B. Wolff and T. E. Boult, Constraining object feature using a polarization reflectance model, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 7, pp , (1991). [74] J. Gu, R. Ramamoorthi, P. Belhumeur, and S. Nayar, Removing image artifacts due to dirty camera lenses and thin occluders, In ACM Transactions on Graphics (TOG), vol. 28, no. 5. ACM, p. 144, (2009). [75] H. G. Barrow and J. M. Tenenbaum, Recovering intrinsic scene characteristics from images, In Computer Vision Systems, (1978). [76] Y. Y. Schechner, N. Kiryati and R. Basri, Separation of transparent layers using focus, In Proc. IEEE International Conf. on Computer Vision, Mumbai, India, pp , (1998). Suresha M and Madhusudhan S 14

15 [77] R. T. Tan and K. Ikeuchi, Separating reflection components of textured surfaces using a single image, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 2, (2005). [78] H. L. Shen and Z. H. Zheng, Real-time highlight removal using intensity ratio, Applied Optics, vol. 52, no. 19, pp , (2013). [79] M. Suresha and S Madhusudhan, Survey on Reflection Removal Methods, National Conference on Computational Sciences and Soft Computing (NCCSSC 2017). Author[s] brief Introduction Dr. Suresha M received the BSc degree in Physics, Chemistry and Mathematics from Kuvempu University, India during 2000 and MCA and Ph.D., from Kuvempu University during 2003 and 2014 respectively. Since 2007, he has been with the Department of Computer Science and MCA, Kuvempu University, India, where he is currently working as Assistant professor. His research interests include Image Processing, Pattern Recognition and Machine Learning. Madhusudhan S obtained his B.E. (Information Science) and M.Tech (Computer Science) during 2004 and 2006 from VTU, Belgaum and Kuvempu University, India, respectively. Currently he is pursuing Ph.D. in Kuvempu University, India. His research area is Digital Image Processing and Pattern Recognition. Suresha M and Madhusudhan S 15