CONTRAST ENHANCEMENT ALGORITHMS FOR FOGGY IMAGES

Transcription

1 International Journal of Electrical and Electronics Engineering Research (IJEEER) ISSN(P): X; ISSN(E): X Vol. 6, Issue 5, Oct 2016, 7-16 TJPRC Pvt. Ltd CONTRAST ENHANCEMENT ALGORITHMS FOR FOGGY IMAGES NAMRATA KHAPARDE 1 & S. S. AGRAWAL 2 1 ME Student, Signal Processing (E&TC), Smt. Kashibai Navale College of Engineering, Pune, India 2 Assistant Professor, Department of E&TC, Smt. Kashibai Navale College of Engineering, Pune, India ABSTRACT An increasing number of applications are making use of image processing technique. It is a non invasive and robust technique. Images are used in the modern applications such as object and lane detection, traffic surveillance, satellite, military and medical imaging need to have clear images that convey correct information. Capturing of image in fog conditions reduces visibility. It degrades quality and contrast of the image. Contrast of the image can be enhanced using modeling methods. A single foggy image is used as input to this work. Translated exponential functions are implemented to find the atmospheric veil and enhancement of contrast. Single foggy image is used as input to the algorithm that takes into account the exponential decay of the fog. In addition to the translated exponential function, some other algorithms have also been implemented in this work for comparison purpose. KEYWORDS: Image Processing, Fog, Contrast Enhancement; Exponential Decay, Translated Exponential Functions Received: Aug 18, 2016; Accepted: Aug 29, 2016; Published: Sep 07, 2016; Paper Id.: IJEEEROCT20162 INTRODUCTION Image processing is a technique that finds applications in almost every field due to its robust nature. Image is used in increasing number of fields such as traffic surveillance, navigation, lane marking identification, satellite imagery and military applications etc. Weather related optical threats cause decreased visibility in case of snow, heavy rain, mist, haze, fog etc. Fog is tiny water droplets suspended in the air. Fog is the most dangerous condition under the weather related optical threats found in cloudy weather condition. The fog coupled with light pose problems for visibility, in driving scenario and in image processing applications. Fog reduces visibility to less than a kilometer. Driving in such condition is next to impossible. It causes accidents and casualties. Most percentages of accidents are caused due to fog. Fog paired with light causes additional lighting in the parts of object where light is actually not present. It happens because of scattering of light through the water droplets in the air. This effect is called as air-light effect. Another name for air-light effect is the atmospheric veil. Original Article Effect of fog on images plays important role in image processing applications. The images captured in bad weather conditions lose their authenticity in terms of contrast and color. The images captured in fog appear blurry and hazy. These images have increased whiteness. Natural colors of the original scene are also altered. Due to the decreased visibility, objects at a considerable distance are not visible at all. Thus it becomes important to remove fog from the image to preserve identity and authenticity of the image for providing true and whole information of the scene. Visibility enhancement techniques in foggy weather conditions are color enhancement based and contrast enhancement based. Color enhancement techniques stress on increasing the color of objects present. editor@tjprc.org

2 8 Namrata Khaparde & S. S. Agrawal This technique is not feasible in the case of fog where at greater distances the object is not visible at all. In such case, enhancing the colors will not be beneficial for reduced visibility. Synthetically enhancing the colors does not preserve the natural colors of the image. Thus, reconstructed image will not be close to the original scene. Next approach is to enhance the contrast of the image. Contrast is an important character of images. It means the range between the brightest grey level and the lowest grey level. When outdoor images are captured in fog, scattering of transmitted light due to atmospheric particles of fog degrades the contrast of these images. Hence, different methods are designed to restore contrast in images. Scattering of light accompanied with fog pose many problems in daily life. It is necessary to know fog and its properties with light in order to identify and remove fog. Some of the points that are important for this purpose are enlisted below: Presence of fog reduces visibility range. Blocked or blurred view in front of the camera. Accidents and casualties are caused due to reduction in visibility. It poses safety concerns in driving. Objects at a greater distance are no longer visible. Images taken under foggy conditions have degraded quality and contrast. Fog has an exponential increase with increase in the distance. Mathematical model must be implemented that takes into account the exponential nature of the fog. When a scene is capture, the scene close to the camera is clearer and the density of the fog increases with increasing distance from the camera. Image is a three dimensional scene converted to two dimensional form. Due to the exponential nature of fog, the bottom part of the image is clearer and uppermost part is white. Related Work It is not sufficient to only classify the images. Identification of fog and its removal is necessary to enhance contrast of images. Contrast is an important part for portraying the visual quality and details in an image. Contrast enhancement methods are broadly categorized as non-model based methods and model based methods. Model based methods are further classified as algorithms with given depth algorithms when depth not given. Figure 1 shows a tree diagram for the classification of contrast restoration techniques. Non-model based techniques for contrast enhancement do not use any specific model for solving the problem of contrast enhancement. In this method no extra information from the user is needed. Non model based perform enhancement using information from the image itself. Non-model based methods use mostly histogram equalization techniques and its various forms. Non model based techniques improve contrast only till a certain level. There is no scope for further enhancement after image has been enhanced to a certain level. Impact Factor (JCC): NAAS Rating: 2.40

3 Contrast Enhancement Algorithms for Foggy Images 9 Figure 1: Classification of Contrast Enhancement Methods Color fidelity is not maintained using these methods and alterations have to be made to reduce the noise content in reconstructed image. Model based contrast enhancement techniques are based on a physical model. A real world problem is converted into a mathematical model. It is then analyzed and solution to the problem is formed. These methods require extra information from the user. Physical world problem is formulated in the understandable form using languages. Next, a mathematical model is created which represents the real world problem. It makes the analysis and understanding of the problem easier because mathematical modelling converts the real world problem in a measurable form using parameters. It helps to understand the extent and characteristics of the problem at hand. Model based methods use mathematical and physical models to find solution to a real world problem. These methods are classified as model based methods with given depth and model based methods with unknown depth. The depth is taken as input from the user. These methods may be erroneous when the value of depth from user is not authentic. It also requires interaction from the user. Model based method where depth is unknown formulate their own technique for calculating depth of a scene. Extensive research has been carried out for defogging of images. The existing algorithms work on fog removal by consideration of atmospheric veil as one entity. Then the atmospheric veil or whiteness is subtracted from image. As image pixels and intensities largely vary throughout the image, this concept will not work. Some other algorithms directly assign a black or a white value to pixels in the case when the edges or corners are not preserved. No Black Pixel Constraint (NBPC) algorithm tries to minimize these directly assigned black and white pixel values. This algorithm acts as a base for many other algorithms used. Different algorithms are either combined or used in accordance with each other to combine the advantages of different methods. Planar assumption (PA) is a technique that takes into account the partition of image as road surface and sky. Planar assumption is combined with NBPC to improve its results. Proposed Methodology The main aim of carrying out this work is to study the effect of fog on the contrast of captured images. Various techniques and methods are available for contrast enhancement and restoration of images under fog. The goal is to learn of these different methods which provide clear image in bad weather conditions and restore the original image by considering the exponential decay of the fog and its effect on the contrast of the images. The main objective is to study and implement the mathematical model for restoration of contrast in images taking into account the atmospheric veil. Different techniques and methods have been proposed and implemented over the years for contrast enhancement. The algorithms implemented in this work are no black pixel constraint (NBPC), no black pixel constraint with planar assumption (NBPCPA or NBPC with PA), no black pixel constraint with squared and modulus exponential functions editor@tjprc.org

4 10 Namrata Khaparde & S. S. Agrawal (NBPC+f s and NBPC+f m respectively), and no black pixel constraint with squared and modulus exponential functions (NBPC+G s and NBPC+G m respectively). These methods are discussed in detail in the following sections. The system block diagram is as shown in Figure 1. Compute Dark Channel Prior: Input to the system is a gray image. A color image is converted to gray image by extracting each of its red green and blue components. This image is called the dark channel prior. Noise Removal and Edge Preserving: Local average filter has a problem that it does not preserve edges. A median filter preserves edges. Noise removal is also performed by median filtering. Thus, a median filtering is applied on the W image. Median Filter along the Column: Effect of fog is greater in the upper part of the image and the bottom part of the image is comparatively clearer. Hence, median filtering along the columns gives good results. Calculate Atmospheric Veil: The algorithms explained in the next sections are used here for image restoration. Image Restoration: Gamma correction is used to restore the natural colors of the image. Image Post Processing: Lastly, Normalization is carried out on the restored image and tone mapping is done. This is done so that the restored image must resemble the colors in original scene. Figure 2: System Block Diagram No Black Pixel Constraint The goal of the algorithm is the reduction in the number of black pixels in the enhanced image. No Black Pixel constraint states that the average of pixels around a pixel position in of the enhanced image must be greater than the standard deviation of those pixels in the enhanced image. The main step for fog removal and visibility enhancement is calculating the intensity of the atmospheric veil in an image. The input foggy image to the algorithm is a normalized image. The maximum intensity of the input image can be seen as corresponding to maximum sky intensity. Hence, the maximum intensity of the input image is set as one. NBPC can be implemented using the atmospheric veil and the input image only. To enhance the fog free image obtained, a parameter called percentage of enhancement, p is used. The more the percentage of enhancement, the clearer the image will be. However, the extent of enhancement also depends on the algorithm being used. This percentage is normally taken between 95 to 99 percent. Local average is calculated as the median of the local intensities in a window of size s v and the standard deviation as the median of the absolute differences between the intensities and the local average using same window size. Impact Factor (JCC): NAAS Rating: 2.40

5 Contrast Enhancement Algorithms for Foggy Images 11 No Black Pixel Constraint with Planar Assumption The points of the road surface in an image can be used to relate a distance d with each vertical line of the image. This assumption is called as Planar Assumption. Planar assumption depends on the calculation of the depth map of an image. It is dependent on the intrinsic and extrinsic parameters of the camera, the horizon line in an image and each vertical line in an input image. Using NBPC algorithm for fog detection does not detect objects located at farther distance and output is not smooth. To overcome this drawback, NBPC is used with planar assumption and it is commonly referred to as NBPCPA or NBPC+PA. Due to fog the visibility distance is reduced between camera and captured scene on the road. NBPCPA considers this distance for calculation of atmospheric veil. Practically, calculation of a distance less than 60 m is rarely found. Thus, when the camera parameters and the position of the horizon line in input image is known, NBPC is combined with planar assumption to find the atmospheric veil at a known distance of 60 m. No Black Pixel Constraint with Exponential Function Fog has non-linear increase with the increase in the distance between the camera and far point. It has low density near the vicinity of the camera and is denser as the distance increases from the camera. Hence it is needed to take into account this exponential nature of fog. For this purpose the NBPC algorithm is further modified as NBPC with the partition of unity functions. This method acts as an exponential filter that treats atmospheric veil as a whole. The function used with NBPC is the squared exponential function, f s and modulus exponential function, f m. Both these functions range from 0 to 1 since the input image is also normalized. Calculation of f s and f m require a parameter, a that depends on the visibility distance and is based on density of fog. Mathematically squared and modulus exponential functions are given as: = (1) = (2) Here, x represents image lines, H is the height of the object and parameter a is used in accordance with the density of the fog. It also controls the shape of the exponential function. The parameters f so (0) and f mo (0) are the squared and modulus partition of unity functions respectively. These are used to keep the exponential in the range from 0 to 1. Mathematically they are given as: = (3) = (4) Figure 3: Translation of Exponential Functions on X Axis editor@tjprc.org

6 12 Namrata Khaparde & S. S. Agrawal No Black Pixel Constraint with Exponential Function NBPC with f s and f m improve results than NBPC combined with PA. Image contrast can be further improved and reconstructed image can be made as close to original image as possible by translating the exponential functions. Translation is done on the X- axis. When a picture is taken in fog conditions, the bottom part of the image that is near to the camera is clearer. The point till which the image intensity matches the sky intensity is given by the vertical horizon line, vh. The image intensity is maximum in this region. From this point the fog has an exponential decay. The clear part is observed at bottom of the image. The point from where the image begins to be clear is taken as Max. The value of Max for every image can be different depending on the amount of fog under which the image is captured. The values for Max that are generally chosen are (1/4), (1/2), (3/4). The whole translation on the x-axis is explained in Fig. 3. It depicts the line segment showing the translation of fog on x axis. Mathematically translated squared exponential function G s is calculated as: 1 "h $% "h' "h, *+ 0 -.h/0 1 2 (5) And c is calculated as: % = (6) To find translated modulus exponential function, G s, f m is used in the same manner as that for cal culation of G m Experimentation This work has been carried out in MATLAB tool 2.10GHz i3 processor. The input images are taken from FRIDA dataset. Implementing the NBPC, NBPC with PA, NBPC with f s and NBPC with f m n images from FRIDA dataset yields the results as shown in fig. 4. Percentage of restoration used for NBPC and NBPCPA is 95% and that used for NBPC with f s, G s and with f m, G m is 99%. For NBPCPA the values of camera parameter and horizon line used are 300 and 300 respectively. The minimum visibility distance is taken as 60. For the NBPC with exponential functions, value of a is taken as 4.5. For quality analysis, each algorithm is trained with the input images having different fog conditions and an absolute difference is calculated taking the clear day images as ground truth images. Four cases of fog are taken in foggy input images that are: homogeneous, heterogeneous, cloudy homogeneous and cloudy heterogeneous type of fog. 18 images for every case of fog are evaluated. The original images are taken as the ground truth images. The output of each algorithm for every case of fog gives the enhanced image. This image is considered as the subject image that has to be compared to the ground truth image. Absolute difference of the pixel values at the same pixel positions in both images is calculated. This absolute difference needs to be aggregated, hence the obtained SAD are summed. RESULTS AND DISCUSSIONS The results for the synthetically added fog in image and for the real life road scene image are shown in Figure 4. Figure 4(a) represents the input foggy image. Fig. 4(b) shows results for NBPC algorithm. This algorithm does not give very clear visibility. This algorithm compromises on the results when a large uniform grey area is present in the image. The resultant image becomes dark. Result of NBPCPA algorithm is shown in Fig. 4(c). As this algorithm takes into account the planar assumption, the bottom part of the image that mostly consists of road is enhanced better than the rest of Impact Factor (JCC): NAAS Rating: 2.40

7 Contrast Enhancement Algorithms for Foggy Images 13 the image. The upper part of the image is left darker than the result from the NBPC algorithm. Fig. 4(d) and 4(e) show results for exponential f s and f m respectively. The results obtained are better than the previous two algorithms. Darkness in these results is reduced as compared to results of NBPC and NBPCPA. The results obtained from operation of translated exponential functions on a foggy image are shown in Fig. 4(f) and 4(g). These results look more close to the original scene. Darkness is the least in these results. The value of a is used as 4.5 for the implementation of all the above algorithms. In addition to implementation of the above mentioned algorithms for enhancement of contrast of foggy images, the results are obtained for three different values of parameter a. Fig. 5(a), 5(b) and 5(c) show the results for the values of a as 1, 3.5 and 7 for implementation of and G s. The same is shown for G m in Fig. 5(d), 5(e) and 5(f). The value of a=1 yields an image where the bottom part of the image is lighter compared to the whole image. a=3.5 gives an image with over compensated bottom part of the image and hence the bottom part is still darker than the rest of the image. Value of a=7 gives better results than the other two values used. Figure 4(A): Shows Input Foggy Image.Figure 4(B): To 4(G) Showing Results of NBPC, NBPCPA, NBPC+F s,nbpc+f m, NBPC+G s, NBPC+G m Figure 5: Variation In Values of A. From (A) To (C) Variation in G s With A=1, A=3.5, A= 4.5; (D) To (F) Variation InG m With A=1, A=3.5, A= 4.5 The algorithms are evaluated based on SAD. Evaluation is done on 18 images from each case of fog for every algorithm. The values obtained of SAD for, homogeneous, heterogeneous, cloudy homogeneous and cloudy heterogeneous conditions are calculated for all algorithms separately. The maximum and minimum values of SAD obtained using all six algorithms for all four cases of fog is given in table 1. Table 1: Summary of SAD Values on 18 Images between Enhanced Images and Ground Truth Images without Fog, for all Six Algorithms, and for the Four Types of Synthetic Fog Algorithm Homogeneous Heterogeneous Cloudy Cloudy Homogeneous Heterogeneous All Types NBPC ± ± ± ± ± NBPC PA ± ± ± ± ± NBPC +fs ± ± ± ± ± NBPC+fm ± ± ± ± ± NBPC+Gs ± ± ± ± ± NBPC+Gm ± ± ± ± ± Figure 6 shows the plot of maximum value of SAD for the four conditions of fog for all algorithms. The plot shows that the maximum values for all conditions is obtained using NBPC algorithm. NBPC PA algorithm gives the minimum value of SAD for heterogeneous fog condition. However, the subjective analysis of the output images shows editor@tjprc.org

8 14 Namrata Khaparde & S. S. Agrawal that using NBPC PA algorithm a part of the image is not enhanced correctly. Also the values for cloudy heterogeneous and cloudy homogeneous condition of fog is maximum using this algorithm. SAD obtained using NBPC with exponential fs, fm, Gs and Gm for homogeneous fog conditions is near about same. Though for the other three conditions it is less for exponential fs. The values for Exponential Gs and Gm for all conditions do not vary much. Figure 7 shows the plot of minimum value of SAD for the four conditions of fog for all algorithms. Maximum value of all is obtained foe NBPC algorithm. And minimum values are obtained for exponential fs and fm algorithms. The values for Gs and Gm do not vary very much for minimum values of SAD as well. Thus, through subjective and objective analysis of results, Exponential fs, fm, Gs and Gm algorithms can be used as pre-processing steps in image processing applications depending upon the fog and the requirement of the application. Figure 6: Graph of Maximum Values of SAD for All Algorithms for Four Conditions of Fog Figure 7: Graph of Minimum Values of SAD for All Algorithms for Four Conditions of Fog CONCLUSIONS Six algorithms are implemented for this work. First algorithm is NBPC that results in a dark image and gives very less improvement in contrast. For second algorithm NBPC is combined with PA. The results are darker than NBPC results in the upper part of the image. The contrast of the bottom part of the image is enhanced better than the rest of the image. In next algorithm squared exponential functions (f s ) and modulus (f m ) exponential functions are implemented to Impact Factor (JCC): NAAS Rating: 2.40

9 Contrast Enhancement Algorithms for Foggy Images 15 take into account the increasing exponential nature of the fog with increasing distance from the camera. The results of these algorithms are better than that obtained from NBPC and NBPCPA. The results of f s and f m are further increased by translating the exponential functions on the x axis. The results of squared and modulus translated exponential functions for different values of parameter a that depends on the density of the fog in input image is implemented. The results are better for value of a=4.5 and 7 than the value of a=1. The objective analysis using SAD values shows that exponential functions and translated exponentail functions enhances the contrast of foggy input image. The output image is closer to the input foggy image using these algorithms as compared to the traditional methods used for contrast enhancement. REFERENCES 1. Mihai Negru, Sergiu Nsedevschi, Exponential Contrast Restoration in Fog Conditions for Driving Assisstance, IEEE Trans.International Transportation systems, ISSN Feb Garima Yadav, Saurabh Maheshwari, Anjali Agarwal, Fog Removal Techniques from Images: A Comparative Review and Future Directions, in IEEE Conf on Signal Propagation and Computer Technology (ICSPCT), pp , ISBN , July M. Pavlic, H. Belzner, G. Rigoll, and S. Ilic, Image based fog detection in vehicles, in Proc. IEEE IV, pp , Jun. 2012, 4. M. Pavlic, G. Rigoll, and S. Ilic, Classification of images in fog and fog-free scenes for use in vehicles, in Proc. IEEE IV, pp , Jun M. Negru and S. Nedevschi, Image based fog detection and visibility estimation for driving assistance systems, in Proc. IEEE Int. Conf. ICCP, pp , Sep N. Hautiere, J.-P. Tarel, J. Lavenant, and D. Aubert, Automatic fog detection and estimation of visibility distance through use of an onboard camera, Mach. Vis. Appl., vol. 17, no. 1, pp. 8 20, Apr J. P. Tarel et al., Vision enhancement in homogeneous and heterogeneous fog, IEEE Intell. Transp. Syst. Mag., vol. 4, no. 2, pp. 6 20, ISSN , Summer K. Zuiderveld, Contrast limited adaptive histogram equalization, in Graphics Gems IV, P. S. Heckbert, Ed. San Diego, CA, pp , USA: Academic, K. Tan and J. Oakley, Enhancement of color images in poor visibility conditions, in Proc. Int. Conf. Image Process., vol. 2, pp , ISSN , Sep editor@tjprc.org

10