Automatic Photograph Enhancement

Transcription

1 Automatic Photograph Enhancement Subhransu Maji University of California at Berkeley Spring 07 Abstract We propose a method of image enhancement, based on a spatial domain technique called histogram equalization. Traditional histogram equalization is only useful for images with backgrounds and foregrounds that are both bright or both dark, whereas our method works for images with high dynamic range. The technique is to segment the image into large regions with high contrast separations which are then enhanced locally. The general problem of finding balanced cuts is NP-Hard and for the special case where the partitions have a special structure we give an polynomial time algorithm for finding good separations. Finally we demonstrate the algorithm on an example. 1 Introduction The aim of image enhancement is to improve the interpretability or perception of information in images for human viewers, or to provide better input for other automated image processing techniques. The very nature of the problem makes it ill posed, as there is no objective measure of quality. Spatial domain techniques enhance the image by finding a transform, T (f(x, y)) = g(x, y), in the image space. Examples include mean and median filters which are used for reducing noise in images. Frequency domain techniques work on the fourier transform of the image. Examples include low pass filtering (analogous to blurring), Homomorphic filtering. Details of various techniques of image enhancement can be found in [10] and [11]. In this work we use a spatial domain filter called histogram equalization. Histogram equalization is a method of contrast adjustment using the image s histogram. It reassigns intensities to the image so that the they are better distributed on a histogram thereby increasing its dynamic range. Excellent description of the theory of histogram equalization can be found in [10]. The method is useful in images with backgrounds and foregrounds that are both bright or both dark. In particular, the method can lead to better views of bone structure in x-ray images, and to better detail in photographs that are over or under-exposed. However Figure 1: Steps in the Enhancement Pipeline. for the images which already have a wide dynamic range, like the one shown Figure 2(a), histogram equalization does not work well (see Figure 5(c)). In this project we propose to enhance the image by breaking down an image into regions of similar intensities and apply local histogram equalization on each of the regions. For the histograms to be reliable each of the regions have to be large, and for histogram equalization to work well the regions should have high inter region contrast and low intra region contrast. The image enhancement pipeline (Figure 1) in the above formulation can be seen a combination of two steps (1) Image Partitioning (2) Image Enhancement. In the sections that follow we describe the choices of algorithms for each of the steps, and present an approximation scheme for (1), when the partition has a special structure. 2 Image Partitioning Theoretical Abstraction: Given a target weight W and a planar graph G a with nodes representing the pixels, and edges denoting the similarity of the pixels. Find a cut through the graph of least weight in G and size of each of the parts should be at least > W. 2.1 Hardness of the Problem The problem of finding minimum b-balanced cut for graphs, is similar to our problem, and is defined as follows; Given a Graph G = (V, E), a vertex-weight function w : V N, an edge-cost function c : E N, and a rational b, 0 < b 1/2. Find a cut C of minimum weight, i.e., a subset 1

2 (a) (b) (c) (d) Figure 2: Lighten the foreground of this picture without losing the background details (a) Input Photo (b) Top Enhanced, Bottom Too Dark (c) Bottom Enhanced, Top Too Bright (d) Combining both Top and Bottom Enhancements C V, such that min{w(c), w(v C)} b w(v ), where w(v ) denotes the sum of the weights of the vertices in V. The weight of the cut is, Σ e δ(c) c(e) where δ(c) = {e = {v 1, v 2 } : e E, v 1 C, v 2 V C}. We can see that our problem is the minimum b-balanced cut for the graph with b = W/ V. This problem is not approximable within 1+ V 2 ε /opt for any ε > 0 [1]. However for planar graphs it is approximable within a factor of 2 for b 1/3 if the vertex weights are polynomially bounded [2]. A possible approach to the problem of partitioning an image into large sized chunks would be to use clustering algorithms which are not biased towards picking small clusters. Min cuts in graphs can be computed quite efficiently using the Ford-Fulkerson method [3] and Wu and Leahy [4] describe a partitioning method using the min cut criteria. However, even the authors notice that this tehnique has a bias to cut small number of nodes which are very different from others. Recently a graph theoretic clustering technique based on normalized cuts [5] has become popular and has been widely used for image segmentation. Max-Cut based techniques seek a cut in the graph with the maximum weight. Though hard to compute on general graphs (one of Karp s original NP-Complete problems [6]) have good approximations and polynomial time algorithms for planar graphs [7]. However none of these methods have an explicit bound on the size of the smaller set. In this work we present a partition algorithm for partitions which satisfy a special requirement. We then experiment with various parameters of the algorithm and show some results. 2.2 Dynamic Programming Approach We give an algorithm for bi-partitioning the image, with the constraint that the partition is x-monotonic, i.e, the cut intersects any column of the image at exactly one point. This is a reasonable choice for outdoor scenes like the example shown in Figure 2, where separate regions like the sky, buildings, ground, sea, etc are vertically separated. The complexity of the algorithm is O(w 3 h 2 ) where [w, h] is the size of the image. This is prohibitive for large images, so can we speed up the procedure by first computing a fast partition of the image( see Figure 3 and requiring that the cut adhere to the edges of the partitioning. This reduces the complexity of the algorithm to O(wh 2 C), where C is the number of regions, a speedup of O( wh C ). An alternate approach which is much easier to implement is to compute the partition for a lower resolution image and then projecting the partition back to the higher resolution image by linearly interpolating the cut. If the s is the scale of the larger image with respect to the smaller one, i.e. s = Worig W scaled then computing the partition on the smaller image and re projecting has a speed up of s 5. We can exploit the x-monotonic structure to formulate a dynamic programming approach for this problem. The partition is uniquely represented by the choices of y s at each x. Thus the problem is to compute the optimal vector y. Let I(x) denote the subimage of I which has the first x columns of I. Let c(w, x, y) be the optimal cost of x-monotonic partition of I(x) ending at row y on the last column such that the size of the upper partition is exactly w. The notion of upper and lower is well defined as the cut intersects the y axis exactly once at any x. Given a contrast function δ, the 2

3 recurrence relations for c(w, x, y) can be written as: c(w, x, y) = max y {c(w y, x 1, y ) + δ(i(x, y), I(x, y + 1)) + max(y,y ) z=min(y,y ) δ(i(x 1, z), I(x, z))} The boundary conditions are { if y w, c(w, 1, y) = δ(i(1, y), I(1, y + 1)) if y = w. The working of the algorithm is illustrated in Figure 3. The summation denoted by V (x, y, y ), can be computed in O(1) time if we precompute the sum of differences of pixels, i.e., D(x, y) denotes the y z=1 δ(i(x 1, z), I(x, z)). This is due to the fact that V (x, y, y ) = max(y,y ) z=min(y,y ) δ(i(x 1, z), I(x, z)) = D(x, max(y, y ) D(x, min(y, y )) D can be computed in one pass over the image.to solve the problem we have to look for the max value of c(w, w, h ) where w {W,..., N W } and h {1,..., h}. Thus each of the entries can be computed in O(h) time leading to an overall complexity of O(w 2 h 3 ). In case the image is partitioned then we have to only compute O(N C) entries, with requiring a time of O(wh 2 C). In practice however the cut resulting from this leads to very jagged edges. This is because large difference in the adjacent positions of the vector y, leads to a larger boundary and hence a higher value of the function c. To prevent this we add a regularization term to the cut, which ensures smoothness. The smoothness cost for a cut y, is given by S(γ, y) = w γ (y i y i 1 ) 2 i=2 The term γ, is a trade off between the smoothness and the contrast. The optimal regularized cut can still can be computed by dynamic programming by observing that the regularized cost c r function can computed recursively in a manner similar to c as follows: c r (w, x, y) = max y {c r (w y, x 1, y ) + δ(i(x, y), I(x, y + 1)) + V (x, y, y ) γ (y y ) 2 } Different low level features can be taken into account for computing the contrast function δ between the adjacent pixels. For the implementation we use δ(x, y) = x y, i.e. difference in intensity values. One may also combine texture which can be measured as the response to DOOG filters [12] at various orientation and scales. Cuts obtained for various values of W and γ are shown in Figure 4. The image is rescaled to about 1000 pixels, and an implementation in MATLAB takes about a minute to compute the matrix c r for a given value of γ. 3 Image Enhancement We look at the problem of enhancing grayscale images. Color images can be enhanced by extracting the luminosity channel of the image and enhancing it separately. We use histogram equalization of local regions obtained in the previous step. The result of histogram equalization on the global image and the local image is shown in Figure 5. Though the local algorithm improves the contrast better it leads to a visible edge along the boundary of the two regions, which is highly undesirable. Simple bilinear interpolation does not reduce the artifact much, hence we use a simple scheme of assigning a grayscale value as a weighted sum of its earlier color and the new color estimated by histogram normalization. I f (x, y) = I o (x, y) w + I e (x, y) (1 w), w [0,1] Where I f, I o, I e is the final, original and enhanced grayscale values respectively. The weight, w for each pixel is 1 at the boundary and 0 far away from it. We model it using a gaussian function centered on the boundaries. Figure 5 shows the result of enhancing the input image. 4 Conclusion In this work we show how one can combine techniques of segmentation and image enhancement to contrast enhance grayscale images with high dynamic range. The special case where the partitions are required to be x-monotonic has a efficient algorithm. Though the final results are visually good, qualitatively measuring the performance of various algorithms is hard due to the lack of a well defined criteria of performance. References [1] Bui, T. N., and Jones, C. (1992), Finding good approximate vertex and edge partitions is NP-hard, Inform. Process. Lett. 42,

4 (x 1,y) (x,y) (a) (b) (c) Figure 3: (a)the optimal substructure of the problem. In addition to the cost of the subproblem, additional cost is incurred as new boundaries are created as shown in the gray regions of the image. (b)input Image on the left has a size of pixels(597x1350), where as after binary partitioning we have reduced the image to 980 regions of the right (c). Notice that the partitioning preserves most of the edges in the image Figure 4: Outputs of partitioning, of various algorithms. The top row is the result of partitioning without any regularization, which leads to jagged edges. Row 2 and 3 are the results for two different values of γ (indicated by the first column). The numbers in the bottom row is the number of pixels in the upper part. For any given W the new partition can be computed in O(w), once the matrix has been computed. 4

5 (a) (b) (c) (d) (e) (f) Figure 5: (a)input Photo in RGB (b)input Photo in Grayscale (c)result of Global Histogram Equalization. Notice that there is hardly any change as the input image has a high dynamic range. (d)separate Enhancement of the top and the bottom, leads to a sharp boundary along the border. (e)final Result obtained by smoothing d, along the boundary. (f)the result of enhancement of the color image, obtained by enhancing the luminosity channel. 5

6 [2] Garg, N., Saran, H., and Vazirani, V. (1994), Finding separator cuts in planar graphs within twice the optimal, Proc. 35th Ann. IEEE Symp. on Foundations of Comput. Sci., IEEE Computer Society, [3] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, ISBN Section 26.2: The Ford- Fulkerson method, pp [4] W. Shih, S. Wu, and Y. Kuo. Unifying maximum cut and minimum cut of a planar graph. IEEE Trans. Computers, 39(5): , [5] Jianbo Shi and Jitendra Malik, Normalized Cuts and Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), , August [6] Richard M. Karp. Reducibility Among Combinatorial Problems. In Complexity of Computer Computations, Proc. Sympos. IBM Thomas J. Watson Res. Center, Yorktown Heights, N.Y.. New York: Plenum, p [7] F. Hadlock, Finding a Maximum Cut of a Planar Graph in Polynomial Time, SIAM, vol.4, no.3,p [8] Morrow, W.M.; Paranjape, R.B.; Rangayyan, R.M.; Desautels, J.E.L. Region-based contrast enhancement of mammograms, Medical Imaging, IEEE Transactions on, Vol.11, Iss.3, Sep 1992 Pages: [9] Sakreya. Chitwong, Fusak Cheevasuvit, Kobchai Dejhan and Somsak Mitatha, Color Image Enhancement based on Segmentation Region Histogram Equalization, [10] Digital Image Processing, Rafael C. Gonzalez, Richard E. Woods, Addison-Wesley Pub (Sd); 3r.e. edition (March 1992) [11] Two-Dimensional Signal and Image Processing, J. S. Lim, Prentice Hall, 1990 [12] J. Malik and P. Perona, Preattentive Texture Discrimination with Early Vision Mechanisms, J. Optical Soc. Am., vol. 7, no. 2, pp , May