Digital image processing

Transcription

1 Digital image processing Image enhancement algorithms: grey scale transformations Any digital image can be represented mathematically in matrix form. The number of lines in the matrix is the number of lines of the digital image (also called image height), whereas the number of columns in the matrix is the number of columns in the digital image (also called image width). In the following we will consider the simplest case, i.e., the case of grey scale image processing; in that case, the elements of the matrix used to represent our digital image are integer positive values in a finite set, which represent the brightness of the pixels in the digital image. The point processing (also known as pixel processing) of grey scale images is, mathematically speaking, the simplest class of algorithms that can be used to process (modify) a digital image. This class of processing algorithms is used for a long time in the digital image processing field. The point processing mathematical algorithms often constitute the "foundation" for much more complex types of image processing algorithms. Since these operations basically just modify the brightness in a spatial location, independent to the surrounding brightness or to the region in the image, they can be simply seen as modifications of the grey levels in the image; therefore they are also called grey scale transformations and can be implemented directly on the image histogram. The point processing (or pixel processing or grey scale processing) operations is the class of algorithms in which the brightness (grey level) of each individual pixel in the output image depends solely on the grey level of the pixel found in the same spatial position in the input image. The point processing algorithms are generally based on simple mathematical operations. Usually, the processing is described by a linear or non-linear transformation of the grey scale range of the input image, and this transformation determines the new grey scale range in the output image. This linear or non-linear transformation is typically described either analytically or graphically. In the most common representation of grey scale digital images, the brightness Y takes integer values in the range [0; 255]; therefore the grey scale transformation must be defined for 256 discrete values. An important class of point processing operations is the one devoted to image enhancement. The goal of image enhancement is to improve the visual appearance of an image or to enhance particular features in the image, to make the examination of its contents easier for the human observer. In this laboratory, we will examine only some of the image enhancement algorithms based on point processing. In the following, the input and output (processed) images are grey scale images, with the minimum grey level corresponding to black represented by the value 0, and the maximum grey level corresponding to white denoted by L Max (typically, L Max =255, for an 8 bit per pixel image representation). The input (original) grey scale image (to be processed) is represented by the matrix U[M N], where M is the number of lines in the image, and N is the number of columns in the image. Let us denote the brightness of any pixel in the input image U by u (regardless of its spatial position in the image), u { 0,1,..., LMax }. In a similar way, we represent the processed (output) grey level image by the matrix V[M N], having the same dimension as the original image. For the output image, let us denote in the same fashion the brightness of any pixel in V by v, v { 0,1,..., LMax}, and consider that v is the brightness obtained in the output image as the result of processing the brightness u from the original (input) image. 1

2 L 3. Image enhancement algorithms: grey scale transformations The grey scale transformation function used for processing the grey levels in the input image U and producing the corresponding grey levels from the processed image V is the mapping: f :{0,1,..., L Max } {0,1,..., LMax}, and the processing is described by the expression v = f ( u), u {0,1,..., LMax}. The function f describes the point processing of the input image. Since the point processing operations are simply some mappings of the grey levels present in the original image into new grey levels, they can also be defined using the grey level histogram of the original image (since the spatial position of these grey levels are not important in processing, just their values matter). The examination of the effect of the point processing can also be done in some cases by comparing the grey level histogram of the input image and the grey level histogram of the output image. 1. The grey level histogram of a digital image The grey level histogram of a digital grey scale image shows the distribution of the pixels in the image on the grey levels in the set {0,1,..., L Max }. Alternatively, one can say that the grey level histogram of the image is a graphical representation of the number of appearances of each grey level in the set {0,1,..., L Max } in the digital image. The grey level histogram provides a global description of the brightness content of the image and may help identifying the grey levels, extent in pixels and uniformity of various components in the image, as e.g. the background, different objects of large sizes, etc. The histogram is one of the most simple global descriptors of an image, although not the most informative descriptor (one can say the histogram provides a rough description of the image content). There are basically two tipes of grey levels histograms of a digital image: the linear histogram and the cumulative histogram. The linear histogram is the most widely used, therefore in the literature, the linear histogram is often referred simply as the image histogram, omitting the term linear. These two types of grey levels histograms are defined as follows. Let a digital grey level image be represented (as considered before) by the matrix U[M N], with M the image height, in pixels, and N the image width, in pixels. Let us consider the usual case when the values of the grey levels in the digital image are represented on 8 bits, i.e., in the range L={0,1,...,255}. The grey level 0 corresponds to black; the grey level 255 corresponds to white. Then: { u( i, j) }, i= 1,2,..., M, j 1,2,...,, U [ M N ] = = N 3.1 where u(i,j) represents the brightness of the pixel found on the line i and column j in the digital image; u(i,j) L. The linear histogram We define the linear histogram (or, shortly, the histogram) of the digital grey level image U, with the range of variation of brightness L as above, as the positive integer valued function H linear, H linear : L {0,1,..., M N}, given by: Hlinear ( k) = nk, k = 0,1,...,255, 3.2 where n k is the number of pixels in the digital image U whose grey level equals k: 2

3 Digital image processing { u( i, j) u( i, j) = k, i = 1,2,..., M, j = 1,2,..., N}, = 0,1,...,255. n k = Card k 3.3 In the Eq. (3.3), the operator Card is defined as follows: if S is a finite set of elements, then Card(S) is the cardinal of the set S, i.e., the number of elements of the set S. One can notice the following property of the function H linear : 255 Hlinear ( k) = M N. k = The equation (3.4) shows that actually there are no pixels in the digital image U distributed on other grey levels than the ones in the set L. Of course, the only brightness values in U are the ones in L. To process a grey scale digital image, is often useful to plot the linear grey level histogram of the image, that is, the function H linear. An example is shown in Fig Figure 3.1 Example of plotting the grey level linear histogram of a digital image Another possible form of the linear grey level histogram is the normalized linear histogram. In this case, the number of appearances of a grey level k, k L, in the digital image U, is normalized to the total number of pixels in the digital image U, denoted by n, n=m N. The normalized linear histogram can then be represented by the function P linear, a real-valued function with its values in the range [0,1], P liniar : L [ 0,1 ], given by: n P ( k) = k linear, k = 0,1,...,255, n 3.5 with n k defined by the expression (3.3). The normalized linear histogram P linear can be thought as an estimate of the probability distribution function of the grey levels in the digital image U: P linear (k) shows the probability for a pixel in the digital image U to have the grey level k. The cumulative histogram For the same grey level digital image considered above, represented by the matrix U[M N], with the grey level range L={0,1,...,255}, one can define another form of grey level histogram, called cumulative grey level histogram, or simply cumulative histogram. The cumulative histogram of the image U is defined by the positive integer valued function H cumulative, : L { 0,1,..., M N}, given by: H cumulative 3

4 L 3. Image enhancement algorithms: grey scale transformations k Hcumulative( k) = nl, l= 0 k = 0,1,...,255, 3.6 where n l is the number of pixels in the digital image U having the grey level l (according to the equation (3.3)). Thus, the value of the cumulative histogram in any value k of its argument, H cumulative (k), shows the number of pixels in the digital image U having the grey level less or equal than k, for any k L. One can see that the value of the cumulative histogram for the maximum grey level in the grey level range L (in our case 255) is always equal to the number of pixels in the digital image U: H cumulative ( 255) = n = M N, 3.7 since all the pixels in the digital image U have their grey level less or at most equal to the maximum grey level in the grey level range L of the image. The normalized cumulative histogram is defined similar to the normalized linear histogram, i.e., it is obtained by normalizing the values of the cumulative histogram function H cumulative (k), k L, by the total number of pixels in the digital image U, n=m N. Since the cumulative histogram function H cumulative varies from 0 to n (acc. to the equation (3.7)), the normalized cumulative histogram will vary from 0 to 1. The normalized cumulative histogram is described by the probability density function P cumulative, P cumulative : L [ 0,1], given by: k n 1 k P ( k) = l cumulative = nl, l= 0 n n l= 0 k = 0,1,...,255, 3.8 where n l is given by the equation (3.3). P cumulative (k) can be seen as the probability of a pixel in the digital image U to have its brigthness at most equal to k (and in any case not larger than k). 2. Contrast enhancement This image enhancement operation is needed to improve the visual appearance of images with low contrast; the low contrast can be due to a poor ambiental illumination or to a too strong ambiental illumination. Some of the most commonly used contrast enhancement methods are presented in the following Histogram equalization: This is probably the most widely used algorithm in image enhancement, as it generally produces a significantly enhanced contrast, without the need of any parameters in the grey scale transformation function (the function is non-parametrical). In principle, histogram equalization provides an optimal redistribution of the grey levels over the entire available grey level range L of the digital image, starting from the linear grey level histogram of the input image and assuming the ideal desired situation of an even distribution of the pixels on all the grey levels in the output image. 4

5 Digital image processing In principle, mathematically speaking, the histogram equalization algorithm implies the definition of a grey scale transformation in the form of an analytical function f Equalize( ), based on the histogram of the input image, to ensure in the largest extent possible that in the histogram of the output image, the grey levels in L are rather equally distributed, so, they appear with approximately the same probability. The mathematical details of the algorithm are given in the lecture notes Contrast enhancement by a piecewise-linear grey scale transformation function: The expression of the grey scale transformation function for this type of processing is the following: v = u tg α, f ( u) = ( u a) tg β + va, ( u b) tg γ + vb, 0 u a a u b b u L Max 3.9 The parameters a and b of the function are obtained by examining the linear histogram of the image. The parameters α, β and γ determine the contrast enhancement degree, as illustrated in Fig The input variable u is the grey level of the pixels in the input image U, whereas the output variable v is the corresponding grey level of a pixel in the processed (output) image V. v L M ax v b γ β v a α a b L M ax u Figure 3.2 Piecewise-linear contrast enhancement function 3. Greyscale clipping; greyscale thresholding Grey scale clipping is a particular useful case of contrast enhancement, obtained from the general form of the piecewise linear greyscale transformation described by the equation (3.9) for the parameters α = γ = 0 : 0, 0 u a v = f ( u) = u tgα, a u b L Max, b u L Max Greyscale thresholding (also called image thresholding or image binarization) is a particular case of greyscale clipping, obtained when a=b=t, for t a parameter called threshold, whose result is a binary image (having only two grey levels). This processing is useful e.g. when one wants to process a scanned printed page with text, and the quality of the scanner is rather poor, so instead of having only characters and background, we have 5

6 L 3. Image enhancement algorithms: grey scale transformations several grey levels in the resulting image. By a suitable selection of the threshold, one can hopefully obtain an black and white image, with the characters represented in black and the background white, with a good separation of the characters from the background. v v Lmax Lmax 0 a b u 0 Lmax a Lmax u a) b) Figure 3.3 Graphical representation of the grey scale clipping (a) and thresholding (b) 4. Image negativation (inversion) This operation is useful when the negative of a digital image is easier to be examined and visually analyzed by a human observer than the original image. This can be the case of some photographic films, radiological images, etc. The negative of the input image is simply computed according to: (see Figure 3.): v = f ( u) = LMax u, 0 u LMax, 3.11 where L Max is the maximum possible grey level in the digital image, corresponding to white (typically, for an 8 bit per pixel image representation, L Max =255). v L Max 0 L Max u Figure 3.4 Image negativation function LabView functions used for image enhancement by point processing operations IMAQ Histogram This function computes the linear grey level histogram of a grey scale image represented in the format 8, 16 or F, but does not allow plotting the histogram directly. However this function provides on the output the values of the linear histogram for any grey level in a tabular form which lists various information regarding the distribution of the grey levels in the image. 6

7 Digital image processing Number of Classes input specifying the number of bins used for the description of the grey levels in the grey level dynamic range of the image. By default, this input has the value 256, corresponding to 8 bits images and to a bin assigned to each grey level. Image input connected to the source image (in our notations the image U) for which the histogram is computed. Image Mask an 8 bit image specifying the region in the input image for which we want to compute the histogram. The non-zero values in the mask indicate that those pixels must be considered when computing the histogram; however the pixels having corresponding zeros in the mask image are not taken into account when computing the histogram. By default, when one wants to compute the histogram of the entire image, this input is not connected. Interval Range is an input used if we want to perform the computation of the grey level histogram only on a sub-range of the grey level range L of the input image U. In this case, a cluster should be connected at this input, specifying the minimal and maximal values of the grey levels defining the sub-range of L. For 8 bits images, the default values at this input (with the input not connected) are 0 (minimum) and 255 (maximum). If e.g. one wants to compute the histogram just for the sub-range of grey levels: {5,6,...,200}, then the minimal value should be set to 5, and the maximal value to 200. Each of these two values are stored in a variable of type SGL and are grouped together into a cluster connected to the input Interval Range. Histogram Report is an output data structure returning the values of the histogram function. IMAQ Histograph This function is also devoted to the computation of the linear grey level histogram of a digital image of type 8, 16 or F, but unlike IMAQ Histogram, it provides on its output a data structure directly compatible with the display components in LabView. This data structure is available at the output terminal Histogram Graph of the component. This component also provides a set of statistical measurements on its output related to the grey levels distribution in the image: the mean value and the standard deviation of the grey level distribution in the input image. Number of Classes just like in the case of IMAQ Histogram function, this input allows specifying the number of bins used for the description of the grey levels in the grey level dynamic range of the image. By default, this input has the value 256, corresponding to 8 bits images and to a bin assigned to each grey level. Image input connected to the source image (in our notations the image U) for which the histogram is computed. Image Mask an 8 bit image specifying the region in the input image for which we want to compute the histogram. The non-zero values in the mask indicate that those 7

8 L 3. Image enhancement algorithms: grey scale transformations pixels must be considered when computing the histogram; however the pixels having corresponding zeros in the mask image are not taken into account when computing the histogram. By default, when one wants to compute the histogram of the entire image, this input is not connected. Interval Range just as in the case of IMAQ Histogram function, is an input used if we want to perform the computation of the grey level histogram only on a sub-range of the grey level range L of the input image U. In this case, a cluster should be connected at this input, specifying the minimal and maximal values of the grey levels defining the subrange of L. For 8 bits images, the default values at this input (with the input not connected) are 0 (minimum) and 255 (maximum). Histogram Graph output data structure providing the values of the linear histogram function in a format compatible with the input of a display component of the type Waveform Graph in LabView; used to display the linear grey level histogram of the image. Mean Value output providing the mean value of the grey levels of the pixels in the input image (or, of the grey levels involved in the computation of the histogram if the inputs Image Mask and/or Interval Range are used). Standard Deviation output providing the standard deviation of the grey levels of the pixels in the input image (or, of the grey levels involved in the computation of the histogram if the inputs Image Mask and/or Interval Range are used). IMAQ Equalize This function implements the histogram equalization algorithm for a grey level image. Basically, image equalization redistributes the pixels in the input grey level image with the greyscale range L so that all the grey levels in L become almost equally probable if possible, i.e., the number of pixels on each grey level is approximately constant in the output image. Therefore, the plot of the linear grey level histogram of the output image tends to be closer to a parallel to the horizontal axis, i.e., the value of the linear histogram function in any grey level in L tends to be close to a constant value. In practice, this ideal case in which Hliniar ( k ) = constant, k = 0,1,...,255, doesn t appear, basically due to the numerical approximations and rounding in the discrete implementation of the histogram equalization algorithm. However, the histogram of the output image is much smoother than the histogram of the input image, and the grey levels of the pixels in the processed image are much more uniformly distributed over L than those of the pixels in the original image. The function IMAQ Equalize can be applied to grey level images of type 8, 16 or F. The function IMAQ Equalize requires at its input a data structure of the type Histogram Report (to describe the grey level linear histogram of the input image); therefore any histogram equalization implementation in LabView using IMAQ Vision requires a call of the IMAQ Histogram function, prior to IMAQ Equalize, to generate this structure. Histogram Report input at which the corresponding output cluster from an IMAQ Histogram component is connected. IMAQ Histogram should have at its input the input image, which is the same as the image connected to the Image Src terminal from IMAQ Equalize. 8

9 Digital image processing Image Src input connected to the source image (in our notations the image U) for which the histogram equalization is performed. Image Mask an 8 bit image specifying the region in the input image for which we want to perform the histogram equalization. The non-zero values in the mask indicate that those pixels must be modified by the histogram equalization algorithm; however the pixels having corresponding zeros in the mask image are not changed. By default, when one wants to apply the histogram equalization on the entire image, this input is not connected. Image Dst on this input, one should connect a reference to an image structure, of the same type as the source (input) image (i.e., of type 8, 16 or F), which will store the output (processed) image resulting from the histogram equalization. If the input Image Dst is not connected, then the input image will be overwritten with the processing result, and therefore we will only have access to the processed image and loose the possibility to display simultaneously the original image and the processed image. Range is a cluster of two values of type SGL, specifying the minimal and maximal grey level values in the input image s greyscale range between which we perform the histogram equalization. The significance of the Range input is the following: only the pixels whose grey levels are found within the Range are processed in the histogram equalization algorithm; the pixels having the grey levels smaller than the minimal value in Range are set to the minimal brightness in the greyscale range L of the input image (i.e., 0 for an 8 bit per pixel representation); the pixels having the grey levels greater than the maximal value in Range are set to the maximal brightness in the greyscale range L of the input image (i.e., 255 for an 8 bit per pixel representation). When the Range input is not connected, the histogram equalization algorithm is applied on all the grey levels in the greyscale range of the original (source) image. Image Dst Out is the output of the function and it represents the reference to the image storing the processing result. If the input Image Dst is not connected, then Image Dst Out refers to the source image. Otherwise, if the input Image Dst is connected to an image structure, then the output Image Dst Out contains the reference to this image structure. IMAQ Add Adds the grey levels of two images, pixel by pixel, or alternatively, adds a constant to the grey level of each pixel in an input image. Constant is the value added to each pixel brightness in the input image whose reference is connected to Image Src A, in case of a constant + image operation. The default value of this input is 0. This operation is performed if the input Image Src B is not connected. Otherwise, the operation is of the type image + image (Image Src A + Image Src B) and the value of Constant has no effect on the processing. Image Src A is an input connected to a first source (input) image. Image Src B is an input connected to a second source (input) image. If not connected, the input image referenced by Image Src A has all the brightness values added by a constant specified by the Constant input. Image Dst is an input where we should connect a reference to an image structure, of the same type as the source images, to store the output (processed) image. If 9

10 L 3. Image enhancement algorithms: grey scale transformations the input Image Dst is not connected, then the input image Image Src A will be overwritten with the processing result. Image Dst Out - is the output of the function and it represents the reference to the image storing the processing result. IMAQ Multiply Multiplies the grey levels in two images, pixel by pixel, or alternatively, multiplies by a constant the grey level of each pixel in an input image. Constant is the value that multiplies each pixel brightness in the input image whose reference is connected to Image Src A, in case of a constant image operation. The default value of this input is 1. This operation is performed if the input Image Src B is not connected. Otherwise, the operation is of the type image image (Image Src A Image Src B) and the value of Constant has no effect on the processing. Image Src A is an input connected to a first source (input) image. Image Src B is an input connected to a second source (input) image. If not connected, the input image referenced by Image Src A has all the brightness values multiplied by a constant specified by the Constant input. Image Dst is an input where we should connect a reference to an image structure, of the same type as the source images, to store the output (processed) image. If the input Image Dst is not connected, then the input image Image Src A will be overwritten with the processing result. Image Dst Out - is the output of the function and it represents the reference to the image storing the processing result. IMAQ BCGLookup This function applies some brightness, contrast and gamma correction, simultaneously, on a grey scale image. Of course, these three corrections change the color palette of the image. BCG Values is an input data structure with the following components: Brightness (default value: 128) sets the average brightness of an image. A value of 128 does not change the grey levels in the image in respect to the average brightness. A value above 128 makes the image brighter, and a value below 128 makes the image darker. Contrast (default value: 45.0) sets the image contrast. The default value of 45 (meaning an angle of 45, describing a line of slope 1) does not change the contrast of the image. However, a value above 45 increases the contrast of the image, whereas a value below 45 decreases the contrast of the image. 10

11 Digital image processing Gamma (default value: 1.0) sets the gamma correction applied to the image. The default value of 1 does not change the image. However, a value above 1 increases the contrast of the dark pixels and decreases the contrast of the bright pixels; a value below 1 has the opposite effect increasing the contrast of the bright pixels and decreasing the contrast of the dark pixels. Image Src is an input connected to the source (input) image. Image Dst is an input where we should connect a reference to an image structure, of the same type as the source image, to store the output (processed) image. If the input Image Dst is not connected, then the input image Image Src will be overwritten with the processing result. Image Dst Out - is the output of the function and it represents the reference to the image storing the processing result. Image mask is an 8 bit type image specifying (by its non-zero values) the input image pixels on which the BCG processing is applied. If this input is not connected, the BCG processing is applied on the entire input image. IMAQ Inverse Computes the negative of a grey scale image by replacing each brightness in the image with its complement in respect to white. Image Src is an input connected to the source (input) image. Image Mask is an 8 bit type image specifying (by its non-zero values) the input image pixels on which the function is applied. To obtain the negative of the entire input image, this input is not connected. Image Dst is an input where we should connect a reference to an image structure, of the same type as the source image, to store the output (processed) image. If the input Image Dst is not connected, then the input image Image Src will be overwritten with the processing result. Image Dst Out - is the output of the function and it represents the reference to the image storing the processing result. Practice Display the histogram of a grey scale image open a new LabView session create a new project in the Diagram window, add the IMAQ components to obtain the diagram below: 11

12 L 3. Image enhancement algorithms: grey scale transformations NOTE: To display the histogram, use IMAQ Histograph the user interface should look as below: NOTE: Remember to set the mechanical action of the two buttons: Load Image and STOP, to Latch when released to display the histogram, use a component from the class Waveform graph Open various images and examine their histograms. Histogram equalization of grey scale images open a new LabView session create a new project in the Diagram window, add the IMAQ components to obtain the diagram below: 12

13 Digital image processing NOTE: To display the histogram, use IMAQ Histograph. The diagram contains also other functions, as IMAQ WindMove, IMAQ GetFileInfo, to render the display of the results more efficient but these functions are optional. The user interface should look as below. Open various grey level images and compare their histogram with the histogram obtained after histogram equalization. Change the range MIN, MAX of the grey levels for which the histogram equalization is performed; notice the differences in the processing results for various images. 13

14 L 3. Image enhancement algorithms: grey scale transformations Brightness and contrast adjustment through greyscale clipping for a grey scale image open a new LabView session create a new project in the Diagram window, add the IMAQ components to obtain the diagram below: the user interface should look as below: Open several images and examine the effect of changing their contrast and their average brightness by this processing solution. Give to the two constants positive and negative values. 14

15 Digital image processing Note: The functions IMAQ ADD and IMAQ Multiply use as inputs only the Image Src A and Image Dst, not Image Mask!! Negativation of a grey scale image open a new LabView session create a new project in the Diagram window, add the IMAQ components to obtain the diagram below: the user interface should look as below: Open some images and notice the effect of the negativation on the images and on their histogram. Questions and exercises 1. Modify the histogram equalization diagram to allow saving the processed image in a BMP file. 2. Implement an application to perform the thresholding of a grey scale image, using: 15

16 L 3. Image enhancement algorithms: grey scale transformations dedicated components from IMAQ Vision general purpose functions from LabView (others than those from IMAQ Vision) 3. Implement a contrast, brightness and gamma correction application using the IMAQ BCGLookup component. Compare its effect with the effect of the greyscale clipping application. 4. Implement a diagram for contrast enhancement according to the greyscale transformation function in Figure 3.2, allowing to set all the parameters of this function through the user interface window. Bibliografie suplimentară [1]. IMAQ Vision for G Reference Manual, National Instruments, 1999 [2]. IMAQ Vision User Manual, National Instruments, 1999 [3]. IMAQ PCI/PXI 1411 User Manual, National Instruments, 1999 [4]. A. VLAICU, Prelucrarea Digitală a Imaginilor, editura Albastră, Cluj Napoca, 1997 [5]. G.X. Ritter, J. N. Wilson, Computer Vision Algorithms in Image Algebra, CRC Press, 1999, second edition 16