Image Processing with Qt BY KLAUDIO VITO

Transcription

1 Image Processing with Qt BY KLAUDIO VITO CSC Senior Design I Prof. George Wolberg May 22, 2016

2 Table of contents 1. Introduction pg Qt pg What is Qt? pg Qt Widgets Used pg Image Processing pg Point Operations pg Neighborhood Operations pg Conclusion pg Appendix pg.58 PAGE 1 OF 133

3 1. Introduction This is the report of the work done in the CSC Senior Design I class during the spring 2016 semester with Prof. George Wolberg. I will explain how I learned about Qt and Image Processing. Qt is cross platform development framework written in C++. I used this framework to construct a GUI application to perform the image processing operations that I learned. The image processing operations that we focused in this class are point operations and neighborhood operations. The point operations include threshold, contrast and brightness, quantization, gamma correction, color inversion, and histogram stretching and matching. Neighborhood operations that we implemented are blurring, sharpening, and median. The following figure is a snapshot of the final application that I will describe throughout this report. Figure 1 GUI Application PAGE 2 OF 133

4 2. Qt 2.1. What is Qt? Qt is a cross-platform application and UI framework. It was founded in 1994 by Trolltech, which was acquired by Nokia in Digia acquired Qt Business from Nokia in This framework is developed for desktop, mobile and embedded environments. Qt is used by more than 350,000 commercial developers as well as open source ones. The following figure shows the Qt virtuous cycle: Figure 2.1 The Qt Virtuous Cycle Qt has many advantage. The main advantage is that the developer writes code once and targets multiple platforms. This allows for compact, high-performance applications. Using Qt, allows developers to focus on innovative ideas, rather than infrastructure coding. The users can choose the license that best fits their purpose, which can be commercial, LGPL, or GLP. Other PAGE 3 OF 133

5 advantages of Qt include professional service, support and training. Here we can mention Qt Assistant which is the official Qt manual where the developers can find detail description of all Qt classes and functions. It extends C++ with macros and introspection while keeping plain C++ code. All Qt applications are built with native look and feel whether you are using them on Windows, Mac, Linux or any other embedded platform. Qt Creator is the main software that we used to create our application. This software provides advanced C++ code editor, project and build management tools, visual debugger, and support for multiple platforms. I will explain how Qt is used to create applications by explaining the classic Hello Qt! program. Since Qt uses C++ as its programming language it has a Main class which defines the application. Let s start by showing the code for this simple program and then explain line by line what it does. Figure 2.2 Hello Qt! The main class of any Qt application is almost the same: it includes the header of every Qt class that is going to be used. In the above example, the first two lines include the definitions for QApplication and QLabel. On line 5, a QApplication object is created takes argc and argv as parameters. PAGE 4 OF 133

6 All visual elements in the user interface are called widgets in Qt. Since QApplication and QLabel are both widgets then we can substitute line 1 and 2 by including the QWidget class to get all widgets. The QLabel created on line 6 is a widget as well and is initialized with the string Hello Qt!. So far we have just created the label but we have not told Qt to display it, thus if we were to run the application now it would simply display a blank window. The next line does the job, it calls the show() method of the label widget. The next line executes the application. This means that the application enters an infinite loop until the user tells it what to do. Even though we have always been told that infinite loops are results of bad coding, in this case it is necessary. The loop will be interrupted when the user clicks the X button to quit the application. The application created from this code is shown in the following figure: Figure 2.3 Hello Qt on Linux Every Qt application consists of the.pro file which is the project file. For the above application we have the helloworld.pro file which is shown below. Figure 2.4 Hello World Project File As we can see, in this case we have the main.cpp file as our source file, we have no headers, and also Qt will use core gui widgets. PAGE 5 OF 133

7 In order for this application to use the qmake tool which creates cross-platform make-files. Qt Creator does everything for us when using the following code to create makefile, compile and link the application, and finally run it. Figure 2.5 qmake Tool 2.2. Qt Widgets Used I stated earlier that every visual element in the UI is called a widget in Qt. The widgets that I have used in this application are numerous. I will explain each of them in detail in this section. We start with the main window which will contain all the widgets that are going to be used. The main window is split into three major components. The Menu Bar, the Label component on the right of the screen used to display the input and output image, and the Control Panel on the left where all the image processing widgets are displayed. Menu Bar Input/Output Image Label Control Panel Figure 2.6 Main Window Components PAGE 6 OF 133

8 The Menu Bar. The menu bar of this application is a built in function in Qt therefore we do need to code it, but we need to add menus to it. We added three menus: File, Point Ops (which stands for Point Operations), and Neighbor Ops (short for Neighborhood Operations). Each of these menus has actions to be performed. The File menu has three actions: Open Image, Save Image and Quit. The Open Image action is used to choose the input image and open it in the program. The Save Image action, which is initially disabled, is used to save a filtered image. Quit is used to exit the application. The Point Ops has six actions: Threshold, Contrast, Quantization, Gamma Correction, Color Inversion and Histogram Stretching and Matching. The last menu, Neighbor Ops has three actions: blurring, sharpening, and median. In order to create a menu we declare it on the header file like so: QMenu* m_menuptops. This is a pointer to a QMenu widget. We define which actions are included in the m_menuptops in the MainWindow.cpp file. The following figure shows the how the actions are added to the Point Ops menu. Figure 2.7 Adding Actions to Point Ops Menu The & before the P character makes it possible to underline the character and use shortcuts to access it. This symbol can be used anywhere in the string to underline any character. PAGE 7 OF 133

9 The actions are created separately. Following the same logic, an action is declared in the header file and then define it in the MainWindow.cpp file. Let s take the example of the Contrast action. We created it in header file using the following code: QAction* m_actioncontrast. Again, this is a pointer to a QAction widget. The following figure shows how this action and the other five actions are defined. Figure 2.8 Definition of Actions in Point Ops The figure shows how the actions are initialized with the first parameter as the name to be displayed in the UI and the second parameter is the parent of the action. In our case the parent PAGE 8 OF 133

10 is the main window. I accessed the setshortcut() function of the QAction class to set the shortcut for the each action. The third line for each definition sets the data for the action. The data comes from an enumerator containing the data imported from the header files. The result of the above code is displayed in the UI as follows: Figure 2.9 Point Ops in the UI PAGE 9 OF 133

11 Input / Output Image Label The area on the right hand side of the screen is a label used to display the image. It can be the input image, or the processed image depending on what we choose from the control panel. The images are stored in a QStackedWidget and then displayed through a label. This is done because the image to be displayed will depend on the choice of the user from the control panel, therefore we stack the input image and the processed one to display whichever the user selects. Even though the image is displayed through a label we need a group box where the label should resign. The following code shows how the image labels are stacked in the group box. Figure 2.10 Group Box to Display Images PAGE 10 OF 133

12 As we can see, the stackwidgetimages are added to a vertical box layout that comes from the QVBoxLayout class. This function returns a GroupBox widget which is used in the createwidgets() function, which in turn is called from the constructor of the MainWindow.cpp file. In order for the widgets to be created, we use the following code: Figure 2.11 Creating Widgets The display group box and the control panel group box are added to a horizontal box layout. We can see that there is a stretch added to the horizontal box. This is done so that we can have a larger display panel rather than having the screen split in two equal halves. Control Panel The control panel that we constructed for this application is split into five components. The first component is a group box containing the Display options, the second group box contains the Mode options, and the third component contains the current Image Filter. The fourth component is used to display the Histogram of the current image displayed, and finally the fifth component is used to contain the Reset and Quit buttons. PAGE 11 OF 133

13 Figure 2.12 The Control Panel The Display, Mode, Histogram, and Reset / Quit buttons will remain the same for all image filters implemented, while the image filter group box will always change according to current filter chosen by the user. For this application we have decided to have a maximum of 50 filter PAGE 12 OF 133

14 types to be stored in the image filter widget. We declare this widget in the header file like so: ImageFilter* m_imagefiltertype[maxfilters]. MAXFILTERS is a constant that we defined to be 50. We also need a stacked widget to store the control panels for each image filter and display them according to the user choice. As usual, we declare this widget in the header file: QStackedWidget* m_stackwidgetpanels. The definition of the image filter and stacked widget is done on the MainWindow.cpp file. We use the creategrouppanel() function to create the image filter types. This function return a QGroupBox which is called in the CreateWidgets() functions mentioned before. Since the creategrouppanel() function is large, I will display the code for only one image filter and then continue with the extension used for the histogram and then the exit buttons group box. QGroupBox* MainWindow::createGroupPanel() init group box QGroupBox *groupbox = new QGroupBox; groupbox->setminimumwidth(400); filter's enum indexes into container of image filters m_imagefiltertype[threshold ] = new Threshold; create a stacked widget to hold multiple control panels m_stackwidgetpanels = new QStackedWidget; add filter control panels to stacked widget m_stackwidgetpanels->addwidget(m_imagefiltertype[threshold]->controlpanel()); display blank dummmy panel initially m_stackwidgetpanels->setcurrentindex(0); assemble display and mode groups into horizontal layout QHBoxLayout *hbox = new QHBoxLayout; hbox->addwidget(createdisplaybuttons()); hbox->addwidget(createmodebuttons ()); create histogram plot m_histogram = new QCustomPlot; set histogram title m_histogram->plotlayout()->insertrow(0); m_histogram->plotlayout()->addelement(0,0,new QCPPlotTitle(m_histogram,"Histogram")); PAGE 13 OF 133

15 assign label axes m_histogram->xaxis->setlabel("intensity"); m_histogram->yaxis->setlabel("frequency"); create extension and insert histogram m_extension = new QWidget; QVBoxLayout *vbox2 = new QVBoxLayout(m_extension); vbox2->addwidget(m_histogram); start dialog with hidden histogram m_extension->hide(); set signal-slot connection connect(m_checkboxhisto, SIGNAL(stateChanged(int)), this, SLOT(setHisto(int))); assemble stacked widget in vertical layout QVBoxLayout *vbox = new QVBoxLayout; vbox->addlayout(hbox); vbox->addwidget(m_stackwidgetpanels); vbox->addwidget(m_extension); vbox->addstretch(1); vbox->addlayout(createexitbuttons()); groupbox->setlayout(vbox); return groupbox; From the above code we can see that we filter the enumerator s indexes as a container of image filters. Each index is associated with a new object of the corresponding filter s header file. The next step is to add each image filter type s control panel into the stacked widget panel so that we can display the filter according to the user choice. The next step is to create a horizontal box layout for the two display buttons and the mode buttons. This box layout will be added to the group box in the left side of the window which we discussed earlier. Then we continue by defining the QCustomPlot. This is the graph to display the histogram of the pixels of the image. What the histogram is and how it is used in this application will be explained in detail in the image processing section of this report. Finally we use a vertical box layout to assemble all the group boxes above mentioned. PAGE 14 OF 133

16 The display panel is composed by two radio buttons to choose between the original image and the processed image, and one checkbox for the histogram. The radio buttons work in such a way that when one is selected, the other is deselected. The display panel is implemented as follows: Figure 2.13 Create Display Buttons In this code we can see that the radio buttons were implemented as an array of radio buttons. Also important is the introduction of the connect function which is unique to Qt. This function is used as an action listener. It will wait for the user to click on one of the radio buttons and it will connect this action to another function to do something. For example, when the 0 th PAGE 15 OF 133

17 radio button is clicked, meaning that the user want to see the input image, the connect function will call the displayin() function, which in turns shows the input image. The opposite happens when the 1 st radio button is clicked which will display the processed image by calling the displayout() function. The mode group box is composed of two radio buttons, one for color image (RGB which stands for Red Green and Blue channels) and gray scale image. This panel is created by using the following code: Figure 2.14 Create Mode Buttons As we can see the buttons are assembled in a vertical box layout and they are connected to their respective functions to change the mode of the image. An important thing to notice is that PAGE 16 OF 133

18 there is an extra radio button which is empty. This is a placeholder so that the display panel and the mode panel are of equal sizes. The final panel to insert into the larger left panel of the window is the exit buttons. These buttons are implemented through the following code: Figure 2.15 Create Exit Buttons In the above figure we can see that the buttons are defined by giving each a name, specifically Reset and Quit. These two buttons are connected to their respective functions to reset the settings of the current filter type and to quit the application. This application will start with disabled menus and filter options since no image is yet opened from the user. The only enabled menu is File. After the image is opened the options are enabled and then the user can start filtering the image through the different filters. PAGE 17 OF 133

19 3. Image Processing 3.1. Point Operations The point operations that were implemented in this application are five: Threshold, Contrast, Quantization, Gamma Correction, Color Inversion, and Histogram Stretching and Matching. In this section I will explain what each filter does. Before starting with each filter I want to explain how these filters are implemented using a look-up table (LUT). The LUT is nothing more than an array of integers from 0 to 255. This range corresponds to the intensity of a pixel where 0 is black and 255 is white. Consider the following figure. Figure 3.1 Look-Up Table In the above figure we can see that we initialize the LUT in the beginning. The indexes of the table represent the pixel intensity of the input image while the value in the table for that index represents the weight of the filtered pixel. The function f(x,y) represents the filter applied to the pixel. The pixel produced by the filter will then go to the output image. PAGE 18 OF 133

20 Threshold This is filter is implemented in our application though a slider. The slider will set the threshold value. The following figure shows how a slider is implemented in Qt. Figure 3.2 Threshold Control Panel PAGE 19 OF 133

21 This is the control panel for the threshold filter. The slider and spin box are both declared in the header file while the definition is done on the MainWindow.cpp file. The slider is defined to be a horizontal slider, with a range of values from 1 to 255. The same range is used for the spin box. The initial value of the slider and spin box are set to be 128 (that is MXGRAY or 256 shift 1 bit right). The spin box and slider are connected together through the connect functions. This means that every time we move the slider the value of the spin box will change accordingly and vice versa. We created a grid layout and added three widgets to it, the label, the slider and the spin box. Figure 3.3 Threshold Control Panel This filter works by taking the threshold value chosen through the slider or the spin box and sets to zero every pixel that is less than the threshold, and sets to 255 all the pixels that are greater than the threshold. PAGE 20 OF 133

22 Figure 3.4 Threshold LUT The above figure shows what the LUT looks like for the threshold filter. This can be implemented in Qt using the following function: Figure 3.5 Threshold Function As we can see the code corresponds to the theory, it sets all values less than the threshold to 0 and sets all values greater than or equal to threshold to 255. Finally, it gets the intensity of PAGE 21 OF 133

23 the output pixels from the look-up table. The following figures show some examples of the threshold filter in our application. Figure 3.6 Gray Scale Threshold 85 Figure 3.7 Gray Scale Threshold 140 PAGE 22 OF 133

24 Figure 3.8 Color Threshold 73 Figure 3.9 Color Threshold 209 PAGE 23 OF 133

25 Contrast This filter includes contrast stretching and brightness. Both of these filters are implemented according to a reference point, therefore we need three sliders and three spin boxes as well as three labels for each slider to implement this filter. Figure 3.10 Contrast Control Panel From the figure we can see that the initial values of the reference, contrast, and brightness sliders are 128, 1.0 and 0 respectively. Notice that in this control panel we have a spin box with double digits. The following figure shows the implementation of a double spin box. Figure 3.11 Double Spin Box Definition PAGE 24 OF 133

26 We created the QDoubleSpinBox and set the contrast range to be from the 0.25 to 5. The issue with the implementation of a double spin box is that there is no such thing as a double slider in Qt, therefore we need to use our mathematical skills to change the values accordingly. We set the slider to have values between -100 and 100. Since the spin box has different range we wrote another function to adjust the values. Figure 3.12 Get Scaled Contrast If the contrast value of the slider is greater than or equal to zero then the value for the spin box will be that value divided by 25 and add 1 to it, so that the new range is from 1 to 5. If the value of the slider is less than zero we divide by 133 and add 1 so that the range is from 0.25 to Contrast is the difference between the maximum and the minimum pixel intensity of an image. Brightness on the other hand is the amount of brightness the pixel has. In terms of pixel intensity, the maximum brightness is 255 and the minimum one is 0. Contrast and brightness were implemented using a reference point on the graph of pixels. PAGE 25 OF 133

27 Figure 3.13 Contrast with Reference Point In the above figure the black curve represents the original graph of pixel intensity, while the red and blue graphs represent different contrast applied to the pixels. The formula that we used to initialize the LUT is: LUT[v] = (v reference) contrast + brightness + reference The following function was used to apply to contrast filter to the image: Figure 3.14 Contrast Function PAGE 26 OF 133

28 The following figures show some examples of contrast filter. Figure 3.15 Color Image Contrast 2.5 Figure 3.16 Color Image Contrast 0.6 PAGE 27 OF 133

29 Figure 3.17 Gray Scale Contrast 3.3 Figure 3.18 Gray Scale Contrast 4.1 PAGE 28 OF 133

30 Quantization The control panel for quantization is composed of one slider and one spin box to adjust the level of quantization. Also, we included a check box for dither quantization. Figure 3.19 Quantization Control Panel To quantize an image means to divide the signal into partitions. This means to digitize the amplitudes of the image (y axis). Quantization is done based on gray levels, meaning that the output image will have as many different colors as the number of levels. Dither quantization on the other hand means adding noise to the input image before quantizing. We are used to think that noise is a bad part of the signal, but in this case it is helpful reduce the quantization error. Adding noise to the signal means adding a random number in the range [-gray level, gray level] to each pixel of the input image before quantization. This filter was implemented with the following code in Qt: PAGE 29 OF 133

31 Figure 3.20 Dither Quantization Function Figure 3.21 Quantization Function Dither quantization is useful when the gray scale level is low, when the gray scale level is very high the dither quantization in almost irrelevant. The following figures show some results of quantization and dither quantization: PAGE 30 OF 133

32 Figure 3.22 Color Quantization with 4 Levels Figure 3.23 Color Quantization with 6 Levels + Dither PAGE 31 OF 133

33 Figure 3.24 Color Quantization with 245 Levels + Dither Figure 3.25 Gray Quantization with 5 Levels + Dither PAGE 32 OF 133

34 Figure 3.26 Gray Quantization with 10 Levels Figure 3.27 Gray Quantization with 190 Levels PAGE 33 OF 133

35 Gamma Correction Gramma correction is implemented using one slider one spin box. Figure 3.28 Gamma Correction Control Panel The range of the gamma correction is from 0.01 to The output pixel intensity is calculated by the following formula: LUT[v] = 255 ( v )gamma. The gamma correction 255 was implemented using the following function: PAGE 34 OF 133

36 Figure 3.29 Gamma Correction Function image: The following figures show some examples of different gamma correction on an input Figure 3.30 Gamma Correction 0.2 PAGE 35 OF 133

37 Figure 3.31 Gamma Correction 7.2 PAGE 36 OF 133

38 Color Inversion Color inversion is done by subtracting the input pixel intensity from 255 to get the output pixel intensity. The control panel for this filter is composed of only one check box. When the check box is checked the inverted color image will be displayed. The following look-up table shows what color inversion is: Figure 3.32 Color Inversion LUT The inversion can be done by the following function: Figure 3.33 Color Inversion Function PAGE 37 OF 133

39 The following figure shows the result of color inversion. Figure 3.34 Color Inversion in RGB Figure 3.35 Color Inversion in Gray Scale PAGE 38 OF 133

40 Histogram Stretching and Matching The histogram is a graph that shows the frequency of the pixel intensity versus the number of pixels with that intensity. For this class we used QCustomPlot to graph our histogram which is an easy to use, modern plotting widget for Qt. This widget was written by Emanuel Eichhammer. Based on the histogram we can do histogram stretching, which means to stretch the actual histogram given a minim and maximum value for the pixel intensity. Furthermore, we implemented histogram equalization. This function allows the histogram to auto stretch to the first non-zero pixel intensity on each side of the histogram. The control panel for this filter is composed from two sliders, two spin boxes, and two check boxes. Figure 3.36 Histogram Stretching Control Panel As we can see the min slider is used to set the minimum value of the pixel intensity and the max slider is used to set the maximum value of pixel intensity. The min slider has range PAGE 39 OF 133

41 from 0 to 254 and the max slider has range from 1 to 255. This is done so that the min can never have a larger value than the max slider. Furthermore, the auto check boxes disable the slider and spin box for min or max. The formula for stretching the histogram is: g = 255 (f min) max min In this formula, g is the intensity of the output pixel while f is the intensity of the input pixel. We slide the histogram from 0 to 255 by subtracting min from f and then we rescale this by multiplying by 255. The histogram is normalized to [0, 1] range by dividing with the difference of max and min. Histogram stretching is implemented with the following function: Figure 3.37 Histogram Stretching Function The first two if statements are there for when the auto min or max are checked. If they are checked then the min value will be the first non-zero pixel intensity starting from 0 and the max value will be the first non-zero pixel intensity starting from 255. PAGE 40 OF 133

42 The following figures are examples of histogram stretching and histogram equalization: Figure 3.38 Histogram Stretching Min 58 and Max 198 Figure 3.39 Histogram Stretching Min 7 and Max 93 PAGE 41 OF 133

43 Figure 3.40 Histogram Stretching Min 0 and Max 204 Figure 3.41 Histogram Equalization Histogram matching is the general case of histogram equalization. To achieve histogram matching we need to match the original image s histogram to a specified histogram. In our PAGE 42 OF 133

44 course we match the original histogram to an exponential histogram. The histogram matching is explained by the following figure: Figure 3.42 Histogram Matching Graph The matching of the histogram was implemented using two functions. The first functions calculated the histogram of an exponential function. I called this function getexphisto. Figure 3.43 Function for Exponential Histogram PAGE 43 OF 133

45 This function takes the exponent, the total number of pixels to be worked with, and an array of doubles to store the result. After the exponential histogram is calculated it returns the array with the exponential values. The second function, called match, gets the input image histogram, calculates the right and left limits, and the pixel that are going to be reserved. Then, it matches the input histogram to the exponential histogram to achieve histogram matching. HistogramMatching::match(ImagePtr I1, int exp, ImagePtr I2) if (exp == 0) exp += 1; IP_copyImageHeader(I1, I2); int w = I1->width(); int h = I1->height(); int total = w * h; int type; int histo[mxgray] = 0, stock[mxgray][mxgray] = 0 ; int limitleft[mxgray] = 0, limitright[mxgray] = 0 ; ChannelPtr<uchar> p1, p11, p2, end; for (int ch = 0; IP_getChannel(I1, ch, p1, type); ch++) int pixtotal = 0; int rightmost = 0; for (end = p1 + total; p1 < end;) histo[*p1++]++; for (int v = 0; v < MXGRAY; v++) limitleft[v] = rightmost; int lastr = rightmost; for (; histo[v] > 0;) if (rightmost!= lastr) pixtotal = 0; lastr = rightmost; double histoexp[mxgray]; getexphisto(exp, total, histoexp); if (histo[v] + pixtotal > histoexp[rightmost] && rightmost<mxgray) if (rightmost!= 255 v!= 255) int i = (histoexp[rightmost] - pixtotal); stock[rightmost][v] = i; histo[v] -= i; rightmost++; else stock[rightmost][v] = histo[v]; histo[v] -= histo[v]; else stock[rightmost][v] = histo[v]; pixtotal += histo[v]; histo[v] -= histo[v]; PAGE 44 OF 133

46 limitright[v] = rightmost; IP_getChannel(I1, ch, p11, type); IP_getChannel(I2, ch, p2, type); for (end = p11 + total; p11<end;) int i = limitleft[*p11]; if (stock[i][*p11] > 0) *p2 = i; stock[i][*p11]--; else if (i + 1 < limitright[*p11]) limitleft[*p11] = ++i; else limitleft[*p11] = limitright[*p11]; stock[limitleft[*p11]][*p11]--; i = limitleft[*p11]; *p2 = i; *p11++; *p2++; This function takes three arguments: the input image, the exponent, and the output image. It is a function but it computes the pixels of the output image by matching them to the exponential histogram. The following figures show some examples of histogram matching: Figure 3.44 Histogram Matching (exp = 1) PAGE 45 OF 133

47 Figure 3.45 Histogram Matching (exp = 7) Figure 3.46 Histogram Matching (exp = -8) The difference among these figures can be seen in the histogram curve of the output image. PAGE 46 OF 133

48 3.2. Neighborhood Operations The point operations that we discussed and implemented in this class are three: blurring, sharpening, and median. Each of these three filters works with the neighborhood of pixels, and hence the name of the operations. The output pixel will be a function of multiple input pixels. The following figure shows how neighborhood operations work: Figure 3.47 Neighborhood Operations As we can see from the above figure, to implement neighborhood operations we need a spatial filter. The filter that we used is called a kernel filter with odd size in order to allow for properly centered pixels. On top of the filter, we use circular buffers to store the intensities. When we deal with edge pixels we also need some padding since we need to consider all neighbors of the pixel, not only some of them. PAGE 47 OF 133

49 Blurring For the blurring filter we used spatial averaging. Blurring is used in preprocessed steps. For example, we use blurring for removal of small details from an image or for bridging small gaps in linear curves. The corresponding output pixel for each input pixel will be the average of its neighborhood pixels. Since input pixels are those located at the edges of the input image we used a circular buffer to pad the image pixels so that no error occurs. The filtering and the blurring itself are implemented separately. This means that we blur rows alone and store the result in a temporary image and then blur the columns. Figure 3.48 Blurring Panel The above figure shows the blurring panel. It is composed of two sliders and one check box. The sliders determine the width and the length of the kernel filter, while the check box makes it possible to have a synchronized movement for both sliders so that the filter is a square. The following figures show the results of the blurring filter. We notice that the effect of having PAGE 48 OF 133

50 width greater than height is the same as moving the camera left-to-right when taking a picture. When the height of the filter is greater than the width, then the effect is that of moving the camera up-and-down when taking the picture. When both sliders are synced we can see blurring in both directions in the same way. The filter function was implemented using the following code: Figure 3.49 Filter Function This function takes an input channel pointer, the size of the direction it is being applied, the incrementing steps, and the size of the filter, and an output channel pointer. In the end it calculates the resulting channel by taking the average of the pixels in the circular buffer. The blurring function uses this filter to blur horizontally, save the blur in a temporary image, then blur vertically and finally get the blurred image. PAGE 49 OF 133

51 Figure 3.50 Blurring Function The blurring filter is shown in the following figures with different width and height. Figure 3.51 Blurring (Width = 30; Height = 0) PAGE 50 OF 133

52 Figure 3.52 Blurring (Width = 10; Height = 40) Figure 3.53 Blurring (Filter size = 10x10) PAGE 51 OF 133

53 Sharpening When we sharpen an image, the intensity of smaller objects fades in the background. In this way we can easily detect larger objects since they become blob-like. Sharpening is done by subtracting a blurred version of the input image and adding a scaled difference back to it. The difference between the original input pixels and the blurred input pixel is multiplied by a factor and then transferred to the output image. Notice that we will use the same function as we used for blurring to get the blurred version of the input image. Figure 3.54 Sharpening Panel The panel of the sharpening filter is composed of two sliders. The sliders control the size of the filter and the multiplying factor. The sharpening function is implemented as shown below: PAGE 52 OF 133

54 Figure 3.55 Sharpening Function This function starts by getting a blurred version of the input image and storing it to output image. Then it goes to every pixel, subtracts the blurred version from the original image using the same blur function that was used in the blurring filter explained earlier. It multiplies this difference by the factor comes from the slider. The factored result is added the pixel intensity. If the factored result is less than zero then we set it to zero, and if the result is more than 255 then we set it to 255. Otherwise, we simply assign the factored result to the output image. The results of the sharpening filter are shown in the following figures, where the examples are chosen with different values of size and factor. The last figure has the maximum size and factor applied, that is where we can spot the sharpening easier. PAGE 53 OF 133

55 Figure 3.56 Sharpening (Size = 10; Factor = 6) Figure 3.57 Sharpening (Size = 3; Factor = 15) PAGE 54 OF 133

56 Figure 3.58 Sharpening (Size = 20; Factor = 20) PAGE 55 OF 133

57 Median The median filter is achieved by using a filter similar to the previous two filters. The neighborhood pixels are all sorted in increasing order. The output pixel is replaced with the median of the neighborhood. The shape of the filter does not need to be a square for this filter. The median filter is very useful in eliminating intensity spikes. There are some very useful properties of the median filter. This filter is an excellent noise reducer and it forces distinct pixels to conform to their neighbors. If we use a square filter, then we can eliminate pixels that are light or dark with respect to their neighbors and their area is less than half of the filter area. Unfortunately, I was unable to finish the implementation of this filter for the final program. I implemented the control panel of the filter but the actual functionality is missing on this one. Figure 3.59 Median Control Panel Even though I was not able to complete the implementation of this filter s functionality, I am confident that I will be able to finish it during the summer break and have it working by the beginning of the next semester. PAGE 56 OF 133

58 4. Conclusion This project is the biggest programming project that I have ever written during my college experience. I really enjoyed working on a project that I can see the benefits. Image processing is a very important field since it is used in such a large variety of fields. We started this semester by learning about Qt. This program was used to write the code for the GUI application. In the first part of this project I described Qt and the widgets that were used in detail for a better understanding of the program. It proved to be very useful in our class since it supports C++ programming language and it is a cross-platform program. During this semester, I was able to learn and understand how images are represented on the computer and how to utilize mathematical and coding skills to implement different filters. I learned how to operate on a point-to-point basis, where we take the image s pixels one at a time and modify their value by passing them through a filter to achieve the output intensity of the pixel. Such point operations proved to be very useful in implementing the threshold, contrast, quantization, gamma correction, color inversion, histogram stretching, histogram equalization, and histogram matching filters. These filters are of utmost importance when working with images. Their importance can be verified by looking at advanced image processing programs, such as Photoshop, where these filters all available. Furthermore, using the neighbors of the pixel is very important as well. We can implement useful filter by taking into consideration the neighborhood of a pixel. In our class we worked on three basic filters: blurring, sharpening, and median filtering. Each of this filtersare implemented using pixel replication methodology. Circular buffers were also used to make up for the neighborhood pixels that are missing from the edges. PAGE 57 OF 133

59 5. Appendix Main.cpp ====================================================================== Computer Graphics Homework Solutions Copyright (C) 2016 by George Wolberg main.c - main() function. Written by: George Wolberg, 2016 ====================================================================== #include "MainWindow.h" int main(int argc, char **argv) QApplication app(argc, argv); MainWindow window; window.showmaximized(); return app.exec(); create application create UI window display window infinite processing loop MainWindow.h ====================================================================== Computer Graphics Homework Solutions Copyright (C) 2016 by George Wolberg MainWindow.h - Header file for MainWindow class Written by: George Wolberg, 2016 ====================================================================== #ifndef MAINWINDOW_H #define MAINWINDOW_H standard include files #include <QtWidgets> #include <iostream> #include <fstream> #include <cstdio> #include <cmath> #include <cstring> #include <cstdlib> #include <cstdarg> #include <cassert> #include <vector> #include <map> #include <algorithm> #include "IP.h" #include "IPtoUI.h" #include "ImageFilter.h" PAGE 58 OF 133

60 #include "qcustomplot.h" #define MAXFILTERS 50 using namespace IP; class MainWindow : public QMainWindow Q_OBJECT public: constructor MainWindow (QWidget *parent = 0); ImagePtr imagesrc () const; ImagePtr imagedst () const; public slots: open (); save (); displayin (); displayout (); modergb (); modegray (); preview (); reset (); quit (); execute (QAction*); protected slots: sethisto(int); protected: createactions (); createmenus (); createwidgets (); QGroupBox* creategrouppanel (); QGroupBox* creategroupdisplay (); QGroupBox* createdisplaybuttons(); QGroupBox* createmodebuttons (); QHBoxLayout* createexitbuttons (); displayhistogram (ImagePtr); display (int); mode (int); enablecontrols (bool); private: QMenu* m_menufile; QMenu* m_menuptops; QMenu* m_menuneighborops; QAction* m_actionopen; QAction* m_actionsave; QAction* m_actionquit; QAction* m_actionopencamera; add open camera action QAction* m_actionthreshold; QAction* m_actioncontrast; QAction* m_actionquantization; QAction* m_actiongamma; PAGE 59 OF 133

61 QAction* QAction* QAction* QAction* QAction* QAction* homework objects ImageFilter* widgets QStackedWidget* QStackedWidget* m_actioninvert; m_actionhistostretching; m_actionhistomatching; m_actionblurring; m_actionsharpening; m_actionmedian; m_imagefiltertype[maxfilters]; m_stackwidgetimages; m_stackwidgetpanels; widgets for image display groupbox QRadioButton* m_radiodisplay[2]; QRadioButton* m_radiomode [2]; QCheckBox* m_checkboxhisto; checkbox: histogram display QWidget* m_extension; extension widget for histogram QCustomPlot* m_histogram; histogram plot int int int QString QString ImagePtr ImagePtr ImagePtr m_width; m_height; m_code; m_file; m_currentdir; m_imagein; m_imagesrc; m_imagedst; histogram variables int m_histocolor; histogram color id: 0=RGB, 1=R, 2=G, 3=B, 4=gray double m_histoxmin[4]; xmin for all histogram channels double m_histoxmax[4]; xmax for all histogram channels double m_histoymin[4]; ymin for all histogram channels double m_histoymax[4]; ymax for all histogram channels ; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::imageSrc: Source image. inline ImagePtr MainWindow::imageSrc() const return m_imagesrc; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::imageDst: Destination image. PAGE 60 OF 133

62 inline ImagePtr MainWindow::imageDst() const return m_imagedst; #endif MAINWINDOW_H MainWindow.cpp =============================================================== Computer Graphics Homework Solutions Copyright (C) 2016 by George Wolberg MainWindow.cpp - MainWindow class Written by: George Wolberg, 2016 =============================================================== #include "MainWindow.h" #include "Dummy.h" #include "Threshold.h" #include "Contrast.h" #include "Quantization.h" #include "Gamma.h" #include "InvertColor.h" #include "Histogram.h" #include "HistogramMatching.h" #include "Blurring.h" #include "Sharpening.h" #include "Median.h" #include "Camera.h" bool flag = false; enum DUMMY, THRESHOLD, CONTRAST, QUANTIZATION, GAMMA, INVERTCOLOR, HISTOGRAM, HISTOGRAMMATCHING, BLURRING, SHARPENING, MEDIAN ;, CAMERA enum RGB, R, G, B, GRAY ; QString GroupBoxStyle = "QGroupBox \ border: 1px solid gray; \ border-radius: 9px; \ margin-top: 0.5em;"; MainWindow *g_mainwindowp = NULL; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::MainWindow: MainWindow constructor. MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), m_code(-1), PAGE 61 OF 133

63 m_histocolor(0) setwindowtitle("capstone Project"); set global variable for main window pointer g_mainwindowp = this; INSERT YOUR ACTIONS AND MENUS createactions(); createmenus (); createwidgets(); enablecontrols(0); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::createActions: Create actions to associate with menu and toolbar selection. MainWindow::createActions() File Actions m_actionopencamera = new QAction("Open &Camera", this); m_actionopencamera->setshortcut(tr("ctrl+c")); m_actionopencamera->setdata(camera); m_actionopen = new QAction("&Open Image", this); m_actionopen->setshortcut(tr("ctrl+o")); connect(m_actionopen, SIGNAL(triggered()), this, SLOT(open())); m_actionsave = new QAction("&Save Image", this); m_actionsave->setshortcut(tr("ctrl+s")); connect(m_actionsave, SIGNAL(triggered()), this, SLOT(save())); m_actionquit = new QAction("&Quit", this); m_actionquit->setshortcut(tr("escape")); connect(m_actionquit, SIGNAL(triggered()), this, SLOT(close())); Point Ops Actions m_actionthreshold = new QAction("&Threshold", this); m_actionthreshold-> setshortcut(tr("ctrl+t")); m_actionthreshold-> setdata(threshold); m_actioncontrast = new QAction("&Contrast", this); m_actioncontrast-> setshortcut(tr("ctrl+c")); m_actioncontrast-> setdata(contrast); m_actionquantization = new QAction("&Quantization", this); m_actionquantization-> setshortcut(tr("ctrl+q")); PAGE 62 OF 133

64 m_actionquantization-> setdata(quantization); m_actiongamma = new QAction("&Gamma Correction", this); m_actiongamma-> setshortcut(tr("ctrl+g")); m_actiongamma-> setdata(gamma); m_actioninvert = new QAction("&Color Inversion", this); m_actioninvert-> setshortcut(tr("ctrl+i")); m_actioninvert-> setdata(invertcolor); m_actionhistostretching = new QAction("&Histogram Stretching", this); m_actionhistostretching-> setshortcut(tr("ctrl+h")); m_actionhistostretching-> setdata(histogram); m_actionhistomatching m_actionhistomatching-> m_actionhistomatching-> = new QAction("H&istogram Matching", this); setshortcut(tr("ctrl+w")); setdata(histogrammatching); Neighborhood Ops Actions m_actionblurring = new QAction("&Blurring", this); m_actionblurring->setshortcut(tr("ctrl+b")); m_actionblurring->setdata(blurring); m_actionsharpening = new QAction("&Sharpening", this); m_actionsharpening->setshortcut(tr("ctrl+r")); m_actionsharpening->setdata(sharpening); m_actionmedian = new QAction("&Median", this); m_actionmedian->setshortcut(tr("ctrl+m")); m_actionmedian->setdata(median); one signal-slot connection for all actions; execute() will resolve which action was triggered connect(menubar(), SIGNAL(triggered(QAction*)), this, SLOT(execute(QAction*))); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::createMenus: Create menus and install in menubar. MainWindow::createMenus() File menu m_menufile = menubar()->addmenu("&file"); m_menufile->addaction(m_actionopen); m_menufile->addaction(m_actionsave); m_menufile->addaction(m_actionopencamera); m_menufile->addaction(m_actionquit); Point Ops menu m_menuptops = menubar()->addmenu("&point Ops"); m_menuptops->addaction(m_actionthreshold); PAGE 63 OF 133

65 m_menuptops->addaction(m_actioncontrast ); m_menuptops->addaction(m_actionquantization); m_menuptops->addaction(m_actiongamma); m_menuptops->addaction(m_actioninvert); m_menuptops->addaction(m_actionhistostretching); m_menuptops->addaction(m_actionhistomatching); Neighborhood Ops menu m_menuneighborops = menubar()->addmenu("&neighbor Ops"); m_menuneighborops->addaction(m_actionblurring); m_menuneighborops->addaction(m_actionsharpening); m_menuneighborops->addaction(m_actionmedian); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::createWidgets: Create widgets for image display and filter control panels. MainWindow::createWidgets() assemble image display widget and control panel in horizontal layout QHBoxLayout *hbox = new QHBoxLayout; hbox->addwidget(creategroupdisplay()); hbox->setstretch(0, 1); hbox->addwidget(creategrouppanel()); create container widget and set its layout QWidget *w = new QWidget; w->setlayout(hbox); set central widget so that it can be displayed setcentralwidget(w); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::createGroupPanel: Create group box for control panel. QGroupBox* MainWindow::createGroupPanel() init group box QGroupBox *groupbox = new QGroupBox; groupbox->setminimumwidth(400); filter's enum indexes into container of image filters m_imagefiltertype[dummy ] = new Dummy; m_imagefiltertype[threshold ] = new Threshold; m_imagefiltertype[contrast ] = new Contrast; m_imagefiltertype[quantization ] = new Quantization; m_imagefiltertype[gamma ] = new Gamma; PAGE 64 OF 133

66 m_imagefiltertype[invertcolor ] = new InvertColor; m_imagefiltertype[histogram ] = new Histogram; m_imagefiltertype[histogrammatching] = new HistogramMatching; m_imagefiltertype[blurring ] = new Blurring; m_imagefiltertype[sharpening ] = new Sharpening; m_imagefiltertype[median ] = new Median; create a stacked widget to hold multiple control panels m_stackwidgetpanels = new QStackedWidget; add filter control panels to stacked widget m_stackwidgetpanels->addwidget(m_imagefiltertype[dummy ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[threshold ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[contrast ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[quantization ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[gamma ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[invertcolor ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[histogram ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[histogrammatching]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[blurring ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[sharpening ]- >controlpanel()); m_stackwidgetpanels->addwidget(m_imagefiltertype[median ]- >controlpanel()); display blank dummmy panel initially m_stackwidgetpanels->setcurrentindex(0); assemble display and mode groups into horizontal layout QHBoxLayout *hbox = new QHBoxLayout; hbox->addwidget(createdisplaybuttons()); hbox->addwidget(createmodebuttons ()); create histogram plot m_histogram = new QCustomPlot; set histogram title m_histogram->plotlayout()->insertrow(0); m_histogram->plotlayout()->addelement(0, 0, new QCPPlotTitle(m_histogram, "Histogram")); assign label axes m_histogram->xaxis->setlabel("intensity"); m_histogram->yaxis->setlabel("frequency"); m_histogram->xaxis->setautotickstep(0); m_histogram->xaxis->settickstep(32.); set axes ranges, so we see all the data m_histogram->xaxis->setrange(0, MXGRAY); m_histogram->yaxis->setrange(0, MXGRAY); PAGE 65 OF 133

67 m_histogram->setminimumheight(350); m_histogram->setinteractions(qcp::irangedrag QCP::iRangeZoom); create extension and insert histogram m_extension = new QWidget; QVBoxLayout *vbox2 = new QVBoxLayout(m_extension); vbox2->addwidget(m_histogram); start dialog with hidden histogram m_extension->hide(); set signal-slot connection connect(m_checkboxhisto, SIGNAL(stateChanged(int)), this, SLOT(setHisto(int))); assemble stacked widget in vertical layout QVBoxLayout *vbox = new QVBoxLayout; vbox->addlayout(hbox); vbox->addwidget(m_stackwidgetpanels); vbox->addwidget(m_extension); vbox->addstretch(1); vbox->addlayout(createexitbuttons()); groupbox->setlayout(vbox); return groupbox; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MainWindow::createGroupDisplay: Create group box for displaying images. QGroupBox* MainWindow::createGroupDisplay() init group box QGroupBox *groupbox = new QGroupBox; groupbox->setminimumwidth(700); groupbox->setminimumheight(700); groupbox->setstylesheet(groupboxstyle); create stacked widget for input/output images m_stackwidgetimages = new QStackedWidget; add QLabels to image stacked widget to display input/output images for(int i = 0; i<2; ++i) m_stackwidgetimages->addwidget(new QLabel); add centering alignment on both labels QLabel *label; label = (QLabel *) m_stackwidgetimages->widget(0); label- >setalignment(qt::aligncenter); label = (QLabel *) m_stackwidgetimages->widget(1); label- >setalignment(qt::aligncenter); set stacked widget to default setting: input image m_stackwidgetimages->setcurrentindex(0); PAGE 66 OF 133