CT336/CT404 Graphics & Image Processing. Section 7 Fundamental Image Processing Techniques

Size: px

Start display at page:

Download "CT336/CT404 Graphics & Image Processing. Section 7 Fundamental Image Processing Techniques"

Clara Barrett
6 years ago
Views:

1 CT336/CT404 Graphics & Image Processing Section 7 Fundamental Image Processing Techniques

2 Graphics Vs. Image Processing COMPUTER GRAPHICS: processing and display of images of objects that exist conceptually rather than physically emphasis on the generation of an image from a model of the objects, illumination, etc. emphasis (often) on rendering efficiency for real-time display DIGITAL IMAGE PROCESSING: processing and analysis of images of the real world emphasis on the modification of the image emphasis on extraction of domain-specific information from the image

5 Image Processing Techniques include.. Image enhancement provide more effective display of data for visual interpretation (e.g. human eye can distinguish c.40 grey shades) increase the visual distinction between features in a scene. "digital darkroom" techniques

6 Image Processing Techniques include.. Image rectification and restoration correction of geometric distortions, calibration of data, elimination of noise correction of blurring

7 Image Processing Techniques include.. Image classification / automatic scene interpretation / machine vision Replace or assist visual (manual) data analysis by automating the identification of features in a scene. Decision rules may be based on: Spectral radiance (e.g. colour or tone) Spatial information (e.g. geometrical shapes, sizes and patterns/textures).

8 Some Application Areas Remote Sensing Spectral classification - e.g. landcover maps Textural classification - e.g. cloud classification maps Knowledge based approaches - e.g. urban maps Robot guidance Automatic target recognition/tracking

9 Some Application Areas Industrial Inspection Measurement Quality control Medical/zoological X-Ray enhancement Counting objects of interest Morphological measurement/ classification

10 Image Processing technology platforms For high-performance image processing, and to use pre-built library functions, one good choice is Open CV Open Source Computer Vision Library (OpenCV) is an open source computer vision and machine learning software library, which is available for C and Java For the purposes of exploring the fundamentals of image processing, we are going to use Canvas and Javascript, in order to avoid learning extra technologies and using other languages

Loading and Displaying images <html> <head> <script> function loadimage() { var textbox = document.getelementbyid("filename"); var filename = textbox.value; var img = new Image(); img.

11 Loading and Displaying images <html> <head> <script> function loadimage() { var textbox = document.getelementbyid("filename"); var filename = textbox.value; var img = new Image(); img.src = filename; img.onload = function() { var wid = img.naturalwidth; var hgt = img.naturalheight; var canvas = document.getelementbyid("canvas"); // this only works if we use width as an attribute rather than a style canvas.width = wid; canvas.height = hgt; var ctx = canvas.getcontext("2d"); ctx.drawimage(img,0,0,wid,hgt); </script> </head> 01-DisplayImage.html <body> <canvas id="canvas" style="border-width:1px; border-style:solid;" width="500" height="500"></canvas><br> <input type="text" id="filename" style="width:600px;" value="images/cookiemonster.jpg"> <input type="button" onclick="loadimage();" value="load Image"> </body> </html>

Manipulating Pixels Using Javascript, we can read and write the pixels in a Canvas object Pixels are received as an array of integers, into which the cells of the image are stored in rows starting

12 Manipulating Pixels Using Javascript, we can read and write the pixels in a Canvas object Pixels are received as an array of integers, into which the cells of the image are stored in rows starting top-left Each cell takes up 4 integers of space (i.e. R,G,B,A components are separate) This is discussed well here: e.g. for an image of 3x2 pixels, the array size will be 3x2x4 = 24

13 Manipulating Pixels To read the data from an Canvas drawing context: var imagedata = ctx.getimagedata(0, 0, width, height); imagedata is an object with the following properties: imagedata.width // width in pixels imagedata.height // height in pixels imagedata.data // pixel data array of (width*height*4) elements To write modified pixel data back into the Canvas drawing context, positioned at 0,0: ctx.putimagedata(imagedata,0,0);

14 <html> <head> <script> Example.. Remove the blue from Cookie Monster (1/2) function loadimage() { var filename = "images/cookiemonster.jpg"; var img = new Image(); img.src = filename; img.onload = function() { var wid = img.naturalwidth; var hgt = img.naturalheight; var canvas = document.getelementbyid("canvas"); // this only works if we use width as an attribute rather than a style canvas.width = wid; canvas.height = hgt; var ctx = canvas.getcontext("2d"); ctx.drawimage(img,0,0,wid,hgt); // read pixels var imagedata = ctx.getimagedata(0, 0, wid, hgt);

15 (2/2) // iterate thru pixels, setting the Blue of each to zero for (x=0; x<wid; x++) { for (y=0; y<hgt; y++) { var index = (x + y*imagedata.width) * 4; /* // if we wanted to inspect the pixel data: var red = imagedata.data[index+0]; var green = imagedata.data[index+1]; var blue = imagedata.data[index+2]; var alpha = imagedata.data[index+3]; */ // actually all we want to do is modify the pixel data: imagedata.data[index+2] = 0; // set Blue component of current pixel to zero </script> </head> // copy the image data back onto the canvas ctx.putimagedata(imagedata, 0, 0); // at coords 0,0 <body onload="loadimage();"> <canvas id="canvas" style="border-width:1px; border-style:solid;" width="500" height="500"></canvas><br> </body> </html>

16 Fundamental Image Processing Techniques Global or point operations modify the value of each pixel independently E.g. histogram manipulation Local operations modify each pixel according to some function involving itself and its neighbours E.g. edge detection

17 Histogram Manipulation Grey-scale histogram: a histogram of the frequencies of each of the grey levels Manipulation of histogram: a point operation Distribution of grey levels: concentrated near a certain level => low contrast/ poor visual interpretability. wide range of levels is represented => high contrast.

18 Histogram Manipulation Contrast stretching spaces out values so that they are more easily distinguishable. Uniform expansion/ linear stretch assigns as many grey levels to rarely occurring values as to frequently occurring ones. Histogram-equalised stretch: image values are assigned to new values on the basis of their frequency of occurrence: more display values (and hence more detail) are assigned to the frequently occurring portions of the histogram. The aim is to maximise the overall contrast: a nearly uniform (i.e. flat) distribution is produced. Special analyses can be made by stretching a particular range of image values over the entire range, while disregarding the other image values. (show IPTK with 9532.bmp)

19 Thresholding A simple segmentation technique that is very useful for separating solid objects from a contrasting background. All pixels above a determined (threshold) grey level are assumed to belong to the object, and all pixels below that level are assumed to be outside the object (or vice versa) The selection of the threshold level is very important, since it will affect any measurements of parameters concerning the object (the exact object boundary is very sensitive to the grey threshold level chosen). Thresholding is a point technique however locally adaptive thresholding" determines different local thresholds for sections of an image

20 Example: thresholding an image in Canvas/Javascript <html> <head> <script src="imageprocessingfunctions.js"></script> <script> // these globals are set in readimage() var imgwidth, imgheight; var ctx; function loadimage() { var textbox = document.getelementbyid("filename"); var filename = textbox.value; var canvas = document.getelementbyid("canvas"); readimage(canvas,filename); function dothreshold() { var level = document.getelementbyid("threshlevel").value; var imagedata = ctx.getimagedata(0, 0, imgwidth, imgheight); greyscale(imagedata); threshold(imagedata, level); ctx.putimagedata(imagedata, 0, 0); </script> </head> <body> <canvas id="canvas" style="border-width:1px; borderstyle:solid;" width="500" height="500"></canvas><br> <input type="text" id="filename" style="width:600px;" value="images/cookiemonster.jpg"> <input type="button" onclick="loadimage();" value="load Image"><br> <input type="text" id="threshlevel" style="width:100px;" value="127"> <input type="button" onclick="dothreshold();" value="threshold"> </body> </html>

21 imageprocessingfunctions.js (1/2) // useful image processing functions for Canvas imagedata function readimage(canvas,filename) { var img = new Image(); img.src = filename; img.onload = function() { var wid = img.naturalwidth; var hgt = img.naturalheight; canvas.width = wid; canvas.height = hgt; ctx = canvas.getcontext("2d"); // global var ctx.drawimage(img,0,0,wid,hgt); imgwidth = wid; // global var imgheight = hgt; // global var

22 function greyscale(imagedata) { var wid = imagedata.width; var hgt = imagedata.height; for (x=0;x<wid;x++) { for (y=0;y<hgt;y++) { var index = (x + y * imagedata.width) * 4; vargrey = Math.round((imageData.data[index+0] + imagedata.data[index+1] + imagedata.data[index+2])/3); imagedata.data[index+0] = imagedata.data[index+1] = imagedata.data[index+2] = grey; function threshold(imagedata, level) { // assumes image is greyscale (i.e. RGB all the same) // so we will only read the Red channel var wid = imagedata.width; var hgt = imagedata.height; for (x=0;x<wid;x++) { for (y=0;y<hgt;y++) { var index = (x + y * imagedata.width) * 4; var grey = imagedata.data[index+0]; var output; if (grey>=level) output=255; else output=0; imagedata.data[index+0] = imagedata.data[index+1] = imagedata.data[index+2] = output; (2/2)

23 Convolution (Filtering) A fundamental (and very flexible) local operation Used for a variety of tasks, including noise removal, blurring, sharpening, and edge detection. Establish a moving window ('kernel') which contains an array of coefficients or weighting factors. Move the kernel across the original image, so that it centres on each pixel in turn. Each coefficient in the kernel is multiplied by the value of the pixel below it, and the addition of each of these values determines the value of the pixel in the output image corresponding to the pixel in the centre of the kernel.

24 Convolution (Filtering) The effect of applying a kernel to an image depends directly upon the size of the kernel used and the values of its coefficients e.g. this kernel would be responsive to vertical lines/edges: e.g. this one would perform a blur: /9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 See:

25 Exercise..? Write a Javascript function for performing an arbitrary 3x3 convolution based on an array of 9 values passed to it Use this function to perform the two filters from the previous slide, using a Canvas object

26 Enhanced vertical edges

27 <html> <head> <script src="imageprocessingfunctions.js"></script> <style> input { width:25px; </style> <script> // these globals are set in readimage() var imgwidth, imgheight; var ctx; function loadimage() { var textbox = document.getelementbyid("filename"); var filename = textbox.value; var canvas = document.getelementbyid("canvas"); readimage(canvas,filename); function doconvolution() { var kernel = new Array(); for (cell=0;cell<9;cell++) kernel[cell]=parseint(document.getelementbyid("cell"+cell).value); var imagedata = ctx.getimagedata(0, 0, imgwidth, imgheight); greyscale(imagedata); convolution3x3(imagedata,kernel); ctx.putimagedata(imagedata, 0, 0); </script> </head> <body> <canvas id="canvas" style="border-width:1px; border-style:solid;" width="500" height="500"></canvas><br> <input type="text" id="filename" style="width:600px;" value="images/cookiemonster.jpg"> <input type="button" style="width:100px;" onclick="loadimage();" value="load Image"><br> <input id='cell0' value='0'><input id='cell1' value='0'><input id='cell2' value='0'><br> <input id='cell3' value='0'><input id='cell4' value='0'><input id='cell5' value='0'><br> <input id='cell6' value='0'><input id='cell7' value='0'><input id='cell8' value='0'><br> <input type="button" style="width:180px;" value="apply Convolution Kernel" onclick="doconvolution();"> </body> </html>

28 function convolution3x3(imagedata,kernel) { // create a temporary array for output var outputdata = new Array(); var sz = imagedata.data.length; for (i=0;i<sz;i++) outputdata[i] = imagedata.data[i]; var wid = imagedata.width; var hgt = imagedata.height; for (x=1;x<wid-1;x++) { for (y=1;y<hgt-1;y++) { var weightedsum = 0; for (xx=-1;xx<=1;xx++) { for (yy=-1;yy<=1;yy++) { var index = (x+xx + (y+yy)*wid) * 4; var grey = imagedata.data[index+0]; var kernelcellindex = (xx+1 + (yy+1)*3); weightedsum += grey*kernel[kernelcellindex]; var index = (x + y * wid) * 4; var outputgrey = weightedsum; outputdata[index] = outputgrey; // red channel outputdata[index+1] = outputgrey; // green channel outputdata[index+2] = outputgrey; // blue channel Additional function in imageprocessingfunctions.js imagedata[x+xx][y+yy][0] // copy data from output array to original array for (i=0;i<sz;i++) imagedata.data[i] = outputdata[i];

Image Processing. HTML5 Canvas JavaScript Pixels

Image Processing. HTML5 Canvas JavaScript Pixels Image Processing HTML5 Canvas JavaScript Pixels Brent M. Dingle, Ph.D. 2015 Game Design and Development Program Mathematics, Statistics and Computer Science University of Wisconsin - Stout Provide Examples