1 Background and Introduction 2. 2 Assessment 2

Transcription

1 Luleå University of Technology Matthew Thurley Last revision: October 27, 2011 Industrial Image Analysis E0005E Product Development Phase 4 Binary Morphological Image Processing Contents 1 Background and Introduction 2 2 Assessment 2 3 Object Detection in Binary Images Detecting Circles Detecting Rectangles Detecting Squares PCB Component Detection 4 5 Blurp Box Incorporated Detecting Circles Optional Exercise Segmentation of Overlapping Objects Tips for Implementing the Distance Transform Reporting Assessment 10 7 Appendix : Calculating a Best-Fit Circle 11 1

2 1 Background and Introduction Read all of these instructions before starting. This week you will create some of the operations used to detect shapes for Blurp Box Incorporated in addition to learning about binary morphological image processing. Parts of this task are marked Optional and you should only do these if you have time. Remember that you need to write a clear and reasoned report outlining how and what you have done as instructed in the previous development phases. 2 Assessment Each Development Phase will be graded fail (U), pass (3), good (4), or very good (5) based on the following criteria; 1. How well does the report answer all the questions outlined in the development plan. Does the student provide a thorough and complete answer? 2. How clearly written is the report and how well reasoned is the explanation? How easy would it be for a technically qualified person to understand what you have done and why, and then follow the development phase instructions and your report and repeat the results? 3. When you are asked to explain or discuss something, explain why your result is this way in relation to the theory from the course material (necessary in order to achieve a grade of 4 or 5) 4. Have the optional parts of the Development Phase been answered and how thoroughly? (necessary in order to achieve a 5 grade) 3 Object Detection in Binary Images Binary morphological image processing can be used to detect objects in images. Load and view the image lab4shapes.tif The image consists of 2 circles, 2 rectangles and 3 squares. The circles have a diameter of 30 pixels, the lengths of the rectangles sides is 12 by 60 pixels and the squares have side lengths of 12 pixels. In this exercise you will implement three functions that will detect the three objects in the image. The functions will use the morphological operations opening and dilation. 3.1 Detecting Circles First, detect the largest objects in the image. As the circles are the largest objects in the image implement a function C = findcircles(im,radius). The circles are detected using 2

3 an opening by a flat circular disk with the radius less than the circles but greater than the size of other objects. For example: % Construct a Euclidean disk structuring element with radius 20 % Do not use structuring element decomposition. Set third argument to 0 sed=strel( disk,20,0); % Open image Im with structuring element sed C=imopen(Im,sed); Now use findcircles to detect the circles in the image and view the result. You will see that the opening by a flat circular disk removes objects where the disk does not fit. To find other objects in the image, the detected circles from the image have to be removed. The circles are removed from the original image by subtracting C from Im using the following Matlab command: D = im2bw(imsubtract(im,c)); Study the image where the circles are removed. You will notice that most of the circles have been removed but a few pixels at the border of the circles are left. So, the function findcircles detects the circles but fails to detect some pixels at the boundary. To make the detection more robust you should dilate the detected circles so that the size will grow with one pixel at the border. That is done by dilating with a diamond shape structuring element of size 1. As the dilated version of the circles may be bigger than the circles in the original image, the circles are finally detected by the intersection of the original image and the dilated version of the detected circles. The above operation is performed with the following MATLAB command: sex1=strel( diamond,1); Cd=imdilate(C,sex1); C2=and(Im,Cd); Now remove the circles that where detected using the more robust method from the original image and call the new image D2. Study the image D2 where you should see that the circles have completely been removed. Use D2 in the next step where you detect rectangles. 3.2 Detecting Rectangles Now implement a function R = findrectangles(im,[sizei sizej]) that detect rectangles in an image with size greater than sizei * sizej. A rectangular structuring element is generated with: ser=strel( rectangle,[sizei sizej]); Use the function findrectangles to open the image D2 by a rectangular structuring element of size 10 * 58 to detect the rectangles. Call the result E and study the result. Remove the detected rectangles from D2, call the new image F and continue to the next step. 3

4 3.3 Detecting Squares Now implement a function R = findsquares(im,size) that detect squares in an image with side lengths greater than a parameter size. A square structuring element whose width is size pixels is generated with: ses=strel( square,size); Use the function findsquares to open the image F by a square structuring element of size 10 * 10 to detect the remaining squares. Call the result G and study the result. Finally remove the detected squares from F and check that no more objects are in the image. Reporting: There is nothing to report for this section as you will need these functions that you have developed in the next section. 4 PCB Component Detection In this section you will use binary mathematical morphological operations to detect different components on a printed circuit board. Load and view the image pcb.tif shown in figure 1. Figure 1: Binary printed circuit board image You will have to detect six different types of components (not in this order); Circular contacts Square contacts Rectangular contacts Thick tracks Thin tracks Holes Use the functions implemented in section 3 to detect all components and use the same iterative strategy outlined in section 3. You have to find suitable sizes of structuring elements and remember that this is an real image where, for instance, a square contact on the printed 4

5 circuit board is not a perfect square. Remember that to make the detection more robust, use the technique explained when circles where detected. Hint: First fill the holes in the image using the morphological operation imfill mentioned in lecture 4. The residue of the subtraction of the filled image without holes and the original image are the holes. It is easier to detect all components using the filled image. When all components have been detected, the holes can be set to zero to show the final result. Reporting Describe in detail how your detection algorithm works. What structuring elements are used and in what order. Provide images that explain the process of detecting all components. Also provide a final image where each component type is labelled with a specific gray level intensity so that it is easy to see and identify each component in the image. You do not have to supply your code for the findcircles, findsquares, findrectangles functions but supply your code for your method that calls these functions so your technical supervisor can see the structure of your detection algorithm. 5 Blurp Box Incorporated 5.1 Detecting Circles You will now construct a useful operation that will be applied to the Blurp Box conveyor belt manufacturing line. Construct images from the conveyor belt as you have previously done with the following commands; load conveyor box= MakeBoxFrame(conveyor); imagesc(box/255) Produce a grayscale version of box as you did in Development Phase 1. Later in the course we will use a more reliable technique to extract a grayscale image and to threshold it, but for now use the following command; box = 255*rgb2gray(box/255); Your job is now to construct a function Out = IsCircle(Im,Tl,Tu,Tc) that takes as input a grayscale image Im and three threshold values Tl,Tu,Tc. The output from the function should be Out = 1 if the image contains a circular blurp box and Out = 0 otherwise. The suggestion is that you perform the following steps (but you are free to chose other ways): Segment the image as you did in the Blurp Box Incorporated section of lab 1 (using the threshold values Tl and Tu). Apply a binary opening on the segmented image (to remove noise points). Use the function bwfill to get rid of the holes in the object. Use the function bwperim to find the perimeter (edges) of the object. 5

6 Now, using the perimeter points, find estimates of the centre point (x 0,y 0 ) and radius R of a best-fit circle (see Appendix 1 for the equation of a circle and how to estimate x 0,y 0 and R). Figure 2 shows an example of the kind of image and circle estimate you might produce. The object shown in figure 2 might not be very accurately approximated by a circle. One possible measure of how wrong the estimate is is how much the circle deviates from the object boundary such as with the weighted sum of the error area (shown in figure 2) and given by equation 1. Use the weighted sum of the error area calculated in equation 1 to produce Out = 1 (i.e. you detected a circle) if ǫ < Tc and Out = 0 otherwise. A value of 0.1 might be a reasonable value for T c. Figure 2: Rectangular shape on the conveyor belt with an estimated circle draw over the top (left image). The error area is shown as the non intersecting parts of both the rectangle and the circle (right image) ǫ = 1 mr m (xk x 0 ) 2 +(y k y 0 ) 2 R 2 (1) k=1 Optional Exercise: You are free to try other solutions than that outlined above. As you discovered in Phase 1 the threshold operation may not always produce a valid object when applied to the grayscale box images, but we must accept this error for now. We will rectify this problem when we learn about Color Image Processing. Reporting: Supply your well commented code for iscircle and make sure to explain what you have done so your technical supervisor can easily see any additions or modifications to the algorithm suggested here. 5.2 Optional Exercise Warning: this is a very unconstrained optional exercise. Be aware of the time you have available. Some tasks you will face in industry can not be completely resolved in the time you will have available. You may only be able to test a few things and make some comment or recommendation to your technical supervisor. Document ALL your actions and findings in this exercise. 6

7 Energy Technology Centre (ETC) is doing some experiments with high speed images of a liquid jet stream captured at 4000 frames per second. The droplets are moving at high speed from left to right in the images and the camera is setup opposite a light source in an arrangement called back lighting. ETC wants to find individual droplets of liquid in the jet stream and classify them into different sizes. How many droplets of each different size. A sequential series of four of these images is named P1 T1 3mm00000Ns.png where N is the sequence number 1,2,3,4. Image 1 is shown in figure 3. Figure 3: Jet stream of droplets The background illumination in these images is very uneven so you will need to attempt to remove this variation. As these images are of the same scene, and the light source is not moving (only the droplets of liquid), you could combine all four images by taking their maximum value to create an image that approximates the light source, without the droplets. C = max(a,b); % calculate a new image C, that contains the maximum intensity pixels from images A and B Once you have an approximation of the light source, lets call it image L, subtract one of the original images from it. You should get an image that has mostly just the droplets of liquid. Consider whether you should contrast enhance this image further, and how. Perform a threshold operation on your image to obtain a binary image of the droplets. Comment on how this works, are you getting too much background, or not enough droplets, do you need to filter the image in any other way? Once you have a binary image of the droplets, consider how you could use morphological operations with a disk structuring element to remove droplets in a similar way to how you have identified and removed the circles on the circuit board. Start by trying to remove very small droplets first. Remember, this is an optional exercise, you need to manage your time to see what you can achieve in the available time. You are trying to give your technical supervisor a good idea of a strategy for solving this problem, you don t have to solve it perfectly, but make sure you explain and comment on the problems you find. Reporting: Show the images and code for all the methods you tried, and provide comments 7

8 and explanation of your reasoning. 5.3 Segmentation of Overlapping Objects Load the Matlab file circles2.tif and look at it. This is a thresholded (i.e. a binary) image from the Blurp Box Incorporated production line, but obviously something has gone wrong. Several circular shapes has been piled on top of each other on the same frame. It is therefore necessary to find a way to separate the four shapes, and this can be done using the morphological watershed transform to do this. The first step is to calculate the distance transform to make a 3D surface upon which the watershed segmentation can be performed (discussed in lecture 4). Figure 4: A binary image Consider the binary image shown in figure 4. Using this image calculate the distance transform d(i,j) as follows; for each pixel in the input binary image Im(i,j) that equals 1, the distance transform equals the distance to the nearest other pixel (i min,j min ) with value 0. Otherwise the distance transform is 0. The distance is calculated using equation 2. d(i,j) = { 0 ;Im(i,j) = 0 (i imin ) 2 +(j j min ) 2 ;Im(i,j) = 1 (2) Write a function d = disttrans(im) that, given a binary image Im, produces the distance transform d. Refer to section 5.4 for some tips for implementing the distance transform. Apply this operation on the image Im and study the result. Test the 3-D MATLAB plot mesh(-d) and see if the analogy with water basins and watersheds is relevant. Use your distance transform function to produce the distance transform image d of the original image circles2.tif. Segment this image with the help of the watershed transform (as discussed in lecture notes 4). An implementation of this segmentation algorithm is available in MATLAB using the command w = watershed(-d); The matrix w will contain a number of different regions (areas of different graylevel values) each representing one object in the original image circles2.tif. Study the transform d 8

9 and examine how well the segmentation has worked by giving the following command (use rescale) image(circles2.*(w==k)); where k = 0,1,2.3,... (you should have 4-5 areas if the segmentation has worked OK). Reporting:Using rescale supply the image (circles2+w), that is, the sum of the original image and the transform. Comment on the result and present your code for disttrans. 5.4 Tips for Implementing the Distance Transform When calculating the distance transform you will need to detect pixels with value one as shown by the region marked inner area in figure 5. Then determine the position of the closest pixel in the outer area (holding the value zero). It is clear that no such closest pixel can be located anywhere else than on the external boundary (see lecture notes 4 for how to calculate the external boundary) of the inner area. It is therefore only necessary to investigate these pixels. The coordinates of all pixels with value one in a binary perimeter image can be found with the find function (see example in the Appendix). Figure 5: Binary image (left image), External boundary (right image) 9

10 6 Reporting Assessment Using the assessment criteria, assess your own development report and write a short evaluation of how well it satisfies the criteria? How well does your report answer all the questions outlined in the development plan. Do you provide a thorough and complete answer? How clearly written is your report and how well reasoned the discussion. How easy would it be for a technically qualified person to follow your report to repeat the results? Have you answered the optional parts of the Development Phase been answered and how thoroughly? Do you have time for this? (necessary in order to achieve a 5 grade) Are there any problems with your solution? That is, situations where it doesn t work as expected? Your technical supervisor will need to know about these situations so that they can be solved and don t cause unexpected problems later during system integration and installation in week 8. The purpose of this assessment question is for you to evaluate your own work like you would have to in a real work situation before handing it over to your technical supervisor. 10

11 7 Appendix : Calculating a Best-Fit Circle The equation for a circle can be written as shown in equation 3 where (x 0,y 0 ) is the circle centre and R the radius. By using the partial derivatives shown in equation 4 it is possible to construct a linear least-squares problem involving a discrete set of m circle coordinate pairs (x 1,y 1 ),(x 2,y 2 ),...,(x m,y m ). f(x 0,y 0,R) = (x x 0 ) 2 +(y y 0 ) 2 R 2 = 0 (3) δf = 0, δf = 0, δf δx 0 δy 0 δr = 0 (4) The linear system Ma = v can be expressed in equation 5 where M is a m by 3 matrix and v is a m long column vector. x 1 y 1 1 x 2 y 2 1 x m y m 1 a b c = x 2 1 y2 1 x 2 2 y2 2 x 2 m y 2 m (5) The solution is given by a = (M t M) 1 (M t v) where the vector a is given by equation 6 from which x 0,y 0 and R can be determined. a b c = 2x o 2y 0 x 2 o +y 2 0 R2 (6) To form the matrix system for a circle in a binary image Im it is convenient to use the command [X Y] = find(im==1) This will give the column vectors X = [x 1,x 2,,x m ] t (i.e. all the possible circle coordinates in the i dimension), and Y = [y 1,y 2,,y m ] t (all the possible circle coordinates in the j dimension). M, v, a can be finally calculated using the following commands M = [X Y ones(size(x))]; v = -X.^2 - Y.^2; a = inv(m *M)*(M *v); 11