Machine Vision. Laboratory Exercise Name: Student ID: S

Transcription

1 Mahine Vision S Laoratory Eerise 2011 Name: Student D:

2 General nformation To pass these laoratory works, you should answer all questions (Q.y) with an understandale handwriting either in English or in Finnish. Continue the answers to an attahment if you will, ut rememer to mark whih answer is for whih question. Return the orretly filled report to Juho Kannala (room TS333) and all the MATLAB sripts (m-files) resulting from tasks (T.y) should e (in)zipped and send to juho.kannala (at) ee.oulu.fi. Name the funtions as Ty.m. lease note that this work should e done individually. 1. BNARY MAGE ANALYSS 1.1. mage Thresholding mage thresholding means that an image is onverted into a inary image y assigning eah piel the values 1 or 0 aording to a threshold (or thresholds). Q1.1. n whih ways a graysale image an e thresholded? (See Chapter 3 in ourse material.) Q1.2. Eplain what kind of information ould e used to determine a suitale threshold automatially. n MATLAB, image thresholding an e implemented in various ways. Here is an eample sript whih thresholds a graysale image aove with a threshold of 128 and displays the images: = imread( kuva.tif ); in = >128; suplot(1,2,1),imshow(),title( Graysale ) suplot(1,2,1),imshow(in),title( Binary ) Two inary images an e omined with logial itwise operators & (and), (or) and ~ (not): A = B & C; T1.1. rite a MATLAB funtion that selets automatially two thresholds for an input image and thresholds the image oth etween and outside these two thresholds. Test your funtion with the image rose.jpg (you should e ale to separate the most of the rose from the akground). 2/13

3 1.2. Conneted Components Laeling A laeling algorithm an e used to ount inary ojets in an image. Usually, morphologial operation(s) preede this. The ojets an e onsidered as 4- or 8-onneted. The outome of a laeling algorithm is an image, with 1 s replaed with numers assigned for separate inary areas. Laeling ould e used, for eample, for ounting alls in an image like Figure 1. Figure 1. mage alls.gif ontaining a large numer of alls. T1.2. Study funtion wlael and write a sript that ounts the alls in Figure 1 (alls.gif). Use morphologial operation(s) to separate the alls. hih operation(s) did you use? How many alls are there? Q1.3. hat kind of properties ould e alulated to inary ojets for further, higher-level analysis? 1.3. attern Reognition attern reognition is a very general term and (inary) image analysis is only one appliation area of it. n this eerise we fous on register plate reognition prolem. Basi onept of the solution is to represent a harater in a register plate with some feature vetor and assign this feature vetor to a predetermined lass aording some riteria. That is, we represent eah lass with a feature vetor etrated from sample haraters (learning phase) and then ompare the feature vetor etrated from an input image to the (learned) lass representatives. Start y studying asi region properties and pattern reognition onepts in leture material (hapter 3, slides 19-21, and hapter 4, respetively). Study also MATLAB mage roessing Toolo funtions regionprops, imsale, and imrotate. T1.3. mplement a register plate reognition program in order to reognize haraters in the 14 plate images register1.mp,, register14.mp. Note, that images are aptured with different lighting and optis settings, as well as in different sales and rotations. So your ode should e (at least 3/13

4 in some amount) invariant to these fators. You an find eamples for region property omputation and image saling in MATLAB sript regplate.m. Your program should inlude: a) A teahing sript (use regmodel.mp for teahing) in whih you ompute feature vetors representatives for all five haraters (R, F, 6, 3, and 8) of the imaged plate. Name this as T13a.m ) A reognition sript in whih you ompute feature vetor for eah harater deteted in atual register plate images, ompare these measured feature vetors to learned ones y using some appropriate method or distane measure (e.g. nearest neighor lassifier and Eulidean distane), and display the haraters reognized. Name this as T13.m. (NOTE: Your program does not need to e ale to reognize all haraters in all given images, see Q1.6. ) Q1.4. hih features did you use for the representation of the haraters? hy just these? Q1.5. hih distane measure or method did you use for the lassifiation? hy? hat kind of limitations the method you hose might have in this ase? Q1.6. Study the reognition performane of your program for given images. n whih ases your ode suess and why, in whih ases it fails and why? 4/13

5 2. COLOR ANALYSS n this setion we onsider prolems related to leture material Chapter 6. n addition of analyzing different olor oding shemes, a method known as histogram akprojetion is introdued for skin detetion in images Color Spaes Color offers lot of information aout an image. Usually, digital image piels are RGB oded, where individual hannels stand for intensity of the wavelengths that are interpreted as red, green and lue y the human visual system. RGB oordinates are the usual output of image apturing hardware, ut for olor analysis it is usually more useful to sort the hromatiity and intensity information to different omponents. Besides the ourse material, information aout olor spaes an e found e.g. from ikipedia: Q2.1. List at least three different olor oding shemes esides RGB. Q2.2. Eplain what the omponents of HS olor spae stands for in your own words. Q2.3. n general olor oding shemes define the intensity and hromatiity of olor what are their physial meanings? 2.2. Skin Detetion Color an e used for image segmentation. For eample, skin areas an e deteted in images ased on their disriminative hromatiity. Net, we uild a simple skin piel lassifier ased on a simple yet effetive method alled the histogram akprojetion, whih is a way of reording how well the piels fit the distriution of piels in a histogram model. The first task is to onvert the images into some olor spae revealing hromatiity. Here we use normalized RGB oordinates, also referred as normalized olor oordinates (NCC). The omponents are formed from RGB oordinates as follows: = R + G + B r = R G g = B =. 5/13

6 Seondly, two-dimensional olor histogram H(n r, n g ) is reated from a sample image. The sample image is onverted to r,g oordinates and quantized to N (numer of ins in the histogram) levels. nitially the histogram is set to zero. The sample image is sanned through pielwise, and for eah piel the in speified y the quantized oordinates is inremented. Now we have a olor model that represents the sample image. Net, a olor likelihood image LH (,y) is reated y projeting the olor model to an image under analysis (,y). The piel values of the likelihood image tell how well a piel mathes with the olor model. For eah piel in the likelihood image, a in value from the olor histogram is assigned. The in is speified y the image olor oordinates of the orresponding piel: LH (,y) = H(n r, n g ), where (n r, n g ) are the normalized olor oordinates of piel (,y). (,y) LH (,y) H(n r, n g ) N N- Figure 2. llustration of histogram akprojetion 1 and how the piel values and the histogram are related. T2.1. rite a funtion that implements the aove mentioned method using the skin.mp as a sample image. The funtion must onvert the sample image to normalized olor oordinates, quantize them to 128 levels and reate a histogram of the sample. Then the funtion transforms and quantizes the image, from whih the skin piels ought to e deteted, as aove. Finally the funtion reates a olor likelihood image whih is thresholded to a inary image, where 1 s and 0 s denote skin and non-skin piels respetively. You an use the prewritten funtion skindetetion.m. Try it on images dudes.jpg and fae.mp in addition with random skin ontaining piture seleted from images.google.fi. Q2.4. hy does the lassifier not detet all the skin piels and why does it make false detetions? 6/13

7 3. CONTENT BASED MAGE RETREVAL mages an e retrieved from a dataase ased on their ontent. Although ontent ased image retrieval still operates on rather primitive data, ommerial and researh systems are already uilt. dea in retrieval is to measure image similarity etween a query image and images in dataase and result most similar images mage Comparison mage Features mage Features Math Resulting image(s) Figure 3. A simple ase of ontent ased image retrieval. Q3.1. hat are the major similarity measures for image omparison? T3.1. Take few minutes to test different measures with a test dataase in the address 7/13

8 3.2. Teture Similarity n this part, we eamine the Loal Binary attern (LB) operator as a measure of teture. LB has numerous variations, ut in this eerise we use the simple 33-neighourhood graysale LB. Eah piel in an image an e assigned with a LB-value y evaluating its 33-neighorhood. Neighoring piels are thresholded as 0 s and 1 s with entral piel value. The thresholded piels are then weighted and summed as illustrated in Figure 4. Figure 4. A simple 33-neighourhood LB. An image an e presented with a LB-histogram, whih is uilt y using a LB-operator for eah piel (eept the ordering piels) in a graysale image. An eample ode for LB-histogram omputation is given in LB.m. The similarity of two histograms an e evaluated y some metri. Among others, a simple measure of similarity is S H1, H 2 n min( H i 1 = n i H [ i], H 2[ i] [] i 1 ). ith this measure, a ontent ased retrieval an e implemented y omparing query-images similarity S H1,H2 to all of the images in the dataase. T3.2. Q3.2. mplement a dataase retrieval funtion that takes an image as an input and displays five losest images in a figure. Use the similarity metri aove and images im1.mp,, im16.mp as a dataase. Test the system y using dataase images as query images and see if similar images result. Use the prewritten funtion retrieve.m. Define the terms preision and reall in the ontet of image retrieval. Q3.3. Mention at least two other teture features esides the LB. 8/13

9 4. MOTON FROM 2-D MAGE SEQUENCES Aquiring motion from image sequenes is ommon researh area in omputer vision. rolems range from simple movement detetion to high-level analysis of multiple ojet shape and motion. The ompleity of the prolem is defined y many things, inluding the numer of ojets eing traked simultaneously and is the amera stati or moving. Bakground Sutration n the ase of stati amera and stati akground, ojets an e traked with a method alled akground sutration. A simple version of that is to detet hanges in images y sutrating them from an image ontaining the akground and thresholding the result as Figure 5 depits. image1 - image1-image2 Deteted ojets image2 Figure 5. Bakground sutration for moving ojet detetion. Usually the inary image aquired y thresholding is enhaned with morphologial operations for eample, losing ould e used to fuse neighoring areas. Also small noise regions should e removed. After morphologial operations, image is laeled for further ojet analysis. Sutration an happen etween susequent images or etween urrent image and an image of an empty sene. Q4.1. n whih situations akground sutration methods ould fail? hy? T4.1. mplement a traffi traker system that uses the akground sutration method. The system should detet moving ojets and display the urrent frame with ounding oes for the ojets. Use the prewritten sript traker.m. 9/13

10 5. CAMERA CALBRATON AND 3D MEASUREMENT Camera aliration means aquiring transformation from real world oordinates to image oordinates. The dependeny etween the two oordinate systems an e presented ompatly y a amera matri C, so that s = y s y C, z s 1 = C, where is the oordinates in the image plane and is the oordinates in the real world. Camera aliration is needed, for eample, when images are used for measuring purposes. Figure 6 illustrates the amera and the world oordinates. Figure 6. orld point projetion to a amera plane. n pratie, aliration means alulating the 11 parameters of the amera matri: C = Best Affine Caliration Matri One approah to estimate the matri C is to solve so alled est affine amera (or aliration) matri as presented in leture slides (Chapter 13). n this setion we try to alirate a we-amera y imaging a normal offie ookshelf. After the aliration we eperiment to use the otained amera matries for 3D measurements. T5.1. rite a MATLAB funtion that omputes the est affine amera (or aliration) matri given the image oordinates of known world points. You an use the method eplained in Chapter 13 of ourse material. Also, hek that your ode produes results similar to the eample of Chapter 13. (Eample data is given in Figure in the ourse ook or on the slide Best affine aliration matri: eample in the leture material.) 10/13

11 T5.2. rite a MATLAB sript that solves amera matries for images ookshelf1.jpg and ookshelf2.jpg with your funtion T51.m (you need to solve the amera matri for oth images separately, eause the amera has een moved etween the shots). Knowledge of the dimensions of the shelf (see Figure 7) together with the image oservations an e used to estimate the amera matri as follows: a) Fi world oordinate system (origin and oordinate ais diretions) somehow to the ookshelf and determine world oordinates of at least si (preferaly more) visile orners of the ookshelf in this oordinate system. Use orners from different depths in the shelf. ) Detet image oordinates orresponding to these orners. You an do this manually, MATLAB funtions ginput and pselet may e useful for this task. List si of the oordinates (inluding origin) into the tale elow. ) Estimate the est affine amera matries using the orresponding world and image oordinates and your funtion T51.m. ookshelf1.jpg ookshelf2.jpg orld y 2 2 y y z Computing 3D oints Using Two Cameras Assuming we have two images with known amera matries B and C and a same 3D point seen from oth images, the following equations an e derived s = s y s y z 1, and t t 2 2 = y t y. z 1 From these, we get y eliminating sale fators s and t y y y y y y = y z y y , or equivalently A =, of whih the oordinates of the 3D point an estimated, for eample, y using the least squares method. 11/13

12 T5.3. rite a MATLAB sript that omputes 3D oordinates of some points visile in oth ookshelf1.jpg and ookshelf2.jpg using prewritten funtion ompute3d.m. Eamine the feasiility of the approah for 3D measurements as follows: a) Detet orresponding image oordinates for some shelf orners visile in oth images. Again, you an do this manually, for eample, y utilizing ginput or pselet. ) Estimate the 3D oordinates separately for eah orner point. ) Study the auray of the estimated oordinates y omparing them to the known ones (study separately points that you used in the amera matri estimation and points that you did not use in that phase). For instane, you an determine the dimensions of the ookshelf ased on the 3D estimates and ompare them to the atual values. Q5.1. How do you omment the auray of the estimates in general? hih may e the main error soures in this ase? rite here your feedak aout the ourse (letures, eerises, and laoratory work). Thank you. 12/13

13 800 mm mm 22 mm 22 mm Figure 7. Dimensions of the ookshelf. 13/13