CS 378 Computer Vision Problem set 4 Out: Thursday, Nov 5 Due: Tuesday, Nov 24, 11:59 PM. See the end of this document for submission instructions.

Size: px

Start display at page:

Download "CS 378 Computer Vision Problem set 4 Out: Thursday, Nov 5 Due: Tuesday, Nov 24, 11:59 PM. See the end of this document for submission instructions."

Gwen Henry
6 years ago
Views:

1 CS 378 Cmputer Visin Prblem set 4 Out: Thursday, Nv 5 Due: Tuesday, Nv 24, 11:59 PM See the end f this dcument fr submissin instructins. I. Shrt answer prblems [30 pints] 1. The SIFT descriptr is frmed using a 4 x 4 grid f histgrams, each f which cunts the underlying pixels gradient directins fr each f 8 pssible rientatins, yielding a 4*4*8 = 128-dimensinal descriptr fr an image patch. If we were t instead use an 8 x 8 grid f histgrams fr the same image patch, we wuld have an 8*8*8 = 512-dimensinal descriptr. What impact wuld this adjustment have n the feature descriptr s invariance prperties? 2. RANSAC is useful fr rbustly estimating the hmgraphy matrix (e.g., fr the msaic stitching applicatin) and fr rbustly estimating the fundamental matrix (fr stere with weakly calibrated cameras). Explain what defines an inlier crrespndence in either scenari. 3. Given a wrld pint measured in the wrld crdinate system, what parameters are needed t cmpute the pint s assciated crdinates in the camera s crdinate system? 4. Briefly describe tw tradeffs (i.e., prs/cns) between perfrming dense stere matching (where we attempt t find crrespndences fr every pixel) vs. sparse stere matching (where we attempt t find crrespndences nly at interest pints). 5. T perfrm autmatic scale selectin with the Laplacian f Gaussian filter, we first filter the image with a Laplacian with increasing values f sigma. Then we lk at the magnitude f the respnse at each pixel, and take as an interest pint any pint at which the respnse is a lcal maximum in bth psitin and scale. Why take the lcal maxima, rather than simply take any respnses that exceed a threshld? II. Prgramming prblem: bject categry detectin [70 pints] The gal f this assignment is t perfrm bject detectin using a vting-based Generalized Hugh Transfrm apprach. Given a set f crpped training images shwing the class f interest, the methd first generates a set f prttypical bject parts. We ll use visual wrds frmed frm lcal image patches extracted at Harris crner interest pints t represent the parts. That is, bject part = visual wrd. Having identified the bject parts, the methd stres the pssible displacement vectrs (translatins) between each part and the bject s center, as bserved in all the training examples. Given a nvel test image, the lcal patches are first mapped t bject parts, by assigning the patch arund each interest pint t the nearest prttype (visual wrd). Then each part uses the stred displacement vectrs t cast vtes fr the psitin f the bject. By lking fr peaks in that vte space, ne r mre detectins are made. Nte that the bject f interest may appear multiple times in a test image, at any lcatin. Hwever, we will assume that the scale and rientatin f the bject is fixed; that is, the scale and rientatin at which it appears in the training images are bth cnstant, and will als be similar in the test images. Thus the vte space is nly 2-dimensinal.

T summarize the main steps required: Training stage - Prepare the visual vcabulary and GHT displacement vectrs: Run the Harris crner detectr n each training example t cllect its interest pints.

2 T summarize the main steps required: Training stage - Prepare the visual vcabulary and GHT displacement vectrs: Run the Harris crner detectr n each training example t cllect its interest pints. At each interest pint, extract a fixed size image patch (e.g., 25 x 25 pixels), and use the vectr f raw pixel intensities as the descriptr. Cluster the patches frm the training images with k-means t frm a visual vcabulary, where each mean represents an bject part discvered in the training data. Having fund the vcabulary, g back ver the training examples and assign their lcal patches t visual wrds (i.e., the prttype bject parts). An image patch is assigned t the visual wrd t which it is clsest using Euclidean distance. Fr each visual wrd ccurrence in the training examples, recrd the pssible displacement vectrs between it and the bject center. Let the bject center be the center f the crpped training image. Nte that a given visual wrd may have multiple displacement vectrs, depending n its appearance. (Fr example, see the wheel-like wrd in the figure abve.) Testing stage - Given a nvel test image, detect instances f the bject: Run the Harris crner detectr n the test image t cllect its interest pints. At each interest pint, extract a fixed size image patch, using the same scale selected abve. Use the vectr f raw pixel intensities as the descriptr.

3 Assign each patch t a visual wrd. Let each visual wrd ccurrence vte fr the psitin f the bject using the stred displacement vectrs. After all vtes are cast, analyze the vtes in the accumulatr array, threshld, and predict where the bject ccurs. (T predict the fixed-size bunding bx placement, assume bject center = bunding bx center.) Nte that the bject f interest may ccur multiple times in the test image. Cmpute the accuracy f the detectr s predictins, based n the verlap between the predicted and true bunding bxes. Specifically, a predicted detectin is cunted as crrect if the area f the intersectin f the bxes, nrmalized by the area f their unin, exceeds 0.5. The fllwing data and cde are prvided n the curse website: CarTrainImages/: directry cntaining 550 crpped training images f cars, each 40 x 100 pixels. (The images are small, but if yu encunter memry issues, it s fine t reduce the training set size as needed.) CarTestImages/: directry cntaining 100 test images. Each has ne r mre cars amidst cluttered backgrunds. Cde/harrisDetectr.m: functin t cmpute crnerness respnse and return Harris interest pints. See dcumentatin fr usage. GrundTruth/CarGrundTruthBundingBxes.mat: data file cntaining grund truth bunding bxes fr the test images. Lad with lad. The variable grundtruth is a struct, and grundtruth(i) cntains the bx fr the i-th test image. The bx is specified in terms f the tp left crner psitin and the width and height f the bx. Each bx gets a rw in the list f lcatins: grundtruth(i).tpleftlcs = [x1, y1; x2, y2,, xn yn] if there are n bxes in image i. Fr example, >> lad CarsGrundTruthBundingBxes >> grundtruth(5) ans = tpleftlcs: [2x2 duble] bxw: 100 bxh: 40 >> grundtruth(5).tpleftlcs ans = means that fr test image #5, there are tw cars present, and their bxes have the tp left crners x=5, y=43 and x = 110, y=45, respectively. Bth have size 40 x 100. Cde/exampleLadingGrundTruth.m: functin t illustrate the frmat explained abve. Other useful Matlab functins: subplt, rectangle, kmeans, im2duble, dist2

4 Nte that yu shuld be able t re-use a prtin f yur cde frm Pset2 (since clustering filter bank respnses with k-means t frm textns clustering lcal image patches t frm visual wrds). After implementing yur detectr and adjusting the parameters t values that prduce reasnable results, answer each f the fllwing: 1. Visualizing the discvered prttypical bject parts [15 pints]: Display example image patches assciated with each f three visual wrds within the vcabulary. Chse three wrds that are distinct t illustrate what the different wrds are capturing, and display enugh sample patches s that the wrd cntent is evident (e.g., say patches per wrd displayed). Explain what yu see. 2. Overall accuracy [15 pints]: Reprt the average accuracy ver all the test images. Measure bth the percentage f crrect detectins per image, as well as the average number f false psitives per image. A false psitive ccurs when the detectr fires, but the bject is nt there. Use the same set f parameters (vte space bin size, vcabulary size, patch size, etc.) fr all test images. 3. Shw example detectins [15 pints]: Pick five test image examples, and display the images with the bunding bx detectins made by yur system. Include sme gd detectin results, as well as at least ne interesting errr case. Briefly explain. 4. Impact f patch size [15 pints]: Run yur full system twice nce with a smaller patch size, nce with a larger patch size. Analyze and describe the impact n the visual vcabulary and the detectr. Use images t illustrate yur pint as needed. 5. Multi-bject detectin [10 pints]: Explain hw yu culd extend yur implementatin t detect multiple types f bjects in a single image (e.g., cars and peple). Yu d nt need t implement this. III. [OPTIONAL] Extra credit [up t 10 pints each, max 30 pints] 1. Train and run yur detectr n the Face data prvided (see prvided FaceTrainImages, FaceTestImages, and GrundTruth/FacesGrundTruthBundingBxes.mat.) Shw the results and briefly explain, nting any cntrasts with the Car data experiment. 2. Add a nn-maximum suppressin step t the detectr that remves highly verlapping detectin windws. Describe and illustrate the results. 3. Design and implement a verificatin stage t be used after vting fr detectins. The verificatin shuld lk fr supprt beynd the sparse pints that vted fr that translatin f the mdel. 4. Extend yur cde frm Pset3 t d autmatic interest pint detectin and feature matching. That is, remve the part where yu prvided manually clicked crrespndences, and insert feature detectin, lcal descriptrs, and matching with RANSAC. Yu can use the Harris crner detectr again, and/r check ut sftware linked frm the class webpage.

5 Acknwledgements: The car images we are using fr this assignment are frm the UIUC Image Database fr Car Detectin, curtesy f Shivani Agarwal and Dan Rth. The face images prvided are frm the Caltech-101 Database, curtesy f Fei-Fei Li et al. Submissin instructins: what t hand in Electrnically: Yur well-dcumented Matlab cde.m files. A pdf file cntaining: Yur name and CS lgin ID at the tp. Yur answers t Sectin I, numbered. Yur image results and accmpanying explanatins fr Sectin II, numbered. (ptinal): any results and descriptins fr extra credit prtins in Sectin III. Submit all the abve with ne call t turnin: >> turnin --submit jaechul pset4 pset4.pdf cdefilexyz.m cdefileabc.m etc. Hardcpy: Print ut the pdf files, and either bring it t class r drp ff at Jaechul s ffice in CSA n Thurs 11/24. D nt print ut cde. As usual, the hardcpy must be the same as what is submitted electrnically.

Municode Website Instructions

Municode Website Instructions Municde Website instructins Municde Website Instructins The new and imprved Municde site allws yu t navigate t, print, save, e-mail and link t desired sectins f the Online Cde f Ordinances with greater