Computer Vision, Laboratory session 1

Transcription

1 Centre for Mathematical Sciences, january 2007 Computer Vision, Laboratory session 1 Overview In this laboratory session you are going to use matlab to look at images, study projective geometry representations of points, lines and conics. You are going to learn more about the camera equation and about how images and features change under coordinate transformations. Some of the instructions here are matlab commands that you are supposed to try. To relieve you of some of the tedious typing, we have collected some of those commands in a script lab1_cheats.m in the folder lab1 that you are going to download and unpack. Computer preparations Download the matlab files needed for the laboration using your favorite web-browser. The homepage for the course is at and the files needed for the laboratory session at Copy this file to your home directory, decompress and unpack it. On unix use: unzip datorseende.zip On pcs use winzip or some other alternative. During this course several directories of matlab routines are going to be downloaded and used. It is practical to have these placed in a common directory datorseende with subdirectories tools data lab1 lab2 lab3 lab4 1

2 These directories are created automatically when you unpack datorseende.zip. A matlab script startup.m is located in the directory datorseende. If you start your matlab session in that directory the search paths are automatically set to the directories above. This simplifies things. If you start matlab in some other way you have to set matlab search paths yourself. Image formats There is an enormous amount of different image formats. These are usually differentiated by their suffixes (.tif,.jpg,.gif,.pgm, etc.). Some of these formats use compression in order to save space. There is also a number of programs that can show and manipulate images. It is convenient to know at least one such program, preferably one that can convert between many different image formats. In the course we are not going to focus so much on these issues. We are mainly going to use a raw format called PGM. Each intensity value of the image is stored either in ascii or using binary encoding. We have written a routine in matlab that can read such images. Matlab also has a image reading routine called imread, that can read images in the formats BMP, HDF, JPEG, PCX, TIFF and XWD. Loading and viewing images Load one of the images into matlab and display it using the matlab command imagesc. bild = readpgm( plc001.pgm ); imagesc(bild); Question: Try zooming in on different parts of the image by klicking on the figure. Do you see anything special? Try zooming in on the sharp edges. Is there anything strange with every 16 th column in the image? The errors are introduced in the frame grabber. This is quite typical. Each frame grabber or camera has its on quirks. We have implemented a special routine that removes these problems (for this specific frame grabber). Try bild = pgmlas( plc001.pgm,1); imagesc(bild); 2

3 Image points A point in the image can be represented by its Cartesian coordinates (x,y). Most often, however, we will use so called extended or homogeneous coordinates x x y, 1 where denotes equality up to scale. Question: What Cartesian coordinates do the following points have: , , ? 1 Load one of the images into matlab and display it using the matlab command imagesc. Then plot a point using a home made command rita. bild = pgmlas( plc001.pgm,1); imagesc(bild); hold on; u=[229.6; 251.6; 1]; rita(u, * ); Question: Where are the coordinates (230,251) located? In the upper left corner of a pixel or in the middle of a pixel? What interpolation method does matlab use when displaying images? Image lines An image line can be written in affine form as {(x,y) x T l = ax + by + c = 0} i.e. all points that fulfill the constraint ax + by + c = 0. We will usually use the representation l=[a;b;c]; 3

4 for this line. Notice that an image point u lies on the line l if and only if l T u = 0. Notice also that two lines are identical if they differ by scale only. The notation is used to denote equality up to scale Try l=[0.413; ; 134.3]; rital(l); Image conics Similarly image conics can be written as {(x,y) x T Cx = C 11 x 2 + 2C 12 xy + 2C 13 x +C 22 y 2 + 2C 23 y +C 33 = 0} i.e. all points that fulfill the constraint. We will usually use the representation C 11 C 12 C 13 C = C 12 C 22 C 23 (1) C 13 C 23 C 33 for this conic. Sometimes it is convenient to represent the conic using all tangent lines to the conic, i.e. the conic is defined as the conic which contains all the following tangent lines {(a,b,c) l T Dl = D 11 a 2 + 2D 12 ab + 2D 13 ac + D 22 b 2 + 2D 23 bc + D 33 c 2 = 0} The two matrices D and C are inverses of each other D = C 1. Try (remember that most matlab code is in the file lab1_cheats.m.) C=[ ; ; ;... ]; ritac(c); Coordinate transformations Many computer vision calculations become better conditioned if one changes coordinate system from pixels to angles. This requires the knowledge of the so called intrinsic parameters. For these images the following change of image coordinates can be used. (u,v) (x,y)=((u 348)/1200, (v 286)/1200). (2) 4

5 Here (u, v) are pixel coordinates and (x, y) are image coordinates corrected for approximate internal calibration. We have subtracted (348, 286) from the pixel coordinates so that the middle of the image (the principal point) becomes the origin. We have also divided by the camera constant This is the distance between the image plane and the focal point of the camera (in pixels). Using homogeneous coordinates the transformation can be represented by a 3 3 matrix K so that x u y = K v. (3) 1 1 What should the matrix K be in this case? Enter this matrix into the matlab workspace. If we change the coordinates of the image points we have to change the coordinate representation of features such as points, lines and conics. Question: What are the coordinate transformation formuli? Calculate the coordinate representations ũ, l and C, of the point u the line l and the conic C that we have studied in the laboratory session. Plot the image in the new coordinate system using: hold off; imagesc(([1 697]-348)/1200,([1 573]-286)/1200,bild) Now plot the point, line and conic in the new coordinate system. u_tilde = K*u; %... calculate l_tilde and C_tilde according to the formuli hold on; rita(u_tilde, * ); rital(l_tilde); ritac(c_tilde); Check that your transformations are correct. If they are now being plotted on the same place relative to the image all is well. Otherwise, check your calculations and do it again. The camera equation Points in a world coordinate system is projected onto points in the image using the camera equation, x PX. Since we only get the image points up to scale it is necessary to normalize the third coordinate to one, before plotting them. This can be done with the command pflat. Try 5

6 X = [randn(3,20);ones(1,20)]; P = [eye(3) [0;0;10]]; x = pflat(p*x); figure(2); rita(x, * ); axis([ ]); This gives the projection of 20 points from one viewpoint. The next script generates a sequence of camera matrices Pt, using a local parametrisation of rotations: Try r 1 r 2 e r 3 0 r 3 r 2 r 3 0 r 1 r 2 r 1 0. R=eye(3); r=0.05*randn(3,1); for i=1:100; R=expm([0 -r(3) r(2);r(3) 0 -r(1);-r(2) r(1) 0])*R; T=[R zeros(3,1);zeros(1,3) 1]; Pt = P*T; xt = pflat(pt*x); rita(xt, * ); axis([ ]); drawnow; pause(0.1); end; If you want to you can try the same script with hold on to see the paths of the points in the images. By using the commmand rita3, it is possible to plot 3D-points as well. Modify the above script such that the 3D-points X and the camera path are plotted in a separate figure in the for-loop. Hint: Use rita3(x, * ) to plot the 3D-points. The homogeneous coordinates of the camera centre is obtained by computing the nullspace of the camera matrix (null(pt)). Vanishing points Question: Consider two lines in the scene. The first line goes through points ( 0.5, 1, 0) and ( 0.5, 1,100). The second line goes through points (0.5, 1,0) and (0.5, 1,100). Do the lines lie in a common plane? Do they intersect? In matlab plot the projection of these four points with the camera matrix P above. 6

7 Xl11 = [ ] ; Xl12 = [ ] ; Xl21 = [ ] ; Xl22 = [ ] ; xl11 = pflat(p*xl11); xl12 = pflat(p*xl12); xl21 = pflat(p*xl21); xl22 = pflat(p*xl22); hold off; rita(x, * ); hold on rita([xl11 xl12]); rita([xl21 xl22]); Question: Calculate the intersection of the two object lines. What are the homogeneous coordinates? What are the Cartesian coordinates? Calculate the projection of the intersection and plot it in the image. Xint = [ ] ; xint = pflat(p*xint); rita(xint, r* ); Notice that the intersection in object space is a strange point (a point at infinity), but the projection is an ordinary point. 7