Take Home Exam # 2 Machine Vision

Transcription

1 1 Take Home Exam # 2 Machine Vision Date: 04/26/2018 Due : 05/03/2018 Work with one awesome/breathtaking/amazing partner. The name of the partner should be clearly stated at the beginning of your report. Do not divide work between you and your partner, every one should work on every question and problem. Reports are written individually. Any overlap is considered plagiarism. Show your work for partial credit. The organization of the paper is very important. Lack of organization will result in deducting points. Number and discuss all figures. Two points are deducted for each figure without a number or title. Discuss the methods and the results. Five points are deducted for each missing discussion. Problem 1: Stereo vision for determining distances (15 pts) We want to use a stereo vision system to determine distances. The cameras are similar and have the following intrinsic parameters: λ =3500 sx λ =3450 sy u0 =2319 v0 =1695 (1) (2) (3) (4) The offset between the cameras is given by 200 T = 0 0 The offset is measured in millimeters. 1) Determine the distance of each pen from the left then the right camera. 2) Determine the height of each pen 3) Determine the length of the red car. The images can be downloaded from the instructor s website. Fig. 1. Image I Fig. 2. Image II (5)

2 2 Problem 2: Camera calibration (22 pts) In this problem we want to determine the intrinsic parameters using four different methods. We are interested in the following parameters: α u, α v, u 0 and v 0. The skew factor γ will be also determined when QR factorization is used. The methods we want to investigate are QR factorization Pseudoinverse method Singular value decomposition (SVD) Graphical solution Note that the results of the different methods are not expected to be exactly the same. In addition to α u, α v, u 0 and v 0, QR factorization can be used to obtain the skew factor. Singular value decomposition will allow to obtain the pixel size and the focal length, from which α u and α v can be deduced. The data are summarized in the table below. Note that (r, c) refer to the pixel coordinates and (x, y, z) are the coordinates of the 3D point in the camera reference frame. The world reference frame is attached to the camera center. r c x(mm) y(mm) z(mm) (6) ) Plot the data points (r, x ) z and (c, y ) in separate graphs. Use visible markers. z 2) Write code to solve for the parameters α u, α v, u 0, v 0 and γ using QR factorization. 3) Write code to solve for the parameters α u, α v, u 0, v 0 using the pseudo-inverse method. 4) Write code to solve for the parameters λ, s x, s y, u 0, v 0 using singular value decomposition. The following command can be used: [U, V, D] = svd(a) (7) sol = D(:, end) (8) where A is an appropriate matrix constructed from the data points. 5) From the graphs of question 1, deduce a reasonable approximation of the parameters α u, α v, u 0, v 0. 6) Based on the results from the previous questions, complete the table below. QR factorization Pseudoinverse method Singular value decomposition Graphical solution α u α v u 0 v 0 γ (9) 7) Deduce the rotation and translation matrices from QR factorization.

3 3 8) We now want to compare between the four methods. We define the errors where M is the total number of data points and B is constant with X i = r i u 0 α u x i + Bγ yi (10) y i Y i = c i v 0 α v (11) i = 1,..., M (12) B = { 1, if QR factorization is used 0, for the other methods (13) The accumulated error is then given by ε = M [ (Xi) 2 + (Y i) 2]. Write code to obtain the accumulated error for the four methods used in this problem. 9) Complete the table below and discuss which method gave the best approximation. i=1 Accumulated error QR factorization Pseudo-inverse Singular value decomposition Graphical method (14) Problem 3 Camera parameters from stereo vision system (8 pts) We want to perform camera calibration based on a simplified approach using a stereo vision system where the real distances of the grid are known. The dimensions of the rectangles in the grid are 41.5mm 31mm. The distance between the grid and cameras is 465mm. The cameras are similar. The images can be downloaded from the instructor s website. a) Determine α u and α v. b) Determine the offset between the cameras in the x and y directions. The offset in the z direction is zero. Fig. 3. Image with the grid Fig. 4. Image with the grid The images are available from the instructor s website.

4 4 Problem 4: Finding the fundamental matrix and the epipolar lines (15 pts) We want to determine and plot the epipolar lines (both right and left). Shifted data points (so that the origin is the center of the image) from the left and right cameras are shown in the table below. Left camera Left camera Right camera Right camera r 1 c 1 r 2 c (15) The camera parameters and the rotation/translation matrices are given by α u = 3500 α v = 3451 T x = 150mm T z = 0 u 0 = 2318 v 0 = 1694 T y = 8mm R = I (16) We are interested in finding the corresponding points to p 1 and p 2, where p 1 is in the left image and p 2 is in the right image. 1) Find the fundamental matrix. If you decide to use singular value decomposition, you will need to use at least eight points. 2) For this question and the next one, you will need to use the fundamental matrix. Feel free to use the results from the previous question or the fundamental matrix given by F = (17) Find the right epipolar line corresponding to point p 1 = [ ) Find the left epipolar line corresponding to point p 2 = [ ] 856 ] 4) Plot the left epipolar line on the left image. 5) Plot the right epipolar line on the right image. The images can be downloaded from the instructor s website. Problem 5: Solving the calibration problem using neural networks (15 pts) Neural networks are universal function approximators. In this problem we want to use two simple neural networks to learn the intrinsic parameters. The networks are shown in figures 5 and 6 for learning (α u, u 0) and (α v, v 0), respectively. It is suggested to use Widrow-Hoff algorithm. Recall that unlike the perceptron where the activation function is a step, the activation function in the Widrow-Hoff algorithm is linear. The output is y = g(x) = w ix i + b (18) The learning equations are w =w + L r(t y)x (19) b =b + L r(t y); (20) where L r is the learning rate. Write code to implement the Widrow-Hoff learning algorithm and determine the numerical values of α u, α v, u 0, v 0. Note that it is suggested to use two different networks, one to learn α u, u 0 and the other one to learn α v, v 0. The data points are the same as those given in problem 2.

5 Fig. 5. Network for learning α u and u 0 Fig. 6. Network for learning α v and v 0 5