APPM 2360 Lab #2: Facial Recognition

Transcription

1 APPM 2360 Lab #2: Facial Recognition Instructions Labs may be done in groups of 3 or less. You may use any program; but the TAs will only answer coding questions in MATLAB. One report must be turned in for each group and must be in PDF format. Labs must include each student s: Name Student number Section number Recitation number This lab is due on Friday, March 20, 2015 at 5pm. Each lab must be turned in through D2L on the course page. When you submit the lab please include each group members information (name, student number, section number and recitation number) in the comments. This allows us to search for a students report. Once the labs are graded you need to specifically check that your grade was entered. If you are missing a grade please bring the issue up with your TA within a week of grading. 1

2 The report must be typed (including equations). Be sure to label all images and graphs so that, independent of the text, it is clear what is in the graph. Simply answering the lab questions will not earn you a good grade. Take time to write your report as up to 20% of your grade may be based on organization, structure, style, grammar, and spelling. Project office hours will be held in ECCR 143 from Monday, March 16, 2015 to Friday, March 20, (Except that on Tuesday March 17, office hours will be in the Stadium as usual.) 1 Objective In this lab we use the singular value decomposition for matrices. We use it to build a facial recognition system. 2 Matrix and Image Manipulation in Matlab It s easy to read grayscale images into Matlab, using the function imread. Matlab stores the image as a matrix. Each entry is an integer between 0 and 255. An entry of 0 corresponds to one black pixel; an entry of 255 corresponds to one white pixel. Download the images from the folder faces included with the lab. Make sure the images are in your current Matlab folder. Then try these Matlab commands: A = imread('face1.pgm'); imshow(a); Next, we want to try manipulating the matrix and then viewing the image again. First, we need to change the datatype of the Matrix to double. Then we can do matrix operations. Then we change the datatype back to uint8 before we use imshow. For example: A = imread('face1.pgm'); A = double(a); newa = -A+255; newa = uint8(newa); imshow(newa); Problem 1. Try the code above, and include the resulting figure in your report. Describe what has happened to the image. In case you are unfamiliar with matrices in Matlab, here are some things to know: If two matrices A and B are the same shape, you can add them in Matlab by typing A + B. 2

3 If A and B are the right shape to be multiplied, Matlab will multiply them with A B. You can take the transpose of a matrix with an apostrophe: A is the transpose of A. You can get the diagonal elements out of a matrix A using diag(a). Singular Value Decomposition The Singular Value Decomposition breaks a matrix A down into three separate matrices A = USV T where V T denotes the transpose of V. The matrices U and V are both orthogonal, which means that U T U = I and V T V = I, where I is the identity matrix. The matrix S is diagonal; and the diagonal elements of S are called the singular values of A. Intuitively, the matrices U and V describe important features of the matrix, and the matrix S gives us a numerical measure of how important those features are. Don t worry about the technical details for now - we ll get a feel for what this decomposition does as we continue to work with it. Below is a brief MATLAB script that takes the SVD of a random 3 3 matrix. After you run it, take a look at what U, S, and V look like! %% Test Matlab's SVD function % Build a random 3x3 matrix A = rand(3); % Get singular value decomposition [U,S,V] = svd(a); % Multiply to get new matrix B = U*S*V'; %Check that we got the same thing we started with A B Low-Dimensional Approximations What does it mean to create a low-dimensional approximation to a set of data? Take a look at Figure 1. The data points shown on the left lie in two dimensions, but we can easily see that they are almost one-dimensional, i.e. they are all pretty close to some line. If we project each point onto this line (shown in blue), we end up with a one-dimensional approximation of the data. Obviously, we have lost some information, but the hope is that most of the important aspects of the data are preserved in this process. The SVD allows us to do a low-dimensional approximation of a matrix. Suppose we take the SVD of a matrix. The diagonal of S will be the singular values, which MATLAB automatically ranks in descending order, so that we might have something like 3

4 Figure 1: A low-dimension approximation to a set of points. diag(s) = {60.4, 12, 1.07, 0.9, 0.05, 0.001, } In the last section, we said that these singular values can be thought of as weighting how important the corresponding features are. So, it seems like the last three features are relatively unimportant, doesn t it? Supposed we replaced S with S where diag( S) = {60.4, 12, 1.07, 0.9, 0, 0, 0} We have set all but the first 4 singular values to zero. Then we set à = U S V T The dimension of à is 4, down from 7 originally. 1 However à is a good approximation of our A, since the features we are ignoring are relatively unimportant. In the next section, we ll explore removing the less important singular values and observe the effects. Remember, the most important singular values are always the first ones. Making Low-Dimensional Approximations to Images Run this code in Matlab: %% A = imread('face1.pgm'); A = double(a); [U,S,V] = svd(a); 1 Technically, when we say dimension here we really mean rank. 4

5 % Make a copy of S Sreduced = S; % Turn all but the first 10 singular values to 0 Sreduced(10:end,10:end) = 0; % remultiply to get lower-dimensional version of A Areduced = U*Sreduced*V'; imshow(uint8(a)); imshow(uint8(areduced)); (a) All the singular values (b) Only 10 singular values Pretty cool! We reduced the dimension all the way down to 10, and we can still see the important features of the face. Problem 2. Follow the above Matlab procedure to project images to lower dimensions. Work with face1.pgm, and try a variety of values of r, where r is the number of singular value you keep. Try keeping most of the singular values, or half, or only a few. Include these figures in your report (try at least 3, and make sure they look different). Problem 3. Follow the above Matlab procedure for 3 of the other faces. How low can you make r while still being able to recognize the face? Include some figures to demonstrate. Problem 4. Find all the singular values for face1.pgm,. Produce a plot of these values. How many of the singular values are larger than 0.01? Facial Recognition Suppose the FBI is searching for a criminal on the run. The identity of the criminal is unknown. However, the FBI suspects that the criminal will try to obtain a new driver s license under a fake name. 5

6 The criminal will enter a DMV (Department of Motor Vehicles, in charge of licensing drivers) and take a photo for a new license. The FBI hopes to catch the criminal by comparing the new photo to a database containing all the other driver s license photos in the US. When they see that the same face is already in the database under a different name, they ll be able to identify the criminal. The process of searching through the database for a match needs to go very quickly, because the database is huge. The FBI wants to find the match before the criminal leaves the DMV, so they can rush in and make an arrest. The FBI hires you to design an algorithm that can identify matching faces very quickly. When a new photo is taken at the DMV, you must compare it to all the existing photos and look for a matching face. If you compare the photos pixel by pixel, you won t be nearly fast enough. It turns out that you can use the SVD to very quickly compare photos. For each photo, simply use the vector of the first r singular values, where r is some suitably chosen integer. This comparatively short vector of numbers captures the most important information about each face. This is called the signature vector for each face. Your strategy is to first make a signature vector for each face in the database. When a new photo is taken at a DMV somewhere in the country, you ll make a new signature vector for it. You ll compare the new signature vector to the old ones in the database. If the new signature vector is close to any of the old ones, you ll alert the FBI. This algorithm is much faster than comparing all the pixels in the photograph. This method works best if the photographs in your database are all taken under similar conditions (similar lighting, similar zoom, face centered). This is true for driver s license photos, so we re in luck. If photo conditions were more variable, we would have to adjust for those variations first.. Problem 5. Set r = 15. Make a signature vector for each of the 6 images face1.pgm through face6.pgm Include the signature vectors for images 5 and 6 in your lab report. Also make a signature vector for the image newface.pgm. In the next problem you will use the signature vectors to test whether newface matches any of the other faces. We want to know whether the signature vector for newface is close to any of the other signature vectors. What does it mean for vectors to be close? You already know how to calculate the distance between points in 3 dimensions: that is, distance = (x 1 x 2 ) 2 + (y 1 y 2 ) 2 + (z 1 z 2 ) 2. There s a similar distance function for our signature vectors, except in 15 dimensions instead of 3. The Matlab function norm calculates the length of a vector. To find the distance between two vectors, we calculate the length of their difference. In Matlab that will look like norm(vector1-vector2). Problem 6. Find the distance between the signature vector for newface and each of the other signature vectors. In your report, include a plot or graph that shows each of these 6 distances. Which image is newface closest to? 6

7 Problem 7. Given this signature vector: ( ) Which of the 6 faces is this closest to? Problem 8. Suppose your job is to build a facial recognition system using SVD, for a database with 100,000 faces. Describe how you would choose the best value of r. What would happen if it was too small? What would happen if it was too large? 7