Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Transcription

1 Colorado School of Mines Computer Vision Professor William Hoff Dept of Electrical Engineering &Computer Science 1

2 Vlfeat and SIFT Examples 2

3 Matlab code SIFT Software Download and put in a directory (such as C:\Users\whoff\Documents\Research\SIFT\vlfeat ) At the Matlab prompt, run( C:\Users\whoff\Documents\Research\vlfeat \toolbox\vl_setup ); Main functions vl_sift extract SIFT features from an image vl_ubcmatch match two sets of SIFT features This temporarily adds the folder containing the vlfeat code, to your Matlab path Also useful vl_plotframe overlay SIFT feature locations on an image vl_plotsiftdescriptor overlay SIFT feature details on an image 3

4 SIFT Feature Detector Function call to detect features (and compute descriptors) [f,d] = vl_sift (I) Returns Arrays f(4,n), d(128,n), where N is the number of features f(1:4,i) is (x,y,scale,angle) for the ith feature d(1:128,i) is the 128 element descriptor for the ith feature Try on graffiti image dataset; see where features are detected Images from evaluation/datasets.html 4

5 clear all close all I1 = imread('img1.png'); imshow(i1, []); if size(i1,3)>1 I1 = rgb2gray(i1); % If color, convert to grayscale end I1 = single(i1); % Convert to single precision floating point SIFT Feature Detector % First make sure the vl_sift code is in the path if exist('vl_sift', 'file')==0 run('c:\users\ahwho_000\documents\research\vlfeat \toolbox\vl_setup'); end [f1,d1] = vl_sift(i1); % Extract SIFT features % Show the SIFT features h = vl_plotframe(f1); set(h,'color','y','linewidth',1); I2 = imread('img2.png'); figure, imshow(i2, []); if size(i2,3)>1 I2 = rgb2gray(i2); % If color, convert to grayscale end I2 = single(i2); % Convert to single precision floating point [f2,d2] = vl_sift(i2); % Extract SIFT features % Show the SIFT features h = vl_plotframe(f2); set(h,'color','y','linewidth',1); 5

6 SIFT Descriptor Let s create a synthetic image of a square clear all close all I = zeros(400,400); I(100:300, 100:300) = 1.0; I = single(i); imshow(i,[]); Then calculate the descriptor at a specified location (ie, at a corner) x = 100; y = 100; scale = 5; ang = 0; % Specify (x;y;scale,angle) of a feature (frame) to extract fc = [x;y;scale;ang]; [f,d] = vl_sift(i,'frames',fc); Use this option to extract just one feature at the specified location, scale, and angle 6

7 Display SIFT Descriptor Function call to display features vl_plotsiftdescriptor(d,f); This shows the gradient directions in the 4x4 cells surrounding each feature % Plot it h = vl_plotsiftdescriptor(d,f); set(h,'color','g'); disp(f); % x,y,scale,angle figure, plot(d); 7

8 Show raw image gradients % Show the image at that scale g = fspecial('gaussian', 6*scale, scale); Is = imfilter(i,g); figure, imshow(is,[]); [gx,gy] = gradient(is); x = 1:size(I,2); y = 1:size(I,1); hold on quiver(x, y, gx, gy); h = vl_plotsiftdescriptor(d,f); set(h,'color','g');

9 clear all close all Invariance to rotation, scale % First make sure the vl_sift code is in the path if exist('vl_sift', 'file')==0 run('c:\users\ahwho_000\documents\research\vlfeat \toolbox\vl_setup'); end 1 of 2 I1 = imread('cameraman.tif'); I1 = single(i1); % Convert to single precision floating point imshow(i1,[]); [f1,d1] = vl_sift(i1); % Find the feature closest to the center of the image dx = size(i1,2)/2 - f1(1,:); dy = size(i1,1)/2 - f1(2,:); distsq = dx.^2 + dy.^2; [~,i1] = min(distsq); % Show the SIFT feature h = vl_plotframe(f1(:,i1)) ; set(h,'color','y','linewidth',2) ;

10 I2 = imresize(i1, 2); % Resize by a factor of 2 I2 = imrotate(i2, 30); figure, imshow(i2,[]); [f2,d2] = vl_sift(i2); % Find the feature closest to the center of the image dx = size(i2,2)/2 - f2(1,:); dy = size(i2,1)/2 - f2(2,:); distsq = dx.^2 + dy.^2; [~,i2] = min(distsq); % Show the SIFT feature h = vl_plotframe(f2(:,i2)) ; set(h,'color','y','linewidth',2) ; disp(f1(:,i1)); % Print (x,y,scale,ang) disp(f2(:,i2)); % Print (x,y,scale,ang) figure, plot(d1(:,i1), 'r'); hold on, plot(d2(:,i2), 'g'); 2 of 2

11 Match SIFT features I1 = imread( test000.jpg'); I1 = single(i1); % Convert to single precision floating point imshow(i1,[]); % These parameters limit the number of features detected peak_thresh = 0; % increase to limit; default is 0 edge_thresh = 10; % decrease to limit; default is 10 [f1,d1] = vl_sift(i1,... 'PeakThresh', peak_thresh,... 'edgethresh', edge_thresh ); fprintf('number of frames (features) detected: %d\n', size(f1,2)); % Show all SIFT features detected h = vl_plotframe(f1) ; set(h,'color','y','linewidth',2) ; 11

12 Display one feature %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Visualize one feature only i = randi(size(f1,2)); % pick any feature fprintf('feature index %d\n', i); disp('(x,y,scale,angle): '); disp(f1(:,i)); figure, plot(d1(:,i)); % Display that feature figure, imshow(i1,[]); h = vl_plotframe(f1(:,i)) ; set(h,'color','y','linewidth',2) ; h = vl_plotsiftdescriptor(d1(:,i),f1(:,i)) ; set(h,'color','g') ; 12

13 Extract SIFT features 2 nd image %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Second image I2 = single( imread('test012.jpg') ); figure, imshow(i2,[]); % These parameters limit the number of features detected peak_thresh = 0; % increase to limit; default is 0 edge_thresh = 10; % decrease to limit; default is 10 [f2,d2] = vl_sift(i2,... 'PeakThresh', peak_thresh,... 'edgethresh', edge_thresh ); fprintf('number of frames (features) detected: %d\n', size(f2,2)); % Show all SIFT features detected h = vl_plotframe(f2) ; set(h,'color','y','linewidth',2) ; 13

14 Match SIFT features Function call [matches, scores] = vl_ubcmatch(d1, d2); Returns Arrays: matches(2,m), scores(m), where M is the number of matches matches(1:2,i) are the indices of the features for the ith match scores(i) is the squared Euclidean distance between the features %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Threshold for matching % Descriptor D1 is matched to a descriptor D2 only if the distance d(d1,d2) % multiplied by THRESH is not greater than the distance of D1 to all other % descriptors thresh = 2.0; % default = 1.5; increase to limit matches [matches, scores] = vl_ubcmatch(d1, d2, thresh); fprintf('number of matching frames (features): %d\n', size(matches,2)); indices1 = matches(1,:); % Get matching features f1match = f1(:,indices1); d1match = d1(:,indices1); indices2 = matches(2,:); f2match = f2(:,indices2); d2match = d2(:,indices2); 14

15 Display matches These are potential matches, based on similarity of local appearance Some may be incorrect %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% % Show matches figure, imshow([i1,i2],[]); o = size(i1,2) ; line([f1match(1,:);f2match(1,:)+o],... [f1match(2,:);f2match(2,:)]) ; for i=1:size(f1match,2) x = f1match(1,i); y = f1match(2,i); text(x,y,sprintf('%d',i), 'Color', 'r'); end for i=1:size(f2match,2) x = f2match(1,i); y = f2match(2,i); text(x+o,y,sprintf('%d',i), 'Color', 'r'); end 15