Lecture 4.1 Feature descriptors. Trym Vegard Haavardsholm

Transcription

1 Lecture 4.1 Feature descriptors Trym Vegard Haavardsholm

2 Feature descriptors Histogram of Gradients (HoG) descriptors Binary descriptors 2

3 Histogram of Gradients (HOG) descriptors Scale Invariant Feature Transform (SIFT) David G. Lowe. "Distinctive image features from scale-invariant keypoints. IJCV 60 (2), pp , 2004 Speeded Up Robust Features (SURF) Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. Computer Vision ECCV Springer Berlin Heidelberg,

4 SIFT detector 4

5 SIFT detections 5

6 Patch at detected position, scale, orientation 6

7 SIFT descriptor Extract patch around detected keypoint Normalize the patch to canonical scale and orientation 7

8 SIFT descriptor Extract patch around detected keypoint Normalize the patch to canonical scale and orientation Resize patch to 16x16 pixels 8

9 SIFT descriptor Compute the gradients 9

10 SIFT descriptor Compute the gradients Unaffected by additive intensity change 10

11 SIFT descriptor Compute the gradients Unaffected by additive intensity change Apply a Gaussian weighting function 11

12 SIFT descriptor Compute the gradients Unaffected by additive intensity change Apply a Gaussian weighting function Weighs down gradients far from the centre Avoids sudden changes in the descriptor with small changes in the window position 12

13 SIFT descriptor Compute the gradients Unaffected by additive intensity change Apply a Gaussian weighting function Weighs down gradients far from the centre Avoids sudden changes in the descriptor with small changes in the window position Divide the patch into 16 4x4 pixels squares 13

14 SIFT descriptor Compute gradient direction histograms over 8 directions in each square 14

15 SIFT descriptor Compute gradient direction histograms over 8 directions in each square Trilinear interpolation Robust to small shifts, while preserving some spatial information 15

16 SIFT descriptor Compute gradient direction histograms over 8 directions in each square Trilinear interpolation Robust to small shifts, while preserving some spatial information 16

17 SIFT descriptor Concatenate the histograms to obtain a 128 dimensional feature vector 17

18 SIFT descriptor Concatenate the histograms to obtain a 128 dimensional feature vector Normalize to unit length Invariant to multiplicative contrast change Threshold gradient magnitudes to avoid excessive influence of high gradients Clamp gradients > 0.2 Renormalize 18

19 Example: Feature comparison 19

20 Example: Feature comparison 20

21 Example: Feature comparison 21

22 SIFT summary Extract a 16x16 patch around detected keypoint Compute the gradients and apply a Gaussian weighting function Divide the window into a 4x4 grid of cells Compute gradient direction histograms over 8 directions in each cell Concatenate the histograms to obtain a 128 dimensional feature vector Normalize to unit length 22

23 SIFT summary 23

24 Binary descriptors Extremely efficient construction and comparison Based on pairwise intensity comparisons Sampling pattern around keypoint Set of sampling pairs Feature descriptor vector is a binary string: F = T( P ) a 0 a N a 2 T( P ) a I P > I P = 0 otherwise r1 r2 1 if ( a ) ( a ) BRISK sampling pattern Matching using Hamming distance: L = XOR F F 0 a N 1 2 ( a, a ) BRISK sampling pairs 24

25 Binary descriptors Method Sampling pattern Orientation calculation Sampling pairs BRIEF None None Random ORB None Moments Learned pairs BRISK Concentric circles, More points on outer rings Comparing gradients of long pairs Short pairs FREAK Overlapping concentric circles, more points on inner rings Comparing gradients of preselected 45 pairs Learned pairs BRIEF sampling pairs Credit: Gil Levi 25

26 Binary descriptors Method Sampling pattern Orientation calculation Sampling pairs BRIEF None None Random ORB None Moments Learned pairs BRISK Concentric circles, More points on outer rings Comparing gradients of long pairs Short pairs FREAK Overlapping concentric circles, more points on inner rings Comparing gradients of preselected 45 pairs Learned pairs ORB sampling pairs Credit: Trzcinski, et al, Learning Image Descriptors with Boosting,

27 Binary descriptors Method Sampling pattern Orientation calculation Sampling pairs BRIEF None None Random ORB None Moments Learned pairs BRISK FREAK Concentric circles, More points on outer rings Overlapping concentric circles, more points on inner rings Comparing gradients of long pairs Comparing gradients of preselected 45 pairs Short pairs Learned pairs BRISK sampling pattern BRISK sampling pairs Credit: Gil Levi 27

28 Binary descriptors Method Sampling pattern Orientation calculation Sampling pairs BRIEF None None Random ORB None Moments Learned pairs FREAK sampling pattern BRISK Concentric circles, More points on outer rings Comparing gradients of long pairs Short pairs FREAK Overlapping concentric circles, more points on inner rings Comparing gradients of preselected 45 pairs Learned pairs FREAK sampling pairs 28

29 Binary descriptors Often achieves very good performance compared to SIFT/SURF Much faster than SIFT/SURF A. Alahi, R. Ortiz, and P. Vandergheynst. FREAK: Fast Retina Keypoint. In IEEE Conference on Computer Vision and Pattern Recognition, 29

30 Binary descriptors Gil Levi s CV blog: 30