Today s lecture. Image Alignment and Stitching. Readings. Motion models
|
|
- Vivien Mitchell
- 5 years ago
- Views:
Transcription
1 Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global alignment automated stitching (recognizing panoramas) ghost and parallax removal compositing and blending Richard Szeliski CSE 576 (Spring 2005): Computer Vision 2 Readings Szeliski & Shum, SIGGRAPH'97 (Sections 1-4). Szeliski, Image Alignment and Stitching, MSR-TR (Sections 2, 4, 5). Recognizing Panoramas, Brown & Lowe, ICCV 2003 Motion models Richard Szeliski CSE 576 (Spring 2005): Computer Vision 3
2 Motion models Motion models What happens when we take two images with a camera and try to align them? translation? rotation? scale? affine? perspective? see interactive demo (VideoMosaic) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 5 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 6 Motion models Homographies Translation Affine Perspective 3D rotation 2 unknowns 6 unknowns 8 unknowns 3 unknowns Richard Szeliski CSE 576 (Spring 2005): Computer Vision 7 Perspective projection of a plane Lots of names for this: homography, texture-map, colineation, planar projective map Modeled as a 2D warp using homogeneous coordinates wx' * * wy' = * * w * * To apply a homography H p H Compute p = Hp (regular matrix multiply) Convert p from homogeneous to image coordinates divide by w (third) coordinate * x * y * 1 p Richard Szeliski CSE 576 (Spring 2005): Computer Vision 8
3 Plane perspective mosaics Rotational mosaics 8-parameter generalization of affine motion works for pure rotation or planar surfaces Limitations: local minima slow convergence difficult to control interactively Directly optimize rotation and focal length Advantages: ability to build full-view panoramas easier to control interactively more stable and accurate estimates Richard Szeliski CSE 576 (Spring 2005): Computer Vision 9 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 10 3D 2D Perspective Projection 3D Rotation Model f u c u (X c,y c,z c ) Projection equations 1. Project from image to 3D ray (x 0,y 0,z 0 ) = (u 0 -u c,v 0 -v c,f) 2. Rotate the ray by camera motion (x 1,y 1,z 1 ) = R 01 (x 0,y 0,z 0 ) 3. Project back into new (source) image (u 1,v 1 ) = (fx 1 /z 1 +u c,fy 1 /z 1 +v c ) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 11 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 12
4 Rotations and quaternions How do we represent rotation matrices? Rotations and quaternions How do we represent rotation matrices? 1. Axis / angle (n,θ) R = I + sinθ [n] + (1- cosθ) [n] 2 (Rodriguez Formula), with [n] = cross product matrix (see paper) 2. Unit quaternions [Shoemake SIGGRAPH 85] q = (n sinθ/2, cosθ/2) = (w,s) quaternion multiplication (division is easy) q 0 q 1 = (s 1 w 0 + s 0 w 1, s 0 s 1 -w 0 w 1 ) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 13 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 14 Incremental rotation update 1. Small angle approximation R = I + sinθ [n] + (1- cosθ) [n] 2 θ [n] = [ω] linear in ω 2. Update original R matrix R R R Image Mosaics (Stitching) [Szeliski & Shum, SIGGRAPH 97] [Szeliski, MSR-TR ] Richard Szeliski CSE 576 (Spring 2005): Computer Vision 15
5 Image Mosaics (Stitching) Image Mosaics (stitching) Blend together several overlapping images into one seamless mosaic (composite) = Full screen panoramas (cubic): Mars: New Years Eve: Richard Szeliski CSE 576 (Spring 2005): Computer Vision 17 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 18 Mosaics for Video Coding Convert masked images into a background sprite for content-based coding = Establishing correspondences 1. Direct method: Use generalization of affine motion model [Szeliski & Shum 97] 2. Feature-based method Compute feature-based correspondence [Lowe ICCV 99; Schmid ICCV 98, Brown&Lowe ICCV 2003] Compute R from correspondences (absolute orientation) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 19 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 20
6 Stitching demo Panoramas What if you want a 360 field of view? mosaic Projection Cylinder Richard Szeliski CSE 576 (Spring 2005): Computer Vision 21 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 22 Cylindrical panoramas Cylindrical Panoramas Map image to cylindrical or spherical coordinates need known focal length Steps Reproject each image onto a cylinder Blend Output the resulting mosaic Image 384x300 f = 180 (pixels) f = 280 f = 380 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 23 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 24
7 Cylindrical projection Map 3D point (X,Y,Z) onto cylinder Cylindrical warping Given focal length f and image center (x c,y c ) Y Z X unit cylinder Convert to cylindrical coordinates Convert to cylindrical image coordinates s defines size of the final image Y (X,Y,Z) (sinθ,h,cosθ) Z X unwrapped cylinder cylindrical image Richard Szeliski CSE 576 (Spring 2005): Computer Vision 25 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 26 Spherical warping Given focal length f and image center (x c,y c ) 3D rotation Rotate image before placing on unrolled sphere (x,y,z) φ cos φ (x,y,z) φ cos φ Y (sinθcosφ,cosθcosφ,sinφ) Z X sin φ cos θ cos φ Y (sinθcosφ,cosθcosφ,sinφ) Z X p = R p sin φ cos θ cos φ Richard Szeliski CSE 576 (Spring 2005): Computer Vision 27 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 28
8 Radial distortion Correct for bending in wide field of view lenses Fisheye lens Extreme bending in ultra-wide fields of view Richard Szeliski CSE 576 (Spring 2005): Computer Vision 29 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 30 Inverse Warping Get each pixel I 0 (u 0 ) from its corresponding location u 1 = h(u 0 ) in I 1 (u 1 ) What if pixel comes from between two pixels? Answer in next lecture Image Stitching 1. Align the images over each other camera pan translation on cylinder! 2. Blend the images together (demo) h(u 0 ) x 1 x I 1 (u 1 ) 0 I 0 (u 0 ) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 31 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 32
9 Project 2 image stitching 1. Take pictures on a tripod (or handheld) 2. Warp images to spherical coordinates 3. Extract features 4. Align neighboring pairs using RANSAC 5. Write out list of neighboring translations 6. Correct for drift 7. Read in warped images and blend them 8. Crop the result and import into a viewer Matching features What do we do about the bad matches? Richard Szeliski CSE 576 (Spring 2005): Computer Vision 33 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 34 RAndom SAmple Consensus RAndom SAmple Consensus Select one match, count inliers Select one match, count inliers Richard Szeliski CSE 576 (Spring 2005): Computer Vision 35 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 36
10 Least squares fit Assembling the panorama Stitch pairs together, blend, then crop Find average translation vector Richard Szeliski CSE 576 (Spring 2005): Computer Vision 37 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 38 Problem: Drift Full-view (360 spherical) panoramas Error accumulation small (vertical) errors accumulate over time apply correction so that sum = 0 (for 360 pan.) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 39
11 Full-view Panoramas Global alignment + + = + + Register all pairwise overlapping images Use a 3D rotation model (one R per image) Use feature based registration of unwarped images Discover which images overlap other images using feature selection (RANSAC) Chain together inter-frame rotations Optimize all R estimates together (next time) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 41 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 42 3D Rotation Model Projection equations 1. Project from image to 3D ray (x 0,y 0,z 0 ) = (u 0 -u c,v 0 -v c,f) 2. Rotate the ray by camera motion (x 1,y 1,z 1 ) = R 01 (x 0,y 0,z 0 ) 3. Project back into new (source) image (u 1,v 1 ) = (fx 1 /z 1 +u c,fy 1 /z 1 +v c ) Absolute orientation [Arun et al., PAMI 1987] [Horn et al., JOSA A 1988] Procrustes Algorithm [Golub & VanLoan] Given two sets of matching points, compute R p i = R p i with 3D rays p i = (x i,y i,z i ) = (u i -u c,v i -v c,f) A = Σ i p i p i T = Σ i p i p it R T = U S V T = (U S U T ) R T V T = U T R T R = VU T Richard Szeliski CSE 576 (Spring 2005): Computer Vision 43 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 44
12 Stitching demo Texture Mapped Model (sphere) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 45 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 46 Texture Mapped Model (cubical) Recognizing Panoramas Matthew Brown & David Lowe ICCV 2003 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 47
13 Recognizing Panoramas Finding the panoramas [Brown & Lowe, ICCV 03] Richard Szeliski CSE 576 (Spring 2005): Computer Vision 49 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 50 Finding the panoramas Finding the panoramas Richard Szeliski CSE 576 (Spring 2005): Computer Vision 51 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 52
14 Finding the panoramas Fully automated 2D stitching Demo Richard Szeliski CSE 576 (Spring 2005): Computer Vision 53 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 54 Get you own copy! System components Feature detection and description more uniform point density Fast matching (hash table) RANSAC filtering of matches Intensity-based verification Incremental bundle adjustment [Brown, Szeliski, Winder, CVPR 05] Richard Szeliski CSE 576 (Spring 2005): Computer Vision 55 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 56
15 Multi-Scale Oriented Patches Interest points Multi-scale Harris corners Orientation from blurred gradient Geometrically invariant to similarity transforms Descriptor vector Bias/gain normalized sampling of local patch (8x8) Photometrically invariant to affine changes in intensity Feature irregularities Distribute points evenly over the image Richard Szeliski CSE 576 (Spring 2005): Computer Vision 57 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 58 Descriptor Vector Probabilistic Feature Matching Orientation = blurred gradient Similarity Invariant Frame Scale-space position (x, y, s) + orientation (θ) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 59 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 60
16 RANSAC motion model RANSAC motion model Richard Szeliski CSE 576 (Spring 2005): Computer Vision 61 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 62 RANSAC motion model Probabilistic model for verification Richard Szeliski CSE 576 (Spring 2005): Computer Vision 63 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 64
17 How well does this work? How well does this work? Test on 100s of examples Test on 100s of examples still too many failures (5-10%) for consumer application Matching Mistakes: False Positive Matching Mistakes: False Positive Richard Szeliski CSE 576 (Spring 2005): Computer Vision 67 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 68
18 Matching Mistakes: False Negative Moving objects: large areas of disagreement Matching Mistakes Accidental alignment repeated / similar regions Failed alignments moving objects / parallax low overlap feature-less regions (more variety?) No 100% reliable algorithm? Richard Szeliski CSE 576 (Spring 2005): Computer Vision 69 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 70 How can we fix these? Tune the feature detector Tune the feature matcher (cost metric) Tune the RANSAC stage (motion model) Tune the verification stage Use higher-level knowledge e.g., typical camera motions Deghosting and blending (optional material) Sounds like a big learning problem Need a large training/test data set (panoramas) Richard Szeliski CSE 576 (Spring 2005): Computer Vision 71
19 Local alignment (deghosting) Use local optic flow to compensate for small motions [Shum & Szeliski, ICCV 98] Local alignment (deghosting) Use local optic flow to compensate for radial distortion [Shum & Szeliski, ICCV 98] Richard Szeliski CSE 576 (Spring 2005): Computer Vision 73 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 74 Image feathering Weight each image proportional to its distance from the edge (distance map [Danielsson, CVGIP 1980] Region-based de-ghosting Select only one image in regions-of-difference using weighted vertex cover [Uyttendaele et al., CVPR 01] Cut out the appropriate region from each image and then blend together Richard Szeliski CSE 576 (Spring 2005): Computer Vision 75 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 76
20 Region-based de-ghosting Select only one image in regions-of-difference using weighted vertex cover [Uyttendaele et al., CVPR 01] Cutout-based de-ghosting Select only one image per output pixel, using spatial continuity Blend across seams using gradient continuity ( Poisson blending ) [Agarwala et al., SG 2004] Richard Szeliski CSE 576 (Spring 2005): Computer Vision 77 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 78 Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: group portraits Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: focus settings Richard Szeliski CSE 576 (Spring 2005): Computer Vision 79 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 80
21 Cutout-based compositing Photomontage [Agarwala et al., SG 2004] Interactively blend different images: people s faces Final thought: What is a panorama? Tracking a subject Repeated (best) shots Multiple exposures Infer what photographer wants? Richard Szeliski CSE 576 (Spring 2005): Computer Vision 81 Richard Szeliski CSE 576 (Spring 2005): Computer Vision 82
Mosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
More informationMore Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.
More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick
More information6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS
6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Bill Freeman Frédo Durand MIT - EECS Lots of slides stolen from Alyosha Efros, who stole them from Steve Seitz
More informationAnnouncements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =
Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
More informationImage stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
More informationPanoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS
Olivier Gondry 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Director of music video and commercial Special effect specialist (Morphing, rotoscoping) Today
More informationAnnouncements. Mosaics. How to do it? Image Mosaics
Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today
More informationImage stitching. Announcements. Outline. Image stitching
Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring
More informationImage stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection
More informationCSE 527: Introduction to Computer Vision
CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature
More informationRecap. DoF Constraint Solver. translation. affine. homography. 3D rotation
Image Blending Recap DoF Constraint Solver translation affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
More informationFeature Matching and RANSAC
Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction
More informationImage Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi
Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish
More informationImage-based Modeling and Rendering: 8. Image Transformation and Panorama
Image-based Modeling and Rendering: 8. Image Transformation and Panorama I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung Univ, Taiwan Outline Image transformation How to represent the
More informationHomographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
More informationAutomatic Image Alignment
Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography
More informationAutomatic Image Alignment
Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com
More informationStitching and Blending
Stitching and Blending Kari Pulli VP Computational Imaging Light First project Build your own (basic) programs panorama HDR (really, exposure fusion) The key components register images so their features
More informationAutomatic Image Alignment (feature-based)
Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature
More informationN-Views (1) Homographies and Projection
CS 4495 Computer Vision N-Views (1) Homographies and Projection Aaron Bobick School of Interactive Computing Administrivia PS 2: Get SDD and Normalized Correlation working for a given windows size say
More information360 Full View Spherical Mosaic
360 Full View Spherical Mosaic Huang Wenfan Huang Yehui Rong Nan U017865B U017844X U018274R Objective Full spherical mosaic 360 x 180. All images are taken with camera mounted on a tripod. Registration
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points
More informationObject Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
More informationImage Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll
Image Warping Man slides from Alosha Efros + Steve Seitz Photo b Sean Carroll Morphing Blend from one object to other with a series of local transformations Image Transformations image filtering: change
More informationImage Warping and Mosacing
Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment
More informationMidterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
More informationPanoramic Image Stitching
Mcgill University Panoramic Image Stitching by Kai Wang Pengbo Li A report submitted in fulfillment for the COMP 558 Final project in the Faculty of Computer Science April 2013 Mcgill University Abstract
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationImage warping and stitching
Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationFeature Based Registration - Image Alignment
Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html
More informationImage warping and stitching
Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment
More informationLocal Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationCS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry
CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from: David Lowe, UBC Jiri Matas, CMP Prague http://cmp.felk.cvut.cz/~matas/papers/presentations/matas_beyondransac_cvprac05.ppt
More informationImage warping and stitching
Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize
More informationAutomatic Panoramic Image Stitching. Dr. Matthew Brown, University of Bath
Automatic Panoramic Image Stitching Dr. Matthew Brown, University of Bath Automatic 2D Stitching The old days: panoramic stitchers were limited to a 1-D sweep 2005: 2-D stitchers use object recognition
More informationFast Image Stitching and Editing for Panorama Painting on Mobile Phones
Fast Image Stitching and Editing for Panorama Painting on Mobile Phones Yingen Xiong and Kari Pulli Nokia Research Center 955 Page Mill Road, Palo Alto, CA 94304, USA {yingen.xiong, kari.pulli}@nokia.com
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based
More informationCS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017
CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo
More informationFast Image Stitching and Editing for Panorama Painting on Mobile Phones
in IEEE Workshop on Mobile Vision, in Conjunction with CVPR 2010 (IWMV2010), San Francisco, 2010, IEEE Computer Society Fast Image Stitching and Editing for Panorama Painting on Mobile Phones Yingen Xiong
More informationCS4670: Computer Vision
CS467: Computer Vision Noah Snavely Lecture 8: Geometric transformations Szeliski: Chapter 3.6 Reading Announcements Project 2 out today, due Oct. 4 (demo at end of class today) Image alignment Why don
More informationAn Algorithm for Seamless Image Stitching and Its Application
An Algorithm for Seamless Image Stitching and Its Application Jing Xing, Zhenjiang Miao, and Jing Chen Institute of Information Science, Beijing JiaoTong University, Beijing 100044, P.R. China Abstract.
More informationVideo Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen
Video Mosaics for Virtual Environments, R. Szeliski Review by: Christopher Rasmussen September 19, 2002 Announcements Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday.
More informationImage Based Rendering. D.A. Forsyth, with slides from John Hart
Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit
More informationCS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know
More informationCS4670: Computer Vision
CS467: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello Szeliski 2.1.3-2.1.6 Reading Announcements Project 2a due Friday, 8:59pm Project 2b out Friday
More informationOverview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers
Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented
More information2D Image Transforms Computer Vision (Kris Kitani) Carnegie Mellon University
2D Image Transforms 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University Extract features from an image what do we do next? Feature matching (object recognition, 3D reconstruction, augmented
More informationImage matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian.
Announcements Project 1 Out today Help session at the end of class Image matching by Diva Sian by swashford Harder case Even harder case How the Afghan Girl was Identified by Her Iris Patterns Read the
More informationImage correspondences and structure from motion
Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationLocal features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
More informationBSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy
BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving
More informationAgenda. Rotations. Camera calibration. Homography. Ransac
Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr
More informationPerspective Projection [2 pts]
Instructions: CSE252a Computer Vision Assignment 1 Instructor: Ben Ochoa Due: Thursday, October 23, 11:59 PM Submit your assignment electronically by email to iskwak+252a@cs.ucsd.edu with the subject line
More informationThe SIFT (Scale Invariant Feature
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical
More informationBuilding a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882
Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk
More informationMobile Panoramic Imaging System
Mobile Panoramic Imaging System Kari Pulli, Marius Tico, Yingen Xiong Nokia Research Center 955 Page Mill Road, Palo Alto, CA, USA firstname.lastname@nokia.com Abstract We introduce a mobile system for
More informationToday s lecture. Structure from Motion. Today s lecture. Parameterizing rotations
Today s lecture Structure from Motion Computer Vision CSE576, Spring 2008 Richard Szeliski Geometric camera calibration camera matrix (Direct Linear Transform) non-linear least squares separating intrinsics
More informationCamera Calibration. COS 429 Princeton University
Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3
More informationImage warping , , Computational Photography Fall 2017, Lecture 10
Image warping http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 10 Course announcements Second make-up lecture on Friday, October 6 th, noon-1:30
More informationCompositing a bird's eye view mosaic
Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition
More informationTargil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects
Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 5 x 35 Targil : Panoramas - Stitching and Blending Some slides from Alexei Efros 2 Slide from Brown & Lowe Why Mosaic? Are you getting
More informationHarder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
More informationDD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication
DD2423 Image Analysis and Computer Vision IMAGE FORMATION Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 8, 2013 1 Image formation Goal:
More informationComputer Vision I - Appearance-based Matching and Projective Geometry
Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation
More informationAgenda. Rotations. Camera models. Camera calibration. Homographies
Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate
More informationIntroduction to Image Processing and Computer Vision. -- Panoramas and Blending --
Introduction to Image Processing and Computer Vision -- Panoramas and Blending -- Winter 2013/14 Ivo Ihrke Panoramas Mosaics and Panoramas - Outline - Perspective Panoramas - Hardware-Based - Software-Based
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationCamera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationCameras and Radiometry. Last lecture in a nutshell. Conversion Euclidean -> Homogenous -> Euclidean. Affine Camera Model. Simplified Camera Models
Cameras and Radiometry Last lecture in a nutshell CSE 252A Lecture 5 Conversion Euclidean -> Homogenous -> Euclidean In 2-D Euclidean -> Homogenous: (x, y) -> k (x,y,1) Homogenous -> Euclidean: (x, y,
More informationProf. Feng Liu. Spring /26/2017
Prof. Feng Liu Spring 2017 http://www.cs.pdx.edu/~fliu/courses/cs510/ 04/26/2017 Last Time Re-lighting HDR 2 Today Panorama Overview Feature detection Mid-term project presentation Not real mid-term 6
More informationLocal Features: Detection, Description & Matching
Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 5: Projection Reading: Szeliski 2.1 Projection Reading: Szeliski 2.1 Projection Müller Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html Modeling
More informationHarder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
More informationAugmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection
More informationPanoramic Image Mosaicing
Panoramic Image Mosaicing Mrs. Hetal M.Patel 1, Asst. Prof. Pinal. J. Patel 2, Asst. Prof. Mr. Sandip G. Patel 3 1 C.S.E. Department, Government College of Engineering, Gandhinagar Gujarat Technology University,
More informationCOSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor
COSC579: Scene Geometry Jeremy Bolton, PhD Assistant Teaching Professor Overview Linear Algebra Review Homogeneous vs non-homogeneous representations Projections and Transformations Scene Geometry The
More informationImage Stitching using Harris Feature Detection
Image Stitching using Harris Feature Detection Shreyas Mistry 1, Prof. Arpita Patel 2 1M. Tech. Student, Electronics & Communication, Chandubhai S. Patel Institute of Technology 2Assi. Professor, Electronics
More informationVisual Recognition: Image Formation
Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know
More informationDD2429 Computational Photography :00-19:00
. Examination: DD2429 Computational Photography 202-0-8 4:00-9:00 Each problem gives max 5 points. In order to pass you need about 0-5 points. You are allowed to use the lecture notes and standard list
More informationFeature Detectors and Descriptors: Corners, Lines, etc.
Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood
More informationImage Mosaicing using MATLAB
Image Mosaicing using MATLAB Rumel M S Pir Assistant Professor Leading University, Sylhet Bangladesh rumelpir@gmail.com Abstract In Image processing Mosaicing is one of the technique which is very useful
More informationCS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing
CS 4495 Computer Vision Features 2 SIFT descriptor Aaron Bobick School of Interactive Computing Administrivia PS 3: Out due Oct 6 th. Features recap: Goal is to find corresponding locations in two images.
More informationImage Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58
Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify
More informationPhoto Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with
More informationCS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 9: Image alignment http://www.wired.com/gadgetlab/2010/07/camera-software-lets-you-see-into-the-past/ Szeliski: Chapter 6.1 Reading All 2D Linear Transformations
More informationIntroduction to Computer Vision. Week 3, Fall 2010 Instructor: Prof. Ko Nishino
Introduction to Computer Vision Week 3, Fall 2010 Instructor: Prof. Ko Nishino Last Week! Image Sensing " Our eyes: rods and cones " CCD, CMOS, Rolling Shutter " Sensing brightness and sensing color! Projective
More informationImplementation of an Image Stitching Algorithm to a Low-Cost Digital Microscope
Implementation of an Image Stitching Algorithm to a Low-Cost Digital Microscope Renan Botan Universidade Federal do Espírito Santo Instituto Federal do Espírito Santo Email: renanbotan@gmail.com Klaus
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationLocal features: detection and description. Local invariant features
Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms
More informationVisual Odometry. Features, Tracking, Essential Matrix, and RANSAC. Stephan Weiss Computer Vision Group NASA-JPL / CalTech
Visual Odometry Features, Tracking, Essential Matrix, and RANSAC Stephan Weiss Computer Vision Group NASA-JPL / CalTech Stephan.Weiss@ieee.org (c) 2013. Government sponsorship acknowledged. Outline The
More informationFeature Matching and Robust Fitting
Feature Matching and Robust Fitting Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Project 2 questions? This
More informationComputer Vision. Coordinates. Prof. Flávio Cardeal DECOM / CEFET- MG.
Computer Vision Coordinates Prof. Flávio Cardeal DECOM / CEFET- MG cardeal@decom.cefetmg.br Abstract This lecture discusses world coordinates and homogeneous coordinates, as well as provides an overview
More informationGlobal Flow Estimation. Lecture 9
Motion Models Image Transformations to relate two images 3D Rigid motion Perspective & Orthographic Transformation Planar Scene Assumption Transformations Translation Rotation Rigid Affine Homography Pseudo
More informationFeature-Based Image Mosaicing
Systems and Computers in Japan, Vol. 31, No. 7, 2000 Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J82-D-II, No. 10, October 1999, pp. 1581 1589 Feature-Based Image Mosaicing Naoki Chiba and
More information