Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

Size: px
Start display at page:

Download "Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac"

Transcription

1 Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

2 Image stitching Stitching = alignment + blending geometrical registration photometric registration

3 Applications of image stitching Video stabilization Video summarization Video compression Video matting Panorama creation

4 Video summarization

5 Video compression

6 Object removal input video

7 Object removal remove foreground

8 Object removal estimate background

9 Object removal background estimation

10 Panorama creation

11 Why panorama? Are you getting the whole picture? Compact Camera FOV = 50 35

12 Why panorama? Are you getting the whole picture? Compact Camera FOV = Human FOV =

13 Why panorama? Are you getting the whole picture? Compact Camera FOV = Human FOV = Panoramic Mosaic =

14 Panorama eamples Similar to HDR, it is a topic of computational photography, seeking ways to build a better camera using either hardware or software. Most consumer cameras have a panorama mode Mars: Earth:

15 What can be globally aligned? In image stitching, we seek for a matri to globally warp one image into another. Are any two images of the same scene can be aligned this way? Images captured with the same center of projection A planar scene or far-away scene

16 A pencil of rays contains all views real camera synthetic camera Can generate any synthetic camera view as long as it has the same center of projection!

17 Mosaic as an image reprojection mosaic projection plane The images are reprojected onto a common plane The mosaic is formed on this plane Mosaic is a synthetic wide-angle camera

18 Changing camera center Does it still work? synthetic PP PP1 PP2

19 What cannot The scene with depth variations and the camera has movement

20 Planar scene (or a faraway one) PP1 PP3 PP2 PP3 is a projection plane of both centers of projection, so we are OK! This is how big aerial photographs are made

21 Motion models Parametric models as the assumptions on the relation between two images.

22 2D Motion models

23 Motion models Translation Affine Perspective 3D rotation 2 unknowns 6 unknowns 8 unknowns 3 unknowns

24 A case study: cylindrical panorama What if you want a 360 field of view? mosaic projection cylinder

25 Cylindrical panoramas Steps Reproject each image onto a cylinder Blend Output the resulting mosaic

26 applet plets/projection.html

27 Cylindrical panorama 1. Take pictures on a tripod (or handheld) 2. Warp to cylindrical coordinate 3. Compute pairwise alignments 4. Fi up the end-to-end alignment 5. Blending 6. Crop the result and import into a viewer It is required to do radial distortion correction for better stitching results!

28 Taking pictures Kaidan panoramic tripod head

29 Translation model

30 Where should the synthetic camera be real camera synthetic camera The projection plane of some camera Onto a cylinder

31 Cylindrical projection Adopted from

32 Cylindrical projection

33 Cylindrical projection Adopted from

34 Cylindrical projection unwrapped cylinder y θ f

35 Cylindrical projection y unwrapped cylinder y f θ z

36 Cylindrical projection y unwrapped cylinder y f z s=f gives less distortion

37 Cylindrical reprojection top-down view Focal length the dirty secret Image f = 180 (piels) f = 280 f = 380

38 A simple method for estimating f w p f d Or, you can use other software, such as AutoStich, to help.

39 Input images

40 Cylindrical warping

41 Blending Why blending: paralla, lens distortion, scene motion, eposure difference

42 Blending

43 Blending

44 Blending

45 Gradient-domain stitching

46 Gradient-domain stitching

47 Panorama weaving

48 Assembling the panorama Stitch pairs together, blend, then crop

49 Problem: Drift Error accumulation small errors accumulate over time

50 Problem: Drift ( 1,y 1 ) ( n,y n ) Solution copy of first image add another copy of first image at the end there are a bunch of ways to solve this problem add displacement of (y 1 y n )/(n -1) to each image after the first compute a global warp: y = y + a run a big optimization problem, incorporating this constraint best solution, but more complicated known as bundle adjustment

51 End-to-end alignment and crop

52 Rectangling panoramas video

53 Rectangling panoramas

54 Rectangling panoramas

55 Viewer: panorama eample:

56 Viewer: teture mapped model eample:

57 Cylindrical panorama 1. Take pictures on a tripod (or handheld) 2. Warp to cylindrical coordinate 3. Compute pairwise alignments 4. Fi up the end-to-end alignment 5. Blending 6. Crop the result and import into a viewer

58 Determine pairwise alignment? Feature-based methods: only use feature points to estimate parameters We will study the Recognising panorama paper published in ICCV 2003 Run SIFT (or other feature algorithms) for each image, find feature matches.

59 Determine pairwise alignment p =Mp, where M is a transformation matri, p and p are feature matches It is possible to use more complicated models such as affine or perspective For eample, assume M is a 22 matri ' m11 m12 y' m21 m22 y Find M with the least square error n Mp p' i1 2

60 Determine pairwise alignment Overdetermined system y m m m m y ' ' ' ' y m y m m y m ' ' ' 2 ' 1 ' n n n n n n y y m m m m y y y y y

61 Normal equation Given an overdetermined system A b the normal equation is that which minimizes the sum of the square differences between left and right sides A T A A T b Why?

62 Normal equation 2 ) ( b A E n m nm n m b b a a a a : : : :... : : : : : : nm, n equations, m variables

63 Normal equation n m j j nj i m j j ij m j j j n i m j j nj m j j ij m j j j b a b a b a b b b a a a : : : : : : b A n i i m j j ij b a E ) ( b A

64 Normal equation n i i m j j ij b a E ) ( b A 1 0 E n i i i n i j m j ij i b a a a i n i i m j j ij a b a ) 2( 0 b A A A T T E b A A A T T

65 Normal equation A b 2

66 Normal equation b b b A b A A A b b b A A b A A b A b A b A b A b A b A b A T T T T T T T T T T T T T T T T T T T 2 b A A A T T 2 2 E

67 Determine pairwise alignment p =Mp, where M is a transformation matri, p and p are feature matches For translation model, it is easier. What if the match is false? Avoid impact of outliers. n i i i i i y y m m E 1 2 ' 2 2 ' m E

68 RANSAC RANSAC = Random Sample Consensus An algorithm for robust fitting of models in the presence of many data outliers Compare to robust statistics Given N data points i, assume that majority of them are generated from a model with parameters, try to recover.

69 RANSAC algorithm Run k times: How many times? (1) draw n samples randomly How big? Smaller is better (2) fit parameters with these n samples (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, c Output with the largest c How to define? Depends on the problem.

70 How to determine k p: probability of real inliers P: probability of success after k trials P 1 (1 p ) n k n samples are all inliers a failure failure after k trials k log(1 P) log(1 p n ) for P=0.99 n p k

71 Eample: line fitting

72 Eample: line fitting n=2

73 Model fitting

74 Measure distances

75 Count inliers c=3

76 Another trial c=3

77 The best model c=15

78 RANSAC for Homography

79 RANSAC for Homography

80 RANSAC for Homography

81 Applications of panorama in VFX Background plates Image-based lighting

82 Troy (image-based lighting)

83 Spiderman 2 (background plate)

84 Reference Richard Szeliski, Image Alignment and Stitching: A Tutorial, Foundations and Trends in Computer Graphics and Computer Vision, 2(1):1-104, December R. Szeliski and H.-Y. Shum. Creating full view panoramic image mosaics and teture-mapped models, SIGGRAPH 1997, pp M. Brown, D. G. Lowe, Recognising Panoramas, ICCV 2003.

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration

More information

Image stitching. Announcements. Outline. Image stitching

Image stitching. Announcements. Outline. Image stitching Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring

More information

Today s lecture. Image Alignment and Stitching. Readings. Motion models

Today s lecture. Image Alignment and Stitching. Readings. Motion models Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global

More information

Mosaics. Today s Readings

Mosaics. Today s Readings Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection

More information

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

More information

6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS

6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Bill Freeman Frédo Durand MIT - EECS Lots of slides stolen from Alyosha Efros, who stole them from Steve Seitz

More information

Panoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS

Panoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS Olivier Gondry 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Director of music video and commercial Special effect specialist (Morphing, rotoscoping) Today

More information

Feature Matching and RANSAC

Feature Matching and RANSAC Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

More information

CSE 527: Introduction to Computer Vision

CSE 527: Introduction to Computer Vision CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

More information

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin. More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick

More information

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position = Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry

More information

Image-based Modeling and Rendering: 8. Image Transformation and Panorama

Image-based Modeling and Rendering: 8. Image Transformation and Panorama Image-based Modeling and Rendering: 8. Image Transformation and Panorama I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung Univ, Taiwan Outline Image transformation How to represent the

More information

Announcements. Mosaics. How to do it? Image Mosaics

Announcements. Mosaics. How to do it? Image Mosaics Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

Image Warping and Mosacing

Image Warping and Mosacing Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

More information

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017 CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo

More information

Image Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll

Image Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll Image Warping Man slides from Alosha Efros + Steve Seitz Photo b Sean Carroll Morphing Blend from one object to other with a series of local transformations Image Transformations image filtering: change

More information

Image warping and stitching

Image warping and stitching Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize

More information

Object Recognition with Invariant Features

Object Recognition with Invariant Features Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user

More information

Warping, Morphing and Mosaics

Warping, Morphing and Mosaics Computational Photograph and Video: Warping, Morphing and Mosaics Prof. Marc Pollefes Dr. Gabriel Brostow Toda s schedule Last week s recap Warping Morphing Mosaics Toda s schedule Last week s recap Warping

More information

Image Based Rendering. D.A. Forsyth, with slides from John Hart

Image Based Rendering. D.A. Forsyth, with slides from John Hart Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit

More information

Lucas-Kanade Image Registration Using Camera Parameters

Lucas-Kanade Image Registration Using Camera Parameters Lucas-Kanade Image Registration Using Camera Parameters Sunghyun Cho a, Hojin Cho a, Yu-Wing Tai b, Young Su Moon c, Junguk Cho c, Shihwa Lee c, and Seungyong Lee a a POSTECH, Pohang, Korea b KAIST, Daejeon,

More information

Automatic Panoramic Image Stitching. Dr. Matthew Brown, University of Bath

Automatic Panoramic Image Stitching. Dr. Matthew Brown, University of Bath Automatic Panoramic Image Stitching Dr. Matthew Brown, University of Bath Automatic 2D Stitching The old days: panoramic stitchers were limited to a 1-D sweep 2005: 2-D stitchers use object recognition

More information

CS4670: Computer Vision

CS4670: Computer Vision CS467: Computer Vision Noah Snavely Lecture 8: Geometric transformations Szeliski: Chapter 3.6 Reading Announcements Project 2 out today, due Oct. 4 (demo at end of class today) Image alignment Why don

More information

CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry

CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from: David Lowe, UBC Jiri Matas, CMP Prague http://cmp.felk.cvut.cz/~matas/papers/presentations/matas_beyondransac_cvprac05.ppt

More information

Agenda. Rotations. Camera models. Camera calibration. Homographies

Agenda. Rotations. Camera models. Camera calibration. Homographies Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate

More information

Instance-level recognition

Instance-level recognition Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction

More information

Chapter 3 Image Registration. Chapter 3 Image Registration

Chapter 3 Image Registration. Chapter 3 Image Registration Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263 Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine

More information

Agenda. Rotations. Camera calibration. Homography. Ransac

Agenda. Rotations. Camera calibration. Homography. Ransac Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253 Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine

More information

Midterm Examination CS 534: Computational Photography

Midterm Examination CS 534: Computational Photography Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)

More information

Instance-level recognition

Instance-level recognition Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction

More information

CS664 Lecture #16: Image registration, robust statistics, motion

CS664 Lecture #16: Image registration, robust statistics, motion CS664 Lecture #16: Image registration, robust statistics, motion Some material taken from: Alyosha Efros, CMU http://www.cs.cmu.edu/~efros Xenios Papademetris http://noodle.med.yale.edu/~papad/various/papademetris_image_registration.p

More information

Video Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen

Video Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen Video Mosaics for Virtual Environments, R. Szeliski Review by: Christopher Rasmussen September 19, 2002 Announcements Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday.

More information

Stereo and Epipolar geometry

Stereo and Epipolar geometry Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka

More information

Stitching and Blending

Stitching and Blending Stitching and Blending Kari Pulli VP Computational Imaging Light First project Build your own (basic) programs panorama HDR (really, exposure fusion) The key components register images so their features

More information

Introduction to Image Processing and Computer Vision. -- Panoramas and Blending --

Introduction to Image Processing and Computer Vision. -- Panoramas and Blending -- Introduction to Image Processing and Computer Vision -- Panoramas and Blending -- Winter 2013/14 Ivo Ihrke Panoramas Mosaics and Panoramas - Outline - Perspective Panoramas - Hardware-Based - Software-Based

More information

Affine and Projective Transformations

Affine and Projective Transformations CS 674: Intro to Computer Vision Affine and Projective Transformations Prof. Adriana Kovaska Universit of Pittsburg October 3, 26 Alignment problem We previousl discussed ow to matc features across images,

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based

More information

Panoramic Image Stitching

Panoramic Image Stitching Mcgill University Panoramic Image Stitching by Kai Wang Pengbo Li A report submitted in fulfillment for the COMP 558 Final project in the Faculty of Computer Science April 2013 Mcgill University Abstract

More information

Multi-stable Perception. Necker Cube

Multi-stable Perception. Necker Cube Multi-stable Perception Necker Cube Spinning dancer illusion, Nobuuki Kaahara Fitting and Alignment Computer Vision Szeliski 6.1 James Has Acknowledgment: Man slides from Derek Hoiem, Lana Lazebnik, and

More information

CS4670: Computer Vision

CS4670: Computer Vision CS467: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello Szeliski 2.1.3-2.1.6 Reading Announcements Project 2a due Friday, 8:59pm Project 2b out Friday

More information

Photo by Carl Warner

Photo by Carl Warner Photo b Carl Warner Photo b Carl Warner Photo b Carl Warner Fitting and Alignment Szeliski 6. Computer Vision CS 43, Brown James Has Acknowledgment: Man slides from Derek Hoiem and Grauman&Leibe 2008 AAAI

More information

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify

More information

calibrated coordinates Linear transformation pixel coordinates

calibrated coordinates Linear transformation pixel coordinates 1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial

More information

N-Views (1) Homographies and Projection

N-Views (1) Homographies and Projection CS 4495 Computer Vision N-Views (1) Homographies and Projection Aaron Bobick School of Interactive Computing Administrivia PS 2: Get SDD and Normalized Correlation working for a given windows size say

More information

Scene Modeling for a Single View

Scene Modeling for a Single View Scene Modeling for a Single View René MAGRITTE Portrait d'edward James with a lot of slides stolen from Steve Seitz and David Brogan, 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Classes

More information

Image correspondences and structure from motion

Image correspondences and structure from motion Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.

More information

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10 Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley

More information

Epipolar Geometry CSE P576. Dr. Matthew Brown

Epipolar Geometry CSE P576. Dr. Matthew Brown Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2

More information

CSE 252B: Computer Vision II

CSE 252B: Computer Vision II CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points

More information

Photo Tourism: Exploring Photo Collections in 3D

Photo Tourism: Exploring Photo Collections in 3D Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with

More information

Computer Vision Lecture 20

Computer Vision Lecture 20 Computer Vision Lecture 2 Motion and Optical Flow Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de 28.1.216 Man slides adapted from K. Grauman, S. Seitz, R. Szeliski,

More information

Structure from Motion

Structure from Motion /8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion

More information

Introduction to Computer Vision

Introduction to Computer Vision Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics

More information

Global Flow Estimation. Lecture 9

Global Flow Estimation. Lecture 9 Motion Models Image Transformations to relate two images 3D Rigid motion Perspective & Orthographic Transformation Planar Scene Assumption Transformations Translation Rotation Rigid Affine Homography Pseudo

More information

Feature Based Registration - Image Alignment

Feature Based Registration - Image Alignment Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html

More information

Scene Modeling for a Single View

Scene Modeling for a Single View Scene Modeling for a Single View René MAGRITTE Portrait d'edward James CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2014 Steve

More information

Fitting a transformation: Feature-based alignment April 30 th, Yong Jae Lee UC Davis

Fitting a transformation: Feature-based alignment April 30 th, Yong Jae Lee UC Davis Fitting a transformation: Feature-based alignment April 3 th, 25 Yong Jae Lee UC Davis Announcements PS2 out toda; due 5/5 Frida at :59 pm Color quantization with k-means Circle detection with the Hough

More information

Image-Based Modeling and Rendering

Image-Based Modeling and Rendering Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski

More information

Rectification and Disparity

Rectification and Disparity Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide

More information

Feature Detectors and Descriptors: Corners, Lines, etc.

Feature Detectors and Descriptors: Corners, Lines, etc. Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood

More information

Image warping/morphing

Image warping/morphing Image warping/morphing Digital Visual Effects, Spring 2007 Yung-Yu Chuang 2007/3/20 with slides b Richard Szeliski, Steve Seitz, Tom Funkhouser and Aleei Efros Image warping Image formation B A Sampling

More information

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review

More information

Geometric camera models and calibration

Geometric camera models and calibration Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October

More information

1-2 Feature-Based Image Mosaicing

1-2 Feature-Based Image Mosaicing MVA'98 IAPR Workshop on Machine Vision Applications, Nov. 17-19, 1998, Makuhari, Chibq Japan 1-2 Feature-Based Image Mosaicing Naoki Chiba, Hiroshi Kano, Minoru Higashihara, Masashi Yasuda, and Masato

More information

Automatic Image Alignment (feature-based)

Automatic Image Alignment (feature-based) Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature

More information

Image-Based Lighting. Inserting Synthetic Objects

Image-Based Lighting. Inserting Synthetic Objects Image-Based Lighting 15-463: Rendering and Image Processing Alexei Efros with a lot of slides donated by Paul Debevec Inserting Synthetic Objects Why does this look so bad? Wrong camera orientation Wrong

More information

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 5 x 35 Targil : Panoramas - Stitching and Blending Some slides from Alexei Efros 2 Slide from Brown & Lowe Why Mosaic? Are you getting

More information

Scene Modeling for a Single View

Scene Modeling for a Single View Scene Modeling for a Single View René MAGRITTE Portrait d'edward James with a lot of slides stolen from Steve Seitz and David Brogan, Breaking out of 2D now we are ready to break out of 2D And enter the

More information

Epipolar Geometry and Stereo Vision

Epipolar Geometry and Stereo Vision CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar

More information

Local Feature Detectors

Local Feature Detectors Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,

More information

A New Representation for Video Inspection. Fabio Viola

A New Representation for Video Inspection. Fabio Viola A New Representation for Video Inspection Fabio Viola Outline Brief introduction to the topic and definition of long term goal. Description of the proposed research project. Identification of a short term

More information

Srikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah

Srikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah School of Computing University of Utah Presentation Outline 1 2 3 Forward Projection (Reminder) u v 1 KR ( I t ) X m Y m Z m 1 Backward Projection (Reminder) Q K 1 q Presentation Outline 1 2 3 Sample Problem

More information

Determining the 2d transformation that brings one image into alignment (registers it) with another. And

Determining the 2d transformation that brings one image into alignment (registers it) with another. And Last two lectures: Representing an image as a weighted combination of other images. Toda: A different kind of coordinate sstem change. Solving the biggest problem in using eigenfaces? Toda Recognition

More information

Multiple Views Geometry

Multiple Views Geometry Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective

More information

Map-Enhanced UAV Image Sequence Registration and Synchronization of Multiple Image Sequences

Map-Enhanced UAV Image Sequence Registration and Synchronization of Multiple Image Sequences Map-Enhanced UAV Image Sequence Registration and Synchronization of Multiple Image Sequences Yuping Lin and Gérard Medioni Computer Science Department, University of Southern California 941 W. 37th Place,

More information

E27 Computer Vision - Final Project: Creating Panoramas David Nahmias, Dan Spagnolo, Vincent Stigliani Professor Zucker Due 5/10/13

E27 Computer Vision - Final Project: Creating Panoramas David Nahmias, Dan Spagnolo, Vincent Stigliani Professor Zucker Due 5/10/13 E27 Computer Vision - Final Project: Creating Panoramas David Nahmias, Dan Spagnolo, Vincent Stigliani Professor Zucker Due 5/10/13 Sources Brown, M.; Lowe, D.G., "Recognising panoramas," Computer Vision,

More information

CS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka

CS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka CS223b Midterm Exam, Computer Vision Monday February 25th, Winter 2008, Prof. Jana Kosecka Your name email This exam is 8 pages long including cover page. Make sure your exam is not missing any pages.

More information

But First: Multi-View Projective Geometry

But First: Multi-View Projective Geometry View Morphing (Seitz & Dyer, SIGGRAPH 96) Virtual Camera Photograph Morphed View View interpolation (ala McMillan) but no depth no camera information Photograph But First: Multi-View Projective Geometry

More information

Image Stitching using Harris Feature Detection

Image Stitching using Harris Feature Detection Image Stitching using Harris Feature Detection Shreyas Mistry 1, Prof. Arpita Patel 2 1M. Tech. Student, Electronics & Communication, Chandubhai S. Patel Institute of Technology 2Assi. Professor, Electronics

More information

Omni Stereo Vision of Cooperative Mobile Robots

Omni Stereo Vision of Cooperative Mobile Robots Omni Stereo Vision of Cooperative Mobile Robots Zhigang Zhu*, Jizhong Xiao** *Department of Computer Science **Department of Electrical Engineering The City College of the City University of New York (CUNY)

More information

COSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor

COSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor COSC579: Scene Geometry Jeremy Bolton, PhD Assistant Teaching Professor Overview Linear Algebra Review Homogeneous vs non-homogeneous representations Projections and Transformations Scene Geometry The

More information

3D Reconstruction from Scene Knowledge

3D Reconstruction from Scene Knowledge Multiple-View Reconstruction from Scene Knowledge 3D Reconstruction from Scene Knowledge SYMMETRY & MULTIPLE-VIEW GEOMETRY Fundamental types of symmetry Equivalent views Symmetry based reconstruction MUTIPLE-VIEW

More information

Prof. Feng Liu. Winter /05/2019

Prof. Feng Liu. Winter /05/2019 Prof. Feng Liu Winter 2019 http://www.cs.pd.edu/~fliu/courses/cs410/ 02/05/2019 Last Time Image alignment 2 Toda Image warping The slides for this topic are used from Prof. Yung-Yu Chuang, which use materials

More information

MAPI Computer Vision

MAPI Computer Vision MAPI Computer Vision Multiple View Geometry In tis module we intend to present several tecniques in te domain of te 3D vision Manuel Joao University of Mino Dep Industrial Electronics - Applications -

More information

Homographies and Mosaics

Homographies and Mosaics Tri reort Homograhies and Mosaics Jeffrey Martin (jeffrey-martin.com) CS94: Image Maniulation & Comutational Photograhy with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 06 Steve Seitz and

More information

Visual Odometry. Features, Tracking, Essential Matrix, and RANSAC. Stephan Weiss Computer Vision Group NASA-JPL / CalTech

Visual Odometry. Features, Tracking, Essential Matrix, and RANSAC. Stephan Weiss Computer Vision Group NASA-JPL / CalTech Visual Odometry Features, Tracking, Essential Matrix, and RANSAC Stephan Weiss Computer Vision Group NASA-JPL / CalTech Stephan.Weiss@ieee.org (c) 2013. Government sponsorship acknowledged. Outline The

More information

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few... STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own

More information

Structure from Motion. Prof. Marco Marcon

Structure from Motion. Prof. Marco Marcon Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)

More information

Prof. Kristen Grauman

Prof. Kristen Grauman Fitting Prof. Kristen Grauman UT Austin Fitting Want to associate a model with observed features [Fig from Marszalek & Schmid, 2007] For eample, the model could be a line, a circle, or an arbitrary shape.

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

Final Exam Study Guide

Final Exam Study Guide Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.

More information