Introduction to Image Processing and Computer Vision. -- Panoramas and Blending --

Size: px
Start display at page:

Download "Introduction to Image Processing and Computer Vision. -- Panoramas and Blending --"

Transcription

1 Introduction to Image Processing and Computer Vision -- Panoramas and Blending -- Winter 2013/14 Ivo Ihrke

2 Panoramas

3 Mosaics and Panoramas - Outline - Perspective Panoramas - Hardware-Based - Software-Based (Multiple Photographs) - Image registration - Image blending

4 Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 50 x 35 Slide from Brown & Lowe

5 Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 50 x 35 Human FOV = 200 x 135 Slide from Brown & Lowe

6 Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 50 x 35 Human FOV = 200 x 135 Panoramic Mosaic = 360 x 180 Slide from Brown & Lowe

7 Single vs. Multiple Viewpoint Single-viewpoint Necessary for creating pure perspective images. Many vision algorithms assume pinhole cameras. Images that aren t perspective images look distorted.

8 In the old days of film photography Single-viewpoint Single exposure Standard 35mm film

9 Omnidirectional Catadioptric Cameras catadioptric = mirror + lens system O-360 EyeSee360

10 Images of an Omnidirectional Camera images: CAVE lab

11 Catadioptric System Full Texture K [Kuthirummal 2006]

12 Cata-Fisheye Camera [Krishnan and Nayhar 2008]

13 Catadioptric System Stereo object center object

14 Multi-camera, Single-viewpoint? Immersive Media Dodeca2000 PointGrey Ladybug

15 Lens image circle scanning Medium format lens (6 cm x 6 cm image circle) Manual Scan plate APS-C (2.4 cm x 1.6 cm) camera

16 Perspective Panoramas Registration

17

18 Single Center of Projection Take a sequence of images from the same position Rotate the camera about its optical center Compute transformation between second image and first Transform the second image to overlap with the first Blend the two together to create a mosaic If there are more images, repeat why don t we need the 3D geometry?

19 Image Reprojection The images are reprojected onto a common plane The mosaic is formed on this plane Mosaic is a synthetic wide-angle camera mosaic PP

20 A pencil of rays contains all views real camera synthetic camera scene Can generate any synthetic camera view as long as it has the same center of projection!

21 No-parallax point Same center of projection can be ensured by rotating camera-lens setup around the entrance pupil (and NOT the nodal point!).

22 Image reprojection How to relate two images from the same camera center? Images contain the same information along the same ray. Use 2D image warp

23 Taxonomy of Projective Transformations

24 Perspective Transformation 3D to 2D projection Point in world coordinates P(x e,y e,z e ) Distance center of projection image plane D(=f) Image coordinates (x s,y s )

25 Homogeneous Coordinates: Point Representation x=(x,y,w) (x,y ) w=1 x' = x w y' = y w

26 Homogeneous Coordinates: Point Representation

27 Hom. Coord.: Line Representation l=(a,b,c) ax+by+c=0

28 Hom. Coordinates: Point on Line x=(x,y,w) l=(a,b,c) xl 0

29 Hom. Coordinates: Intersection of Lines l l x l'l x

30 Hom. Coordinates: Line through 2 Points l=(a,b,c) x=(x,y,w) x =(x,y,w ) x'x l

31 Embedding of R 2 into P 2 For the time being Representation of transformations by 3x3 matrices Mathematical trick convenient representation to express rotations and translations as matrix multiplications Easy to find line through points, point-line/line-line intersections Easy representation of projective transformation (homography) Homogeneous Coordinates for 2D W Y W X W Y X y x y x R / / and, P 1 2 2

32 Projective Transformations Projecting one plane onto another using one projection center

33 Examples of Projective Transformations Projection between 2 images via a world plane Concatenating two projective transforms gives another projective transform Projection between 2 images with the same camera center Rotating camera or camera with varying focal length Shadow projection of a plane onto another plane

34 Taxonomy of Projective Transformations

35 Taxonomy of Projective Transformations

36 Distortions under Central Projection Similarity: circle remains circle, square remains square line orientation is preserved Affine: circle becomes ellipse, square becomes rhombus parallel lines remain parallel Projective: imaged object size depends on distance from camera parallel lines converge

37 Removing Projective Distortion Projective transformation in inhomogeneous form 4 general point correspondences (x,y ->x,y ) on the planar facade lead to eight linear equations of the type Sufficient to solve for H up to multiplicative factor

38 The Direct Linear Transform (DLT) Algorithm Given: 4 2D point correspondences Objective: estimate the projective transform matrix H x x x x i ' ' ' ' x x x x i

39 The DLT Algorithm II Re-phrasing H Re-ording into h vector gives 0 Estimating matrix H from point correspondences is equivalent to i i i w y x x i ' ' ' ' i i i w y x x i i i i w y x h h h h h h h h h

40 The DLT Algorithm III Only rows 1 and 2 are linearly independent omit row 3 Inhomogeneous solution: set one matrix entry equal to 1 (e.g. h33) Solve by Gaussian elimination or least-squares techniques i i i w y x x i ' ' ' ' i i i w y x x i

41 Estimating Homographies

42 Homography or not? Coincidences between 3D points at different depths are preserved Pure camera rotation about camera center 2D Homography Different depths are imaged to different image positions Camera rotates and translates Motion Parallax, no Homography

43 Panoramic Mosaicing Rotation about camera center: homography choose one image as reference compute homography to map neighboring image to reference image plane projectively warp image, add to reference plane repeat for all images bow tie shape

44 Alternative Panoramas Project images onto different surfaces: Cylindrical Spherical Cubic (think of cube map) Images

45 Example My former office Register left and right image to the middle one using two homographies

46 Example My former office all images registered to the central one (2 homographies)

47 Example My former office ghost seams

48 Image Blending

49 Linear Blending «Moyenne» entre deux images Pas la moyenne de l image des objets mais une image de la moyenne des objets et une moyenne évoluant au cours du temps. Comment savoir ce qu est la bonne moyenne? On n en sait rien! Mais les artistes peuvent nous aider 49

50 Fondu «cross-dissolve» Interpolation de l image complète I t = (1-t) * I 1 + t * I 2 Mais que se passe-t-il si les images ne sont pas alignées? 50

51 Aligner puis faire le fondu

52 Image Blending slides from Alexei Efros

53 Feathering = Blending

54 Effect of Window Size 1 left 1 0 right 0

55 Effect of Window Size

56 Good Window Size 1 0 Optimal Window: smooth but not ghosted

57 What is the Optimal Window? To avoid seams window >= size of largest prominent feature To avoid ghosting window <= 2*size of smallest prominent feature Natural to cast this in the Fourier domain largest frequency <= 2*size of smallest frequency do blending in different frequency bands

58 Bandpass Computations

59 Bandpass Computations filtered images Fourier space filter shape Octave = doubling frequency low-pass 1 st octave 2 nd octave 3 rd octave

60 Lowpass

61 First Octave

62 Second Octave

63 Third Octave

64 Zoom-In 3 rd Octave - Jpeg-Artifacts

65 Reconstruction filtered images = low-pass 1 st octave 2 nd octave 3 rd octave original Fourier space filter shape = low-pass 1 st octave 2 nd octave 3 rd octave full freq. range

66 Spatial Domain Interpretation / Implementation original

67 What does blurring take away? smoothed (5x5 Gaussian)

68 High-Pass Filter smoothed minus original

69 Image Pyramids mipmap or precursor of wavelets Gaussian Pyramid

70 Create by Image Sub-sampling 1/8 1/4 Throw away every other row and column to create a 1/2 size image

71 Improper Image Sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Why does this look so bad? Aliasing!

72 Proper Sub-Sampling First, band-limit, then sub-sample! Repeat Filter Subsample Until minimum resolution reached filter mask Whole pyramid is only 4/3 the size of the original image!

73 Implementation by Gaussian pre-filtering G 1/8 G 1/4 Gaussian 1/2 Filter size should double for each ½ size reduction.

74 Subsampling with Gaussian pre-filtering Gaussian 1/2 G 1/4 G 1/8 Solution: filter the image, then subsample Filter size should double for each ½ size reduction.

75 Compare with... 1/2 1/4 (2x zoom) 1/8 (4x zoom)

76 Band-pass filtering Gaussian Pyramid (low-pass images) Laplacian Pyramid (subband images) Created from Gaussian pyramid by subtraction

77 Laplacian Pyramid Original image Need this! How can we reconstruct (collapse) this pyramid into the original image?

78 Laplacian Pyramid Need this!

79 Pyramid Blending Left pyramid blend Right pyramid

80 Blending Apples and Oranges original apple original orange blend scale 1 blend scale 2 blend scale 3 pyramid blending

81 Blending Apples and Oranges blend scale 1 pyramid blending

82 Blending Apples and Oranges blend scale 2 pyramid blending

83 Blending Apples and Oranges blend scale 3 pyramid blending

84 Different Frequency Bands

85 Simplification: Two-band Blending Brown & Lowe, 2003 Only use two bands: high freq. and low freq. Blends low freq. smoothly Blend high freq. with no smoothing: use binary mask

86 2-band Blending Low frequency (l > 2 pixels) High frequency (l < 2 pixels)

87 Linear Blending

88 2-band Blending

89 Still Some Artifacts Left Ghosting objects move in the scene. Differing exposures between images. Pyramid blending does not solve this.

90 De-Ghosting In regions with differences don t blend - crop. [Uyttendaele et al. 2001]

91 Gradient Domain Blending In Pyramid Blending, we decomposed our image into 2 nd derivatives (Laplacian) and a low-res image Let us now look at 1 st derivatives (gradients): No need for low-res image captures everything (up to a constant) easy to deal with low-frequency differences Idea: Differentiate Blend Reintegrate

92 Poisson Image Editing original mask Poisson Inpainting result

93 Poisson Image Editing original original to paste copy and paste Poisson Image Editing result

94 Gradient Domain Blending (2D) Take partial derivatives dx and dy (the gradient field) Fiddle around with them (copy, smooth, blend, feather, etc) Reintegrate But now integral(dx) might not equal integral(dy) Find the most agreeable solution Equivalent to solving Poisson equation

95 Gradient Domain Blending (2D) - But now integral(dx) might not equal integral(dy): INCONSISTENCY - There is no UNIQUE SOLUTON! - Poisson-solver (most widely used) can produce artifacts. This is how it looks like when we directly integrate an inconsistent gradient field (row-by-row in this case) +10 ¹ +70

96 Comparisons [Levin et al 2004]

97 End

98 Acknowledgements Many slides by Steve Seitz, Rick Szeliski Histogram slides by Samir H. Abdul-Jauwad Some histogram-matching results by Paul Bourke More slides by Pierre Bénard, Hendrik Lensch

99 The End

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 5 x 35 Targil : Panoramas - Stitching and Blending Some slides from Alexei Efros 2 Slide from Brown & Lowe Why Mosaic? Are you getting

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection

More information

Image Compositing and Blending

Image Compositing and Blending Computational Photography and Capture: Image Compositing and Blending Gabriel Brostow & Tim Weyrich TA: Frederic Besse Vignetting 3 Figure from http://www.vanwalree.com/optics/vignetting.html Radial Distortion

More information

Announcements. Mosaics. How to do it? Image Mosaics

Announcements. Mosaics. How to do it? Image Mosaics Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today

More information

Recap from Monday. Frequency domain analytical tool computational shortcut compression tool

Recap from Monday. Frequency domain analytical tool computational shortcut compression tool Recap from Monday Frequency domain analytical tool computational shortcut compression tool Fourier Transform in 2d in Matlab, check out: imagesc(log(abs(fftshift(fft2(im))))); Image Blending (Szeliski

More information

Mosaics. Today s Readings

Mosaics. Today s Readings Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections

More information

Image Warping and Mosacing

Image Warping and Mosacing Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

More information

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.

More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin. More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick

More information

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position = Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

More information

Today s lecture. Image Alignment and Stitching. Readings. Motion models

Today s lecture. Image Alignment and Stitching. Readings. Motion models Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

More information

Midterm Examination CS 534: Computational Photography

Midterm Examination CS 534: Computational Photography Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)

More information

6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS

6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Bill Freeman Frédo Durand MIT - EECS Lots of slides stolen from Alyosha Efros, who stole them from Steve Seitz

More information

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

Panoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS

Panoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS Olivier Gondry 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Director of music video and commercial Special effect specialist (Morphing, rotoscoping) Today

More information

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

More information

Recap. DoF Constraint Solver. translation. affine. homography. 3D rotation

Recap. DoF Constraint Solver. translation. affine. homography. 3D rotation Image Blending Recap DoF Constraint Solver translation affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine

More information

CS4670: Computer Vision

CS4670: Computer Vision CS467: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello Szeliski 2.1.3-2.1.6 Reading Announcements Project 2a due Friday, 8:59pm Project 2b out Friday

More information

Stitching and Blending

Stitching and Blending Stitching and Blending Kari Pulli VP Computational Imaging Light First project Build your own (basic) programs panorama HDR (really, exposure fusion) The key components register images so their features

More information

COSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor

COSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor COSC579: Scene Geometry Jeremy Bolton, PhD Assistant Teaching Professor Overview Linear Algebra Review Homogeneous vs non-homogeneous representations Projections and Transformations Scene Geometry The

More information

Image stitching. Announcements. Outline. Image stitching

Image stitching. Announcements. Outline. Image stitching Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring

More information

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration

More information

Photometric Processing

Photometric Processing Photometric Processing 1 Histogram Probability distribution of the different grays in an image 2 Contrast Enhancement Limited gray levels are used Hence, low contrast Enhance contrast 3 Histogram Stretching

More information

Image warping and stitching

Image warping and stitching Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize

More information

Image Blending and Compositing NASA

Image Blending and Compositing NASA Image Blending and Compositing NASA CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2016 Image Compositing Compositing Procedure 1. Extract Sprites (e.g using Intelligent

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry

More information

calibrated coordinates Linear transformation pixel coordinates

calibrated coordinates Linear transformation pixel coordinates 1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial

More information

N-Views (1) Homographies and Projection

N-Views (1) Homographies and Projection CS 4495 Computer Vision N-Views (1) Homographies and Projection Aaron Bobick School of Interactive Computing Administrivia PS 2: Get SDD and Normalized Correlation working for a given windows size say

More information

Feature Matching and RANSAC

Feature Matching and RANSAC Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction

More information

Multiresolution Image Processing

Multiresolution Image Processing Multiresolution Image Processing 2 Processing and Analysis of Images at Multiple Scales What is Multiscale Decompostion? Why use Multiscale Processing? How to use Multiscale Processing? Related Concepts:

More information

Single-view 3D Reconstruction

Single-view 3D Reconstruction Single-view 3D Reconstruction 10/12/17 Computational Photography Derek Hoiem, University of Illinois Some slides from Alyosha Efros, Steve Seitz Notes about Project 4 (Image-based Lighting) You can work

More information

Vision Review: Image Formation. Course web page:

Vision Review: Image Formation. Course web page: Vision Review: Image Formation Course web page: www.cis.udel.edu/~cer/arv September 10, 2002 Announcements Lecture on Thursday will be about Matlab; next Tuesday will be Image Processing The dates some

More information

Video Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen

Video Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen Video Mosaics for Virtual Environments, R. Szeliski Review by: Christopher Rasmussen September 19, 2002 Announcements Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday.

More information

Geometric camera models and calibration

Geometric camera models and calibration Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October

More information

Broad field that includes low-level operations as well as complex high-level algorithms

Broad field that includes low-level operations as well as complex high-level algorithms Image processing About Broad field that includes low-level operations as well as complex high-level algorithms Low-level image processing Computer vision Computational photography Several procedures and

More information

Panoramic Image Stitching

Panoramic Image Stitching Mcgill University Panoramic Image Stitching by Kai Wang Pengbo Li A report submitted in fulfillment for the COMP 558 Final project in the Faculty of Computer Science April 2013 Mcgill University Abstract

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 5: Projection Reading: Szeliski 2.1 Projection Reading: Szeliski 2.1 Projection Müller Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html Modeling

More information

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few... STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own

More information

Modeling Light. On Simulating the Visual Experience

Modeling Light. On Simulating the Visual Experience Modeling Light 15-463: Rendering and Image Processing Alexei Efros On Simulating the Visual Experience Just feed the eyes the right data No one will know the difference! Philosophy: Ancient question: Does

More information

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication DD2423 Image Analysis and Computer Vision IMAGE FORMATION Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 8, 2013 1 Image formation Goal:

More information

Image-Based Rendering

Image-Based Rendering Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome

More information

CSE 252B: Computer Vision II

CSE 252B: Computer Vision II CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe: Jayson Smith LECTURE 4 Planar Scenes and Homography 4.1. Points on Planes This lecture examines the special case of planar scenes. When talking

More information

Image Formation. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania

Image Formation. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania Image Formation Antonino Furnari Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania furnari@dmi.unict.it 18/03/2014 Outline Introduction; Geometric Primitives

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253 Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine

More information

Filtering Applications & Edge Detection. GV12/3072 Image Processing.

Filtering Applications & Edge Detection. GV12/3072 Image Processing. Filtering Applications & Edge Detection GV12/3072 1 Outline Sampling & Reconstruction Revisited Anti-Aliasing Edges Edge detection Simple edge detector Canny edge detector Performance analysis Hough Transform

More information

Object Recognition with Invariant Features

Object Recognition with Invariant Features Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user

More information

Agenda. Rotations. Camera models. Camera calibration. Homographies

Agenda. Rotations. Camera models. Camera calibration. Homographies Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate

More information

11/28/17. Midterm Review. Magritte, Homesickness. Computational Photography Derek Hoiem, University of Illinois

11/28/17. Midterm Review. Magritte, Homesickness. Computational Photography Derek Hoiem, University of Illinois Midterm Review 11/28/17 Computational Photography Derek Hoiem, University of Illinois Magritte, Homesickness Major Topics Linear Filtering How it works Template and Frequency interpretations Image pyramids

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263 Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine

More information

Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration

Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1

More information

CS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003

CS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003 CS 664 Slides #9 Multi-Camera Geometry Prof. Dan Huttenlocher Fall 2003 Pinhole Camera Geometric model of camera projection Image plane I, which rays intersect Camera center C, through which all rays pass

More information

Image Based Rendering. D.A. Forsyth, with slides from John Hart

Image Based Rendering. D.A. Forsyth, with slides from John Hart Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit

More information

Cameras and Stereo CSE 455. Linda Shapiro

Cameras and Stereo CSE 455. Linda Shapiro Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation

More information

The 7d plenoptic function, indexing all light.

The 7d plenoptic function, indexing all light. Previous Lecture The 7d plenoptic function, indexing all light. Lightfields: a 4d (not 5d!) data structure which captures all outgoing light from a region and permits reconstruction of arbitrary synthetic

More information

Image-based Modeling and Rendering: 8. Image Transformation and Panorama

Image-based Modeling and Rendering: 8. Image Transformation and Panorama Image-based Modeling and Rendering: 8. Image Transformation and Panorama I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung Univ, Taiwan Outline Image transformation How to represent the

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V

More information

Compositing a bird's eye view mosaic

Compositing a bird's eye view mosaic Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition

More information

Warping, Morphing and Mosaics

Warping, Morphing and Mosaics Computational Photograph and Video: Warping, Morphing and Mosaics Prof. Marc Pollefes Dr. Gabriel Brostow Toda s schedule Last week s recap Warping Morphing Mosaics Toda s schedule Last week s recap Warping

More information

Computational Photography

Computational Photography Computational Photography Matthias Zwicker University of Bern Fall 2010 Today Light fields Introduction Light fields Signal processing analysis Light field cameras Application Introduction Pinhole camera

More information

CSE 527: Introduction to Computer Vision

CSE 527: Introduction to Computer Vision CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature

More information

But First: Multi-View Projective Geometry

But First: Multi-View Projective Geometry View Morphing (Seitz & Dyer, SIGGRAPH 96) Virtual Camera Photograph Morphed View View interpolation (ala McMillan) but no depth no camera information Photograph But First: Multi-View Projective Geometry

More information

Stereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz

Stereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes

More information

Computer Vision I - Appearance-based Matching and Projective Geometry

Computer Vision I - Appearance-based Matching and Projective Geometry Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation

More information

Final Exam Study Guide CSE/EE 486 Fall 2007

Final Exam Study Guide CSE/EE 486 Fall 2007 Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.

More information

Image-Based Lighting. Inserting Synthetic Objects

Image-Based Lighting. Inserting Synthetic Objects Image-Based Lighting 15-463: Rendering and Image Processing Alexei Efros with a lot of slides donated by Paul Debevec Inserting Synthetic Objects Why does this look so bad? Wrong camera orientation Wrong

More information

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017 CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo

More information

Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski

Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Today Go over Midterm Go over Project #3

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester

More information

Today. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography

Today. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry

More information

Projective Geometry and Camera Models

Projective Geometry and Camera Models /2/ Projective Geometry and Camera Models Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Note about HW Out before next Tues Prob: covered today, Tues Prob2: covered next Thurs Prob3:

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week

More information

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004 Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,

More information

Application questions. Theoretical questions

Application questions. Theoretical questions The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography

More information

Introduction to Computer Vision. Week 3, Fall 2010 Instructor: Prof. Ko Nishino

Introduction to Computer Vision. Week 3, Fall 2010 Instructor: Prof. Ko Nishino Introduction to Computer Vision Week 3, Fall 2010 Instructor: Prof. Ko Nishino Last Week! Image Sensing " Our eyes: rods and cones " CCD, CMOS, Rolling Shutter " Sensing brightness and sensing color! Projective

More information

Automatic Image Alignment (feature-based)

Automatic Image Alignment (feature-based) Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature

More information

All good things must...

All good things must... Lecture 17 Final Review All good things must... UW CSE vision faculty Course Grading Programming Projects (80%) Image scissors (20%) -DONE! Panoramas (20%) - DONE! Content-based image retrieval (20%) -

More information

Scene Modeling for a Single View

Scene Modeling for a Single View Scene Modeling for a Single View René MAGRITTE Portrait d'edward James with a lot of slides stolen from Steve Seitz and David Brogan, 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Classes

More information

Camera Calibration. COS 429 Princeton University

Camera Calibration. COS 429 Princeton University Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3

More information

Image-Based Modeling and Rendering

Image-Based Modeling and Rendering Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski

More information

Computer Vision CS 776 Fall 2018

Computer Vision CS 776 Fall 2018 Computer Vision CS 776 Fall 2018 Cameras & Photogrammetry 1 Prof. Alex Berg (Slide credits to many folks on individual slides) Cameras & Photogrammetry 1 Albrecht Dürer early 1500s Brunelleschi, early

More information

A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India

A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract

More information

Agenda. Rotations. Camera calibration. Homography. Ransac

Agenda. Rotations. Camera calibration. Homography. Ransac Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr

More information

Local Feature Detectors

Local Feature Detectors Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,

More information

The SIFT (Scale Invariant Feature

The SIFT (Scale Invariant Feature The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical

More information

CS201 Computer Vision Camera Geometry

CS201 Computer Vision Camera Geometry CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the

More information

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014

SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image

More information

Image Composition. COS 526 Princeton University

Image Composition. COS 526 Princeton University Image Composition COS 526 Princeton University Modeled after lecture by Alexei Efros. Slides by Efros, Durand, Freeman, Hays, Fergus, Lazebnik, Agarwala, Shamir, and Perez. Image Composition Jurassic Park

More information

360 Full View Spherical Mosaic

360 Full View Spherical Mosaic 360 Full View Spherical Mosaic Huang Wenfan Huang Yehui Rong Nan U017865B U017844X U018274R Objective Full spherical mosaic 360 x 180. All images are taken with camera mounted on a tripod. Registration

More information

Introduction to Computer Vision

Introduction to Computer Vision Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics

More information

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town

Computer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town Recap: Smoothing with a Gaussian Computer Vision Computer Science Tripos Part II Dr Christopher Town Recall: parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of

More information

Image-based Lighting

Image-based Lighting Image-based Lighting 10/17/15 T2 Computational Photography Derek Hoiem, University of Illinois Many slides from Debevec, some from Efros Next week Derek away for ICCV (Venice) Zhizhong and Aditya will

More information

Scene Modeling for a Single View

Scene Modeling for a Single View Scene Modeling for a Single View René MAGRITTE Portrait d'edward James CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2014 Steve

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

Blending and Compositing

Blending and Compositing 09/26/17 Blending and Compositing Computational Photography Derek Hoiem, University of Illinois hybridimage.m pyramids.m Project 1: issues Basic tips Display/save Laplacian images using mat2gray or imagesc

More information

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving

More information

CMPSCI 670: Computer Vision! Image formation. University of Massachusetts, Amherst September 8, 2014 Instructor: Subhransu Maji

CMPSCI 670: Computer Vision! Image formation. University of Massachusetts, Amherst September 8, 2014 Instructor: Subhransu Maji CMPSCI 670: Computer Vision! Image formation University of Massachusetts, Amherst September 8, 2014 Instructor: Subhransu Maji MATLAB setup and tutorial Does everyone have access to MATLAB yet? EdLab accounts

More information