Homographies and Mosaics

Transcription

1 Tri reort

2 Homograhies and Mosaics Jeffrey Martin (jeffrey-martin.com) CS94: Image Maniulation & Comutational Photograhy with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 06 Steve Seitz and Rick Szeliski

3 Why Mosaic? Are you getting the whole icture? Comact Camera FOV = 50 x 35 Slide from Brown & Lowe

4 Why Mosaic? Are you getting the whole icture? Comact Camera FOV = 50 x 35 Human FOV = 00 x 35 Slide from Brown & Lowe

5 Why Mosaic? Are you getting the whole icture? Comact Camera FOV = 50 x 35 Human FOV = 00 x 35 Panoramic Mosaic = 360 x 80 Slide from Brown & Lowe

6 Mosaics: stitching images together virtual wide-angle camera

7 Naïve Stitching left on to right on to Translations are not enough to align the images

8 A encil of rays contains all views real camera synthetic camera Can generate any synthetic camera view as long as it has the same center of rojection!

9 Image rerojection mosaic PP The mosaic has a natural interretation in 3D The images are rerojected onto a common lane The mosaic is formed on this lane Mosaic is a synthetic wide-angle camera

10 How to do it? Basic Procedure Take a sequence of images from the same osition Rotate the camera about its otical center Comute transformation between second image and first Transform the second image to overla with the first Blend the two together to create a mosaic If there are more images, reeat but wait, why should this work at all? What about the 3D geometry of the scene? Why aren t we using it?

11 Image rerojection Basic question How to relate two images from the same camera center? how to ma a ixel from PP to PP Answer Cast a ray through each ixel in PP Draw the ixel where that ray intersects PP PP But don t we need to know the geometry of the two lanes in resect to the eye? PP Observation: Rather than thinking of this as a 3D rerojection, think of it as a D image war from one image to another

12 Back to Image Waring Which t-form is the right one for waring PP into PP? e.g. translation, Euclidean, affine, rojective Translation Affine Persective unknowns 6 unknowns 8 unknowns

13 Homograhy A: Projective maing between any two PPs with the same center of rojection rectangle should ma to arbitrary quadrilateral arallel lines aren t but must reserve straight lines same as: unroject, rotate, reroject called Homograhy PP wx' wy' w = * * * * * * H * x * y * To aly a homograhy H Comute = H (regular matrix multily) Convert from homogeneous to image coordinates PP

14 Image waring with homograhies image lane in front black area where no ixel mas to image lane below

15 Image rectification To unwar (rectify) an image Find the homograhy H given a set of and airs How many corresondences are needed? Tricky to write H analytically, but we can solve for it! Find such H that best transforms oints into Use least-squares!

16 Least Squares Examle ' ' x x x x = + = + = ' ' x x

17 Least Squares Examle ' ' x x x x = + = + = ' ' x x = ' 3 ' ' 3 x x overconstrained min b Ax

18 Least-Squares Solve: A x = b (N,d)(d,) = (N,) N d A Normal equations A T A x = A T b (d,n)(n,d)(d,) = (d,n)(n,) Solution: x = (A T A) - A T b rank(a) min(d,n) assume rank(a)=d imlies rank(a T A)=d A T A is invertible

19 Solving for homograhies wx' wy' w = H a b = d e g h c f i x y Can set scale factor i=. So, there are 8 unkowns. Set u a system of linear equations: Ah = b where vector of unknowns h = [a,b,c,d,e,f,g,h] T Need at least 8 eqs, but the more the better Solve for h. If overconstrained, solve using least-squares: Can be done in Matlab using \ command see hel lmdivide min Ah b

20 Fun with homograhies Original image St.Petersburg hoto by A. Tikhonov Virtual camera rotations

21 Analysing atterns and shaes What is the shae of the b/w floor attern? Slide from Criminisi The floor (enlarged) Automatically rectified floor

22 Analysing atterns and shaes Automatic rectification From Martin Kem The Science of Art (manual reconstruction) atterns have been discovered! Slide from Criminisi

23 Analysing atterns and shaes What is the (comlicated) shae of the floor attern? St. Lucy Altariece, D. Veneziano Slide from Criminisi Automatically rectified floor

24 Analysing atterns and shaes Automatic rectification From Martin Kem, The Science of Art (manual reconstruction) Slide from Criminisi

25 Julian Beever: Manual Homograhies htt://users.skynet.be/j.beever/ave.htm

26 Holbein, The Ambassadors

27 Panoramas. Pick one image (red). War the other images towards it (usually, one by one) 3. blend

28 changing camera center Does it still work? synthetic PP PP PP

29 Planar scene (or far away) PP PP3 PP PP3 is a rojection lane of both centers of rojection, so we are OK! This is how big aerial hotograhs are made

30 Planar mosaic

31 Programming Project #7 (art ) Homograhies and Panoramic Mosaics Cature hotograhs (and ossibly video) Might want to use triod Comute homograhies (define corresondences) will need to figure out how to setu system of eqs. (un)war an image (undo ersective distortion) Produce anoramic mosaics (with blending) Do some of the Bells and Whistles

32 Bells and Whistles Blending and Comositing use homograhies to combine images or video and images together in an interesting (fun) way. E.g. ut fake graffiti on buildings or chalk drawings on the ground relace a road sign with your own oster roject a movie onto a building wall etc.

33 Bells and Whistles Video Panorama Cature two (or more) stationary videos (either from the same oint, or of a lanar/far-away scene). Comute homograhy and roduce a video mosaic. Need to worry about synchronization (not too hard). e.g. caturing a football game from the sides of the stadium Other interesting ideas? talk to me

34 From revious year s classes Ben Hollis, 004 Ben Hollis, 004 Matt Pucevich, 004 Eunjeong Ryu (E.J), 004

35 Bells and Whistles Cature creative/cool/bizzare anoramas Examle from UW (by Brett Allen): Ever wondered what is haening inside your fridge while you are not looking? Cature a 360 anorama (quite tricky )