Lecture 8: Camera Models

Size: px

Start display at page:

Download "Lecture 8: Camera Models"

Ilene Wood
5 years ago
Views:

1 Lecture 8: Camera Models Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei- Fei Li Stanford Vision Lab Oct- 15

2 What we will learn today? Pinhole cameras Cameras & lenses The geometry of pinhole cameras ProjecPon matrix Intrinsic parameters Extrinsic parameters Reading: [FP] Chapters 1 3 [HZ] Chapter Oct- 15

3 What we will learn today? Pinhole cameras Cameras & lenses The geometry of pinhole cameras ProjecPon matrix Intrinsic parameters Extrinsic parameters Reading: [FP] Chapters 1 3 [HZ] Chapter Oct- 15

4 Camera and World Geometry How tall is this woman? How high is the camera? What is the camera rotation? What is the focal length of the camera? Which ball is closer? Slide credit: J. Hayes

5 How do we see the world? Let s design a camera Idea 1: put a piece of film in front of an object Do we get a reasonable image? Oct- 15

6 Pinhole camera Add a barrier to block off most of the rays This reduces blurring The opening known as the aperture Oct- 15

7 Camera obscura: the pre- camera Known during classical period in China and Greece (e.g. Mo- Tsi, China, 47BC to 39BC) Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill Photo by Seth Ilys Slide credit: J. Hayes

8 Camera Obscura used for Tracing Lens Based Camera Obscura, 1568 Slide credit: J. Hayes

Niepce, 1826 Stored at UT Austin Niepce later teamed up with

9 First Photograph Oldest surviving photograph Took 8 hours on pewter plate Photograph of the first photograph Joseph Niepce, 1826 Stored at UT Austin Niepce later teamed up with Daguerre, who eventually created Daguerrotypes Slide credit: J. Hayes

10 Dimensionality ReducPon Machine (3D to 2D) 3D world 2D image Point of observation Figures Stephen E. Palmer, 22

11 ProjecPon can be tricky Slide source: Seitz

12 ProjecPon can be tricky Slide source: Seitz

13 What is lost? Length ProjecPve Geometry Who is taller? Which is closer? Slide credit: J. Hayes

14 Length is not preserved A B C Figure by David Forsyth

15 ProjecPve Geometry What is lost? Length Angles Parallel? Perpendicular? Slide credit: J. Hayes

16 ProjecPve Geometry What is preserved? Straight lines are spll straight Slide credit: J. Hayes

17 Vanishing points and lines Parallel lines in the world intersect in the image at a vanishing point Vanishing Line Vanishing Point o Vanishing Point o Slide credit: J. Hayes

18 Vanishing points and lines Vertical vanishing point (at infinity) Vanishing line Vanishing point Vanishing point Slide from Efros, Photo from Criminisi

19 14- Oct Pinhole camera = = z y f y z x f x ' ' ' ' ʹ ʹ = ʹ y x P = z y x P Derived using similar triangles Note: z is always negapve.

20 Pinhole camera f f Common to draw image plane in front of the focal point Moving the image plane merely scales the image. x' = y' = f f x z y z Oct- 15

21 Pinhole camera Is the size of the aperture important? Kate lazuka Oct- 15

$be too small? - Less light passes through - DiffracPon effect Adding lenses!$

22 Cameras & Lenses Shrinking aperture size - Rays are mixed up - Why the aperture cannot be too small? - Less light passes through - DiffracPon effect Adding lenses! Oct- 15

23 What we will learn today? Pinhole cameras Cameras & lenses The geometry of pinhole cameras ProjecPon matrix Intrinsic parameters Extrinsic parameters Reading: [FP] Chapters 1 3 [HZ] Chapter Oct- 15

24 Cameras & Lenses A lens focuses light onto the film Oct- 15

25 Cameras & Lenses focal point f A lens focuses light onto the film Rays passing through the center are not deviated All parallel rays converge to one point on a plane located at the focal length f Oct- 15

26 Cameras & Lenses circle of confusion A lens focuses light onto the film There is a specific distance at which objects are in focus [other points project to a circle of confusion in the image] Oct- 15

$planar RefracPon: when a ray passes from one medium to another Snell s law n 1 sin α 1 = n 2 sin α 2 α 1 = incident angle α 2 = refracpon angle n i = index of refracpon 27 14-$

27 Cameras & Lenses Laws of geometric oppcs Light travels in straight lines in homogeneous medium ReflecPon upon a surface: incoming ray, surface normal, and reflecpon are co- planar RefracPon: when a ray passes from one medium to another Snell s law n 1 sin α 1 = n 2 sin α 2 α 1 = incident angle α 2 = refracpon angle n i = index of refracpon Oct- 15

28 Thin Lenses z o z ' = f + z o f = R 2(n 1) Snell s law: n 1 sin α 1 = n 2 sin α 2 Small angles: n 1 α 1 n 2 α 2 n 1 = n (lens) n 1 = 1 (air) x' = y' = z' z' x z y z Oct- 15

29 Cameras & Lenses Source wikipedia Oct- 15

$Issues with lenses: Chroma>c Aberra>on Lens has different refracpve indices$

30 Issues with lenses: Chroma>c Aberra>on Lens has different refracpve indices for different wavelengths: causes color fringing f = R 2(n 1) Oct- 15

31 Issues with lenses: Chroma>c Aberra>on Rays farther from the oppcal axis focus closer Oct- 15

32 Issues with lenses: Chroma>c Aberra>on DeviaPons are most nopceable for rays that pass through the edge of the lens No distortion Pin cushion Barrel (fisheye lens) Image magnificapon decreases with distance from the oppcal axis Oct- 15

33 What we will learn today? Pinhole cameras Cameras & lenses The geometry of pinhole cameras ProjecPon matrix Intrinsic parameters Extrinsic parameters Oct- 15

34 Rela>ng real- world point to a point on a camera f P c P f = focal length c = center of the camera P = (x, y, z) P' = ( f x z, f y z ) R 3 E R Oct- 15

35 Rela>ng real- world point to a point on a camera Is this a linear transformapon? P = (x, y, z) P' = ( f x z, f y z ) No division by z is nonlinear! How to make it linear? Oct- 15

36 Homogeneous coordinates a reminder homogeneous image coordinates homogeneous scene coordinates ConverPng from homogeneous coordinates Oct- 15

37 14- Oct RelaPng a real- world point to a point on the camera P = (x, y, z) P' = ( f x z, f y z ) In Cartesian coordinates: In homogeneous coordinates: = = 1 1 ' z y x f f z y f x f P P = M P ' M Projec>on matrix 3 H 4 R R

38 Interlude: why does this maser?

39 Object RecogniPon (CVPR 26) Slide credit: J. Hayes

40 InserPng photographed objects into images (SIGGRAPH 27) Original Created Slide credit: J. Hayes

14- Oct- 15 41 RelaPng a real- world point to a point on the camera In homogeneous coordinates: = = 1 1 ' z y x f f z y f x f P M

41 14- Oct RelaPng a real- world point to a point on the camera In homogeneous coordinates: = = 1 1 ' z y x f f z y f x f P M ideal world Intrinsic Assumptions Unit aspect ratio Optical center at (,) No skew Extrinsic Assumptions No rotation Camera at (,,)

42 RelaPng a real- world point to a point on the camera In homogeneous coordinates:! # P' = # # "# f x f y z $! & # & = # & # %& " f f 1 K $! & # & # & # %# " x y z 1 Intrinsic Assumptions Unit aspect ratio Optical center at (,) No skew $ & & = K & [ I ]P & % Extrinsic Assumptions No rotation Camera at (,,) Oct- 15

43 Remove assumppon: known oppcal center Intrinsic Assumptions Optical center at (,) Optical center at (u, v) Square pixels No skew P' = K! I " Extrinsic Assumptions No rotation Camera at (,,) u x y # $ P w v = f v z 1 f u 1 1 Slide inspirapon: S. Savarese

44 Remove assumppon: square pixels = z y x v u v u w β α Extrinsic Assumptions No rotation Camera at (,,) Intrinsic Assumptions Optical center at (u, v) Square pixels Rectangular pixels No skew P' = K I! " # $ P Slide inspirapon: S. Savarese

45 Remove assumppon: non- skewed pixels Intrinsic Assumptions Optical center at (u, v) Rectangular pixels No skew Small skew P' = K! I " # $ P y y c Extrinsic Assumptions No rotation Camera at (,,) u w v 1 = α s β u v 1 x y z 1 x c Slide inspirapon: S. Savarese v C=[c x, c y ] x

46 Remove assumppon: non- skewed pixels = z y x v u s v u w β α Extrinsic Assumptions No rotation Camera at (,,) Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew Intrinsic parameters P' = K I! " # $ P Slide inspirapon: S. Savarese

47 Real world camera: Translate + Rotate R j w t k w O w i w Slide inspirapon: S. Savarese, J. Hayes

48 Remove assumppon: allow translapon Extrinsic Assumptions No rotation Camera at (,,) -> (tx,ty,tz) Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew P' = K I t [ ] P = z y x t t t v u v u w z y x β α Slide inspirapon: S. Savarese

49 Remove assumppon: allow rotapon Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew z y P Rotation around the coordinate axes, counter-clockwise Slide inspirapon: S. Savarese γ P Extrinsic Assumptions No rotation Camera at (tx,ty,tz) 1 R x ( α) = cosα sinα sinα cosα cos β sin β R = y ( β ) 1 sin β cos β cosγ sin γ R ( γ ) = z sin γ cosγ 1

50 Remove assumppon: allow rotapon Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew P' = K [ R t ] P Extrinsic Assumptions No rotation Camera at (tx,ty,tz) u α s u r11 r12 r13 w v = β v r 21 r22 r r31 r32 r33 t t t x y z x y z 1 Slide inspirapon: S. Savarese

51 A generic projecpon matrix Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew P' = K [ R t ] P Extrinsic Assumptions Allow rotation Camera at (tx,ty,tz) u α s u r11 r12 r13 w v = β v r 21 r22 r r31 r32 r33 t t t x y z x y z 1 Slide inspirapon: S. Savarese

52 A generic projecpon matrix Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew P' = K [ R t ] P Extrinsic Assumptions Allow rotation Camera at (tx,ty,tz) u α s u r11 r12 r13 w v = β v r 21 r22 r r31 r32 r33 t t t x y z x y z 1 Degrees of freedom?? Slide inspirapon: S. Savarese

53 A generic projecpon matrix Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew P' = K [ R t ] P Extrinsic Assumptions Allow rotation Camera at (tx,ty,tz) u α s u r11 r12 r = w v β v r 21 r22 r r31 r32 r33 t t t x y z x y z 1 Degrees of freedom?? Slide inspirapon: S. Savarese

54 CS231a: Camera CalibraPon espmate all intrinsic and extrinsic parameters Intrinsic Assumptions Optical center at (u, v) Rectangular pixels Small skew Extrinsic Assumptions Allow rotation Camera at (tx,ty,tz) u α s u r11 r12 r = P' = K [ R t ] P w v β v r 21 r22 r r31 r32 r33 t t t x y z x y z 1 Slide inspirapon: S. Savarese

55 Orthographic ProjecPon Special case of perspecpve projecpon Distance from the COP to the image plane is infinite Also called parallel projecpon What s the projecpon matrix? Image World Slide credit: Steve Seitz = z y x v u w

56 Scaled Orthographic ProjecPon Special case of perspecpve projecpon Object dimensions are small compared to distance to camera Also called weak perspecpve What s the projecpon matrix? Image World = 1 1 z y x s f f v u w Slide credit: Steve Seitz

57 Field of View (Zoom)

58 Things to remember Vanishing points and vanishing lines Vanishing line Vanishing point Pinhole camera model and camera projecpon matrix M Intrinsic parameters Extrinsic parameters P ' = K!" R Vertical vanishing point (at infinity) Vanishing point t #$ P Homogeneous coordinates Slide inspirapon: J. Hayes

59 What we have learned today? Pinhole cameras Cameras & lenses The geometry of pinhole cameras ProjecPon matrix Intrinsic parameters Extrinsic parameters Reading: [FP] Chapters 1 3 [HZ] Chapter Oct- 15

Projective Geometry and Camera Models

Projective Geometry and Camera Models Computer Vision CS 43 Brown James Hays Slides from Derek Hoiem, Alexei Efros, Steve Seitz, and David Forsyth Administrative Stuff My Office hours, CIT 375 Monday and