A Selected Primer on Computer Vision: Geometric and Photometric Stereo & Structured Light

Transcription

1 A Seected Primer o Computer Visio: Geometric ad Photometric Stereo & Structured Light CS334 Sprig 2012 Daie G. Aiaga Departmet of Computer Sciece Purdue Uiversit

2 Defiitios Camera geometr (=motio) Give correspoded poits o 2 views, what are the poses of the cameras? Correspodece geometr (=correspodece) Give a poit i oe view, what are the costraits of its positio i aother view? Scee geometr (=structure) Give correspoded poits o 2 views ad the camera poses, what is the 3D ocatio of the poits?

3 Geometric Stereo: Eampe Resut Usig dese feature-based stereo [Poefes99]

4 Geometric Stereo X, Y, Z scee eft image Assume that we kow f P L Y b P L, Z L L X baseie correspods to right image, PR P R R R Usig perspective projectio (defied usig coordiate sstem show) X b L 2 f Z X b R 2 f Z f L f R Y Z

5 Geometric Stereo X, Y, Z scee X eft image X b L 2 f Z X b R 2 f Z bl R bl R Y 2 bf Z 2 L R P L b, L L baseie L R right image, P R f L R R f R L R Y Z

6 Geometric Stereo X, Y, Z scee X eft image X b L 2 f Z X b R 2 f Z bl R bl R Y 2 bf Z 2 L R P L b, L L baseie L R right image, P R f L R R f R L R Y Z d L R is the disparit betwee correspodig eft ad right image poits iverse proportioa to depth disparit icreases with baseie b

7 Correspodece scee poit optica ceter image pae optica ceter

8 Correspodece scee poit epipoar ie optica ceter image pae optica ceter

9 Correspodece scee poit epipoar ie epipoar ie optica ceter image pae optica ceter

10 Epipoar Geometr scee poit epipoar ie epipoar pae epipoar ie optica ceter image pae optica ceter Epipoar Costrait: reduces correspodece probem to 1D search aog cojugate epipoar ies

11 Epipoar Geometr scee poit epipoar ie epipoar pae epipoar ie optica ceter image pae optica ceter Epipoar Costrait: ca be epressed usig the fudameta matri F

12 Epipoar Geometr covergig cameras

13 Epipoar Geometr motio parae with image pae

14 Epipoar Geometr Forward motio e e

15 Epipoar Geometr Correspodece reduced to ookig i a sma eighborhood of a ie

16 Correspodece: Epipoar Geometr scee poit epipoar ie epipoar pae epipoar ie optica ceter image pae optica ceter Epipoar costrait ca be epressed as T F 0 Fudameta matri

17 Photometric Stereo A techique for estimatig the surface ormas of objects b observig that object uder differet ightig coditios The, usig the surface ormas, a pausibe surface geometr ca be recostructed Woodham i 1980 Reated: whe usig a sige image, it is caed shape from shadig B. K. P. Hor i 1989

18 Photometric Stereo What are the vaues for i? poit i [ 1, P] i surface

19 Photometric Stereo Lambertia (sige chae) mode: ( ) c surface poit

20 Photometric Stereo (kow ights) Lambertia (sige chae) mode: ( ) c How ma ukows per poit? 3+1 How to get more equatios? More ights! poit surface

21 Photometric Stereo ( c ) 1 1 surface poit

22 Photometric Stereo surface poit ) ( c ) ( c

23 Photometric Stereo surface poit ) ( c ) ( c ) ( c

24 Photometric Stereo ) ( c ) ( c ) ( c Usig 1 Usig 2 Usig 3

25 Photometric Stereo ) ( c ) ( c ) ( c Usig 1 Usig 2 Usig 3 1 c 2 c 3 c

26 Photometric Stereo ) ( c ) ( c ) ( c What is?,, L c Write as ad sove c L 1 Where T T c c c c What is the surface orma at the poit? / What is (the abedo at the surface poit)?

27 Lambertia Photometric Stereo with Kow Lights Take three pictures of a static Lambertia object with a static camera I each picture move the ight to a differet but kow positio For distat ights, coud kow ight directio istead of ight positio At pie i, sove 1 i L c i Use ormas to itegrate a surface

28 How? Normas -> Surface

29 Normas -> Surface [Basri et a., IJCV, 2007]

30 Fudameta Ambiguit Behumeur et a. 1999

31 Fudameta Ambiguit Behumeur et a. 1999

32 Fudameta Ambiguit NL C NRR 1 L C NAA 1 ( NA)( A L 1 L) C C A RG Rotatio matri Geeraied Bas Reief (GBR) Ambiguit matri

33 Photogeometric Approach Combie photometric stereo with geometric stereo High resoutio of photometric stereo Accurac of geometric method Ca ead to sef-caibratio of etire acquisitio process

34 Photogeometric Upsampig 1. Itegrate surface ormas photo-surface

35 Photogeometric Upsampig 2. Compute sparse geometric mode geo-surface

36 Photogeometric Upsampig 3. Warp photometric surface to geometric surface geo-surface photo-surface

37 Photogeometric Upsampig 3. Warp photometric surface to geometric surface photo-geo surface

38 Photogeometric Upsampig 4. Triaguate ad proceed to optimiatio photo-geo surface true surface

39 Photogeometric Optimiatio Liear sstem i the ukow 3D poits (p i ) Supports muti-view recostructio Weighted combiatio of three error terms: e = 1 λ 1 τ κ g e g + λκ p e p + τκ r e r 0 where e g = error of reprojectio e p = error of perpedicuarit of orma-to-taget e r = error of reative distace chage

40 Photogeometric Optimiatio Liear sstem i the ukow 3D poits (p i ) Supports muti-view recostructio Weighted combiatio of three error terms: e = 1 λ 1 τ κ g e g + λκ p e p + τκ r e r 0 where e g = j i p ij u ij p ij f p ij v ij p ij f e p = δ ik ( i p i p k ) i e r = δ ik ( p i p ik d ik ) i

41 Photogeometric Recostructio photographs recostructio

42 Passive vs. Active Acquisitio Passive + Just take pictures + Does ot itrude i the eviromet (=passive) Some surfaces caot be acquired Robustess is probematic Active + Emit ight ito the scee so as to force the geeratio of robust correspodece Eviromet is itruded (=active)

43 Active Acquisitio Some optios: Laser scaig Structured Light

44 Laser Scaig

45 Light Stripe Scaig (Sige Stripe) Light pae Source Camera Surface Optica triaguatio Project a sige stripe of aser ight Sca it across the surface of the object This is a ver precise versio of structured ight scaig Good for high resoutio 3D, but eeds ma images ad takes time

46 Stripe Triaguatio Object Light Pae A B C D 0 Laser Camera Project aser stripe oto object

47 Stripe Triaguatio Object Light Pae A B C D 0 Laser Image Poit ( ', ') Camera Depth from ra-pae triaguatio: Itersect camera ra with ight pae ' / ' / f f Df A' B' Cf

48 Eampe: Laser scaer + ver accurate < 0.01 mm more tha 10sec per sca Cberware face ad head scaer

49 Eampe: Laser scaer Digita Micheageo Project

50 Eampe: Laser scaer Portabe scaer b Miota

51 Digita Projector Structured Light Goa: geerate correspodeces so as to eabe a robust 3D recostructio

52 Digita Projector Structured Light Method: Use the projector as a patter geerator Have the camera see the patter ad geerate 1 or more correspoded poits

53 Digita Projector Structured Light What are possibe patters? Spatia patters Tempora patters Coor patters Ad combiatios of the above

54 Digita Projector Structured Light Lets focus o biar striped patters

55 Biar Patter Structured Light 59

56 Biar Codig Assig each pie a uique iumiatio code over time [Posdamer 82] Time X coordiate

57 Biar Codig Assig each pie a uique iumiatio code over time [Posdamer 82] Time Y coordiate

58 Biar Codig 2 1 stripes i images Projected over time Patter 3 Eampe: 3 biar-ecoded patters which aows the measurig surface to be divided i 8 sub-regios Patter 2 Patter 1

59 Biar vs Gra Codes Decima Biar Gra Code

60 Biar vs Gra Codes Patter 3 Patter 2 Patter 1 Biar code Gra code

61 Pie Cassificatio Chaeges? Iumiated (ON) No-iumiated (OFF)?

62 Stadard Pie Cassificatio Iterva P off P o Commo methods Simpe threshod Abedo threshod Dua patter 66