Sereo and Moion The Sereo Problem Reconsrc scene geomer from wo or more calibraed images scene poin focal poin image plane Sereo The Sereo Problem Reconsrc scene geomer from wo or more calibraed images Basic Principle: Trianglaion Gives reconsrcion as inersecion of wo ras Reqires poin correspondence This is he hard par 1
Sereo Correspondence Deermine Piel Correspondence Pairs of poins ha correspond o same scene poin epipolar line epipolar plane epipolar line Epipolar Consrain Redces correspondence problem o 1D search along conjgae epipolar lines Sereo recificaion: make epipolar lines horizonal his is wha he prewarp did in view morphing Correspondence and Opical Flow Sereo reqires js 1D moion esimaion B in general he moion field is 2D Epipolar lines no known in advance Non-rigid moion (no epipolar lines) Tre moion field: projeced poin displacemens Opical flow is apparen moion in he image Generall hese will no be he same 2
3 The Aperre Problem We can measre he re 2D moion field from local image measremens Eample: Barber Pole llsion hp://www.sandloscience.com Opical Flow Eqaion Several of he following slides adaped from P. Anandan, 1999 = (,v) v d d = = = = 0 ),, ( ),, ( v δ δ δ = Assmpions Brighness Consanc: inensi of a moving poin is consan over ime Piel inensi is linear in (for small ime seps)
Normal Flow v v T = 0 Opical flow eqaion is a line consrain Normal componen can be comped Tangen componen T is ndefined negraing Neighborhood nformaion Lcas and Kanade Mehod v We wan o minimize : This corresponds o solving : i i i ( v (i.e., if poins are on a line - -js ge normal flow) 2 2 = v Mari on he lef is singlar if all gradiens poin in same direcion i ) 2 4
Limis of he Gradien Mehod Fails When No enogh variaion in local neighborhood Moion is large (mch greaer han a piel) Linear brighness assmpion is no me For larger displacemens, mach emplaes insead Define a small area arond a piel as he emplae Search locall for emplae in ne image Use a mach measre sch as correlaion, normalized correlaion, or sm-of-sqares difference (SSD) Choose he maimm (or minimm) as he mach Window size is imporan small windows lead o false maches big windows lead o over-smoohing SSD Srface Tered area 5
SSD Srface -- Edge SSD Srface homogeneos area 6
Coarse o Fine Esimaion Firs se large windows and search over large displacemen range Refine hese esimaes sing smaller windows Can do his more efficienl b sing: A PYRAMD! Seps: Convolve image wih a small kernel Tpicall 55 Gassian or Laplacian filer Sbsample o ge lower resolion image Repea for more levels Resl: A seqence of low-pass or band-pass filered images Pramids Pramids were inrodced as a mli-resolion image compaion paradigm in he earl 80s. The mos poplar pramid is he Br pramid, which foreshadows waveles Two kinds of pramids: Low pass or Gassian pramid Band-pass or Laplacian pramid 7
8
Coarse-o-Fine Flow Esimaion (Anandan) Consrc pramids from each image (Gassian) Sar a coarses level, iniialize flow o 0 1. Do local search (33 or 55 area) sing small (55) emplaes 2. Arond he peak perform sbpiel refinemen 1. Eiher analicall, sing he Lcas-Kanade formlaion or 2. Nmericall b fiing qadraic srface o he peak and inerpolaing o find he sb-piel peak 3. Warp one image oward he oher sing he flow field 4. Repea seps 1,2, and 3 a few imes (sall 5-10) 5. Projec he flow field o ne finer level 6. Move o he ne finer level and repea 1-5. Sop when o finish he ieraions a he fines level Sereo Maching Sereo Pair Qanized Deph Map normalized cross-correlaion search 9
Sereo Maching Algorihms Pifalls speclariies (non-lamberian srfaces) ambigi (aperre problem, low-conras regions) missing daa (occlsions) inensi error (qanizaion, sensor error) posiion error (camera calibraion) Nmeros approaches corse-o-fine [Anandan 89] edge-based [Marr-Poggio] dnamic programming [Baker-Binford 81] MRF s, graph cs [Zabih] adapive windows [Kanade 91] mli-baseline [Okomi 93] man more... Normalized cross-correlaion Graph cs [Zabih 99] 10
Acive Sereo (Laser Scanning) One wa o solve he aperre problem Creae or own ere b projecing ligh paerns ono he objec Mos precise wa is o se a laser Trianglae as before, b beween laser and sensor Figres b Brian Crless, 1999 Sanford s Digial Michelangelo Projec hp://graphics.sanford.ed/projecs/mich/ maimm heigh of ganr: 7.5 meers weigh inclding sbbase: 800 kilograms 11
Saisics abo he scan 480 individall aimed scans 2 billion polgons 7,000 color images 32 gigabes 30 nighs of scanning 1,080 man-hors 22 people 12