Image warping Li Zhang CS559

Transcription

1 Wha is an image Image arping Li Zhang S559 We can hink of an image as a funcion, f: R 2 R: f(, ) gives he inensi a posiion (, ) defined over a recangle, ih a finie range: f: [a,b][c,d] [,] f Slides solen from rof Yungu huang hp://.csie.nu.edu./~c/courses/vf/7spring/overvie/ color image r(, ) f (, ) g (, ) b(, ) digial image We usuall operae on digial (discree) images: Sample he 2D space on a regular grid Quanize each sample (round o neares ineger) If our samples are D apar, e can rie his as: f[i,j] Quanize{ f(i D, j D) The image can no be represened as a mari of ineger values Image arping hange piels locaions o creae a ne image: f() g(h()) h([,])[,/2] f h g arameric (global) arping Nonparameric (local) arping Eamples of parameric arps: ranslaion roaion aspec affine perspecive clindrical Original Warped

2 arameric (global) arping T p (,) p (, ) Transformaion T is a coordinae-changing machine: p T(p) Wha does i mean ha T is global? can be described b jus a fe numbers (parameers) he parameers are he same for an poin p Represen T as a mari: p Mp M Scaling Scaling a coordinae means mulipling each of is componens b a scalar Uniform scaling means his scalar is he same for all componens: f g Scaling Non-uniform scaling: differen scalars per componen: f g 2.5 Scaling Scaling operaion: a b Or, in mari form: a b 2,.5 Wha s inverse of S? scaling mari S 2-D Roaion (, ) (, ) cos() - sin() sin() cos() 2-D Roaion This is eas o capure in mari form: cos sin ( ) sin( ) ( ) cos( ) R Even hough sin() and cos() are nonlinear o, is a linear combinaion of and is a linear combinaion of and Wha is he inverse ransformaion? Roaion b For roaion marices, de(r) so T R R 2

3 3 22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 2D Ideni? 2D Scale around (,)? s s s s 22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 2D Roae around (,)? cos sin sin cos cos sin sin cos 2D Shear? sh sh sh sh 22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 2D Mirror abou Y ais? 2D Mirror over (,)? ll 2D Linear Transformaions Linear ransformaions are combinaions of Scale, Roaion, Shear, and Mirror roperies of linear ransformaions: Origin maps o origin Lines map o lines arallel lines remain parallel Raios are preserved losed under composiion d c b a 22 Marices Wha pes of ransformaions can no be represened ih a 22 mari? 2D Translaion? Onl linear 2D ransformaions can be represened ih a 22 mari NO! Translaion Eample of ranslaion 2 Homogeneous oordinaes

4 ffine Transformaions ffine ransformaions are combinaions of Linear ransformaions, and Translaions roperies of affine ransformaions: Origin does no necessaril map o origin Lines map o lines arallel lines remain parallel a b c Raios are preserved d e f losed under composiion Models change of basis rojecive Transformaions rojecive ransformaions ffine ransformaions, and rojecive arps roperies of projecive ransformaions: Origin does no necessaril map o origin Lines map o lines arallel lines do no necessaril remain parallel Raios are no preserved a b losed under composiion Models change of basis d e g h Ver Useful In Teure Mapping! / / c f i Image arping Given a coordinae ransform T() and a source image I(), ho do e compue a ransformed image I ( ) I(T())? Forard arping Send each piel I() o is corresponding locaion T() in I ( ) T() T() I() I ( ) I() I ( ) Forard arping farp(i, I, T) { for (; <I.heigh; ) for (; <I.idh; ) { (, )T(,); I (, )I(,); I I T Forard arping Send each piel I() o is corresponding locaion T() in I ( ) Wha if piel lands beeen o piels? Will be here holes? nser: add conribuion o several piels, normalize laer (splaing) h() f() g( ) 4

5 Forard arping farp(i, I, T) { for (; <I.heigh; ) for (; <I.idh; ) { (, )T(,); Splaing(I,,,I(,),kernel); I I T Inverse arping Ge each piel I ( ) from is corresponding locaion T - ( ) in I() T - ( ) I() I ( ) Inverse arping iarp(i, I, T) { for (; <I.heigh; ) for (; <I.idh; ) { (,)T - (, ); I (, )I(,); I T - I Inverse arping Ge each piel I ( ) from is corresponding locaion T - ( ) in I() Wha if piel comes from beeen o piels? nser: resample color value from inerpolaed (prefilered) source image f() g( ) Inverse arping iarp(i, I, T) { for (; <I.heigh; ) for (; <I.idh; ) { (,)T - (, ); I (, )Reconsruc(I,,,kernel); I T - I Sampling band limied 5

6 Reconsrucion Reconsrucion Reconsrucion generaes an approimaion o he original funcion. Error is called aliasing. sampling sample value reconsrucion The reconsruced funcion is obained b inerpolaing among he samples in some manner sample posiion Reconsrucion ompued eighed sum of piel neighborhood; oupu is eighed average of inpu, here eighs are normalized values of filer kernel k k( q i i ) f ( qi ) f ( p) k( q i i ) color; eighs; p idh for all q s dis < idh d dis(p, q); d kernel(d); color q.color; q eighs ; p.olor color/eighs; Reconsrucion (inerpolaion) ossible reconsrucion filers (kernels): neares neighbor bilinear bicubic sinc (opimal reconsrucion) ilinear inerpolaion (riangle filer) simple mehod for resampling images Non-parameric image arping 6

7 7 Non-parameric image arping Specif a more deailed arp funcion Splines, meshes, opical flo (per-piel moion) Non-parameric image arping Mappings implied b correspondences Inverse arping? Non-parameric image arping arcenric coordinae arcenric coordinaes Non-parameric image arping arcenric coordinae Non-parameric image arping radial basis funcion 2 ) ( r e r β ρ ) log( ) ( 2 r r r ρ Gaussian hin plae spline Δ Δ i X X i k K i ) (

8 Demo hp://.colonize.com/arp/arp4-2.php Warping is a useful operaion for mosaics, video maching, vie inerpolaion and so on. Image morphing Image morphing The goal is o snhesize a fluid ransformaion from one image o anoher. ross dissolving is a common ransiion beeen cus, bu i is no good for morphing because of he ghosing effecs. Image morphing Wh ghosing? Morphing arping cross-dissolving shape (geomeric) color (phoomeric) image # dissolving image #2 Image morphing Morphing sequence image # cross-dissolving image #2 arp morphing arp 8

9 Muli-source morphing Muli-source morphing Face averaging b morphing The average face hp://.uniregensburg.de/fakulaeen/phil_fak_ii/schol ogie/s_ii/beaucheck/english/inde.hm average faces Image morphing creae a morphing sequence: for each ime. reae an inermediae arping field (b inerpolaion) 2. Warp boh images oards i 3. ross-dissolve he colors in he nel arped images Warp specificaion (mesh arping) Ho can e specif he arp?. Specif corresponding spline conrol poins inerpolae o a complee arping funcion.33 eas o implemen, bu ma no be epressive enough 9

10 Warp specificaion Ho can e specif he arp 2. Specif corresponding poins inerpolae o a complee arping funcion Soluion: conver o mesh arping. Define a riangular mesh over he poins Same mesh in boh images! No e have riangle-o-riangle correspondences 2. Warp each riangle separael from source o desinaion Ho do e arp a riangle? 3 poins affine arp! Jus like eure mapping Warp specificaion (field arping) Ho can e specif he arp? 3. Specif corresponding vecors inerpolae o a complee arping funcion The eier & Neel lgorihm eier&neel (SIGGRH 992) Single line-pair Q o Q : lgorihm (single line-pair) For each X in he desinaion image:. Find he corresponding u,v 2. Find X in he source image for ha u,v 3. desinaionimage(x) sourceimage(x ) Eamples: Muliple Lines D X X i i i ffine ransformaion lengh lengh of he line segmen, dis disance o line segmen The influence of a, p, b. The same as he average of X i

11 Full lgorihm Resuling arp omparison o mesh morphing ros: more epressive ons: speed and conrol Warp inerpolaion Ho do e creae an inermediae arp a ime? linear inerpolaion for line end-poins u, a line roaing 8 degrees ill become lengh in he middle One soluion is o inerpolae line mid-poin and orienaion angle nimaion nimaed sequences Specif keframes and inerpolae he lines for he inbeeen frames Require a lo of eaking

12 Resuls roblem ih morphing So far, e have performed linear inerpolaion of feaure poin posiions u ha happens if e r o morph beeen o vies of he same objec? Michael Jackson s MTV lack or Whie hp://.michaeljackson.com/quickime_blackorhie.hml Vie morphing Seiz & Der hp://.cs.ashingon.edu/homes/seiz/vmorph/vmorph.hm Inerpolaion consisen ih 3D vie inerpolaion Main rick rearp ih a homograph o "prealign" images So ha he o vies are parallel ecause linear inerpolaion orks hen vies are parallel morph morph prearp prearp homographies inpu oupu inpu 2

13 3