Image arping/morphing Image arping Digial Visual Effecs Yung-Yu Chuang ih slides b Richard Szeliski, Seve Seiz, Tom Funkhouser and leei Efros Image formaion Sampling and quanizaion B
Wha is an image We can hink of an image as a funcion, f: R 2 R: f(, gives he inensi i a posiion (, defined over a recangle, ih a finie range: f: [a,b][c,d] [ [0,1] f digial image We usuall operae on digial (discree images: Sample he 2D space on a regular grid Quanize each sample (round o neares ineger If our samples are D apar, e can rie his as: f[i,j] = Quanize{ f(i D, j D } The image can no be represened as a mari of ineger values color image r(, f (, g (, b(, Image arping image filering: change range of image g( ( = h(f( h(f( f g h h(=0.5+0.5 image arping: change domain of image g( = f(h( h(=2 f h g Image arping image filering: change range of image g( ( = h(f( h(f( f g h(=0.5+0.5 h image arping: change domain of image g( = f(h( h([,]=[,/2] f h g
Parameric (global arping Eamples of parameric arps: ranslaion roaion aspec affine perspecive clindrical Parameric (global arping T p = (, p = (, Transformaion T is a coordinae-changing machine: p = T(p Wha does i mean ha T is global? Is he same for an poin p can be described b jus a fe numbers (parameers Represen T as a mari: i p = M*p M Scaling Scaling a coordinae means mulipling each of is componens b a scalar Uniform scaling means his scalar is he same for all componens: f g 2 2 Scaling Non-uniform scaling: differen scalars per componen: f 2 0. 5 g 2 2, 0.5
Scaling Scaling operaion: Or, in mari form: Wha s inverse of S? a b a 0 0 b scaling mari S 2-D Roaion This is eas o capure in mari form: cos sin sin cos R Even hough sin( and cos( are nonlinear o, is a linear combinaion of and is a linear combinaion of and Wha is he inverse ransformaion? Roaion b For roaion marices, de(r = 1 so 11 T R R 22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 2D Ideni? 1 0 1 0 2D Scale around (0,0? s * s 0 s * 0 s 2D Roae around (0,0? cos * sin * sin * cos * 2D Shear? sh * sh * cos sin sin cos 1 sh sh 1
22 Marices Wha pes of ransformaions can be represened ih a 22 mari? 2D Mirror abou Y ais? 1 0 0 1 2D Mirror over (0,0? 1 0 0 1 ll 2D Linear Transformaions Linear ransformaions are combinaions of Scale, Roaion, Shear, and Mirror Properies of linear ransformaions: Origin maps o origin Lines map o lines Parallel lines remain parallel Raios are preserved Closed under composiion a c b d 22 Marices Translaion Wha pes of ransformaions can no be represened ih a 22 mari? Eample of ranslaion Homogeneous Coordinaes 2D Translaion? NO! Onl linear 2D ransformaions can be represened ih a 222 mari = 2 = 1 1 0 0 1 1 0 0 1 1 1
ffine Transformaions ffine ransformaions are combinaions of Linear ransformaions, and Translaions Properies of affine ransformaions: Origin does no necessaril map o origin Lines map o lines Parallel lines remain parallel Raios are preserved Closed under composiion a Models change of basis d 0 b e 0 c f 1 Projecive Transformaions Projecive ransformaions ffine ransformaions, and Projecive arps Properies of projecive ransformaions: i Origin does no necessaril map o origin Lines map o lines Parallel lines do no necessaril remain parallel Raios are no preserved Closed under composiion a b c Models change of basis d g e h f i Image arping Given a coordinae ransform = T( and a source image I(, ho do e compue a ransformed image I ( =I(T(? Forard arping Send each piel I( o is corresponding locaion = T( in I ( T( I( I ( T( I( I (
Forard arping farp(i, I, T { for (=0; <I.heigh; ++ for (=0; <I.idh; ++ { (, =T(,; I (, =I(,; I( ; } } I I T Forard arping Send each piel I( o is corresponding locaion = T( in I ( Wha if piel lands beeen o piels? Will be here holes? nser: add conribuion o several piels, normalize laer (splaing h( f( g( Forard arping Inverse arping farp(i, I, T { for (=0; <I.heigh; ++ for (=0; <I.idh; ++ { (, =T(,; Splaing(I,,,I(,,kernel; I( } } I I T Ge each piel I ( from is corresponding locaion = T -1 ( in I( T -1 ( I( I (
Inverse arping iarp(i, I, T { for (=0; <I.heigh; ++ for (=0; <I.idh; ++ { (,=T -1 (, ; I (, =I(,; I( ; } } I T -1 I Inverse arping Ge each piel I ( from is corresponding locaion = T -1 ( in I( Wha if piel comes from beeen o piels? nser: resample color value from inerpolaed (prefilered source image f( g( Inverse arping iarp(i, I, T { for (=0; <I.heigh; ++ for (=0; <I.idh; ++ { (,=T -1 (, ; I (, =Reconsruc(I,,,kernel; } } I T -1 I Sampling band limied
Reconsrucion Reconsrucion Reconsrucion generaes an approimaion o he original funcion. Error is called aliasing. sampling sample value reconsrucion The reconsruced funcion is obained b inerpolaing among he samples in some manner sample posiion i Reconsrucion Compued eighed sum of piel neighborhood; oupu is eighed average of inpu, here eighs are normalized values of filer kernel k k ( q i i qi p k( q i i color=0; eighs=0; p idh for all q s dis < idh d = dis(p, q; = kernel(d; d color += *q.color; q eighs += ; p.color = color/eighs; Reconsrucion (inerpolaion Possible reconsrucion filers (kernels: neares neighbor bilinear bicubic sinc (opimal reconsrucion
Bilinear inerpolaion (riangle filer simple mehod for resampling images Non-parameric image arping Specif a more deailed arp funcion Splines, meshes, opical flo (per-piel moion Non-parameric image arping Non-parameric image arping Mappings implied b correspondences Inverse arping P B C B C P B B C C Barcenric coordinae? P P P
Barcenric coordinaes 1 3 2 1 3 3 2 2 1 1 P 1 3 2 1 Non-parameric image arping C B P C B C B P Barcenric coordinae C B P C B Non-parameric image arping 2 ( r e r Gaussian i X X P k K P i ( 1 radial basis funcion log( ( 2 r r r hin plae spline i K Demo hp://.colonize.com/arp/arp04-2.php W i i f l i f i id Warping is a useful operaion for mosaics, video maching, vie inerpolaion and so on.
Image morphing The goal is o snhesize a fluid ransformaion from one image o anoher. Image morphing Cross dissolving is a common ransiion beeen cus, bu i is no good for morphing because of he ghosing effecs. image #1 dissolving i image #2 rifacs of cross-dissolving Image morphing Wh ghosing? Morphing = arping + cross-dissolving i shape (geomeric color (phoomeric hp://.salavon.com/
Image morphing Morphing sequence image #1 cross-dissolving image #2 arp morphing arp Face averaging b morphing Image morphing creae a morphing sequence: for each ime 1. Creae an inermediae arping field (b inerpolaion 2. Warp boh images oards i 3. Cross-dissolve he colors in he nel arped images average faces =0 =0.33 =1
n ideal eample (in 2004 n ideal eample =0 morphing =0.25 =0.75 =0.5 =1 =0 middle face (=0.5 =1 Warp specificaion (mesh arping Ho can e specif he arp? 1. Specif corresponding spline conrol poins inerpolae o a complee arping funcion Warp specificaion Ho can e specif he arp 2. Specif corresponding poins inerpolae o a complee arping funcion eas o implemen, bu less epressive
Soluion: conver o mesh arping Warp specificaion (field arping Ho can e specif he arp? 3. Specif corresponding vecors inerpolae o a complee arping funcion The Beier & Neel lgorihm 1. Define a riangular mesh over he poins Same mesh in boh images! No e have riangle-o-riangle i l correspondences 2. Warp each riangle separael from source o desinaion Ho do e arp a riangle? 3 poins = affine arp! Jus like eure mapping Beier&Neel (SIGGRPH 1992 Single line-pair PQ o P Q : lgorihm (single line-pair For each X in he desinaion image: 1. Find he corresponding u,v 2. Find X in he source image for ha u,v 3. desinaionimage(x = sourceimage(x Eamples: ffine ransformaion
Muliple Lines Full lgorihm D i X i X i lengh = lengh of he line segmen, dis = disance o line segmen The influence of a, p, b. The same as he average of X i Resuling arp Comparison o mesh morphing Pros: more epressive Cons: speed and conrol
Warp inerpolaion nimaion Ho do e creae an inermediae arp a ime? linear inerpolaion for line end-poins Bu, a line roaing i 180 degrees ill become 0 lengh in he middle One soluion is o inerpolae line mid-poin and orienaion angle =0 =1 nimaed sequences Resuls Specif keframes and inerpolae he lines for he inbeeen frames Require a lo of eaking Michael Jackson s MTV Black or Whie
Muli-source morphing Muli-source morphing References Thaddeus Beier, Shan Neel, Feaure-Based Image Meamorphosis, SIGGRPH 1992, pp35-42. Delef Ruprech, Heinrich Muller, Image Warping ih Scaered Daa Inerpolaion, IEEE Compuer Graphics and pplicaions, March 1995, pp37-43 43. Seung-Yong Lee, Kung-Yong Cha, Sung Yong Shin, Image Meamorphosis Using Snakes and Free-Form Deformaions, SIGGRPH 1995. Seungong Lee, Wolberg, G., Sung Yong Shin, Polmorph: morphing among muliple images, IEEE Compuer Graphics and pplicaions, Vol. 18, No. 1, 1998, pp58-71. Peinsheng Gao, Thomas Sederberg, ork minimizaion approach o image morphing, The Visual Compuer, 1998, pp390-400. George Wolberg, Image morphing: a surve, The Visual Compuer, 1998, pp360-372.