Image warping introduction 1997-2015 Josef Pelikán CGG MFF UK Praha pepca@cgg.mff.cuni.cz http://cgg.mff.cuni.cz/~pepca/ Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 1 / 22
Warping.. image deformation texture mapping in 3D rendering (after rasterization) perspective distortion, mapping textures to arbitrary shapes correction of geometric distortion (digital image acquirement) satellite and aerial photography scanning of deformed documents special effects in TV, film and advertisement Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 2 / 22
Mathematical model image function f: U R 2 R n y U f: x, y a 1, a 2,... a n x position on the plane image attributes (color, transparency) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 3 / 22
Spatial discretization Digitized raster image: I: 0.. m 1 0.. n 1 R n Digitization using filter d: I i, j f x, y d x i, y j dx dy f 2 R d.. device characteristics (optical system, CCD element) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 4 / 22
Digital image reproduction Reconstruction of a discrete image: r m1 n1 i0 j0 f x, y I i, j r i x, j y f r.. output device characteristics (impulse response).. we need f r to be similar to f (in frequency range defined by the Nyquist law) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 5 / 22
Geometric transform y f U g y f x 2 2 g: R R, f g x, y f x y x 1,, f g u v f u v But what about raster images? G I f I f Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 6 / 22
Transform with interpolation digitization = sampling digitizing filter = Dirac delta attributes (color) of transformed pixel computed by approximation or interpolation inverse transform function g -1 is needed rounding, polynomial interpolation bilinear to bicubic interpolation/approximation is sufficient Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 7 / 22
Interpolation in source coord. system i+1 j i i+1 g -1 (u,v) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 8 / 22
Transform with filtering pixel-area model digitizing filter has areal support (e.g. box or conical filter) source pixels are projected to target coordinate system only g is needed suitable also for contractive transforms isometric transform image is blurred big contraction high computing time (speedup needed) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 9 / 22
Transform with filtering v+1 v v-1 u-1 u u+1 g(i,j) Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 10 / 22
MIP map (multum in parvo) pyramidal representation of the source image (hierarchy) preprocessing = pre-filtering useful mostly for transforms with big contraction substantial speedup used mostly for texture mapping on the GPU (distant objects) compact storage only 4/3 size of original RGB image Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 11 / 22
MIP map G 0 R 0 B 0 G 1 R 1 B 1 G 2 R 2 B 2 Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 12 / 22
Multi-pass algorithms transform is factored into several consecutive steps each step works either on rows or on columns of an image faster computation simpler filtering two 1D filters are faster than a 2D one bottleneck problem partial mapping is highly contractive or even non-injective Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 13 / 22
Two-pass rotation [x,y] [u,v] = f(x,y) = [f 1 (x,y),y] [r,s] = g(u,v) = [u,g 1 (u,v)] = [f 1 (x,y),g 1 (f 1 (x,y),v)] f [u,v] g [r,s] Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 14 / 22
Derivation Target transform (rotation by ): r x cos y sin s x sin y cos First transform f: u f x y v y 1, Second transform g: r u f x, y s g u, v g f x, y, y 1 1 1 1 Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 15 / 22
Derivation Horizontal shear for the first pass: f x y x y 1, cos sin Vertical shear for the second pass: g u v x y x v 1, sin cos sin cos u f 1 x, v x cos v sin x u v sin cos g u v u v 1, tan sec Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 16 / 22
Three-pass rotation Rotation matrix factoring: cos sin sin cos 1 tan 2 1 0 0 1 sin 1 1 tan 2 0 1 sufficient for angles: 2 2 Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 17 / 22
Arbitrary angle multi-pass methods cannot be used for wide range of angles. Suitable angle ranges: -45 o 45 o for two-pass rotation -90 o 90 o for three-pass rotation auxiliary rotation (multiple of 90 o ) simple, fast no image degradation Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 18 / 22
General separable transform Separable transform: h x, y h x, y, h x, y g f x, y f x, y f x, y, y, u, g u, v g u v 1 1 2 1 f x, y h x, y 1 1 g u, v h u, v, v 1 2 if (u,v) such that x u, v Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 19 / 22
Image degradation Inapplicable areas: low values of the derivative h 1 x, y x or high values of the derivative g 1 u, v v Sometimes even the inverse function could not be defined (u,v)! concurrent processing of images I and I T Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 20 / 22
Optimized algorithm concurrent processing of original I and transposed I T image both branches use the same method (two-pass algorithm using separable transforms h and h T ) result pixels are composed from h(i) and (h T (I T )) T comparing degradation pixel by pixel even for non-injective deformations! image folding.. Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 21 / 22
The End More info: J. Foley, A. van Dam, S. Feiner, J. Hughes: Computer Graphics, Principles and Practice, 815-832 J. Gomes et al.: Warping and Morphing of Graphical Objects, Course Notes - SIGGRAPH 95 Warping 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 22 / 22