Image interpolation. A reinterpretation of low-pass filtering. Image Interpolation

Transcription

1 Imge interpoltion A reinterprettion of low-pss filtering Imge Interpoltion Introduction Wht is imge interpoltion? (D-A conversion) Wh do we need it? Interpoltion Techniques 1D zero-order, first-order, third-order 2D = two sequentil 1D (divide-nd-conquer) Directionl(Adptive) interpoltion* Interpoltion Applictions Digitl zooming (resolution enhncement) Imge inpinting (error concelment) Geometric trnsformtions (where our imgintion cn fl)

2 Introduction Wht is imge interpoltion? An imge f(,) tells us the intensit vlues t the integrl lttice loctions, i.e., when nd re both integers Imge interpoltion refers to the guess of intensit vlues t missing loctions, i.e., nd cn be rbitrr Note tht it is just guess (Note tht ll sensors hve finite smpling distnce) A Sentimentl Comment Hven t we just lerned from discrete smpling (A-D conversion)? Yes, imge interpoltion is bout D-A conversion Recll the gp between biologicl vision nd rtificil vision sstems Digitl: cmer + computer Anlog: retin + brin

3 Engineering Motivtions Wh do we need imge interpoltion? We wnt BIG imges When we see video clip on PC, we like to see it in the full screen mode We wnt GOOD imges If some block of n imge gets dmged during the trnsmission, we wnt to repir it We wnt COOL imges Mnipulte imges digitll cn render fnc rtistic effects s we often see in movies Scenrio I: Resolution Enhncement Low-Res. High-Res.

4 Scenrio II: Imge Inpinting Non-dmged Dmged Scenrio III: Imge Wrping

5 Imge Interpoltion Introduction Wht is imge interpoltion? Wh do we need it? Interpoltion Techniques 1D liner interpoltion (elementr lgebr) 2D = 2 sequentil 1D (divide-nd-conquer) Directionl (dptive) interpoltion Interpoltion Applictions Digitl zooming (resolution enhncement) Imge inpinting (error concelment) Geometric trnsformtions Upsmpling* This*imge*is*too*smll*for*this*screen:* How*cn*we*mke*it*10*:mes*s*big?* Simplest*pproch:* **repet*ech*row* **nd*column*10*:mes* ( Nerest*neighbor* ***interpol:on )* *

6 Imge*interpol:on* d = 1 in this emple 1* 2* 3* 4* 5* Recll*how**digitl*imge*is*formed* * It*is**discrete*pointEsmpling*of**con:nuous*func:on* If*we*could*somehow*reconstruct*the*originl*func:on,*n*new* imge*could*be*generted,*t*n*resolu:on*nd*scle** Adpted*from:*S.*Seitz* Imge*interpol:on* d = 1 in this emple 1* 2* 3* 4* 5* Recll*how**digitl*imge*is*formed* * It*is**discrete*pointEsmpling*of**con:nuous*func:on* If*we*could*somehow*reconstruct*the*originl*func:on,*n*new* imge*could*be*generted,*t*n*resolu:on*nd*scle** Adpted*from:*S.*Seitz*

7 Imge*interpol:on* 1* d = 1 in this emple 1* 2* 3* 4* 5* 2.5* Wht*if*we*don t*know****?* Guess*n*pproim:on:* Cn*be*done*in**principled*w:*filtering* Convert******to**con:nuous*func:on:** * Reconstruct*b*convolu:on*with**reconstruc)on*filter,*h Adpted*from:*S.*Seitz* Imge interpoltion Idel *reconstruc:on* NerestEneighbor* interpol:on* Liner*interpol:on* Gussin*reconstruc:on* Source:*B.*Curless*

8 Idel reconstruction Idel reconstruction

9 Idel reconstruction The idel reconstruction filter is squre window in the F-domin nd sinc function in the signl domin Its implementtion is unprcticl since it is prone to trunction rtifcts nd hs reltivel high computtionl compleit. Other low-pss filters cn be used insted Imge interpoltion Originl*imge:***********10* NerestEneighbor*interpol:on* Biliner*interpol:on* Bicubic*interpol:on*

10 1D Zero-order (Repliction) f(n) n f() 1D First-order Interpoltion (Liner) f(n) n f()

11 Liner Interpoltion Formul Heuristic: the closer to piel, the higher weight is ssigned Principle: line fitting to polnomil fitting (nlticl formul) f(n) f(n+) f(n+1) 1- f(n+)=(1-) f(n)+ f(n+1), 0<<1 Note: when =0.5, we simpl hve the verge of two Numericl Emples f(n)=[0,120,180,120,0] Interpolte t 1/2-piel f()=[0,60,120,150,180,150,120,60,0], =n/2 Interpolte t 1/3-piel f()=[0,20,40,60,80,100,120,130,140,150,160,170,180, ], =n/6

12 1D Third-order Interpoltion (Cubic)* f(n) n f() Cubic spline fitting From 1D to 2D Engineers wisdom: divide nd conquer 2D interpoltion cn be decomposed into two sequentil 1D interpoltions. The ordering does not mtter (row-column = column-row) Such seprble implementtion is not optiml but enjos low computtionl compleit If ou don t know how to solve problem, there must be relted but esier problem ou know how to solve. See if ou cn reduce the problem to the esier one. - rephrsed from G. Pol s How to Solve It

13 Grphicl Interprettion of Interpoltion t Hlf-pel row column f(m,n) g(m,n) Numericl Emples b zero-order c d b b b b c c d d c c d d first-order (+b)/2 b (+c)/2 (+b+c+d)/4 (b+d)/2 c (c+d)/2 d

14 Numericl Emples (Con t) Col n Col n+1 row m X(m,n) b 1- Y X(m,n+1) 1-b row m+1 X(m+1,n) X(m+1,n+1) Q: wht is the interpolted vlue t Y? Ans.: (1-)(1-b)X(m,n)+(1-)bX(m+1,n) +(1-b)X(m,n+1)+bX(m+1,n+1) Limittion with biliner/bicubic Edge blurring Jgged rtifcts Edge blurring Jgged rtifcts Z X X Z

15 Edge-Sensitive Interpoltion Step 1: interpolte the missing piels long the digonl b Since -c = b-d blck or white? c d hs equl probbilit of being blck or white Step 2: interpolte the other hlf missing piels d b Since -c > b-d =(b+d)/2=blck c Imge Interpoltion Introduction Interpoltion Techniques 1D zero-order, first-order, third-order 2D zero-order, first-order, third-order Directionl interpoltion* Interpoltion Applictions Digitl zooming (resolution enhncement) Imge inpinting (error concelment) Geometric trnsformtions (where our imgintion cn fl)

16 Piel Repliction low-resolution imge ( ) high-resolution imge ( ) Biliner Interpoltion low-resolution imge ( ) high-resolution imge ( )

17 Bicubic Interpoltion low-resolution imge ( ) high-resolution imge ( ) Edge-Directed Interpoltion (LiOrchrd 2000) low-resolution imge ( ) high-resolution imge ( )

18 Imge Demosicing (Color-Filter-Arr Interpoltion) Ber Pttern Imge Emple Ad-hoc CFA Interpoltion Advnced CFA Interpoltion

19 Error Concelment* dmged interpolted Imge Inpinting

20 Imge mosicing Geometric Trnsformtion MATLAB functions: griddt, interp2, mketform, imtrnsform

21 Bsic Principle (,) (, ) is geometric trnsformtion We re given piel vlues t (,) nd wnt to interpolte the unknown vlues t (, ) Usull (, ) re not integers nd therefore we cn use liner interpoltion to guess their vlues MATLAB implementtion: interp2 Rottion θ ' cosθ = ' sinθ sinθ cosθ

22 MATLAB Emple z=imred('cmermn.tif'); originl coordintes [,]=meshgrid(1:256,1:256); new coordintes =2; for i=1:256;for j=1:256; 1(i,j)=*(i,j); 1(i,j=(i,j)/; end;end Do the interpoltion z1=interp2(,,z,1,1,'cubic'); Rottion Emple θ=3 o

23 Scle = 1/ 0 0 ' ' =1/2 Affine Trnsform + = d d ' ' squre prllelogrm

24 Affine Trnsform Emple + = ' ' Sher + = d d s ' ' squre prllelogrm

25 Sher Emple + = ' ' Projective Trnsform 1 ' = 1 ' = qudrilterl squre A B C D A B C D

26 Projective Trnsform Emple [ 0 0; 1 0; 1 1; 0 1] [-4 2; -8-3; -3-5; 6 3] Polr Trnsform r = θ = tn 1 = r cosϑ = rsinϑ 2 r = + 2

27 Iris Imge Unwrpping θ r Use Your Imgintion r -> sqrt(r)

28 Free Form Deformtion Seung-Yong Lee et l., Imge Metmorphosis Using Snkes nd Free-Form Deformtions, SIGGRAPH 1985, Pges Appliction into Imge Metmorphosis

29 Summr of Imge Interpoltion A fundmentl tool in digitl processing of imges: bridging the continuous world nd the discrete world Wide pplictions from consumer electronics to biomedicl imging Remins hot topic fter the IT bubbles brek