Image Reconstruction. Prof. George Wolberg Dept. of Computer Science City College of New York

Transcription

1 Image Reconstruction Pro. George Wolberg Dept. o Computer Science Cit College o New Yor

2 Objectives In this lecture we describe image reconstruction: - Interpolation as convolution - Interpolation ernels or: Nearest neighbor Triangle ilter Cubic convolution B-Spline interpolation Windowed sinc unctions

3 Introduction Reconstruction is snonmous with interpolation. Determine value at position ling between samples. Strateg: it a continuous unction through the discrete input samples and evaluate at an desired set o points. Sampling generates ininite bandwidth signal. Interpolation reconstructs signal b smoothing samples with an interpolation unction (ernel. 3

4 Interpolation For equi-spaced data, interpolation can be epressed as a convolution: ( K c h( r where K is the number o neighborhood piels and c are coeicients or ernel h Kernel Samples: h(-d, h(--d, h(-d, h(-d I h is smmetric, h(d, h(+d, h(-d, h(-d

5 Interpolation Kernel Set o weights applied to neighborhood piels Oten deined analticall Usuall smmetric: h( = h(- Commonl used ernels: - Nearest neighbor (piel replication - Triangle ilter (linear interpolation - Cubic convolution (smooth; used in digital cameras - Windowed sinc unctions (highest qualit, more costl 5

6 Nearest Neighbor Interpolating Polnomial: ( (.5 Interpolating Kernel: h( Other names: bo ilter, sample-and hold unction, and Fourier window. Poor stopband. NN achieves magniication b piel replication. Ver bloc. Shit errors o up to / piel are possible. Common in hardware zooms. 6

7 7 Triangle Filter Other names or h: triangle ilter, tent ilter, roo unction, chateau unction, and Bartlett window. h a a or Solve a a a a ( Interpolation Kernel: ( (, ] [ ] [ ( Interpolating Polnomial: - + * = Sinc Sinc = Sinc In the requenc domain:

8 Cubic Convolution ( Third degree approimation to sinc. Its ernel is derived rom constraints imposed on the general cubic spline interpolation ormula. 3 a3 a a a 3 h( a3 a a a Determine coeicients b appling ollowing constraints:. h( = and h( = or =,. h must be continuous at =,, 3. h must have a continuous irst derivative at =,, 8

9 Cubic Convolution ( Constraint ( states that when h is centered on an input sample, the interpolation unction is independent o neighboring samples. First constraints give equations: h( a h( a3 a a a h( a3 a a a h( 8a a a 3a a a a a a a h ( h ( h ( h ( h ( a 3 more equations are obtained rom constraint (3 : h ( a 3a a a Total :7 equations, 8 unnowns ree variable ( a a 3 ( a ( a 3 3 h( a 5a 8a a 3 3 9

10 Cubic Convolution (3 How to pic a? Add some heuristics (mae it resemble Sinc unction: h ( = -(a+3 < a> -3 Concave downward at = h ( = -a > Concave upward at = This bounds a to the [-3, ] range. Common choices: a = - matches the slope o sinc at = (sharpens image a = -.5 maes the Talor series approimatel agree in as man terms as possible with the original signal a = -.75 sets the second derivative o the cubic polnomials in h to (continuous nd derivative at =

11 Cubic Splines ( 6 polnomial segments, each o 3 rd degree. s are joined at (or =,, n- such that,, and are continuous. (proo in App. 3 3 ( ( ( ( a a a a 3 3 a a where a a

12 Cubic Splines ( The derivatives ma be determined b solving the ollowing sstem o linear equations: Introduced b the constraints or continuit in the irst and second derivatives at the nots. dependencies global 5 3 3( 3( 3( 3( n n n n n n n

13 B-Splines To analze cubic splines, introduce cubic B-Spline interpolation ernel: B bo ilter B = B * B * B B = B * B B 3 = B * B * B * B h( Parzen Window: Not interpolator because it does not satis h( =, and h( = h( =. Indeed, it approimates the data. 3

14 Interpolator B-Splines K - inverse o tridiagonal matri; Computation is O(n All previous methods used data values or c rom C = K - F. F K C C K F c c c c c c c c h h h h h h c n n n n j j j j j j j j 6 ( 6 ( have we ( (, 6 ( (, 6 ( Since ( (

15 Truncated Sinc Function Alternative to previous ernels: use windowed sinc unction. Truncated Sinc X Rect(.5.5 Truncating in spatial domain = convolving spectrum (bo with a Sinc unction. Ringing can be mitigated b using a smoothl tapering windowing unction. Popular window unctions: Hann, Hamming, Blacman, Kaiser, and Lanczos. 5

16 Hann/Hamming Window ( cos Hann/Hamming( N N o / w N = number o samples in windowing unction. Hann: =.5; Hamming: =.5. Also nown as raised cosine window. h( (sinc( ( Hann( H( is sinc+ shited sincs. These cancel the right and let side lobes o Rect(. 6

17 Blacman Window..5cos.8cos Blacman( N N N o / w 7

18 Lanczos Window ( sinc Lanczos( h( = sinc( Lanczos( sinc( sinc(/ Rect(/ window Spatial Domain = sinc(/ Rect(/ h(=sinc(lanczos( H( = Rect( * Rect( * sinc( Frequenc Domain 8

19 Lanczos Window ( Generalization to N lobes: sinc LanczosN ( N N N Let N = 3, this lets 3 lobes pass under the Lanczos window. Better passband and stopband response 9

20 Comparison o Interpolation Methods NN, linear, cubic convolution, windowed sinc, sinc poor..> ideal (bloc, blurred, ringing, no artiacts

21 Convolution Implementation. Position (center ernel in input.. Evaluate ernel values at positions coinciding with neighbors. 3. Compute products o ernel values and neighbors.. Add products; init output piel. Step ( can be simpliied b incremental computation or space-invariant warps. (newpos = oldpos + inc. Step ( can be simpliied b LUT.

22 Interpolation with Coeicient Bins Implementation #: Interp. with Coeicient Bins (or space-invariant warps Strateg: accelerate resampling b precomputing the input weights and storing them in LUT or ast access during convolution. n intervals (bins between input piels Input Output u bin: (quantize u h -, h - h,h Kernel values i n- Interleaved h - - h - - h h Let d = i/n (ud< h = h(d; h - = h(-d h = h(+d; h - = h(-d

23 Uninterleaved Coeicient Bins i n- i n- i n- Oversample Oversample Oversample ii = bin # ii < oversample val = IN[-] * ern[ * oversample + ii] + IN[-] * ern[ * oversample + ii] + IN[ ] * ern[ii] + IN[ ] * ern[ * oversample - ii] + IN[ ] * ern[ * oversample - ii] + IN[ 3] * ern[3 * oversample - ii]; i(ii == val +=IN[-3] * ern[3 * oversample - ii]; ern[3+ii] ern[+ii] ern[+ii] Kern[ii] Kern[-ii] Kern[-ii] Kern[3-ii] Reer to code on p. 5 ii IN[-3] IN[-] IN[-] IN[] IN[] IN[] IN[3] 3

24 Kernel Position Since we are assuming space invariance, the new position or the ernel = oldpos + oset. Subpiel (bin INlen * oversample oset dii d ; dii # whole bins OUTlen d partial bin ( INlen * oversample % OUTlen Piel I J K L Current pos. Let dii = d =.6 I J K L Piel Blow-up o bins. Oset must be accurate to avoid accrual o error in the incremental repositioning o the ernel. Old pos. New pos. Oset =.6 bins

25 Forward vs. Inverse Mapping v Input Y Output u Forward mapping: = X(u, v; = Y(u, v Inverse mapping: u = U(,; v = V(, X Ch. 3, Sec. Input Output (accumulator Forward mapping Input Output Inverse mapping OV OV 3OV -d d Coeicient Bins or ernel eval or ast Convolution or image reconstruction. LHS d*ov OV OV OV RHS 5

26 Fant s Algorithm Implementation #: Fant s Resampling Algorithm (or space-var. warps Resampler Input and output are streams o piels that are consumed and generated at rate determined b the spatial mapping. Three conditions per stream: Current input piel is entirel consumed without completing an output piel. The input is entirel consumed while completing the output piel. 3 Output piel computed without entirel consuming the current input piel. Algorithm uses linear interpolation or image reconstruction and bo iltering (unweighted averaging or antialiasing. Code on p.56. 6

27 Eample m(u Input Output A A A A 3 A 3 ((... A ( (6 (.7.7 A A.. ( (6 (.3 (6( (6 (9 (. (9(. (9(