Sampling: Application to 2D Transformations

Transcription

1 Sampling: Application to 2D Transformations University of the Philippines - Diliman August Diane Lingrand lingrand@polytech.unice.fr

2 Sampling Computer images are manipulated in a digitalized form for: Image processing Visualization (on screen, on printer...) Transmission and back-up Antagonist needs: Good visual quality, Reduce image size as much as possible. 2

3 Signal sampling How to transform a 2D continuous signal into a discrete signal? Technological solution: Digital camera Scanner for paper documents Theoretical solution : Sampling theory 3

4 Let us sample Santa-Claus! 300 x 260 pixels 20 x 17 pixels 8 x 6 pixels 4

5 Some definitions Vertical resolution : number of rows Horizontal resolution : number of columns Spatial resolution = vertical resolution * horizontal resolution Resolution density : number of pixels by length unit pixels per inch (ppi) or dots per inch (dpi) 5

6 Dirac pulse 1D Dirac pulse δ(x) = 1 if x=0 δ(x) = 0 else 2D Dirac pulse 1 0 δ(x,y) = 1 if x=0 and y=0 δ(x,y) = 0 else which corresponds to : δ(x,y) = δ(x) δ(y) 6

7 Sampling : theory (1) Image : seen as a set of Dirac pulses pixel (x,y) : Dirac pulse centered in (x,y) with the intensity as amplitude 7

8 Sampling : theory (2) 1D sampling: Dirac comb (or Shah function) 2D sampling : Dirac «brush» 8

9 Extended comb and comb y Extended comb : Comb : δx x y x 9

10 Brush or 2D comb Brush = product of 2 extended combs δy y x δx 10

11 Sampled function g : sampled function 11

12 2D Fourier transform Inverse transform : 12

13 Interpretation of a Fourier image (coefficients norm) v High frequencies ω 0 θ Low frequencies u 13

14 Fourier and Santa Claus 14

15 Synthetic image

16 A property of the 2D Fourier transform translation / phase shift Exercise: demonstrate this property 16

17 2D Convolution Convolution with a 2D Dirac pulse f2(x, y) = f1(x, y) Convolution a Dirac pulse shifted by (x0,y0) f2(x, y) = f1(x x0, y y0) Fourier transform F2(u, v) = F1(u, v) H(u, v) and vice versa g(x,y) = f1(x, y) f2(x, y) then G(u,v) = F1(u, v) * F2(u, v) 17

18 Reconstruction Retrieve a continuous signal from its sampled form Standard method : retrieve the continuous signal's Fourier transform from the discrete signal's Fourier transform. spatial domain Fourier domain x Fourier domain u spatial domain u x original samples Fourier transform window extracting inverse Fourier transform 18

19 Fourier transform of a brush Fourier transform of extended combs : Fourier transform of a brush : 19

20 Sampled signal Fourier transform Convoluting the Fourier transform with a brush Fourier transform. The Fourier transform of a brush with step δx and δy is a brush with step 1/δx and 1/δy Convolution with a brush is equivalent to suming convolutions with shifted Dirac pulses 20

21 Aliasing 2/δx Gibbs effect (halo) Watch out shirts with rays on TV and caravans on Westerns! 21

22 Aliasing (2) The reconstruction quality depends on the sampling step : We expect 1/δx and 1/δy as big as possible Thus δx and δy have to be small : Many sensors With few inter-spacing 22

23 Nyquist (1928) Shannon (1948) theorem The Fourier transform is bounded The sampling frequency has to be at least twice the maximal frequency of the image Exemple : 768 pixels per line sensors, line duration = 52 µs : fsampling=768/(52*10-6)=14.75mhz Aliasing frequency : 7.37 MHz 23

24 Preserving Nyquist Shannon condition Signal low-pass filtering This sets up the maximal frequency Increase the sampling step However, the signal spectrum has to be bounded oversampling: more samples than pixels 24

25 Low-pass filtering on Santa Claus 25

26 26

27 ESSI /

28 Optical low-pass filtering Performed by cameras Using properties of thin quartz plates Consequences : contours weakening Requires to strengthen contours 28

29 CCD sensors Only 1/3 of a CCD cell is used for acquiring light Too smalls details are not correctly captured The «green» sensors are shifted with ½ pixel from the «red» and «blue» sensors 29

30 CRT reconstruction CRT = Cathodic Ray Tube response = Gaussian Frequency domain Spatial domain The larger the spot, the narrower its Fourier transform, the more high frequencies are lost and low frequencies are weakened 30

31 Ideal reconstruction filter Infinite spatial extension Bounded image Each pixel contribution Computational complexity Truncating the Σ : Gibbs effect δy δx Frequency domain Spatial domain 31

32 Reconstruction filter g : sampled function h : ideal filter f : reconstructed signal

33 Reconstruction filters shortcomings Low-pass weakening Blur effect Addition of high frequencies «ringing» or strengthening of the sampling grid 33

34 Ideal filter approximation (1) 0-order approximation of the sinus cardinal: crenelation sin (fs/2) c sinc(x) = sin(πx)/πx Spatial domain Frequency domain 34

35 Ideal filter approximation (2) Bi-linear or tent filter High frequencies are weakened Easy to compute Produces artefacts T.F. 35

36 Ideal filter approximation (3) spline Mitchell s 2-parameter spline family Segmented degree 3 polynomials : k k1(x) = A1 x 3+B1x2+C1 x +D1 k2(x) = A2 x 3+B2x2+C2 x +D2 1 Symmetric function : k(x) = k(-x) Continuity : k1(1) = k2(1) et k2(2) = 0 k'1(0) = 0 et k'1(1) = k'2(1) et k'2(2) = 0 Sum : Σ k(x-n)=k2(1+ε)+k1(ε)+k1(ε-1)+k2(ε-2)=1 k 2 36

37 Mitchell's 2-parameters spline family k1(1) = k2(1) => A1+B1+C1+D1 = A2+B2+C2+D2 (1) k2(2) = 0 => 8A2+4B2+2C2+D2 = 0 (2) k'1(0) = 0 => C1= 0 (3) k'2(2) = 0 => 12 A2+4 B2+C2 = 0 => C2 = - 12 A2-4 B2 (4) k'1(1) = k'2(1) => 3 A1+2 B1 = -9A2-2B2 (5) (2)et(4) => D2 = -8A2-4 B2 +24A2+8 B2=16A2 + 4 B2 (1)et(2) => A1+B1+D1 = A2+B2-12A2-4B2+16A2+4B2=5A2+B2 => D1 = 5A2+B2-A1-B1 sum : 9A2 + 5B2 + 3C2 + 2D2 + A1+ B1+ 2D1 = 1 thus : 15A2 + 3B2 - A1- B1=1 hence: A1=-39A2-8B2 + 2 B1=54A2 + 11B2 3 C1=0 D1=-10A2 2B2 + 1 C2=-12A2 4B2 D2=16A2 + 4B2 6A2+B2 = - C et By setting : 5A2+B2 = B/6 one get... (turn page please)

38 Mitchell's 2-parameters spline family (solution) Family : if if anywhere else Other examples : if B=0 then k(0)=1 and k(1)=0 since sinc Cardinal splines : B=0; C=-a Catmull-Rom Spline : B=0; C=0.5 Cubic B-spline : B=1; C=0

39 Ideal filter approximation (4) Other trade-off : truncated sinus cardinal sin(πx)/ x. sin(πy)/ y => discontinuity problem located at the truncature and causing ripples => this problem can be avoided by using a cubic convolution function (cf Mitchell) Circular reconstruction filter with Bessel's function impulsional functions 39

40 Applications Changing an image scale Performed by Java libraries Plotting geometrical objects Lines, curves Moving geometrical objects Decimal step (non-integer) translation rotation Scale changes 40

41 Interpolation? int getpixel(double x, double y) {... }

42 Problem illustration Rotation example (angle θ ) : x1 y1 x2= cos(θ ) x1 + sin(θ ) y1 y2= - sin(θ) x1 + cos(θ ) y1 x2 y2 and I(x2,y2) = I(x1,y1) thus I(x2,y2) = I(cos(θ )x2-sin(θ )y2, sin(θ )x2+cos(θ )y2)

43 Problem illustration (continued) x2 x1 y1 y2 where x1 and y1 are not necessarily integers! Intuitive solutions: Get the nearest integer values Weight with the integer values in the neighborhood by the distance to the neighbors

44 How does this relate to sampling? The original image is resampled to get the color value of a point with non-integer coordinates (x1,y1) int getpixel(double x, double y, int interpolationmode) {... }

45 1D Interpolation 0 order 1st order 2nd order 3rd order... Gaussian!

46 Interpolations en 1D (suite) 1st order 0 order 2nd order 3rd order

47 2D Interpolation Reconstruction filter : R(x,y) = R(x)R(y) Reconstructed function : Color of pixel (m,n)

48 O Order 0 : nearest neighbor Reconstruction filter : R(x,y) = 1 pour x ½ et y ½ Reconstructed function : There exists only 1 value m such as x-m ½ There exists only 1 value n such as y-n ½ thus : f(x,y) = g(m,n) m n

49 st 1 order : bilinear interpolation Reconstruction filter : R(x,y)=(1+ x)(1+ y) for -1 x 0 and -1 y 0 R(x,y)=(1+ x)(1 y) for -1 x 0 and 0 y 1 R(x,y)=(1 x)(1+ y) for 0 x 1 and -1 y 0 and 0 anywhere else R(x,y)=(1 x)(1 y) for 0 x 1 and 0 y 1 P00 εx P10 εy P P01 P11 I(P) = (1-εx)(1-εy) I(P00) +(1-εx) εy I(P01) +εx (1-εy) I(P10) +εx εy I(P11)

50 nd 2 order (bell) Reconstruction filter : 3 * 3 = 9 cases depending on x and y values Similar to the previous order 4 cases depending on εx,y 0.5 or >0.5 P00 P10 P20 P01 εx P εy : point taken into account to compute the color of point

51 nd 2 order (continued) Suppose that : 0 εx,y 0.5 m = 0 : R(x) = ½ (x - 3/2 )2=0.5(εx-0.5)2 Other cases are identical up to a symmetry m = 1 : R(x-1) = ¾ - (x-1)2 = ¾ - εx2 m = 2 : R(x-2) = R(εx-1) = ½ (εx+ ½)2 I(P)= Σm,n R(x-m,y-n)Imn = Σm,n R(x-m)R(y-n)Imn = R(x)R(y)I00 + R(x-1)R(y)I10 + R(x-2)R(y)I20 + R(x)R(y-1)I01 + R(x)R(y-2)I02 + R(x-1)R(y-1)I11 R(x-1)R(y-2)I12 + R(x-2)R(y-1)I21 + R(x-2)R(y-2)I22 +

52 rd 3 order or cubic B-spline Reconstruction filter : 2 * 2 = 4 cases depending on x and y values Similar to the 3rd order P00 P01 P11 εx : point taken into account for computing the value of point εy P22 P23 P33

53 rd 3 order (continued) if if anywhere else m=0 : R(x) = (1-εx)3/6 m=1 : R(x-1) = 2/3+(½ εx-1)εx2 m=2 : R(x-2) = 2/3 - ½(1+εx)(1- εx)2 m=3 : R(x-3) = εx3/6 I(P) = I00R(x)R(y) + I10 R(x-1)R(y) I33R(x-3)R(y-3)

54 Improved cubic B-spline if if anywhere else Mitchell polynomials family with B=0 (sinc) For C=1 : m=0 : R(x) = (-εx3+3εx2-3εx+1)/6 m=1 : R(x-1) = (εx/2-1)εx2 +2/3 m=2 : R(x-2) = (-3εx3+3εx2+3εx+1)/6 m=3 : R(x-3) = εx3/6 I(P) = I00R(x)R(y) + I10 R(x-1)R(y) I33R(x-3)R(y-3)

55 Rotation example (10 angle) o

56 Nearest neighbor interpolation

57 Bilinear Interpolation

58 nd bell Interpolation (2 order)

59 rd 3 order Interpolation (B=1,C=0)

60 rd Comparing 0 and 3 orders

61 rd 3 order Interpolation (B=0,C=1)

62 9 rotations with 40o angle 0 order 2nd order 1st order 3rd order B=1 C=0 B=0 C=1

63 36 rotations with 10o angle 3rd order interpolation B=1 C=0 B=0 C=1

64 Zoom example (2.4 zoom factor)

65 0 order

66 st 1 order

67 nd 2 order

68 rd 3 order B=1 C=0

69 rd 3 order B=0 C=1

70 Other example : zoom 1.4

71

72

73

74

75

76 On-computer exercise Rotation Around the image upper left corner Around the window center (or the image center) Scaling Interpolation Nearest neighbor Bilinear 3rd order (considering Mitchell's polynomials with B=0 and C=0.5)