Lecture 2: 2D Fourier transforms and applications

Transcription

1 Lecture 2: 2D Fourier transforms and applications B14 Image Analysis Michaelmas 2017 Dr. M. Fallon Fourier transforms and spatial frequencies in 2D Definition and meaning The Convolution Theorem Applications to spatial filtering The Sampling Theorem and Aliasing Much of this material is a generalization of the 1D Fourier analysis course.

2 Reminder: 1D Fourier Series Spatial frequency analysis of a step edge x Fourier decomposition

3 Fourier series reminder Example = +

4 Fourier series for a square wave = # $ = % 1 './0,2,3, sin ('$)

5 Fourier series for a square wave = + =

6 Fourier series for a square wave = + =

7 Fourier series for a square wave = + =

8 Fourier series for a square wave = + =

9 Fourier series for a square wave = + =

10 Fourier series: just a change of basis M # $ = 6(7). =...

11 Inverse FT: Just a change of basis M = # $. =...

12 Action of spatial frequency filters: low pass high pass low pass eliminates details, maintains overall (smooth) shape high pass maintains details, eliminates overall shape

13 Frequency filters Nomenclature filter types (idealized) u 0 u Example: sinusoidal sweep image 0 u low pass band pass high pass

14 1D Fourier Transform Reminder transform pair - definition Example x u

15 2D Fourier transforms

16 2D Fourier transform Definition

17 Sinusoidal Waves 1D Fourier transform basis functions

18 slide: B. Freeman To get some sense of basis elements: we plot the real part of basis element as a function of x,y for some fixed u, v. We get a function that is constant when (ux+vy) is constant. magnitude of the vector (u, v) gives a frequency, direction gives an orientation. The function is a sinusoid with this frequency along the direction, and constant perpendicular to the direction. v u

19 slide: B. Freeman Here u and v are larger than in the previous slide. v u

20 slide: B. Freeman And larger still... v u

21 Summary f(x,y) = a +b +g +

22 The Discrete Fourier transform Forward transform Inverse transform

23 Some important Fourier Transform Pairs

24 Reminder: 1D Fourier Transform Reminder transform pair - definition Example x u

25 FT pair example 1 v rectangle centred at origin with sides of length X and Y u f(x,y) F(u,v) separability F(u,v)

26 FT pair example 3 Circular disk unit height and radius a centred on origin f(x,y) F(u,v) rotational symmetry a 2D rotational version of a sinc

27 FT pair example 2 Gaussian centred on origin f(x,y) FT of a Gaussian is a Gaussian Note inverse scale relation F(u,v)

28 FT pairs example 4 f(x,y) F(u,v)

29 Fourier transform practice Image Magnitude FT

30 Image Magnitude FT Fourier transform practice

31 Sinusoidal Waves

32 u u u u = a ( 2 ) 0 u = a ( 1 ) u = b( 1) 0-1

33 Summary f(x,y) = a +b +g +

34 Example: action of filters on a real image original low pass high pass f(x,y) F(u,v)

35 Example 2D Fourier transform Image with periodic structure f(x,y) F(u,v) FT has peaks at spatial frequencies of repeated texture

36 Example Forensic application F(u,v) remove peaks Periodic background removed

37 Example Image processing Lunar orbital image (1966) F(u,v) remove peaks join lines removed

38 Review 1. Match the spatial domain image to the Fourier magnitude image 1 A 2 B 3 4 C 5 D E

39 Magnitude vs Phase F(u,v) cross-section f(x,y) phase F(u,v) f(u,v) generally decreases with higher spatial frequencies phase appears less informative

40 The importance of phase phase magnitude phase

41 A second example phase magnitude phase

42 Transformations As in the 1D case FTs have the following properties Linearity Similarity Shift

43 In 2D can also rotate, shear etc Under an affine transformation: Example How does F(u,v) transform if f(x,y) is rotated by 45 degrees? f(x,y) F(u,v)

44 The convolution theorem

45 Filtering vs convolution in 1D filtering f(x) with h(x) f(x) h(x) /4 1/2 1/4 molecule/template/kernel g(x) 150 convolution of f(x) and h(x) after change of variable note negative sign (which is a reflection in x) in convolution h(x) is often symmetric (even/odd), and then (e.g. for even)

46 for convolution, reflect filter in x and y axes Filtering vs convolution in 2D convolution filtering image f(x,y) filter / kernel h(x,y) g(x,y) =

47 slide: K. Grauman Filtering Filtering with h h f

48 slide: K. Grauman Convolution Convolution: Flip the filter in both dimensions (bottom to top, right to left) Convolution with h h f

49 Filtering vs convolution in 2D in Matlab 2D filtering g=filter2(h,f); 2D convolution g=conv2(h,f); ], [ ], [ ], [, l n k m f l k h n m g l k - - = å f=image h=filter ], [ ], [ ], [, l n k m f l k h n m g l k + + = å

50 Convolution theorem Space convolution = frequency multiplication In words: the Fourier transform of the convolution of two functions is the product of their individual Fourier transforms Proof: exercise Why is this so important? Because linear filtering operations can be carried out by simple multiplications in the Fourier domain

51 The importance of the convolution theorem It establishes the link between operations in the frequency domain and the action of linear spatial filters Example smooth an image with a Gaussian spatial filter * 1. Compute FT of image and FT of Gaussian 2. Multiply FT s 3. Compute inverse FT of the result.

52 f(x,y) g(x,y) * Fourier transform Inverse Fourier transform x F(u,v) G(u,v)

53 f(x,y) Gaussian scale=3 pixels g(x,y) * Fourier transform Inverse Fourier transform x F(u,v) G(u,v)

54 There are two equivalent ways of carrying out linear spatial filtering operations: 1. Spatial domain: convolution with a spatial operator 2. Frequency domain: multiply FT of signal and filter, and compute inverse FT of product Why choose one over the other? The filter may be simpler to specify or compute in one of the domains Computational cost

55 Exercise What is the FT of? 2 small disks

56 Exercise What is the FT of frequency domain f(x,y) F(u,v) F(u,v) * n x

57 The sampling theorem

58 Discrete Images - Sampling f(x) X x x x

59 Fourier transform pairs

60 Sampling Theorem in 1D spatial domain frequency domain x F(u) * u replicated copies of F(u)

61 How to do the IFT? Apply a box filter 1/X u f(x) F(u) x The original continuous function f(x) is completely recovered from the samples provided the sampling frequency (1/X) exceeds twice the greatest frequency of the band-limited signal. (Nyquist sampling limit)

62 The Sampling Theorem and Aliasing if sampling frequency is reduced spatial domain frequency domain x u Frequencies above the Nyquist limit are folded back corrupting the signal in the acceptable range. The information in these frequencies is not correctly reconstructed.

63 Sampling Theorem in 2D frequency domain 1/X 1/Y * F(u,v) = * =

64 The sampling theorem in 2D If the Fourier transform of a function ƒ(x,y) is zero for all frequencies beyond u b and v b, (i.e. if the Fourier transform is band-limited) then the continuous function ƒ(x,y) can be completely reconstructed from its samples as long as the sampling distances w and h along the x and y directions are such that: 1 w 2 u b h 1 2 v b

65 Aliasing

66 Aliasing : 1D example If the signal has frequencies above the Nyquist limit Insufficient samples to distinguish the high and low frequency aliasing: signals travelling in disguise as other frequencies

67 Aliasing in video Example: Slide by Steve Seitz

68 Aliasing in 2D under sampling example original reconstruction signal has frequencies above Nyquist limit

69 Aliasing in images Aliased shirt:

70 What s happening? Input signal: Plot as image: x = 0:.05:5; imagesc(sin((2.^x).*x)) Aliasing Not enough samples

71 Anti-Aliasing Increase sampling frequency e.g. in graphics rendering cast 4 rays per pixel Reduce maximum frequency to below Nyquist limit e.g. low pass filter before sampling

72 Example 4 x zoom down sample by factor of 4 convolve with Gaussian * down sample by factor of 4

73 Hybrid Images

74 Filtering application: Hybrid Images Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006

75 Clues from Human Perception Early processing in humans filters for various orientations and scales of frequency Perceptual cues in the mid frequencies dominate perception When we see an image from far away, we are effectively subsampling it Early Visual Processing: Multi-scale edge and blob filters slide: A. Efros

76 slide: A. Efros Frequency Domain and Perception Campbell-Robson contrast sensitivity curve

77

78 Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006

79

80 Example 1 Gaussian Filter A. Oliva, A. Torralba, P.G. Schyns, Hybrid Images, SIGGRAPH 2006 Laplacian Filter unit impulse Gaussian Laplacian of Gaussian

81

82

83

84 Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006