Multiresolution Image Processing

Transcription

1 Multiresolution Image Processing 2 Processing and Analysis of Images at Multiple Scales What is Multiscale Decompostion? Why use Multiscale Processing? How to use Multiscale Processing? Related Concepts: Subbands, Wavelets

2 3 Motivation Images can not be adequately modeled with a single description, at a single scale

3 5 6 Multiscale Decomposition: Pyramids We can model images as being composed from a combination of simpler images at increasing scales. Gaussian Pyramid: Laplacian Pyramid: Good for compression 3

4 7 The Laplacian Pyramid Synthesis preserve difference between upsampled Gaussian pyramid level and Gaussian pyramid level band pass filter - each level represents spatial frequencies largely unrepresented at other levels Analysis reconstruct Gaussian pyramid, take fine-scale layer 8 4

5 9 Pyramids Construction: An Overcomplete Redundant Representation Original Invention by: Burt, Adelson, 983 5

6 Image Processing with Pyramids: 2 Image Blending 6

7 3 Feathering + Encoding transparency = Ix,y = ar, ag, ab, a I blend = I left + I right 4 Effect of Window Sie left right 7

8 5 Effect of Window Sie 6 Good Window Sie Optimal Window: smooth but not ghosted 8

9 7 What is the Optimal Window? To avoid seams window >= sie of largest prominent feature To avoid ghosting window <= 2*sie of smallest prominent feature Natural to cast this in the Fourier domain largest frequency <= 2*sie of smallest frequency image frequency content should occupy one octave power of two FFT 8 Pyramid Blending Left pyramid blend Right pyramid 9

10 9 Pyramid Blending 2 laplacian level 4 laplacian level 2 laplacian level left pyramid right pyramid blended pyramid

11 2 Simplification: Two-band Blending Brown & Lowe, 23 Only use two bands: high freq. and low freq. Blends low freq. smoothly Blend high freq. with no smoothing: use binary mask 22 2-band Blending Low frequency l > 2 pixels High frequency l < 2 pixels

12 23 Linear Blending 24 2-band Blending 2

13 25 Direct Merge: Multiscale Merge: 26 Very early computational approach to creating large depth-of-field 3

14 27 Image Analysis with Pyramids: Detection Recognition Segmentation Etc. 28 Related Notion: Subband Coding Decomposition of a Signal/Image into a set of complementary bandlimited components Analysis: Filter + Downsample Synthesis: Upsample + Filter 4

15 5 29 Subband Coding: Perfect Reconstruction What relationship between the filters guarantees perfect reconstruction? Key Tool: The -transform n n n x X Key Idea: Aliasing Cancellation 3 Perfect reconstruction conditions: Key relationships: Downsampling: Upsampling: 2 n x n x d otherwise for even 2 / n n x n x u 2 2 / 2 / X X X d 2 X X u 2 2 ˆ X X H X H G X H X H G X Perfect Reconstruction: 2 G H G H G H G H

16 3 Example: 2-Channel with Perfect Reconstruciton 32 Family of Solutions: The whole approach can be extended to M subbands The same arguments can be applied in a separable fashion to image decomposition along rows and columns. 6

17 33 Now, in 2 dimensions Horiontal high pass Frequency domain Horiontal low pass 34 Apply the wavelet transform separable in both dimensions Horiontal high pass, vertical high pass Horiontal high pass, vertical low-pass Horiontal low pass, vertical high-pass Horiontal low pass, Vertical low-pass 7

18 35 Simoncelli and Adelson, in Subband coding, Kluwer, 99. To create 2-d filters, apply the -d filters separably in the two spatial dimensions 36 Wavelet/QMF representation 8

19 37 Good and bad features of wavelet/qmf filters Bad: Aliased subbands Non-oriented diagonal subband Good: Not overcomplete so same number of coefficients as image pixels. Good for image compression JPEG 2 38 Example: 4 subband image decomposition 9

20 39 Steerable filters Analye image with oriented filters Avoid preferred orientation Said differently: We want to be able to compute the response to an arbitrary orientation from the response to a few basis filters By linear combination Notion of steerability 4 Reprinted from Shiftable MultiScale Transforms, by Simoncelli et al., IEEE Transactions on Information Theory, 992, copyright 992, IEEE 2

21 4 42 Fourier construction Slice Fourier domain Concentric rings for different scales Slices for orientation Feather cutoff to make steerable Tradeoff steerable/orthogonal 2

22 43 But we need to get rid of the corner regions before starting the recursive circular filtering Simoncelli and Freeman, ICIP Non-oriented steerable pyramid 22

23 45 3-orientation steerable pyramid 46 Steerable pyramids Good: Oriented subbands Non-aliased subbands Steerable filters Bad: Overcomplete Have one high frequency residual subband, required in order to form a circular region of analysis in frequency from a square region of support in frequency. 23

24 47 Gaussian Image pyramids Progressively blurred and subsampled versions of the image. Adds scale invariance to fixed-sie algorithms. Laplacian Shows the information added in Gaussian pyramid at each spatial scale. Useful for noise reduction & coding. Wavelet/QMF Steerable pyramid Bandpassed representation, complete, but with aliasing and some non-oriented subbands. Shows components at each scale and orientation separately. Non-aliased subbands. Good for texture and feature analysis. 48 Related Notion: Wavelet Transform Simplest case: Discrete Haar Wavelet Transform in -D y y2 y 3 y x x2 x 3 2x4 Transform of signal H 4 Given signal y Hx 24

25 49 Related Notion: Wavelet Transform Important points: Note the action of each row of H y gives information about the signal at different scales of resolution Rows of H are the coefficients of the corresponding QMF system Orthogonal Transformation Basis vectors are finite support H HH Can be applied in separable way in 2-D Non-redundant square transformation H H T T I 5 Discrete Haar Wavelet Example 25

26 5 Continuous Wavelet Series Expansion f Arbitrary starting coarse scale x c j k j k x d j k, j, k k j j k x Scaling functions Scaling coeffs. Detail coeffs. Wavelet functions Coarse Scale Approximation Fine-scale details 52 The Scaling Functions x V j, k j j 2 j x 2 2 x k h n 2 2 n Span x k j, k x n Haar Example: 26

27 53 The Wavelet Functions x W j, k j j 2 j x 2 2 x k h n 2 n Span x k j, k 2x n Haar Example: 54 Their Relationships Haar Example: 27

28 55 The Discrete Case: The Fast WT If fx is composed of discrete samples k is discrete, the transform is similar. f x W j, k j, k x W j, k j, k k j j k And the coefficients can be obtained as: Finer scale coefficients x Coarser scale detail coeffs. HPF Coarser scale approximation coeffs. LPF 56 The Discrete Case: The Fast WT Resemblance to the QMF setup is not coincidental! 28

29 57 2-D Wavelet Analysis Scaling Functions: x, y x y Wavelet Functions: Horiontal Vertical Diagonal x, y x y H x, y y x V x, y x y D 58 2-D Wavelet Analysis: Example 29

30 59 Multiscale Motion Estimation Construct a Gaussian Pyramid and estimate motion from coarse-to-fine levels. Compute motion estimates at each scale. Coarse Significantly better than nonmultiscale Fine 6 Multiscale Methods Details. Estimate motion at coarsest scale 2. Undo motion in the sequence at the next level. 3. Estimate residual motion 4. Update motion estimate. 5. Repeat in a coarse-tofine fashion. Offers much better performance than non-multiscale 3