Natural image modeling using complex wavelets

Size: px

Start display at page:

Download "Natural image modeling using complex wavelets"

Nickolas Horn
5 years ago
Views:

1 Natural image modeling using complex wavelets André Jalobeanu Bayesian Learning group, NASA Ames - Moffett Field, CA Laure Blanc-Féraud, Josiane Zerubia Ariana, CNRS / INRIA / UNSA - Sophia Antipolis, France

2 Summary The complex wavelet transform Properties of natural images Consequences on image modeling Subband modeling Choice of the wavelet basis A hierarchical Bayesian multiscale model Denoising algorithm Deblurring algorithm and wavelet packets Results: aerial, outdoors scene, astronomy Conclusion & future work Natural image modeling & complex wavelets 2 / 25

3 The complex wavelet transform (1) First level: non-decimated biorthogonal transform a 1 d 1 1 a 1A d 1 1A a 1B d 1 1B a 0 (image) d 2 1A da 3 1C 1A d 1 d 2 1B d 3 1C a 1D 1B d 1 1D d 2 1 d 3 1 d 2 1C d 3 1C d 2 1D d 3 1D Non-decimated transform 4 parallel trees ABCD Perfect reconstruction : mean (A+B+C+D)/4 A B C D A B C D A B C D A B C D A A B A A A A A A A A A A B C B B B A B A A A B B B C D B B B C C C B C B B B C C C D C C C D D D D C C C C D D D D D D D D D D Natural image modeling & complex wavelets 3 / 25

4 The complex wavelet transform (2) Level j: separated filter banks for each tree A,B,C,D different length filters : h o, g o, h e, g e shift < pixel a j,a a j,b a j,c a j,a h e h e h o h o g e g e g o g o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o h e h o h e g e h o g o g e h e g o h o h e g e h o g o g e g o 2 e 2 o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o 2 e 2 o a j+1,a a j+1,b a j+1,c d 1 a j+1,a j+1,d d 1 j+1,b d 1 d 2 j+1,c d 1 j+1,a d 2 j+1,d j+1,b d 2 d 3 j+1,c d 2 j+1,a d 3 j+1,d j+1,b d 3 j+1,c d 3 j+1,d Natural image modeling & complex wavelets 4 / 25

5 Building complex coefficients from the 4 trees The complex wavelet transform (3) 4 real subbands d k j,a d k j,b d k j,c d k j,d M Z + = (A - D) + i (B + C) Z - = (A + D) + i (B - C) Z k j+ Z k j- 2 symmetric complex subbands! The wavelet function is not a complex function. Not exactly complex wavelets! Natural image modeling & complex wavelets 5 / 25

1+ z 3 1+ Impulse responses (level 4) a 1 a 1 separates 6 directions

6 Complex wavelets: directional selectivity Re z 1 1- z 1 2- a 2- a 1 a 2+ a 1 z 1 2+ z 1 1+ z 3 2- z 2 2- z 2 2+ z 3 2+ Im z 3 1- z 2 1- z 2 1+ z 3 1+ Impulse responses (level 4) a 1 a 1 separates 6 directions frequency space partition Natural image modeling & complex wavelets 6 / 25

7 Image modeling Image models : sets / probabilistic, discrete / continuous, single scale / multiscale Build a new image model : probabilistic, discrete, multiscale Study natural images Axioms Axioms Wavelet representation Necessary properties Choice of the wavelet New hierarchical model P(X) is complex: simplify using augmented process P(ω,X) ω X Natural image modeling & complex wavelets 7 / 25

8 Axiom A1 : self-similarity Vannes Toulouse (1) (2) Vannes(2) (1) IMAGE scale invariance or self-similarity Spectrum log w energy w Power spectrum decay w = w0 r -q log r radial frequency r Natural image modeling & complex wavelets 8 / 25

9 Axiom A2 : spatial adaptivity textures homogeneous areas small features edges Natural image modeling & complex wavelets 9 / 25

10 Consequences on image modeling A1 Fractional Brownian Motion (w 0,q) fractal model A2 Spatially adaptive parameters A1 frequency space A2 image space Wavelet transform reduce the coefficients dependence (K-L) multiresolution analysis [Mallat, 89] subband histogram frequency space partition A2 heavy-tailed distribution Natural image modeling & complex wavelets 10 / 25

11 Subband modeling 1 : spatial adaptivity ξ = transform coefficients : stochastic process P(ξ t )? Marginal subband distribution : Generalized Gaussian, independent coefficients P ( ξt) Z 1 α,p Propose a spatially adaptive model s t ξ t = e ξ t / α p p = nonstationarity parameter Augmented process P(s t,ξ t ) marginal ~ P(ξ t ) discrete Gaussian Mixture, parameters ν P(s t ) discrete pdf. Major drawback: global model P(ξ t s t ) Gaussian pdf. std. dev. s t homogeneous areas edges, textures Natural image modeling & complex wavelets 11 / 25

12 Interscale dependence (real) wavelet transform scale 3 scale 2 scale 1 Interscale persistence of edges and smooth areas Natural image modeling & complex wavelets 12 / 25

13 Subband modeling 2: interscale dependence Joint histogram P(ξ t, ξ π(t) )? Implicit dependence, using hidden variables s Augmented process P(s t, s π(t), ξ t, ξ π(t) ) Hidden Markov Tree (HMT) s π(t) ξ π(t) discrete Gaussian mixture ξ π(t) ξ t s t ξ t P(s t,s π(t) ) discrete pdf. matrix ε P(ξ t s t ) Gaussian pdf. Natural image modeling & complex wavelets 13 / 25

14 Intrascale dependence (real) wavelet transform horizontal details vertical details diagonal details Intrascale persistence of edges and smooth areas Natural image modeling & complex wavelets 14 / 25

15 Subband modeling 3: intrascale dependence Joint histogram P(ξ t, ξ t )? Implicit dependence, shared hidden variable Augmented process P(s t, s π(t), {ξ ti }, {ξ π(ti) }) ξ π(t1) s π(t) ξ π(t2) ξ π(t3) ξ t1 Hidden Markov Tree + hidden variable shared btw. orientations ξ t ξ t s t ξ t3 ξ t2 Natural image modeling & complex wavelets 15 / 25

Choice of the wavelet basis 1: compactness and

different wavelet functions Haar [Haar, 10]

small number of wavelet coefficients Complex

Asymptote E~N -1/2 Haar Symmlet-8 log # wavelet

16 Choice of the wavelet basis 1: compactness and detection Optimal shape representation using different wavelet functions Haar [Haar, 10] Symmlet-8 [Daubechies, 88] Representation by a small number of wavelet coefficients Complex [Kingsbury, 98] image log approximation error Asymptote E~N -1/2 Haar Symmlet-8 log # wavelet coefficients Natural image modeling & complex wavelets 16 / 25

17 Choice of the wavelet basis 2: invariance properties Shift invariance? Shifted image Haar Spline Symmlet 8 Complex Rotational invariance? Haar Spline Symmlet 8 Complex Natural image modeling & complex wavelets 17 / 25

18 A hierarchical Bayesian multiscale model q w 0 ν,ρ ω X p s ξ X ε P(ω,X) = P(w 0,q,p) P(ε) P(ν,ρ w 0,q,p, ε) P(s ε,ν,ρ) P(ξ s) P(X ξ) Hierarchical Bayesian model Multiscale approach : wavelets In practice : simplified models and custom optimized algorithms Natural image modeling & complex wavelets 18 / 25

19 Bayesian estimators Bayes estimator : θˆ = argmin E [C( θ, θ*)] θ* Y, θ cost function C( θ, θ*) = 1 δ ( θ θ*) C( θ, θ*) = θ θ* 2 estimator MAP Maximum A Posteriori θ= ˆ argmaxp( θ/y) cost : binary simple estimation θ MF Mean Field θˆ = θp( θ/y) dθ cost restoration difficult estimation Empirical Bayesian estimation : in general, MF estimation difficult ( αˆ, ωˆ ) = argmaxp( α, ω/y) α, ω Laplace approx. P(α,ω Y) ~ δ(α α 0, ω ω 0 ) Xˆ = X P(X/ αˆ,ˆ ω,y)dx ~ ~argmaxp(x/ αˆ, ωˆ,y) X Natural image modeling & complex wavelets 19 / 25

20 w 0,q,p ν,ρ ε s ξ o HMTAD (denoising algorithm) «Hidden Markov Tree Automatic Denoising» In the wavelet domain : estimate ξ from the noisy observation o Observation equation in the wavelet domain : o = ξ + N Estimate Compute P(s o) o, σ k Compute ν,ρ Estimate ε Denoising p, w 0 :, Mean q Field (MF) o (independent) w 0,q,p ν,ρ ε P(s o) ξ Empirical Bayesian estimation Estimate w 0, q, p : use moments on subbandscompute P(s o) backward forward Compute ν,ρ (Gaussian mixture parameters) from w 0, q, p Estimate ε (transition matrices) : Estimate P(s o) Compute P(s o) in the independent subband case (MF) Compute the joint histogram : (s t in state m, s π(t) in state n) Compute P(s o) : from ν, ε and o Denoise o (MF) Baum-Welch method: forward/backward [Crouse et al, 98] = shrinkage - Backward: terminal nodes root nodes (descendents) - Forward: root nodes terminal nodes (parent) s2 ^ Denoising step : MF using P(ξ s) : ξ ˆ = o ξ s2+σ2 Natural image modeling & complex wavelets 20 / 25

21 HMT ξ X Y COWPATH 1 (deconvolution) «COmplex Wavelet Packets Automatic THresholding» Estimate X from the blurred and noisy observation Y Complex Wavelet Packets (CWP) COWPATH 1 HMTAD Fast deblurring algorithm : simultaneously diagonalize - the blur Estimate (convolution) Estimate operator Y, h, σ Wavelets: - the covariance compact p, w 0 representation, q matrix of noise the of σimage k natural model images (space/frequency) Wavelet packets: good representation of the deblurred noise Complex Approximate wavelets : method invariance Rough : deconvolution properties (shift, step rotation) Non-regularized Generalized DCT deconvolution DCT colored noise inverse φ 2 φ filter φ 2 φ Change the basis : wavelet packet transform - near-diagonalization of the deblurred noise [Rougé 95, Kalifa 99] Denoising step - natural basis for CWP the image model : wavelets packets CWP HMTAD -1 White iid. noise denoising : HMTAD X^ Natural image modeling & complex wavelets 21 / 25

22 Results: aerial image PSF Observed image Y blurred and noisy, σ=1.4 Deblurred image SNR improvement = 5.68dB Natural image modeling & complex wavelets 22 / 25

23 Results: cameraman image Observed image Y (same PSF) blurred and noisy, σ=1.4 Deblurred image SNR improvement = 6.75dB Natural image modeling & complex wavelets 23 / 25

24 Complex wavelet deconvolution example Reference (original) X Wiener 17.3, db Edge-pres., ϕ HL, 17.2 db Observed Y, 14.9 db Edge-pres., ϕ HS, 17.4 db Complex wav., 16.8 db Natural image modeling & complex wavelets 24 / 25

25 Conclusion New image model Based on properties of natural images (scale invariance, spatial adaptivity) Uses complex wavelets to avoid artifacts (invariance properties) Hierarchical Bayesian approach: easy step by step parameter estimation Deconvolution algorithm Use complex wavelet packets to do denoising instead of deblurring Denoising method: fast coefficient shrinkage / posterior mean estimate One pass estimation technique vs. slow EM method: low complexity Nonstationary & multiscale: the quality is superior to classical algorithms Future work? Generalization by introducing multiple wavelet bases Use a model map within the hierarchical framework, empirical Bayes estimation Extend to 3D imaging (confocal microscopy) / surface models (meshes) Natural image modeling & complex wavelets 25 / 25

Bayesian Tree-Structured Image Modeling Using Wavelet-Domain Hidden Markov Models

1056 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 10, NO. 7, JULY 2001 Bayesian Tree-Structured Image Modeling Using Wavelet-Domain Hidden Markov Models Justin K. Romberg, Student Member, IEEE, Hyeokho