Sparse wavelet expansions for seismic tomography: Methods and algorithms

Transcription

1 Sparse wavelet expansions for seismic tomography: Methods and algorithms Ignace Loris Université Libre de Bruxelles International symposium on geophysical imaging with localized waves July 2011 (Joint work with: G. Nolet, I.Daubechies, F.J.Simons, C. Verhoeven et al.)

2 Theme Discussion of: fewer data than unknowns -problem recent advances in regularization methods in context of global seismic tomography How to exploit such insight in seismic inversions? Philosophy: Ockham s razor the simplest explanation is usually the best In applied mathematics: principle of sparsity (here sparsity refers to a property of the model, NOT of a matrix)

3 Overview Mathematical developments: How can model sparsity help in solving an inverse problem? Illustration on toy example Algorithms Application to seismic tomography: Good model parametrization: so-called wavelets Mathematical microscope : analyze seismic heterogeneity at different length scales Computationally efficient (fast algorithm) Can express seismic heterogeneity efficiently Formulate solution method

4 Mathematical framework Solve linear relations between unknown x and measurement data y: Kx = y Here: K =matrix with sensitivity kernels (known) y =data vector (known) x =model vector (coefficients representing the wave speed anomaly) Problems: insufficient data, inconsistent data, ill-conditioning of K No solution or no unique solution Way out: minimize a penalized least-squares functional: x rec = arg min Kx y 2 + penalty x

5 Penalization strategy Traditional: quadratic penalties penalty x 2 or penalty x 2 = smoothing operator Advantage: leads to linear equations K T Kx + λx = K T y or K T Kx + λ T x = K T y However: model can be too smooth... Can better reconstructions be found by using other information? Increased interest in using sparsity as a priori information (sparsity solution described by few nonzero coefficients)

6 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

7 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

8 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) x 2 Kx = y x 1 # nonzero 1 combinatorial D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

9 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) x 2 Kx = y x 2 Kx = y x 1 x 1 # nonzero 1 x1 + x 2 1 combinatorial algebraic D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

10 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) x 2 Kx = y x 2 Kx = y x 2 Kx = y x 1 x 1 x 1 # nonzero 1 x1 + x 2 1 x 1 + x 2 1 combinatorial algebraic convex D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

11 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) x 2 Kx = y x 2 Kx = y x 2 Kx = y x 2 Kx = y x 1 x 1 x 1 x 1 # nonzero 1 x1 + x 2 1 x 1 + x 2 1 x1 2 + x2 2 r 2 combinatorial algebraic convex linear D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

12 Penalization strategy How to enforce sparsity on solutions of Kx = y? Minimize a penalized least-squares functional: x rec 1 = arg min Kx x 2 y 2 + penalty with a judiciously chosen sparsity promoting penalty. A trade-off between sparsity promotion and tractability (convexity) x 2 Kx = y x 2 Kx = y x 2 Kx = y x 2 Kx = y x 1 x 1 x 1 x 1 # nonzero 1 combinatorial algebraic x1 + x 2 1 convex x 1 + x 2 1 linear x x2 2 r 2 l 1 norm penalty promotes sparsity and is tractable ( x 1 = i x i ) D. Donoho (2006) DOI: /TIT A. Bruckstein et al. (2009) DOI: /

13 Sparsity and l 1 norm Unit l 1 ball in 2-D 1 2

14 Sparsity and l 1 norm Unit l 1 ball in N-D 1 N

15 Sparsity and l 1 norm Unit l 1 ball in N-D 1 N looks like (when N is large)

16 Illustration of the efficiency of sparse recovery x input K

17 Illustration of the efficiency of sparse recovery x input K Synthetic noisy data: y = Kx input + n

18 Illustration of the efficiency of sparse recovery 30 K Synthetic noisy data: y = Kx input + n

19 Illustration of the efficiency of sparse recovery 30 K Synthetic noisy data: y = Kx input + n min x Kx y 2 + λ x 2 min x Kx y 2 + 2λ x 1

20 Illustration of the efficiency of sparse recovery 30 K Synthetic noisy data: y = Kx input + n min Kx y 2 + λ x 2 min x x Kx y 2 + 2λ x

21 Illustration of the efficiency of sparse recovery x input K Synthetic noisy data: y = Kx input + n min Kx y 2 + λ x 2 min x x Kx y 2 + 2λ x

22 Analysis sparsity vs. synthesis sparsity Analysis-style sparsity: Require that certain linear combinations Ax of unknown x are sparse: x with many (Ax) i = 0 Here A is explicitly known. Synthesis-style sparsity: Express unknown x as a sparse linear combination of a set of known basis/frame vectors: x = Su with many u i = 0 If AS = SA = 1 then: synthesis sparsity x rec = analysis sparsity x rec (algorithms may still be different)

23 Analysis sparsity vs. synthesis sparsity Analysis-style sparsity: Require that certain linear combinations Ax of unknown x are sparse: x with many (Ax) i = 0 Here A is explicitly known. Synthesis-style sparsity: Express unknown x as a sparse linear combination of a set of known basis/frame vectors: x = Su with many u i = 0 If AS = SA = 1 then: synthesis sparsity x rec = analysis sparsity x rec (algorithms may still be different)

24 Analysis sparsity vs. synthesis sparsity Analysis-style sparsity: Require that certain linear combinations Ax of unknown x are sparse: x with many (Ax) i = 0 Here A is explicitly known. Synthesis-style sparsity: Express unknown x as a sparse linear combination of a set of known basis/frame vectors: x = Su with many u i = 0 If AS = SA = 1 then: synthesis sparsity x rec = analysis sparsity x rec (algorithms may still be different)

25 Analysis sparsity vs. synthesis sparsity Analysis-style sparsity: Require that certain linear combinations Ax of unknown x are sparse: x with many (Ax) i = 0 Here A is explicitly known. Synthesis-style sparsity: Express unknown x as a sparse linear combination of a set of known basis/frame vectors: x = Su with many u i = 0 If AS = SA = 1 then: synthesis sparsity x rec = analysis sparsity x rec (algorithms may still be different)

26 Cost functions for analysis and synthesis sparsity Analysis-style sparsity (require that Ax is sparse): x rec 1 = arg min Kx x 2 y 2 + λ Ax 1 (1) Primary example of (1) is total variation (TV) penalty: A =grad Synthesis-style sparsity (express x = Su with u sparse): u rec 1 = arg min KSu u 2 y 2 + λ u 1 and x rec = Su rec (2) Example of (2): sparse combination of wavelets (2) is a special case of (1): A 1 and K KS Variational equations are nonlinear but problem is convex: convex minimization techniques can be applied

27 Cost functions for analysis and synthesis sparsity Analysis-style sparsity (require that Ax is sparse): x rec 1 = arg min Kx x 2 y 2 + λ Ax 1 (1) Primary example of (1) is total variation (TV) penalty: A =grad Synthesis-style sparsity (express x = Su with u sparse): u rec 1 = arg min KSu u 2 y 2 + λ u 1 and x rec = Su rec (2) Example of (2): sparse combination of wavelets (2) is a special case of (1): A 1 and K KS Variational equations are nonlinear but problem is convex: convex minimization techniques can be applied

28 Cost functions for analysis and synthesis sparsity Analysis-style sparsity (require that Ax is sparse): x rec 1 = arg min Kx x 2 y 2 + λ Ax 1 (1) Primary example of (1) is total variation (TV) penalty: A =grad Synthesis-style sparsity (express x = Su with u sparse): u rec 1 = arg min KSu u 2 y 2 + λ u 1 and x rec = Su rec (2) Example of (2): sparse combination of wavelets (2) is a special case of (1): A 1 and K KS Variational equations are nonlinear but problem is convex: convex minimization techniques can be applied

29 Cost functions for analysis and synthesis sparsity Analysis-style sparsity (require that Ax is sparse): x rec 1 = arg min Kx x 2 y 2 + λ Ax 1 (1) Primary example of (1) is total variation (TV) penalty: A =grad Synthesis-style sparsity (express x = Su with u sparse): u rec 1 = arg min KSu u 2 y 2 + λ u 1 and x rec = Su rec (2) Example of (2): sparse combination of wavelets (2) is a special case of (1): A 1 and K KS Variational equations are nonlinear but problem is convex: convex minimization techniques can be applied

30 Cost functions for analysis and synthesis sparsity Analysis-style sparsity (require that Ax is sparse): x rec 1 = arg min Kx x 2 y 2 + λ Ax 1 (1) Primary example of (1) is total variation (TV) penalty: A =grad Synthesis-style sparsity (express x = Su with u sparse): u rec 1 = arg min KSu u 2 y 2 + λ u 1 and x rec = Su rec (2) Example of (2): sparse combination of wavelets (2) is a special case of (1): A 1 and K KS Variational equations are nonlinear but problem is convex: convex minimization techniques can be applied

31 Algorithm for analysis style sparsity To solve: 1 arg min Kx x 2 y 2 + λ Ax 1 Explicit iterative algorithm: x n+1 = x n + K T (y Kx n ) A T w n w n+1 ( = P λ w n + A x n+1) x n+1 = x n + K T (y Kx n ) A T w n+1, λ u u > λ with P λ (u) = u (component-wise) u u λ Explicit: uses only K, K T, A and A T at every step, No equation solving at every step No non-trivial sub-problem at every step No smoothing of u 1 with small ǫ: u i ui 2 + ǫ 2 I. Loris and C. Verhoeven (2011).

32 Algorithm for synthesis style sparsity When A = 1 then the previous algorithm reduces to iterative soft-thresholding : ) x n+1 = S λ (x n + K T (y Kx n 1 ) for arg min Kx x 2 y 2 + λ x 1 S λ (x) with S λ = soft-thresholding: Advantages: 1 very simple 2 F(x n ) F(x ) x 0 x 2 3 Can be accelerated 2n λ I. Daubechies et al. (2004). DOI: /cpa λ x n > 0

33 Example: Total variation penalty Total Variation: A =grad 1 arg min Kx x 2 y 2 + λ grad(x) 1 grad(x) is sparse grad(x) is piece-wise zero x is piece-wise constant

34 Toy TV regularization problem Seismic surface wave tomography (2D): T = receiver source (ray approximation) ds c receiver perturbation on c 1 δc δt = ds source c b c b y = Kx This example: 8490 data for unknowns. Sum of rows of K Min=0 Max= Sum of rows of K

35 (10 5 iterations of Generalized IST, 10% noise on data) Synthetic model and reference TV reconstruction Input noisy data Reconstruction (A =grad) Min= 1 Max= grad(input) Min=0 Max=

36 (10 5 iterations of Generalized IST, 10% noise on data) Synthetic model and reference TV reconstruction Input noisy data Reconstruction (A =grad) Min= 1 Max= grad(input) Min= Max= grad(reconstruction) Min=0 Max= Min=2.7207e 014 Max=

37 Alternatives for Total Variation (1) TV will impose piece-wise constant reconstruction Good: sharp edge will be preserved Bad: Smooth transition replaced by succession of sharp edges ( staircasing effect ) Improvements: generalized TV, Huber prior,... Other possibility: Try synthesis sparsity with an appropriate basis: express x = Su with u sparse S = basis or frame that can represent an overall smooth model with some sharp edges in a sparse manner S may be overcomplete

38 Alternatives for Total Variation (1) For synthesis style sparsity, faster algorithms exist Fast Iterative Soft-thresholding algorithm: with u n+1 = T(u n + β n (u n u n 1 )) 1 T(u) S λ [ u + S T K T (y KSu) ], 2 β n = n 1 n+2, 3 S λ = soft-thresholding 4 F(x n ) F(x ) 4 x 0 x 2 Sλ(u) λ (n + 1) 2 n > 0 A. Beck and M. Teboule (2009). DOI: / Yu E. Nesterov, Soviet Math. Dokl. 27 (1983) λ u

39 Application: Seismic tomography of Mantle Model parametrization is key to success of sparse recovery: Spherical harmonics well-adapted to spherical geometry good spectral localization poor spatial localization sparse spherical harmonic parametrization very smooth model Wavelet basis Constructed by local averaging/differencing and by subsampling (iterated filter bank) In other words: based on translating and resizing (scaling) of a single function Good compromise between spectral and spatial localization Sparse in wavelet basis = some discontinuities (edges) allowed Fast algorithms exist (O(N) for length N signal) Not well-adapted to spherical geometry Wavelet basis is a good candidate for setting-up a sparse regularization method for seismic tomography

40 Application: Seismic tomography of Mantle Model parametrization is key to success of sparse recovery: Spherical harmonics well-adapted to spherical geometry good spectral localization poor spatial localization sparse spherical harmonic parametrization very smooth model Wavelet basis Constructed by local averaging/differencing and by subsampling (iterated filter bank) In other words: based on translating and resizing (scaling) of a single function Good compromise between spectral and spatial localization Sparse in wavelet basis = some discontinuities (edges) allowed Fast algorithms exist (O(N) for length N signal) Not well-adapted to spherical geometry Wavelet basis is a good candidate for setting-up a sparse regularization method for seismic tomography

41 Application: Seismic tomography of Mantle Model parametrization is key to success of sparse recovery: Spherical harmonics well-adapted to spherical geometry good spectral localization poor spatial localization sparse spherical harmonic parametrization very smooth model Wavelet basis Constructed by local averaging/differencing and by subsampling (iterated filter bank) In other words: based on translating and resizing (scaling) of a single function Good compromise between spectral and spatial localization Sparse in wavelet basis = some discontinuities (edges) allowed Fast algorithms exist (O(N) for length N signal) Not well-adapted to spherical geometry Wavelet basis is a good candidate for setting-up a sparse regularization method for seismic tomography

42 Application: Seismic tomography of Mantle Model parametrization is key to success of sparse recovery: Spherical harmonics well-adapted to spherical geometry good spectral localization poor spatial localization sparse spherical harmonic parametrization very smooth model Wavelet basis Constructed by local averaging/differencing and by subsampling (iterated filter bank) In other words: based on translating and resizing (scaling) of a single function Good compromise between spectral and spatial localization Sparse in wavelet basis = some discontinuities (edges) allowed Fast algorithms exist (O(N) for length N signal) Not well-adapted to spherical geometry Wavelet basis is a good candidate for setting-up a sparse regularization method for seismic tomography

43 Wavelets for seismic tomography of Mantle Many wavelet families exist These will yield different reconstructions Important: the ability to easily switch between different wavelet bases Rather not use a custom-designed wavelet basis for the sphere Preference goes to using cartesian geometry (allows for off-the-shelve wavelets but also curvelets, shearlets etc) Needs a special parametrization of the sphere: cubed sphere (Ronchi et al, 1996)

44 Wavelets for seismic tomography of Mantle Many wavelet families exist These will yield different reconstructions Important: the ability to easily switch between different wavelet bases Rather not use a custom-designed wavelet basis for the sphere Preference goes to using cartesian geometry (allows for off-the-shelve wavelets but also curvelets, shearlets etc) Needs a special parametrization of the sphere: cubed sphere (Ronchi et al, 1996)

45 Wavelets for seismic tomography of Mantle Many wavelet families exist These will yield different reconstructions Important: the ability to easily switch between different wavelet bases Rather not use a custom-designed wavelet basis for the sphere Preference goes to using cartesian geometry (allows for off-the-shelve wavelets but also curvelets, shearlets etc) Needs a special parametrization of the sphere: cubed sphere (Ronchi et al, 1996)

46 The cubed sphere Sphere is divided into six identical regions (by projecting the sides of a circumscribed cube) All coordinate lines: arcs of great circles 4 subdivisions 8 subdivisions 16 subdivisions In radial dimension: simple subdivisions are used Unwraps to 6 Cartesian chunks with tolerable area distortion

47 The cubed sphere unfolded: coastline Min=0 Max=

48 The cubed sphere unfolded: topography

49 Wavelets on the cubed sphere (CDF 4-2) Min= Max= NB: Basis functions can cross edges smoothly

50 Wavelets on the cubed sphere (CDF 4-2) Min= Max=

51 Spherical harmonics Min= 1 Max=

52 Why wavelets for sparse seismic tomography? A model that is sparse in a wavelet basis is generally smooth but can have some sharp discontinuities Because wavelets are localized in space and scale, sharp discontinuities do not affect all wavelet coefficients Illustration: Developing an existing model and throwing away most of the coefficients still allows for a faithful model

53 An existing Earth model model: 100% nonzero coefficients, err=0%. Min= 1 Max= (Montelli, 2004)

54 An existing Earth model: sparsified Compressed model: 80% zero coefficients, error=3%. Min= Max= (Montelli, 2004)

55 An existing Earth model: sparsified Compressed model: 90% zero coefficients, error=5%. Min= Max= (Montelli, 2004)

56 An existing Earth model: sparsified Compressed model: 95% zero coefficients, error=10%. Min= Max= (Montelli, 2004)

57 An existing Earth model Los Angeles on top. montelli from129.mat Depth: 6371km n=(34 o N, 118 o E) x =20km y =20km (Montelli, 2004)

58 An existing Earth model: sparsified Los Angeles on top. Spars.: 20%, Err=1.967% Depth: 6371km n=(34 o N, 118 o E) x =20km y =20km (Montelli, 2004)

59 An existing Earth model: sparsified Los Angeles on top. Spars.: 10%, Err=3.8699% Depth: 6371km n=(34 o N, 118 o E) x =20km y =20km (Montelli, 2004)

60 An existing Earth model: sparsified Los Angeles on top. Spars.: 5%, Err=6.5745% Depth: 6371km n=(34 o N, 118 o E) x =20km y =20km (Montelli, 2004)

61 An existing Earth model: sparsified Los Angeles on top. Spars.: 2%, Err= % Depth: 6371km n=(34 o N, 118 o E) x =20km y =20km (Montelli, 2004)

62 Opportunity: Exploit model sparsity in reconstruction An accurate model can be obtained using few nonzero wavelet coefficients The location of these few nonzero coefficients is not, and does not have to be specified in advance As wavelets are localized in space, this is an automatic procedure of allowing more degrees of freedom where the data calls for it In contrast: With a standard spherical harmonics based reconstruction, an a priori choice is made of nonzero spherical harmonic coefficients (namely does belonging to the lowest orders)

63 Handling chunk boundaries (surface wave toy problem): Importance of using wavelet functions that cross chunk edges smoothly: Synthetic input model Ray density Min= 1 Min=0 Max=1595 Max= Synthetic output model (incorrect handling of boundaries) Synthetic output model (correct handling of boundaries) Min= Min= Max= Max=

64 Two pillars 1 Expected sparsity of the model: Seismic velocity anomalies can be sparsely described by wavelets Fast algorithms exist for wavelet transforms 2 Sparse recovery algorithms exist: l 1 penalty for recovering sparse solutions to linear equations Functional to be minimized: where Kx y 2 + 2λ u 1 1 x = Earth model 2 K = matrix containing sensitivity kernels 3 y = data 4 u = wavelet coefficients of x; so that x = Su 5 S = suitable basis or frame Together: better reconstructions than usual smoothing regularization

65 Conclusion Sparse reconstruction useful for seismic tomography (model is sparse in wavelet basis) Good parametrization essential Use l 1 penalty on wavelet coefficients to promote sparsity in wavelet basis Now: parallel implementation and inverting real data More: igloris I. Loris and C. Verhoeven (2011). F. J. Simons, I. Loris et al. (2011).