On the Sparsity of Wavelet Coefficients for Signals on Graphs

Size: px

Start display at page:

Download "On the Sparsity of Wavelet Coefficients for Signals on Graphs"

Samuel Byrd
5 years ago
Views:

1 On the Sparsity of Wavelet Coefficients for Signals on Graphs Benjamin Ricaud, David Shuman, and Pierre Vandergheynst August 29, 23 SPIE Wavelets and Sparsity XV San Diego, CA

2 2 Signal Processing on Graphs Social Networks Energy Networks Biological Networks Transportation Networks Irregular Data Domains 2

3 3 Some Typical Processing Problems Compression / Visualization Denoising Semi-Supervised Learning Earth data source: Frederik Simons Analysis / Information Extraction 3

4 4 Simple Motivating Examples l Tikhonov regularization for denoising: argmin f f y f T Lf Original Noisy Denoised.8 4

5 4 Simple Motivating Examples l Tikhonov regularization for denoising: argmin f f y f T Lf l Original Noisy Denoised Wavelet denoising:.6.8 argmin a f W a a,µ Decay of wavelet coefficients

6 5 Research Questions l For signals on Euclidean data domains, we have results characterizing classes of signals that are well-approximated by different transforms - e.g., piecewise-smooth D signals by wavelets, 2D cartoons with curvilinear discontinuities by curvelets/shearlets l Which multiscale transforms for signals on graphs are wellsuited for which signal processing tasks, which classes of signals, and which types of graphs? l Connections between properties of graph signals and the decay of their wavelet coefficients? 5

7 6 Smoothness of Graph Signals G G 2 G 3 To identify and exploit structure in the data, we need to account for the intrinsic geometric structure of the underlying graph data domain 6

8 7 Global Regularity of Graph Signals 7

9 8 Notions of Global Regularity for Graph Signals Discrete Calculus, Grady and Polimeni, 2 m := p w(m, n)[f(n) f(m)] 8

10 8 Notions of Global Regularity for Graph Signals Discrete Calculus, Grady and Polimeni, 2 m := p w(m, n)[f(n) f(m)] # Graph Gradient O m f m e2e s.t. e=(m,n) Local Variation O m f 2 = " X n2n m w(m, n)[f(n) f(m)] 2 # 2 8

8 Notions of Global Regularity for Graph Signals Discrete Calculus, Grady and Polimeni, 2 Edge Derivative @f @e m := p w(m, n)[f(n) f(m)] " @f # Graph Gradient O m f := @e m

11 8 Notions of Global Regularity for Graph Signals Discrete Calculus, Grady and Polimeni, 2 m := p w(m, n)[f(n) f(m)] # Graph Gradient O m f m e2e s.t. e=(m,n) Local Variation O m f 2 = " X n2n m w(m, n)[f(n) f(m)] 2 # 2 Quadratic Form 2 X m2v O m f 2 2 = X (m,n)2e w(m, n)[f(n) f(m)] 2 = f T Lf f L := L 2 f 2 = p f T Lf 8

12 9 Smoothness of Graph Signals Revisited G G 2 G 3 ˆf ( λ ) ˆf ( λ ) ˆf ( λ ) λ λ λ f T L f =.4 f T L 2 f =.3 f T L 3 f =.8 Example (Importance of the underlying graph): 9

13 Notions of Global Regularity for Graph Signals Generalizations p-dirichlet Form (Elmoataz et al., 28) p X m2v O m f p 2 = p X m2v " X n2n m w(m, n)[f(n) f(m)] 2 # p 2 Discrete Sobolev Semi-Norm kfk H p := kl p fk 2 = k d L p fk 2 = s X ` ` 2p ˆf(`) 2 In the continuous setting, the space W p (R) of p-times differentiable Sobolev functions are those satisfying Z In the graph setting,! 2p ˆf(!) 2 d! < kfk H p f 2 apple p max for all f 2 R N Mallat, 28, pp

14 Wavelet Coefficient Decay of Globally Regular Graph Signals

15 2 Spectral Graph Wavelets Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 l Generalized translation Classical setting: Graph setting: (T s g)(t) =g(t s) = (T n g)(i) := NX `= Z R ĝ( `)u ` (n)u`(i) ĝ( )e 2 i s e 2 i t d 2

16 2 Spectral Graph Wavelets Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 l Generalized translation Classical setting: Graph setting: (T s g)(t) =g(t s) = (T n g)(i) := NX `= Z R ĝ( `)u ` (n)u`(i) ĝ( )e 2 i s e 2 i t d 2

17 2 Spectral Graph Wavelets Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 l Generalized translation Classical setting: Graph setting: (T s g)(t) =g(t s) = (T n g)(i) := NX `= Z R ĝ( `)u ` (n)u`(i) ĝ( )e 2 i s e 2 i t d 2

18 2 Spectral Graph Wavelets Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 l Generalized translation Classical setting: Graph setting: (T s g)(t) =g(t s) = (T n g)(i) := NX `= Z R ĝ( `)u ` (n)u`(i) ĝ( )e 2 i s e 2 i t d 2

19 3 Spectral Graph Wavelets Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 l Generalized translation Classical setting: Graph setting: (T s g)(t) =g(t s) = (T n g)(i) := NX `= Z R ĝ( `)u ` (n)u`(i) ĝ( )e 2 i s e 2 i t d l Generalized dilation: dd s g( )=ĝ(s ).5 ĝ(s )

20 Further Reading Gaussian noise, and wish to recover f. To enforce a priori informati the underlying graph, we include a regularization term of the form problem 4 Tutorial Overviews argmin kf Spectral Graph Wavelets f yk22 + f D. I Shuman, S. K. Narang, P. Frossard, A. Ortega, P. Vandergheynst, Signal processing on graphs: The first-order optimality conditions of the convex objective function Extending high-dimensional data analysis to networks and other irregular domains, Signal Process. Mag., [?, Proposition ]) the optimal reconstruction is given by to appear May 23. Spectral Graph Theory Generalized Operators N XTransforms R. Rubinstein, A. M. Bruckstein, and M. Elad, Dictionaries for sparse representation modeling, Proc. f (i) = y + ` `= IEEE, vol. 98, no. 6, pp , Jun. 2. Hammond et al., Wavelets on graphs via spectral graph theory, ACHA, 2 Example: Image Denoising by Lo or, equivalently, f = h (L)y, where h ( ) := can be viewed as Generalized translation Spectral Graph Theory and Graph Laplacian Eigenvectors Z l + As an example, in the figure below, we take the 52 x 52 cam Gaussian with mean zero and standard deviation. to get a 2 i snoise 2 i t Classical setting: F. K. Chung, Spectral Graph Theory, vol. 92 of the CBMS Reg. Conf. Ser. Math., AMS Bokstore, / 997. / / s methods the signal. In/ the fˆ( first GFT ` ) method, weg apply a symg f (n) to denoise size 72 x 72 with two different standard deviations:.5 and 3.5. In R T. Bıyıkog lu, J. Leydold, and P. F. Stadler, Laplacian of Graphs, Springer, 27. N Eigenvectors the pixels by connecting each pixel to its horizontal, vertical, and di (??) between neighboring pixels according to the similarity of D. Spielman, Spectral graph theory in Combinatorial Scientific Computing, Chapman Hall,two 22. Graph setting: n ` ` ` edgesand of the semi-local graph are independent of the noisy image, Tikh Semi-Local Graphpixel values in the noisy image. For the between the neighboring Ga Semi-Local Graph Dictionaries for Signals on Graphs `= We then perform the low-pass graph filtering (??) to reconstruct the anisotropic diffusion image smoothing method of [?]. kf argmin y k22 R. R. Coifman and M. Maggioni, Di usion wavelets, Appl. Comput. Harmon. Anal.,Invol. 2, no., pp. all image displays, we threshold the values to the [,] int f 53 94, Jul. 26. l zoomed-in versions of the top row of images. Comparing the results s smooth sufficiently in smoother areas of the =) image,g ( the classical G D. K. Hammond, P. Vandergheynst, and R. Gribonval, Wavelets on graphs via spectral graph theory, `) = The graph spectral filtering method does not smooth as much acros + Appl. Comput. Harmon. Anal., vol. 3, no. 2, pp. 29 5, Mar. 2. image is encoded in the graph Laplacian via the noisy image. (T g)(t) = g(t (T g)(i) := l X s) = g ( )e e d g ( )u (n)u (i) dg( ) = g (s ) Generalized dilation: D Spectral graph wavelet at scale s, S. K. Narang and A. Ortega, Perfect reconstruction two-channel wavelet filter banks for graph structured data, IEEE Trans. Process., centered at Signal vertex n:vol. 6, pp , Jun. 22. Original Image Noisy Image Gaussian-Filtered (Std. Dev. =.5) D. I Shuman, B. Ricaud, and P. Vandergheynst, A windowed graph Fourier transform, in Proc. IEEE Stat. =22. Signal Process. Wrkshp., Ann Arbor, MI, N Aug. X (i) := (Tn Ds g)(i) Signal = Processing g (son `Graphs )u ` (n)u` (i) David s,nshuman February, / 35 `= David Shuman Signal Processing on G 4

21 Wavelet Coefficient Decay of Globally Regular 5 Graph Signals Proposition Let p, and assume that C p := R ĝ(s) 2 /s 2p ds <. Then Z s 2p X n hf, s,n i 2 ds = C p f H (2p )/2. Proposition 2 Assume that ĝ( )= P q k=p a k k for some p (implyingĝ = ) Then qx f(s, n) = hf, s,n i apple a k s k f H k. k=p 5

22 6 Ongoing Work: Local Regularity and Wavelet Coefficient Decay of Locally Regular Graph Signals 6

23 7 Notions of Local Regularity Local Variation O m f 2 = " X n2n m w(m, n)[f(n) f(m)] 2 # 2 Hölder Regularity A graph signal f is (C,, r)-hölder regular with respect to the graph G at vertex n 2V if f(n) f(m) applec[d G (m, n)], 8m 2N(n, r) Gavish et al. ICML, 2 Laplacian as Derivative (L k f)(n) as a measure of local regularity of f in a neighborhood of radius k around vertex n For polynomial kernel: f(s, n) = qx k=p a k s k L k f (n) 7

24 8 Spectral Graph Wavelet Localization exact wavelet (naive forward transform) s=7.8 s2=6.7 exact wavelet (naive forward transform).5 gbj ( ).5 exact wavelet (naive forward transform).4 s3=2. s4=.5 s = s,n 2 3 exact wavelet (naive forward transform).6 Intro Signal Transforms s2,n Problem Spectral Graph Theory Further ReadingProblem Intro Signal Transforms Generalized Operators WGFT Scale Operators s3generalized,n WGFT Conclusion. Spectral Graph Theory 4 exact wavelet (naive forward transform). Conclusion Tutorial Overviews Further Reading D. I Shuman, S. K. Narang, P. Frossard, A. Ortega, P. Vandergheynst, Signal processing on graphs: s4,n Tutorial Overviews Extending high-dimensional data analysis to networks and other irregular domains, Signal Process. Mag., to appear May 23. D. I Shuman, S. et K. Narang, P. Frossard, on A. Ortega, P. Vandergheynst, Signal processing on graphs: Hammond Wavelets viaforspectral R. Rubinstein, A. M.al., Bruckstein, and M. Elad,graphs Dictionaries sparse representation modeling, Proc. Extending high-dimensional data analysis to networks and other irregular domains, Signal Process. Mag., graph 2 IEEE, vol.theory, 98, no. 6, pp , Jun. 2. to appear May 23. Characterizations of this localization Shuman eta. al., Vertex-frequency analysisforon graphs, 23modeling, Proc. R. Rubinstein, M. Bruckstein, and Graph M. Elad, Dictionaries sparse representation Spectral Graph Theory and Laplacian Eigenvectors s5,n IEEE, vol. 98, no. 6, pp , Jun. 2. F. K. Chung, Spectral Graph Theory, vol. 92 of the CBMS Reg. Conf. Ser. Math., AMS Bokstore, 997. EPFL Laplacian Signal Processing Laboratory (LTS2) Spectral Graph Theory and Graph Eigenvectors T. Bıyıkog lu, J. Leydold, and P. F. Stadler, Laplacian Eigenvectors of Graphs, Springer, 27. F. Chung, Spectral Spectral Graph 92 of the CBMS Reg. Conf. Ser. Math., AMS and Bokstore, 997. D. K. Spielman, graph Theory, theory vol. in Combinatorial Scientific Computing, Chapman Hall, 22. 8

25 Wavelet Coefficient Decay of Locally Regular Graph Signals 9 High-level intuition l Far away from vertex n, for small scales s, f(s, n) is small because is highly localized around n s,n l Close to vertex n, f(s, n) is small because f is locally regular Proposition 3 Assume that f is (C,, r)-hölder regular for some r, and let ĝ( )= P q k=r a k k for some coe cients {a k } k=r,r+,...,q. Then there exist constants C 2 and s such that for all s< s, wehave f(s, n) applecr X m2n (n,r) s,n (m) + C 2 s r+ X m/2n (n,r) f(m) f(n). 9

2 Outlook Generalized+ Operators+ Applica(ons+

Graph+Theory+ ++ Numerical+Linear+Algebra+ Signal+

Scalable+ Algorithms+ Application of graph signal

problems is in its infancy Theoretical connections between

26 2 Outlook Generalized+ Operators+ Applica(ons+ Computa(onal+ Harmonic+Analysis+ ++ Spectral+and+Algebraic+ Graph+Theory+ ++ Numerical+Linear+Algebra+ Signal+ Transforms+/+ Dic(onaries+ Theore(cal+ Underpinnings+ Scalable+ Algorithms+ Application of graph signal processing techniques to real science and engineering problems is in its infancy Theoretical connections between classes of graph signals, the underlying graph structure, and sparsity of transform coefficients 2

Gabor and Wavelet transforms for signals defined on graphs An invitation to signal processing on graphs

Gabor and Wavelet transforms for signals defined on graphs An invitation to signal processing on graphs Benjamin Ricaud Signal Processing Lab 2, EPFL Lausanne, Switzerland SPLab, Brno, 10/2013 B. Ricaud