Wavelets Made Easy
Yves Nievergelt Wavelets Made Easy Springer Science+Business Media, LLC
Yves Nievergelt Department of Mathematics Eastem Washington University Cheney, WA 99004-2431 USA Library of Congress Cataloging-in-Publieation Data Nievergelt, Yves. Wavelets made easy / Yves Nievergelt. p.em. Includes bibliographieal references and index. ISBN 978-1-4612-6823-9 3-7643-4061-4 (acid-free paper) 1. Wavelets (Mathematics) 1. Title. QA403.3.N54 1999 515'.2433-dc2! 98-29994 CIP AMS Subject Classifications: 42 Printed on acid-free paper 1999 Springer Seience+Business Media New York Originally published by Birkhăuser Boston in 1999 Softeover reprint of the hardeover! st edition 1999 2nd printing with eorreetions cr» Birkhiiuser fl&j All rights reserved. This work may not be translated or copied in whole or in part without the written permission ofthe publisher (Springer Scienee+Business Media, LLC), except for brief excerpts in eonnection with reviews or scholarly analysis. Vse in eonnection with any form of inforrnation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may aceordingly be used freely byanyone. ISBN 978-1-4612-6823-9 ISBN 978-1-4612-0573-9 (ebook) SPIN 10787109 DOI 10.1007/978-1-4613-0573-9 Formatted from the author's TEX files by Integre Technical Publishiog Company, loc., Albuquerque, NM. 987 654 3 2
Contents Preface Outline ix xi A 1 Algorithms for Wavelet Transforms Haar's Simple Wavelets 1.0 Introduction... 1.1 Simple Approximation........ 1.2 Approximation with Simple Wavelets 1.2.1 The Basic Haar Wavelet Transform 1.2.2 1.2.3 Significance of the Basic Haar Wavelet Transfonn Shifts and Dilations of the Basic Haar Transfonn 1.3 The Ordered Fast Haar Wavelet Transform..... 1.3.1 Initialization.... 1.3.2 The Ordered Fast Haar Wavelet Transform 1.4 The In-Place Fast Haar Wavelet Transfoffil..... 1.4.1 In-Place Basic Sweep.... 1.4.2 The In-Place Fast Haar Wavelet Transform 1.5 The In-Place Fast Inverse Haar Wavelet Transfonn 1.6 Examples.... 1.6.1 Creek Water Temperature Analysis.. 1.6.2 Financial Stock Index Event Detection. 1 3 3 4 8 8 10 II 14 14 15 21 22 23 28 31 31 33 2 Multidimensional Wavelets and Applications 36 2.0 Introduction... 36 2.1 Two-Dimensional Haar Wavelets................ 37 2.1. I Two-Dimensional Approximation with Step Functions 37 2.1.2 Tensor Products of Functions.............. 39 2.1.3 The Basic Two-Dimensional Haar Wavelet Transfonn 42 2.1.4 Two-Dimensional Fast Haar Wavelet Transfonn. 46 2.2 Applications of Wavelets 49 2.2.1 Noise Reduction................. 49 v
vi Contents 2.2.2 Data Compression 2.2.3 Edge Detection.. 2.3 Computational Notes... 2.3.1 Fast Reconstruction of Single Values 2.3.2 Operation Count.... 2.4 Examples................... 2.4.1 Creek Water Temperature Compression 2.4.2 Financial Stock Index Image Compression 2.4.3 Two-Dimensional Diffusion Analysis. 2.4.4 Three-Dimensional Diffusion Analysis 52 58 60 60 63 65 65 67 68 69 3 Algorithms for Daubechies Wavelets 73 3.0 Introduction... 73 3.1 Calculation of Daubechies Wavelets.......... 73 3.2 Approximation of Samples with Daubechies Wavelets. 82 3.2.1 Approximate Interpolation. 83 3.2.2 Approximate Averages............. 84 3.3 Extensions to Alleviate Edge Effects.......... 85 3.3.1 Zigzag Edge Effects from Extensions by Zeros 85 3.3.2 Medium Edge Effects from Mirror Reflections 88 3.3.3 Small Edge Effects from Smooth Periodic Extensions. 90 3.4 The Fast Daubechies Wavelet Transform...... 95 3.5 The Fast Inverse Daubechies Wavelet Transform.. 101 3.6 Multidimensional Daubechies Wavelet Transforms 107 3.7 Examples...................... 110 3.7.1 Hangman Creek Water Temperature Analysis 110 3.7.2 Financial Stock Index Image Compression. 112 B Basic Fourier Analysis 115 4 Inner Products and Orthogonal Projections 117 4.0 Introduction... 117 4.1 Linear Spaces..... 117 4.1.1 Number Fields 117 4.1.2 Linear Spaces. 120 4.1.3 Linear Maps. 122 4.2 Projections... 123 4.2.1 Inner Products 124 4.2.2 Gram-Schmidt Orthogonalization 129 4.2.3 Orthogonal Projections...... 131 4.3 Applications of Orthogonal Projections. 134 4.3.1 Application to Three-Dimensional Computer Graphics 134 4.3.2 Application to Ordinary Least-Squares Regression.. 136
Contents vii 4.3.3 Application to the Computation of Functions 4.3.4 Applications to Wavelets.... 138 142 5 Discrete and Fast Fourier Transforms 147 5.0 Introduction... 147 5.1 The Discrete Fourier Transform (DFT). 147 5.1.1 Definition and Inversion. 148 5.1.2 Unitary Operators........ 155 5.2 The Fast Fourier Transform (FFT)... 157 5.2.1 The Forward Fast Fourier Transform. 157 5.2.2 The Inverse Fast Fourier Transform. 161 5.2.3 Interpolation by the Inverse Fast Fourier Transform. 161 5.2.4 Bit Reversal...................... 163 5.3 Applications of the Fast Fourier Transform.......... 165 5.3.1 Noise Reduction Through the Fast Fourier Transform. 165 5.3.2 Convolution and Fast Multiplication....... 167 5.4 Multidimensional Discrete and Fast Fourier Transforms. 171 6 Fourier Series for Periodic Functions 175 6.0 Introduction... 175 6.1 Fourier Series....................... 176 6.1.1 Orthonormal Complex Trigonometric Functions 176 6.1.2 Definition and Examples of Fourier Series.... 177 6.1.3 Relation Between Series and Discrete Transforms. 182 6.1.4 Multidimensional Fourier Series... 183 6.2 Convergence and Inversion of Fourier Series. 185 6.2.1 The Gibbs-Wilbraham Phenomenon. 185 6.2.2 Piecewise Continuous Functions... 187 6.2.3 Convergence and Inversion of Fourier Series 191 6.2.4 Convolutions and Dirac's "Function" 0 192 6.2.5 Uniform Convergence of Fourier Series 194 6.3 Periodic Functions.... 200 C Computation and Design of Wavelets 203 7 Fourier Transforms on the Line and in Space 205 7.0 Introduction.....205 7.1 The Fourier Transform.................... 205 7.1.1 Definition and Examples of the Fourier Transform. 205 7.2 Convolutions and Inversion of the Fourier Transform. 209 7.3 Approximate Identities..... 213 7.3.1 Weight Functions..... 214 7.3.2 Approximate Identities.. 215 7.3.3 Dirac Delta (0) Function. 219
viii 7.4 Further Features of the Fourier Transform.... 7 A.1 Algebraic Features of the Fourier Transform 7A.2 Metric Features of the Fourier Transform. 7A.3 Uniform Continuity of Fourier Transforms 7.5 The Fourier Transform with Several Variables 7.6 Applications of Fourier Analysis...... 7.6.1 Shannon's Sampling Theorem... 7.6.2 Heisenberg's Uncertainty Principle Contents.220. 221.223.227.229.234.234.236 8 Daubechies Wavelets Design 238 8.0 Introduction.....238 8.1 Existence, Uniqueness, and Construction..... 238 8.1.1 The Recursion Operator and Its Adjoint. 239 8.1.2 The Fourier Transform of the Recursion Operator.. 243 8.1.3 Convergence of Iterations of the Recursion Operator. 245 8.2 Orthogonality of Daubechies Wavelets. 253 8.3 MaHafs Fast Wavelet Algorithm................ 258 9 Signal Representations with Wavelets 9.0 Introduction.... 9.1 Computational Features of Daubechies Wavelets.... 9.1.1 Initial Values of Daubechies' Scaling Function 9.1.2 Computational Features of Daubechies' Function 9.1.3 Exact Representation of Pol ynomials by Wavelets. 9.2 Accuracy of Signal Approximation by Wavelets.... 9.2.1 Accuracy of Taylor Polynomials.... 9.2.2 Accuracy of Signal Representations by Wavelets. 9.2.3 Approximate Interpolation by Daubechies' Function 262.262.262.263.266.273.274.274.278. 281 D Directories 285 Acknowledgments Collection of Symbols Bibliography Index 287 289 291 295
Preface This book explains the nature and computation of mathematical wavelets, which provide a framework and methods for the analysis and the synthesis of signals, images, and other arrays of data. The material presented here addresses the audience of engineers, financiers, scientists, and students looking for explanations of wavelets at the undergraduate level. It requires only a working knowledge or memories of a first course in linear algebra and calculus. The first part of the book answers the following two questions: What are wavelets? Wavelets extend Fourier analysis. How are wavelets computed? Fast transforms compute them. To show the practical significance of wavelets, the book also provides transitions into several applications: analysis (detection of crashes, edges, or other events), compression (reduction of storage), smoothing (attenuation of noise), and synthesis (reconstruction after compression or other modification). Such applications include one-dimensional signals (sounds or other time-series), two-dimensional arrays (pictures or maps), and three-dimensional data (spatial diffusion). The applications demonstrated here do not constitute recipes for real implementations, but aim only at clarifying and strengthening the understanding of the mathematics of wavelets. The second part of the book explains orthogonal projections, discrete and fast Fourier transforms, and Fourier series, as a preparation for the third part and as an answer to the following question: How are wavelets related to other methods of signal analysis? The third part of the book invokes occasional results from advanced calculus and focuses on the following question, which provides a transition into the theory and research on the subject: How are wavelets designed? (Designs use Fourier transforms.) More details appear in the chapter summaries on the following page. The material has been taught in various forms for a decade in an undergraduate course at Eastern Washington University, to engineers and students majoring in mathematics or computer science. I thank them for their patience in reading through several drafts. Eastern Washington University Cheney, WA YVES NIEVERGELT ix
Outline Part A, which can be read before or after Part B, provides an immediate and very basic introduction to wavelets. Chapter 1 gives a first elementary yet rigorous explanation of the nature of mathematical wavelets, in particular, Alfred Haar's wavelets, without either calculus or linear algebra. Chapter 2 presents multidimensional wavelets and some applications, with one use of matrix algebra but without calculus. Chapter 3 introduces computational features of Ingrid Daubechies' wavelets with one and more dimensions, with some matrix algebra but without calculus. Chapter 3 also aims at justifying the need for some clarification of Daubechies wavelets through theory, which will be the subject of Parts Band C. Part B presents the mathematical context in which wavelets arose: Joseph Fourier's analysis of signals and functions. Chapter 4 reviews the topics from linear algebra that explain how wavelets approximate functions: linear spaces, inner products, norms, orthogonal projections, and least-squares regression. Chapter 5 focuses on the discrete fast Fourier transform of James W. Cooley and John W. Tukey, which provides a framework simpler than Daubechies wavelets to explain fast transforms. Chapter 6 treats Fourier series, which demonstrate least-squares approximations of functions within a framework simpler than, but similar to, that of Daubechies wavelets in Chapter 8. Part C explains Fourier transforms and their use in wavelet design. Chapter 7 presents the Fourier transform and its inverse on the real line, in the plane, and in space. This is the essential concept for the design and the mathematical foundations of wavelets. Chapter 8 explains how Ingrid Daubechies applied the Fourier Transform to design wavelets. The explanations also show how the Fourier Transform applies to the design of other wavelets. Chapter 9 shows how accurately wavelets can approximate signals. xi