MULTIDIMENSIONAL SIGNAL, IMAGE, AND VIDEO PROCESSING AND CODING

Transcription

1 MULTIDIMENSIONAL SIGNAL, IMAGE, AND VIDEO PROCESSING AND CODING JOHN W. WOODS Rensselaer Polytechnic Institute Troy, New York»iBllfllfiii.. i. ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Academic Press is an imprint of Elsevier

2 CONTENTS Preface xiii Acknowledgments xvii v 1 TWO-DIMENSIONAL SiGNALS AND SYSTEMS Two-Dimensional Signals Separable Signals Periodic signals D Discrete-Space Systems Two-Dimensional Convolution Stability of 2-D Systems D Discrete-Space Fourier Transform Inverse 2-D Fourier Transform Fourier Transform of 2-D or Spatial Convolution Symmetry Properties of Fourier Transform Continuous-Space Fourier Transform Conclusions Problems 31 References 33 2 SAMPLING IN TWO DIMENSIONS Sampling Theorem Rectangular Case Reconstruction Formula Ideal Rectangular Sampling Sampling Theorem General Regulär Case Hexagonal Reconstruction Formula Change of Sample Rate Downsampling by Integers Mi x M Ideal Decimation Upsampling by Integers L\ x Lj Ideal Interpolation Sample-Rate Change General Case General Downsampling 64

3 VI CONTENTS 2.5 Conclusions Problems 66 References 70 3 TwO-DiMENSIONAL SYSTEMS AND Z-TRANSFORMS Linear Spatial or 2-D Systems Z-Transforms Regions of Convergence More General Case Some Z-Transform Properties Linear Mapping of Variables Inverse Z-Transform D Filter Stability First-Quadrant Support Second-Quadrant Support Root Maps Stability Criteria for NSHP Support Filters Conclusions Problems 101 References TwO-DiMENSIONAL DiSCRETE TRANSFORMS Discrete Fourier Series Properties of the DFS Transform Periodic Convolution Shifting or Delay Property Discrete Fourier Transform DFT Properties Relation of DFT to Fourier Transform Effect of Sampling in Frequency Interpolating the DFT D Discrete Cosine Transform Review of 1-D DCT Some 1-D DCT Properties Symmetrie Extension in 2-D DCT Subband/Wavelet Transform (SWT) Ideal Filter Case D SWT with Finite-Order Filter D SWT with FIR Filters Relation of SWT to DCT Relation of SWT to Wavelets 138

4 CONTENTS vi/ 4.5 Fast Transform Algorithms Fast DFT Algorithm Fast DCT Methods Sectioned Convolution Methods Conclusions Problems 144 References TWO-DIMENSIONAL FiLTER DESIGN FIR Filter Design FIR Window Function Design Design by Transformation of 1-D Filter Projection-Onto-Convex-Sets Method HR Filter Design D Recursive Filter Design Fully Recursive Filter Design Subband/Wavelet Filter Design Wavelet (Biorthogonal) Filter Design Method Conclusions Problems 182 References INTRODUCTORY IMAGE PROCESSING Light and Luminance Still Image Visual Properties Weber's Law Contrast Sensitivity Function Local Contrast Adaptation Time-Variant Human Visual System Properties Image Sensors Electronic Film Image and Video Display Gamma Simple Image Processing Filters Box Filter Gaussian Filter Prewitt Operator Sobel Operator Laplacian Filter 209

5 vüi CONTENTS 6.7 Conclusions Problems 211 References IMAGE ESTIMATION AND RESTORATION D Random Fields Filtering a 2-D Random Field Autoregressive Random Signal Models Estimation for Random Fields Infinite Observation Domain D Recursive Estimation D Kaiman Filter D Kaiman Filtering Reduced Update Kaiman Filter Approximate RUKF Steady-State RUKF LSI Estimation and Restoration Examples with RUKF Inhomogeneous Gaussian Estimation Inhomogeneous Estimation with RUKF Estimation in the Subband/Wavelet Domain Bayesian and MAP Estimation Gauss Markov Image Models Simulated Annealing Image Identification and Restoration Expectation-Maximization Algorithm Approach EM Method in the Subband/Wavelet Domain Color Image Processing Conclusions Problems 263 References DIGITAL IMAGE COMPRESSION Introduction Transformation DCT SWT DPCM Quantization Uniform Quantization Optimal MSE Quantization 278

6 CONTENTS ix Vector Quantization LBG Algorithm [7] Entropy Coding Huffman Coding Arithmetic Coding ECSQandECVQ DCT Coder SWT Coder Multiresolution SWT Coding Nondyadic SWT Decompositions Fully Embedded SWT Coders Embedded Zero-Tree Wavelet (EZW) Coder Set Partitioning in Hierarchical Trees (SPIHT) Coder Embedded Zero Block Coder (EZBC) JPEG Color Image Coding Scalable Coder Results Comparison Robustness Considerations Conclusions Problems 312 References THREE-DIMENSIONAL AND SPATIOTEMPORAL PROCESSING D Signals and Systems Properties of 3-D Fourier Transform D Filters D Sampling and Reconstruction General 3-D Sampling Spatiotemporal Signal Processing Spatiotemporal Sampling Spatiotemporal Filters Intraframe Filtering Intraframe Wiener Filter Interframe Filtering Interframe Wiener Filter Spatiotemporal Markov Models Causal and Semicausal 3-D Field Sequences Reduced Update Spatiotemporal Kaiman Filter Conclusions Problems 338 References 339

7 x CONTENTS 10 DIGITAL VIDEO PROCESSING Interframe Processing Motion Estimation and Motion Compensation Block Matching Method Hierarchical Block Matching Overlapped Block Motion Compensation Pel-Recursive Motion Estimation Optical flow methods Motion-Compensated Filtering MC-Wiener Filter MC-Kalman Filter Frame-Rate Conversion Deinterlacing Bayesian Method for Estimating Motion Joint Motion Estimation and Segmentation Conclusions Problems 378 References Appendix: Digital Video Formats 380 SIF 381 CIF 381 ITU 601 Digital TV (aka SMPTE Dl and D5) 381 ATSC Formats DIGITAL VIDEO COMPRESSION Intraframe Coding M-JPEG Pseudo Algorithm DV Codec Intraframe SWT Coding M-JPEG Interframe Coding Generalizing 1-D DPCM to Interframe Coding MC Spatiotemporal Prediction Interframe Coding Standards MPEG MPEG 2 "a Generic Standard" The Missing MPEG 3 High-Definition Television MPEG 4 Natural and Synthetic Combined Video Processing of MPEG-Coded Bitstreams H.263 Coder for Visual Conferencing H.264/AVC Video Coder Mode Control Network Adaptation 410

8 CONTENTS XI 11.4 Interframe SWT Coders Motion-Compensated SWT Hybrid Coding D or Spatiotemporal Transform Coding Scalable Video Coders MoreonMCTF Detection of Covered Pixels Bidirectional MCTF Obj ect-based Video Coding Comments on the Sensitivity of Compressed Video Conclusions Problems 430 References VIDEO TRANSMISSION OVER NETWORKS Video on IP Networks Overview of IP Networks Error-Resilient Coding Transport-Level Error Control Wireless Networks Joint Source-Channel Coding Error Concealment Robust SWT Video Coding (Bajic) Dispersive Packetization Multiple Description FEC Error-Resilience Features of H.264/AVC Syntax Data Partitioning Slice Interleaving and Flexible Macroblock Ordering Switching Frames Reference Frame Selection Intrablock Refreshing Error Concealment in H.264/AVC Joint Source-Network Coding Digital Item Adaptation (DIA) in MPEG Fine-Grain Adaptive FEC Conclusions Problems 469 References 471 Index 477