VIDEO CODING. The Second Generation Approach

Transcription

1 VIDEO CODING The Second Generation Approach

2 VIDEO CODING The Second Generation Approach EDITED BY luis Torres Department of Signal Theory and Communications, Universitat Politecnica de Catalunya Barcelona, Spain and Murat Kunt Signal Processing Laboratory, Swiss Federal Institute of Technology Lausanne, Switzerland KLUWER ACADEMIC PUBLISHERS Boston/London/Dordrecht

3 A C.I.P. Catalogue record for this book is available from the Library of Congress ISBN-13: : / e-isbn-13: Published by Kluwer Academic Publishers, P.O. Box 17, 3300 AA Dordrecht, The Netherlands. Kluwer Academic Publishers incorporates the publishing programmes of D. Reidel, Martinus Nijhoff, Dr W. Junk and MTP Press. Sold and distributed in the U.S.A. and Canada by Kluwer Academic Publishers, 101 Philip Drive, Norwell, MA 02061, U.S.A. In all other countries, sold and distributed by Kluwer Academic Publishers Group, P.O. Box 322, 3300 AH Dordrecht, The Netherlands. Printed on acid-free paper All Rights Reserved 1996 Kluwer Academic Publishers Boston Softcover reprint of the hardcover 1 st edition 1996 No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the copyright owner.

4 CONTENTS Preface Acknow ledgments Contributors IX Xl xiii 1 2 SECOND GENERATION VIDEO CODING TECHNIQUES Luis Torres and Murat K unt 1 Introduction 2 The problems of the canonical representation of images 3 First generation video coding techniques 4 Second generation video coding techniques 5 Segmentation-based video coding results 6 MPEG-4 functionalities 7 Conclusions and new trends PIXEL-BASED VIDEO COMPRESSION SCHEMES Narciso Garcia, Fernando Jaureguizar and Jose Ignacio Ronda 1 Introduction 2 Fundamentals of pixel-based techniques 3 Video coding standards 4 Pixel-based schemes: Properties and possible improvements 5 Conclusions CODING-ORIENTED SEGMENTATION OF VIDEO SEQUENCES Ferran Marques, Montse Pardas and Philippe Salembier 1 Introduction 2 Problem statement 3 A general structure for segmentation v

5 VI VIDEO CODING 4 An implementation based on compound random fields 99 5 An implementation based on morphological tools Conclusions CODING OF PARTITION SEQUENCES Philippe Salembier, Ferran Marques and Antoni Gasull Introduction Problem statement Intra-frame coding Inter-frame coding Conclusions REGION-ORIENTED TEXTURE CODING Michael Gilge Introduction Data compression by approximation Orthogonal basis functions for arbitrarily shaped regions Region-oriented transform coding Region-oriented video coding Conclusions SEGMENTATION-BASED MOTION ESTIMATION FOR SECOND GENERATION VIDEO CODING TECHNIQUES Frederic Dufaux and Fabrice M oscheni Introduction Motion models Segmentation-based fully parametric motion estimation Spatio-temporal segmentation Tracking Conclusions FRACTAL-BASED IMAGE AND VIDEO CODING Mohammad Gharavi-Alkhansari and Thomas S. Huang Introduction 265

6 Contents Vll 2 Basic theory Implementations Conclusions MODEL-BASED VIDEO CODING Kiyoharu Aizawa Introduction: A dream in Model-based approaches to image coding D model-based coding of a person's face D model-based approach using 2-D deformable triangles for advanced motion compensation Hybrid of 3-D and 2-D model-based coding schemes Applications of 3-D model-based coding A PERCEPTUALLY MOTIVATED THREE-COMPONENT IMAGE MODEL Nariman Farvardin and Xiaonong Ran Introduction Psychovisual aspects of the human visual system Characterization of strong edges Extraction of strong edge information and the three-component image model Coding of primary component Coding of smooth and texture components Image coding systems Perceptual weighting of distortions Video coding using the three-component image model Conclusions EXTENDED SIGNAL-THEORETIC TECHNIQUES FOR VERY LOW BIT-RATE VIDEO CODING H aibo Li and Robert Forchheimer Introduction Motion information Visual perception 397

7 Vlll VIDEO CODING 4 Signal-dependent coding Spatio-temporal information Acknowledgments 425 Index 429

8 PREFACE In recent years there has been an increasing interest in Second Generation image and video coding techniques. These techniques, introduce new concepts from image analysis that greatly improve the performance of the coding schemes for very high compression. This interest has been further emphasized by the future MPEG-4 standard. Second Generation image and video coding techniques are the ensemble of approaches proposing new and more efficient image representations than the conventional canonical form. As a consequence, the human visual system becomes a fundamental part of the encoding/decoding chain. More insight to distinguish between First and Second Generation can be gained if it is noticed that image and video coding is basically carried out in two steps. First, image data are converted into a sequence of messages and, second, code words are assigned to the messages. Methods of the First Generation put the emphasis on the second step, whereas methods of the Second Generation put it on the first step and use available results for the second step. As a result of including the human visual system, Second Generation can be also seen as an approach of seeing the image composed by different entities called objects. This implies that the image or sequence of images have first to be analyzed and/or segmented in order to find the entities. It is in this context that we have selected in this book three main approaches as Second Generation video coding techniques: Segmentation-based schemes Model-based schemes Fractal-based schemes We have compiled the necessary information to further facilitate developments in this exciting field. The first chapter provides an introduction to the subsequent technical chapters. Furthermore, a self-contained chapter is included which covers the basic concepts of image and video coding and describes the existing standards. This will assist the reader in understanding the evolution from First Generation to Second Generation schemes. ix

9 x We hope you enjoy reading this book as much as we did writing and preparing it. Luis Torres, Barcelona Murat K unt, Lausanne November 1995

10 ACKNOWLEDGEMENTS We are indebted to many people who have contributed one way or the other to this book. In the first place we want to thank all the authors of the chapters for their outstanding work, and for suffering the red pencil of the often too tough and picky editors. We also believe that the ideas contained in this book are the fruits of the work of many people, too many to mention here, who in recent years have contributed to many journal papers and conferences to disseminate new image and video coding techniques. We are also thankful to the Directorate General XIII of the European Union for their support through the RACE and ACTS projects that have greatly contributed to the advances in this new field. Within this Directorate, special thanks go to Eric Badique for his continual belief in what some people occasionally refer to as, "exotic video coding techniques". We are also indebted to Mike Casey, acquisitions editor of Kluwer Academic Publishers, for the help provided while editing and compiling this book. Carlos Nistal from UPC, has provided all the computer support needed to handle and solve the variety of problems that have arisen in the electronic editing of this book. Our final debt of gratitude is to ours wives, Merce and Jocelyne, for their support during the last twelve months we have spent editing and writing this book. This book now appears, thanks also to these Second Generation wives. xi

11 CONTRIBUTORS Kiyoharu Aizawa Department of Electrical Engineering University of Tokyo Tokyo 113, Japan Frederic Dufaux The Media Laboratory Massachusetts Institute of Technology Cambridge, Massachusetts 02139, USA N ariman Farvardin Electrical Engineering Department and Institute for Systems Research University of Maryland College Park, Maryland 20742, USA Robert Forchheimer Image Coding Group Department of Electrical Engineering Linkoping University S Linkoping, Sweden Narciso Garcia Escuela Tecnica Superior de Ingenieros de Telecomunicacion Universidad Politecnica de Madrid Madrid, Spain Antoni Gasull Department of Signal Theory and Communications Universitat Politecnica de Catalunya Barcelona, Spain Mohammad Gharavi-Alkhansari Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign Urbana, Illinois 61801, USA Michael Gilge Philips Communication Systems Nuremberg, Germany Thomas S. Huang Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign Urbana, Illinois 61801, USA Fernando Jaureguizar Escuela Tecnica Superior de Ingenieros de Telecomunicacion Universidad Politecnica de Madrid Madrid, Spain xiii

12 XIV CONTRIBUTORS Murat Kunt Signal Processing Laboratory Swiss Federal Institute of Technology 1015 Lausanne, Switzerland Haibo Li Image Coding Group Department of Electrical Engineering Linkoping University S Linkoping, Sweden Ferran Marques Department of Signal Theory and Communications Universitat Politecnica de Catalunya Barcelona, Spain Fabrice Moscheni Signal Processing Laboratory Swiss Federal Institute of Technology 1015 Lausanne, Switzerland Jose Ignacio Ronda Escuela Tecnica Superior de Ingenieros de Telecomunicaci6n Universidad Politecnica de Madrid Madrid, Spain Philippe Salembier Department of Signal Theory and Communications Universitat Politecnica de Catalunya Barcelona, Spain Luis Torres Department of Signal Theory and Communications Universitat Politecnica de Catalunya Barcelona, Spain Montse Pardas Department of Signal Theory and Communications Universitat Politecnica de Catalunya Barcelona, Spain Xiaonong Ran Systems Technology Corporate Technology Group National Semiconductor Corporation 2900 Semiconductor Drive, Santa Clara California, USA