VIDEO DISTORTION MEASUREMENT USING PSNR IN WAVELET DOMAIN MOK YUNG LENG Bachelor of Engineering with Honors (Electronics & Computer Engineering) 2009/2010
UNIVERSITI MALAYSIA SARAWAK R13a BORANG PENGESAHAN STATUS TESIS Judul: VIDEO DISTORTION MEASUREMENT USING PSNR IN WAVELET DOMAIN SESI PENGAJIAN: 2009/2010 Saya MOK YUNG LENG (HURUF BESAR) mengaku membenarkan tesis * ini disimpan di Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak dengan syarat-syarat kegunaan seperti berikut: 1. Tesis adalah hakmilik Universiti Malaysia Sarawak. 2. Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak dibenarkan membuat salinan untuk tujuan pengajian sahaja. 3. Membuat pendigitan untuk membangunkan Pangkalan Data Kandungan Tempatan. 4. Pusat Khidmat Maklumat Akademik, Universiti Malaysia Sarawak dibenarkan membuat salinan tesis ini sebagai bahan pertukaran antara institusi pengajian tinggi. 5. ** Sila tandakan ( ) di kotak yang berkenaan SULIT TERHAD (Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972). (Mengandungi maklumat TERHAD yang telah ditentukan oleh organisasi/ badan di mana penyelidikan dijalankan). TIDAK TERHAD Disahkan oleh (TANDATANGAN PENULIS) (TANDATANGAN PENYELIA) Alamat tetap: 5, JLN ANGGERIK VANILLA 31/98Q, KOTA KEMUNING, 40460 SHAH ALAM, SELANGOR IR. DAVID BONG BOON LIANG Nama Penyelia Tarikh: 10 APRIL 2010 Tarikh: 12 APRIL 2010 CATATAN * Tesis dimaksudkan sebagai tesis bagi Ijazah Doktor Falsafah, Sarjana dan Sarjana Muda. ** Jika tesis ini SULIT atau TERHAD, sila lampirkan surat daripada pihak berkuasa/organisasi berkenaan dengan menyatakan sekali sebab dan tempoh tesis ini perlu dikelaskan sebagai SULIT dan TERHAD.
This Final Year Project attached here: Title : Video Distortion Measurement Using PSNR In Wavelet Domain Student Name : Mok Yung Leng Matric No : 16744 has been read and approved by: Ir. David Bong Boon Liang Date (Supervisor)
Video Distortion Measurement Using PSNR In Wavelet Domain MOK YUNG LENG This project is submitted in partial fulfillment of The requirements for the degree of Bachelor of Engineering with Honors (Electronics and Computer Engineering) Faculty of Engineering UNIVERSITI MALAYSIA SARAWAK 2009/2010
Dedicated to Mom, my friends and my family
ACKNOWLEDGEMENT I would like take the opportunity to thank my supervisor, Ir. David Bong for his encouragement and support, as well as his comments, suggestions and advice on the course of developing this project. With his involvement in this project, I am able to complete the project within the scheduled time. I would also like to give credit to my beloved friends and family, who gave support to me throughout the years in the University, both financially and in the form of moral support. Without them, it would not be easy to go through my rough times along the four years in my university life. I am also grateful to UNIMAS and the Engineering Faculty for giving me the chance to receive my tertiary education here. Finally, I would also like to express my gratitude to in the individuals who directly or indirectly helped me in the development of this project.
ABSTRAK Pada era digital, teknologi imej digital semakin maju. Oleh itu, algoritma pengekodan video yang berprestasi baik penting untuk menghasilkan video yang berkualiti tinggi. Analisis kualiti video objektif dapat menambahbaikan algoritma pengekodan video. Satu cara ukuran herotan video yang baru akan dicadangkan dalam tesis ini. Ukuran herotan ini adalah ditujukan kepada video dalam domain wavelet. Video yang diujikan dalam projek ini adalah daripada video yang diperolehi dalam pangkalan data video Laboratory for Image and Video Engineering (LIVE). Wavelet Cohen-Daubechies-Feauveau (CDF) 9/7 dalam 2D akan diapplikasikan dalam semua imej dalam semua video yang akan diuji. Ukuran objektif yang digunakan dalam projek ini adalah Peak Signal-to-Noise Ratio (PSNR). Projek ini akan membuat taksiran dengan mengunakan PSNR sebagai ukuran objektif dalam perbezaan antara video rujukan dan video yang mempunyai herotan dalam domain wavelet. Satu skor keseluruhan untuk video yang mempunyai herotan akan ditentukan daripada analisis ini. Dalam analsis ini, nilai-nilai PSNR video dalam domain wavelet juga akan dibandingkan dengan nilai-nilai PSNR video dalam domain ruangan. Prestasi, keutuhan, dan ketekalan skema ukuran herotan yang dicadangkan ini juga dianalisa dengan membuat perbandingan dengan nilai-nilai PSNR video dalam domain ruangan. Perisian MATLAB digunakan untuk mendapatkan nilai-nilai PSNR. Perisian Microsoft Excel digunakan untuk analisis nilai-nilai PSNR. i
ABSTRACT With the advancement of digital imaging, video coding algorithms that has good performance is important for producing videos in high quality. Objective video quality analysis can improve the video coding algorithms. In this thesis, a new objective method for distortion measurement of videos is proposed. The distortion measurement is based on videos in wavelet domain. The test videos used in the project are test videos provided by Laboratory for Image and Video Engineering (LIVE) video database. 2D Cohen-Daubechies-Feauveau (CDF) 9/7 wavelet is applied to the video frames. The objective measurement used in this project is Peak Signal-to-Noise Ratio (PSNR). The project calculates the differences of the reference video and the distorted video in wavelet domain, by implementing PSNR values as the objective measurement. An overall PSNR score for a distorted video is also determined from the analysis. PSNR values of the video in wavelet domain are compared to the PSNR values of the videos in spatial domain. Performance, reliability, and consistency of the proposed video distortion measurement scheme are also analysed in this thesis by comparison to the PSNR values of videos in spatial domain. The PSNR values are calculated using MATLAB and the values are exported to Microsoft Excel to perform analysis. ii
TABLE OF CONTENTS CONTENT PAGES ACKNOWLEDGEMENT ABSTRAK ABSTRACT TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES ABBREVIATIONS i ii iii ix x xii CHAPTER 1 INTRODUCTION 1.1 Introduction 1 1.2 Problem Statement 3 1.3 Project Objectives 4 1.4 Project Scope 5 1.5 Project Outline 5 CHAPTER 2 LITERATURE REVIEW 2.1 Overview 7 2.2 Types of distortions in digital videos 7 iii
2.2.1 Blocking effect 8 2.2.2 Blurring 9 2.2.3 Colour bleeding 10 2.2.4 Posterisation 11 2.2.5 Ringing effect 12 2.2.6 Mosquito noise 13 2.2.7 Ghosting 13 2.2.8 Random noise 14 2.2.9 Unstableness 15 2.2.10 Jerkiness 16 2.3 Full-reference (FR), no-reference (NR), and reduced-reference (RR) video quality assessment 17 2.4 MOS, MSE and PSNR 18 2.5 Wavelet transform 21 2.5.1 CDF 9/7 wavelet transform 23 2.5.2 Construction of CDF 9/7 wavelets 26 CHAPTER 3 METHODOLOGY 3.1 Overview 29 3.2 Video distortion measurement in wavelet domain 31 3.3 Video distortion measurement in spatial domain 35 3.4 MATLAB 37 iv
CHAPTER 4 RESULTS, ANALYSIS AND DISCUSSIONS 4.1 Overview 40 4.2 Analysis on PSNR values of videos in wavelet domain 41 4.2.1 Discussion on the analysis on PSNR values of videos in wavelet domain 46 4.3 Analysis on PSNR values of videos in spatial domain 47 4.3.1 Discussion on the analysis on PSNR values of videos in spatial domain 53 4.4 Discussion 54 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS 5.1 Overview 56 5.2 Conclusion 56 5.3 Recommendations 58 REFERENCES 59 APPENDIX A: MATLAB source codes 62 APPENDIX B: PSNR values of video frames of bs2 to bs16 in wavelet domain 68 APPENDIX C: PSNR values of video frames of mc2 to mc16 in wavelet domain 72 APPENDIX D: PSNR values of video frames of pa2 to pa16 in wavelet domain 81 v
APPENDIX E: PSNR values of video frames of pr2 to pr16 in wavelet domain 85 APPENDIX F: PSNR values of video frames of rb2 to rb16 in wavelet domain 93 APPENDIX G: PSNR values of video frames of bs2 to bs16 in spatial domain 98 APPENDIX H: PSNR values of video frames of mc2 to mc16 in spatial domain 101 APPENDIX I: PSNR values of video frames of pa2 to pa16 in spatial domain 108 APPENDIX J: PSNR values of video frames of pr2 to pr16 in spatial domain 112 APPENDIX K: PSNR values of video frames of rbr2 to rb16 in spatial domain 119 APPENDIX L: Line graph of PSNR values of video frames in bs2 to bs16 in wavelet domain 123 APPENDIX M: Line graph of PSNR values of video frames in mc2 to mc16 in wavelet domain 124 APPENDIX N: Line graph of PSNR values of video frames in pa2 to pa16 in wavelet domain 125 APPENDIX O: Line graph of PSNR values of video frames in pr2 to pr16 in wavelet domain 126 APPENDIX P: Line graph of PSNR values of video frames in pr2 to pr16 in wavelet domain 127 vi
APPENDIX Q: Line graph of PSNR values of video frames in bs2 to bs16 in spatial domain 128 APPENDIX R: Line graph of PSNR values of video frames in mc2 to mc16 in spatial domain 129 APPENDIX S: Line graph of PSNR values of video frames in pa2 to pa16 in spatial domain 130 APPENDIX T: Line graph of PSNR values of video frames in pr2 to pr16 in spatial domain 131 APPENDIX U: Line graph of PSNR values of video frames in rb2 to rb16 in spatial domain 132 APPENDIX V: Comparison of PSNR values of video frames in bs2 to bs16 in wavelet domain to PSNR values of video frames in bs2 to bs 16 in spatial domain 133 APPENDIX W: Difference of PSNR values of video frames in bs2 to bs16 in wavelet domain to PSNR values of video frames in bs2 to bs 16 in spatial domain and the percentage of difference 148 APPENDIX X: Correlation of mean PSNR, median PSNR, RMS PSNR and DMOS in video sequence bs 156 APPENDIX Y: Correlation of mean PSNR, median PSNR, RMS PSNR and DMOS in video sequence mc 157 APPENDIX Z: Correlation of mean PSNR, median PSNR, RMS PSNR and DMOS in video sequence pa 159 APPENDIX AA: Correlation of mean PSNR, median PSNR, RMS PSNR and DMOS in video sequence pr 160 vii
APPENDIX AB: Correlation of mean PSNR, median PSNR, RMS PSNR and DMOS in video sequence rb 162 APPENDIX AC: Spatial RMS vs. Wavelet RMS of PSNR values of bs2 bs16 164 APPENDIX AD: Spatial RMS vs. Wavelet RMS of PSNR values of mc2 mc16 164 APPENDIX AE: Spatial RMS vs. Wavelet RMS of PSNR values of pa2 pa16 165 APPENDIX AF: Spatial RMS vs. Wavelet RMS of PSNR values of pr2 pr16 165 APPENDIX AG: Spatial RMS vs. Wavelet RMS of PSNR values of rb2 rb16 166 viii
LIST OF TABLES TABLE PAGES 3.1 Number of frames and frame rate of the test videos 31 4.1 DMOS, Mean PSNR, RMS PSNR, and Median PSNR of bs2 bs16 43 4.2 Spatial RMS vs. Wavelet RMS of PSNR values of bs2 bs16 51 ix
LIST OF FIGURES FIGURE PAGE 2.1 Lena reference image 8 2.2 Blocking effect 8 2.3 Blurring 9 2.4 Lena reference image in colour 10 2.5 Colour bleeding 10 2.6 Posterisation 11 2.7 Cropped Lena reference image 12 2.8 Ringing effect 12 2.9 Random noise 14 2.10 Unstableness 15 2.11 Jerkiness 16 2.12 Two level 2D wavelet transform 23 2.13 Lifting algorithm for forward wavelet transform 24 2.14 Lifting algorithm for inverse wavelet transform 25 3.1 Flow chart of the video distortion measurement in wavelet domain 32 3.2 Three-scale wavelet decomposition 34 3.3 Flow chart of the video distortion measurement in spatial domain 36 3.4 Layout of the M-file editor 39 x
4.1 PSNR values of video frames of bs1 vs. bs2-bs16 in wavelet domain 42 4.2 Correlation of mean, median and RMS PSNR in video sequence bs 44 4.3 Correlation of DMOS to mean, median and RMS PSNR in video sequence bs 45 4.4 PSNR values of video frames of bs1 vs. bs2-bs16 in spatial domain 48 4.5 Wavelet PSNR vs. spatial PSNR (bs2) 49 4.6 Difference and percentage of difference of wavelet PSNR values to spatial PSNR values. 50 4.7 Spatial RMS vs. Wavelet RMS of PSNR values of bs2 bs16 52 xi
ABBREVIATIONS LIST OF NOTATIONS PSNR LIVE MPEG FR NR RR MOS MSE DMOS CDF IP AVI JPEG RGB YCbCr RMS Peak signal-to-noise ratio Laboratory for Image and Video Engineering Motion Picture Experts Group Full-reference No-reference Reduced-reference Mean opinion score Mean squared error Difference Mean Opinion Score Cohen-Daubechies-Feauveau Internet Protocol Audio Video Interleave Joint Photographic Experts Group Red, Green, Blue Luminance, Blue difference, Red difference Root Mean Square xii
CHAPTER 1 INTRODUCTION 1.1 Introduction Video is the technology of capturing and recording a sequence of still images representing scenes in motion using electronic devices like digital camera and camcorders. Video also involves in processing, storing, transmitting, and reconstructing such sequences of images [1]. Videos are prone to distortions. Distortions reduce the quality of a video. The first type of distortion is introduced at the video acquisition stage. This is due to the limitations of camera devices. Such distortions are introduced by camera optics, sensor noise, colour calibration, exposure control etc [2]. The second type of distortion is caused by video processing and transmission. Raw video occupies large bandwidth, thus it must be compressed using different video compression schemes before storage or transmission for better efficiency [3]. The compressed video generally has a certain degree of distortions or loss of quality compared to the raw 1
video. When a video is transmitted over a channel, bit errors will occur and this also introduce distortions to the transmitted video. With the technology advancement in electronics and digital imaging, many digital video coding techniques are implemented in different digital video coding products. These products, which cover a broad range of applications, have different quality and bandwidth requirements. Thus, it has become eventually more important to develop video quality/distortion measurement techniques that can help to evaluate, to compare and to improve the video coding techniques and products that provide effective and high quality digital video services. [4] To measure the quality of a video, researchers use two major methods of video quality analysis. The first method is called the subjective video quality assessment. This method evaluates the quality of a video by seeking opinion from human observers [5]. However, this method is not practical in application because there are a lot of videos in the real world and cannot be evaluated one by one. The other reason is that researchers want to incorporate such quality measurement techniques into algorithms that can be used to process videos, thus further enhance the efficiency of the process to achieve a better quality of video with a given set of resources [2]. As an alternative, researchers look into a more efficient method of video quality analysis, namely objective video quality assessment. The purpose of objective video quality analysis is to develop quantitative measures that can predict apparent video quality by using a computer or other electronic devices [2]. Due to this property, objective video quality assessment can be incorporated into different video 2
processing algorithms; that can improve the output of a processed video if such a technique is used. 1.2 Problem Statement Objective video assessment scheme is useful as it can be incorporated into different video processing algorithms to improve the output of a processed video. By implementing distortion measurement in a video processing algorithm, a processed video can be further enhanced to give a higher quality output of videos. Hence, an objective video assessment scheme is very much needed in the field of digital imaging. As of today, there is no standardised objective assessment scheme accepted for measuring distortion. Many researchers had studied and proposed many different objective assessment schemes, but none is accepted as a standard. The aim of this project is to develop an objective assessment scheme to measure distortion in videos in wavelet domain. This method utilizes full-reference method and will use an unprocessed video as a reference to measure the degree of distortion of the distorted videos. Wavelet transform is a popular method used for image/video compression and analysis, and is used in JPEG2000 compression standard. 3
1.3 Project Objectives The objectives of the project are: I. To evaluate and to compare different existing video distortion measurement techniques. There are many types of video distortion measurement techniques available currently. For this project, two different techniques are studied and compared thoroughly. A suitable technique is used to apply in the distortion measurement of the video. II. To develop an assessment scheme and apply the techniques in 2-D wavelet domain. Distortion measurement in 2-D wavelet domain is chosen as the suitable technique for the video distortion measurement. An assessment scheme or a methodology is developed to implement such technique into the measurement. III. To implement statistical analysis in the distortion measurement of the video. Peak signal-to-noise ration (PSNR) is used as a statistical analysis for this project. PSNR is a popular and widely accepted objective measurement due to their easy-to-calculate features. 4
1.4 Project Scope This project is focused on the distortion measurement of videos by measuring the distortion introduced in the processed video in wavelet domain. The distortions measured are distortions that were introduced in the compression and transmission process of the videos. Wavelet transform is a popular method used for video analysis and compression. Full-reference method is also used to make direct comparison between a reference video and a processed video. PSNR is used as statistical analysis for the project. MATLAB is used as a programming tool for the computation of algorithms required for the distortion measurement by implementing the suitable toolboxes available in MATLAB. Five test videos from Laboratory for Image and Video Engineering (LIVE) Video Quality Database, provided by University of Texas are used for analysis in this project. 1.5 Project Outlines This thesis covers the details of processes involved in developing an assessment scheme for measuring distortion in videos. The thesis is divided into five main chapters; introduction, literature review, methodology, results, analysis and discussions, conclusion and recommendations. Brief description for each chapter is as below: Chapter 1: Introduction This chapter briefly describes the background of the project title, problem statements and the objectives of the project, as well as the scope of the project. 5
Chapter 2: Literature Review This chapter is basically a summary of the researches done to gain knowledge and information for the purpose of developing the project. Literature review from different sources such as journals, books, internet sources and conference papers are compiled and summarized in this chapter. Chapter 3: Methodology This chapter focuses on the proposed methodology for this. The methodology gives an overall idea on how the assessment scheme is implemented using PSNR in wavelet domain. Chapter 4: Results, Analysis and Discussion This chapter contains the experiment results, analysis and discussion of the result. This chapter also discusses problems that occurred along the development of the project. Chapter 5: Conclusion and Recommendations This chapter is the summary of the overall findings of the project. Future implementations and suggestions for further improvement for the project are also covered in this chapter. 6