Robust Video ing for MPEG Compression and DA-AD Conversion Jong-Uk Hou, Jin-Seok Park, Do-Guk Kim, Seung-Hun Nam, Heung-Kyu Lee Division of Web Science and Technology, KAIST Department of Computer Science, KAIST Graduate School of Information Security, KAIST 335 Gwahangno, Yuseong-gu, Daejeon, Republic of Korea {juheo, jspark, dgkim, shnam, hklee}@mmc.kaist.ac.kr ABSTRACT In this paper, we proposed a robust watermarking system. Using DCT domain spread-spectrum watermarking, our system achieved robustness to various non-hostile processings such as MPEG compression, and DA-AD conversion. Our proposed system guarantees that 16-bit data can be extracted in any 15 second interval, even in a long, by using adaptive frame selection. To verify the performance of the proposed system, various experiments were carried out under a DA-AD conversion environment. Experimental results show that the proposed method is not only robust to non-hostile processings but also achieves invisibility. Categories and Subject Descriptors I.4.9 [Image Processing and Computer Vision]: Applications Keywords Digital watermarking; watermark; compression; signal processing 1. INTRODUCTION Digital watermarking is a process that alters a cover work imperceptibly to embed a watermark message [4]. One of the main applications of digital watermarking is copyright protection. In particular, digital watermarking is robust against analog hole piracy [5], which is difficult to prevent with other copyright protection techniques, such as Digital Rights Management (DRM). In the case of digital watermarking, an embedded watermark should endure various non-hostile processings. Photometric attacks, spatial desynchronization, temporal desynchronization, and editing are included in Corresponding author Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. IWIHC 14, June 3, 2014, Kyoto, Japan. Copyright 2014 ACM 978-1-4503-2803-6/14/06...$15.00. http://dx.doi.org/10.1145/2598908.2598909. non-hostile processings [6]. Photometric attacks are attacks which modify the pixel values in a frame. Noise addition and Digital/Analog - Analog/Digital (DA-AD) conversion belong to this category. Spatial desynchronization can occur with changes of display format and changes of spatial resolution. Temporal desynchronization is mainly due to changes of frame rate. Video editing involves various editing techniques, such as cut-and-splice and fade-and-dissolve. This paper presents a robust watermarking system. The proposed watermarking system achieves robustness to temporal desynchronization by using adaptive selection of watermarking frames. In the adaptive frame selection, a sample watermark is embedded, and the watermarked is compressed. Then, frames that are robust to the compression are selected for watermarking. We adopt a DCT-based spread-spectrum image watermarking [1] which is robust to lossy compression, noise addition, DA-AD conversion, and uniform scaling. Therefore, the embedded watermark can endure various non-hostile processing. Experimental results show that the proposed method achieves robustness and perceptual invisibility. The rest of the paper is organized as follows. The proposed watermarking method is described in Section 2. The experimental setup and results are shown in Section 3. Finally, the conclusion is presented in Section 4. 2. PROPOSED METHOD 2.1 Overall Process In the proposed watermarking system, we extended the DCT domain spread-spectrum image watermarking [1] to work for the environment. A pseudo-random sequence of length N is used as the watermark. The overall process of the proposed method is summarized in Figure 1. The frames are adaptively selected for watermarking. For every 7.5 seconds in the, a one second interval is chosen for watermarking. Before watermarking of the selected frames, 512 watermarks are generated to encode 16 bits, that is, 256 watermarks for encoding the front 8 bits, and the other 256 watermarks for encoding the back 8 bits. Therefore, two watermarks are embedded in the selected frames. s are embedded in the blue channel, since variation in the blue channel is less perceptible than variation in other channels [7]. In the watermarking extraction step, watermarks are extracted from the entire frames. If two watermarks are precisely extracted, 16 bits of data are recovered from these watermarks. 2
Adaptive Frame selection Video compression Frame selection Embedding process ed Video All Frames... Sample watermark Embedding process DCT DCT Original Selected frames embedding Set of pseudo random sequences extraction 16-bit data Sequence selection s Inverse DCT Decoding s Sequence generation Set of Pseudo random sequences Data encoding ed 16-bit data (a) Embedding process (b) Extraction process Figure 1: ing embedding and extraction processes 2.2 Adaptive frame selection In our proposed scheme, watermarks are embedded in the frames robust to compression. The details of the adaptive frame selection process are summarized as following. First, a sample watermark is repeatedly embedded in each frame by using the algorithm introduced in Section 2.4, and the file is compressed by a compression algorithm. Then, the extractor responses in each frame are computed by using the procedure described in Section 2.5. Figure 2 shows an example of the computed extractor responses. The extractor responses are degraded in the middle interval which includes a lot of motion. In the case of compressed, the extractor responses are entirely degraded, and there are periodic peaks in the sequence of the extractor responses. These peaks appear in the I-frames in group of pictures (GOP) [2]. I-frames are more robust to compression than other kinds of frames in GOP, since it is less affected by motion estimation. Using this property, watermarks can be embedded efficiently. For instance, it is inefficient to embed watermarks in frames that are not robust to compression. Therefore, the one second intervals with the highest extractor responses are selected from every 7.5 second interval (See Figure 3). In the watermark embedding step, the selected frames are watermarked, and the other frames remain unchanged. After the adaptive frame selection, a watermarked interval exists in any 15-second interval in the, since the maximum value of the time between two consecutive watermarked intervals is 13 seconds. Therefore, the proposed Correlation 80 60 40 20 Original MPEG (QF=58) MPEG (QF=0) 0 0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 Frame number Figure 2: MPEG 4 compression effect on extractor response ( : ITE/ARIB test sequence #2) system guarantees that 16 bits of data can be extracted from any 15-second interval in the. 2.3 Data encoding process In our proposed system, two pseudo-random sequences X, Y are used for watermarking: sequence X for encoding the front 8 bits, and sequence Y for encoding the back 8 bits. To represent 8 bits information, 256(= 2 8 ) unique codes are generated. From this, 16 bits of data can be encoded with two sets X, Y of 256 pseudo-random sequences. X = {X 0,X 1,X 2,...,X 255}, Y = { Y 0,Y 1, Y 2,..., Y 255}. Therefore, two sequences X i,y j areselectedineachsequence set for data encoding where index i, and j correspond to input data. For instance, if input data is 0x00FF, twosequences X i,y j are selected where i = 0x00 and j = 0xFF. 2.4 embedding In the watermarking embedding process, the discrete cosine transform (DCT) of frame F is computed. Then, the N coefficients in fixed position are used to form vector V.The vector of the original coefficients V = {v 1,v 2,v 3,..., v N } is watermarked, and it becomes a vector of watermarked coefficients V = {v 1,v 2,v 3,..., v N } according to the following embedding rule: v i = v i + α v i (1) xi + yi, (1 i N), (2) 2 ed frames Unchanged frames... X, Y X, Y X, Y... 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16... time (sec) Frame sequence of input Figure 3: Example of watermarked sequence. ed intervals are selected from every 7.5 second. 3
Data 16bits Video #2, #8, #20, #23, #46 embedder Video encoder (MPEG-4) Decompressed D-A converter Analog signal A-D converter extractor Video encoder (MPEG-4) Decompressed PSNR & SSIM Extracted data 16bits bit error rate Figure 4: Overall process of our test system where α is the scaling factor which decides the strength of the watermark signal, and X = {x 1,x 2,x 3,..., x N }, Y = {y 1,y 2,y 3,..., y N } are selected watermarks. To encode 16 bits, two watermarks X, Y are embedded in each selected frame. The coefficients in the mid-band frequency spectrum are used for embedding so that a trade-off between robustness to compression and perceptual invisibility is achieved. After the coefficient value modification, inverse DCT is performed, and the watermarked frame F is obtained. 2.5 extraction The watermarking extraction process is carried out as follows. The DCT of the possibly corrupted watermarked frame F is computed, and vector V is extracted from the DCT coefficients in the fixed position. Then, the correlation between vector V and the watermarks from the sets X, Y are computed and used as a measure of the presence of the watermark. The correlation z is defined as z = X V N = 1 N N x i vi. (3) The presence of watermark X is decided by comparison between the correlation and a threshold T. The threshold is computed by T = 1 20 N i=1 N vi. (4) The threshold equation is obtained from the experimental results to minimize a false-positive rate. The presence of each watermark among the total 512 watermarks from the sets X, Y in frame F is determined by thresholding. In this manner, the watermarks in all frames are extracted. Then, each extracted watermark is matched to the proper 8 bits of data and the whole 16 bits of data are recovered. i=1 with 16 GB main memory was used to measure the performance. For our test set, we used clips from the ITE/ARIB Hi-Vision Test Sequence 1st Edition: numbers 2, 8, 20, 23, and 46. First, we converted raw YPbPr into uncompressed RGB by the following equation: R = Y +1.576P r G = Y (0.159P b +0.334P r)/0.701 B = Y +1.826P b. The original frames of each clip were 20 seconds in duration with 600 frames and 1920 1035 display resolution. The MPEG-4 part 10 (H.264) codec in MATLAB automatically resizes the resolution to 1920 1032; thus, we cropped the last three rows. Then, we removed the first 150 frames to get a 15-second. Finally, we got uncompressed AVI s: display resolution 1920 1032; 30 frames per second; 15 seconds; 1, 430, 826 kbps; and 2.49 GB size. The experiment was conducted in the following manner. First, we inserted 16-bit data into the original clips by using the proposed embedding algorithm. After the embedding process, the watermarked clips were compressed using the MPEG-4 part 10 (H.264) codec with various quality factors. Then the compressed was converted into analog signal using a digital-to-analog (DA) converter. Analog component (YPbPr) was selected as the format of analog signal because YPbPr accommodates full HD display resolution (1080p). To process DA conversion, a âăÿfosmon HD- MI to component (YPbPr) / VGA & SPDIF output converter boxâăź was used (See Figure 5(a)). The converted YPbPr signal was converted into digital signal again, and the converted digital was saved in YUV422 format, which is an uncompressed format. For this process, we used âăÿskyhd capturex HDMIâĂŹ for the AD converter (See Figure 5(b)). The converted di- (5) 3. EXPERIMENTAL RESULTS 3.1 Experimental setup In this section, the test environment for the proposed method is explained. Figure 4 shows the overall process of our test for verification of our proposed scheme. The test included the processes of watermark embedding, watermark extraction, compression (MPEG-4 part 10), DA-AD conversion, and calculation of the average bit error rate of the embedded information. The proposed watermarking algorithm was implemented in the MATLAB 2012b environment, and a machine consisting of Intel i7-3770 (3.40 GHz) (a) DA converter (b) AD converter Figure 5: Converters for DA-AD conversion. (a) Fosmon HDMI to component converter box, (b) skyhd capturex HDMI 4
Figure 6: Selected DCT coefficients (red area) for watermarking process. gital had different display resolution (1280 720) and frame rate in comparison with the original. Lastly, we extracted the watermarks embedded in the clips through the watermark extracting system. The extracted watermarks were decoded into 16-bit data, and the bit error rate of the extracted data was calculated. To achieve robustness and invisibility, several parameters of our algorithm have to be carefully set. To implement our watermarking process, we selected N = 25000 coefficients from the fixed position of DCT domain described in Figure 6. It was found experimentally that this choice yields maximum watermark robustness while preserving invisibility. 3.2 Experimental results 3.2.1 Image quality assessment To assess image quality, we inserted 16 bits of data into the original clips by using the proposed embedding algorithm in various watermark strengths α. The watermarked s were generated to yield an average PSNR of more than 36dB. Figure 7 shows the example of the original and watermarked images. After the embedding process, the watermarked clips were compressed using the MPEG-4 part 10 (H.264) codec with various quality factors between 56 and 59 (See Table 1). Video clips compressed with quality factor between 56 and 59 satisfy the bit rate of 1/100 compared to the uncompressed. Also, the unwatermarked original clips were compressed using the same quality factor. Both watermarked clips and unwatermarked original clips were decompressed, and the SSIM [8] and PSNR were calculated for each pair of the luminance signal calculated by following equation: Y =0.7152G +0.0722B +0.2126R. (6) Table 2 shows the bit error rate (BER), PNSR, and SSIM results for various watermark strengths α. According to the results, no error occured in our tests when α 0.4. The ITE/ARIB test sequence Quality factor ITE / ARIB #2 58 ITE / ARIB #8 59 ITE / ARIB #20 58 ITE / ARIB #23 58 ITE / ARIB #46 56 Table 1: Quality factor to generate less than 1/100 of the original bit rate Figure 7: Compressed version of original images (Left), and images from Set-4 (α =0.4, PSNR 44.03 db, SSIM 0.9945) (Right) result of #2 showed the best detection ratio with the highest PSNR. In contrast, the test result of #23 showed the worst performance because it had a lot of motion throughout the whole playing time. In addition, Table 3 compares the PSNR and SSIM values of the uncompressed original and uncompressed watermarked. These results indicate visual degradation of the test s by the compression algorithm. 3.2.2 Tolerance assessment A watermarking system must be robust, which means that a watermark should be difficult to eliminate by common signal processing including compression, such as MPEG-4 part 10, and DA-AD conversion. Therefore, an embedded watermark should be barely affected by these signal processing. Robustness is evaluated by calculation of the average bit error rate for an embedded watermark after undergoing common signal processing. Tests were conducted with various watermark strengths α = 0.4, 0.9, and 1.5. Tables 4, 5, and 6 show the detection results and compressibilities for various MPEG quality factors. In the test with α =0.9, the embedded data bits were successfully extracted from all s except #46, while the average compressibility was 1/604. The embedded data bits were precisely extracted for all tested MPEG quality factors when α =1.5. 5
4. CONCLUSION In this paper, we proposed a robust watermarking system. Our system achieves robustness to non-hostile processings, such as temporal desynchronization, MPEG compression, and DA-AD conversion. We could obtain this property by using adaptive frame selection and DCT domain spread-spectrum watermarking. To verify the performance of the proposed system, various experiments were performed under DA-AD conversion environment. The experimental results demonstrated that the proposed method not only is robust to non-hostile processings but also achieves invisibility. However, our scheme is not secure in some respect because of the spread spectrum watermarking is known to be sensitive to attacks [3]. Therefore, enhancing the security of our system is one of the most important task for our future improvement. Acknowledgments This research project was supported by Government Fund from Korea Copyright Commission and by the National Research Foundation of Korea(NRF) grant funded by the Korea government(mest) (No. 2012R1A2A1A05026327) 5. REFERENCES [1] M. Barni, F. Bartolini, V. Cappellini, and A. Piva. A dct-domain system for robust image watermarking. Signal Processing, 66(3):357 372, 1998. [2] V. Bhaskaran and K. Konstantinides. Image and compression standards: algorithms and architectures. Springer, 1997. [3] M. Chaumont. Ensuring security of h.264 s by using watermarking. volume 8063, pages 806303 806303 10, 2011. [4] I.J.Cox.Digital watermarking and steganography. Elsevier Science Limited, 2008. [5] E. Diehl and T. Furon. : closing the analog hole. In Consumer Electronics, 2003. ICCE. 2003 IEEE International Conference on, pages 52 53, 2003. [6] G. DoÃńrr and J.-L. Dugelay. A guide tour of watermarking. Signal Processing: Image Communication, 18(4):263 282, 2003. [7] A. K. Jain. Fundamentals of Digital Image Processing. Prentice Hall, 1988. [8] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Transactions on, 2004. 6
Test set No. Set-1 Set-2 Set-3 Set-4 Set-5 Set-6 Set-7 Set-8 WM strength α 0.1 0.3 0.35 0.4 0.9 1.5 3 5 PSNR(dB) 53.53 50.48 50.43 50.40 49.26 47.17 45.32 43.82 #2 SSIM 0.9994 0.9987 0.9987 0.9987 0.9983 0.9974 0.9949 0.9912 BER 0.000 0.000 0.000 0.000 0.000 0.000 0.000 PSNR(dB) 48.35 46.62 46.59 46.58 41.66 41.22 39.40 37.80 #8 SSIM 0.9986 0.9980 0.9980 0.9979 0.9934 0.9920 0.9846 0.9710 BER 0.000 0.000 0.000 0.000 0.000 0.000 0.000 PSNR(dB) 45.06 44.94 44.91 44.89 35.62 35.61 36.87 36.21 #20 SSIM 0.9958 0.9641 0.9954 0.9953 0.9611 0.9606 0.9626 0.9436 BER 0.000 0.000 0.000 0.000 0.000 0.000 PSNR(dB) 40.75 33.7 33.7 33.69 32.28 32.31 31.93 30.82 #23 SSIM 0.9973 0.9865 0.9865 0.9864 0.9780 0.9771 0.9705 0.9346 BER 0.000 0.000 0.000 0.000 0.000 PSNR(dB) 41.79 44.66 44.63 44.58 41.06 41.22 35.65 35.41 #46 SSIM 0.9878 0.9944 0.9944 0.9943 0.9851 0.9857 0.9428 0.9360 BER 0.000 0.000 0.000 0.000 0.000 0.000 0.000 PSNR(dB) 45.89 44.08 44.05 44.03 39.98 39.50 37.83 36.81 average SSIM 0.9957 0.9883 0.9946 0.9945 0.9831 0.9825 0.9710 0.9552 BER 0.400 0.200 0.000 0.000 0.000 0.000 0.000 Compression quality factor - #2 : 58, #8 : 59, #20 : 58, #23 : 58, #46 : 56 : watermark is not detected (BER is considered as 1.000) Table 2: Result of test for various watermark strength α. Test set No. Set-1 Set-2 Set-3 Set-4 Set-5 Set-6 Set-7 Set-8 WM strength α 0.1 0.3 0.35 0.4 0.9 1.5 3 5 PSNR(dB) 76.94 70.28 69.70 68.91 65.07 63.17 57.56 54.00 SSIM 1.0000 1.0000 1.0000 1.0000 0.9999 0.9998 0.9994 0.9984 Table 3: Result of test for uncompressed original and uncompressed watermarked MPEG Quality factor 50 40 30 20 10 0 #2 BER 0.000 0.000 0.000 0.000 0.000 compressibility 1/139 1/191 1/232 1/307 1/432 1/1029 #8 compressibility 1/139 1/193 1/253 1/321 1/445 1/606 #20 BER 0.000 0.000 0.000 0.000 compressibility 1/139 1/190 1/253 1/321 1/444 1/460 #23 BER 0.000 0.000 0.000 compressibility 1/138 1/193 1/249 1/322 1/400 1/395 #46 BER 0.000 0.000 0.000 compressibility 1/129 1/183 1/236 1/296 1/386 1/443 BER 0.000 0.000 0.200 0.200 0.600 0.800 compressibility 1/137 1/190 1/245 1/313 1/422 1/587 : watermark is not detected (BER is considered as 1.000) Table 4: Tolerance test results for the test Set-4 (α =0.4, PSNR 44.03 db, SSIM 0.9945) 7
MPEG Quality factor 50 40 30 20 10 0 #2 #8 #20 #23 #46 compressibility 1/139 1/188 1/236 1/310 1/444 1/1001 compressibility 1/139 1/192 1/253 1/321 1/446 1/631 compressibility 1/139 1/190 1/239 1/317 1/438 1/501 compressibility 1/138 1/191 1/250 1/322 1/419 1/413 BER 0.000 0.000 0.000 0.000 0.000 compressibility 1/130 1/186 1/236 1/297 1/407 1/473 BER 0.000 0.000 0.000 0.000 0.000 0.200 compressibility 1/137 1/189 1/243 1/313 1/431 1/604 : watermark is not detected (BER is considered as 1.000) Table 5: Tolerance test results for the test Set-5 (α =0.9, PSNR 39.98 db, SSIM 0.9831) MPEG Quality factor 50 40 30 20 10 0 #2 #8 #20 #23 #46 compressibility 1/136 1/194 1/248 1/315 1/413 1/1029 compressibility 1/140 1/194 1/253 1/323 1/446 1/603 compressibility 1/134 1/193 1/254 1/317 1/446 1/460 compressibility 1/137 1/193 1/253 1/320 1/383 1/380 compressibility 1/131 1/184 1/237 1/297 1/387 1/443 compressibility 1/136 1/192 1/249 1/314 1/415 1/583 Table 6: Tolerance test results for the test Set-6 (α =1.5, PSNR 39.50 db, SSIM 0.9825) 8