Detecting Re-captured Videos using Shot-Based Photo Response Non-Uniformity

Transcription

1 Detecting Re-captured Videos using Shot-Based Photo Response Non-Uniformity Dae-Jin Jung 1, Dai-Kyung Hyun 1, Seung-Jin Ryu 1, Ji-Won Lee 1, Hae-Yeoun Lee 2, and Heung-Kyu Lee 1 1 Department of CS, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, Republic of Korea {djjung,dkhyun,sjryu,jwlee,hklee}@mmc.kaist.ac.kr 2 Department of Computer Software Engineering, Kumoh National Institute of Technology, Sanho-ro 77, Gumi, Gyeongbuk, Republic of Korea haeyeoun.lee@kumoh.ac.kr Abstract. With advances in digital camcorders, re-capturing commercial videos called camcorder theft is getting a big problem. In this paper, we propose an automatic detection method for re-captured videos based on the photo response non-uniformity (PRNU). To discern a re-captured video, a given video is divided into shots first. Several usable shots are selected and PRNU is estimated from each of the shots. Using peak-tocorrelation energy (PCE), a connection matrix, which indicates which shots were recorded with a specific camcorder, is constructed. Then, false negative connections are corrected by using Warshall s algorithm. With the number of connections from connection matrix, the given video is determined whether it was the re-captured or not. The experimental results show that the proposed method performs well even with compressed and scaled re-captured videos. Keywords: Forensics, Photo Response Non-Uniformity (PRNU), Recaptured video 1 Introduction With highly sophisticated IT technologies, digital camcorders that are capable of producing high quality footage with low prices and easy usage have been developed. Those advantages of using digital camcorders make many people use digital camcorders more common. Furthermore, traditional analog videos in the movie industry are also replaced by digital videos since digitally recorded movies are cheap and easy to be edited and stored compared with the traditional ones. Digital camcorders come into wide use due to their great benefits, however, increase in digital camcorder use brought many misuses. The most common abuse is re-capturing the commercial videos, called camcorder theft. Approximately 90% of newly released movies are re-captured in the theater with digital camcorders. The illegally re-captured videos are the largest source of fake DVDs and unauthorized copies distributed through the Internet [1]. As a result, the

2 2 Dae-Jin Jung et al. camcorder theft causes a great loss on movie industry and becomes a big problem. (a) (b) Fig. 1. An example of captured shots from a movie : (a) captured shot from original video, (b) captured shot from re-captured video In early days, re-captured videos had low quality so they could be easily detected by naked eyes. However, with the highly functional optical device technology, the quality of re-captured videos is improved. As shown in Fig. 1, the re-captured video is now comparable to the original video. Therefore, we need an automatic technique which can detect re-captured videos. Some studies were proposed for protecting videos using watermarking techniques against camcorder theft. The representative study was introduced by Lee et al. [2]. Their scheme was designed to be robust to camcorder theft and showed robustness. However, the watermark degrades the quality of videos. Also, the watermarking way requires an embedding process during movie playback. Cao et al. proposed a method that identifies re-captured images on LCD screens [3]. Forensic features such as local binary pattern, multi-scale wavelet statistics, and color features were extracted from image sets. By using the extracted features, a probability support vector machine classifier was trained and then tested. Their scheme could discriminate re-captured images with good qualities from original images with equal error rate lower than 0.5%. However, their method took too much time and could not be applied to video directly. The re-projected video detection by estimating a skew parameter was proposed by Wang et al. [4]. Their method could detect the re-projected video with some frames and could have much lower false positive by extracting more feature points. However, the feature points needed to be positioned in ridged body geometry. In this step, some feature points not on the ridged body geometry should be removed manually since it is hard to check those points automatically. In this study, we propose a method to discriminate the re-captured video based on the shot-based photo response non-uniformity (PRNU). The proposed method can discriminate re-captured vieo without any additive information and it is designed for videos. Moreover, the entire procedure of the proposed method performs automatically.

3 Detecting Re-captured Videos 3 The rest of this paper is structured as follows. The differences between original videos and re-captured videos are analyzed in Sec. 2. Then, the detail of the proposed method is explained in Sec. 3. Experimental results are exhibited in Sec. 4 and Sec. 5 concludes. 2 Differences between original and re-captured videos In this chapter, we describe the differences between original and re-captured videos. These differences are caused by the following factors: 1) Different recording devices: The original videos can be recorded by analog cameras or digital camcorders. Even though digital camcorders provide several benefits such as editing efficiency, reducing film cost, easy process to insert CGs, and etc., analog film cameras are still used because of their own characteristics such as high quality, soft shades of colors, and so on. On the other hand, the re-captured videos are mostly recorded by digital camcorders. Compact size, light weight, and easy manipulation make easier for pirates to handle digital camcorders in theaters without being observed. 2) The number of cameras used in recording: In the original videos, multiple cameras are used to record shots. For example, two or more cameras are used to shoot talking two actors; one for one actor, another for another actor, and the other for both actors. It means that each shot in the original videos has high probability to be recorded by different cameras. On the contrary, only a single digital camcorder is used to re-capture the original videos because pirates do not need multiple camcorders to re-capture videos. 3) Different post-processing: Original videos are edited by huge amount of postprocessing in general. As discussed above, original videos are recorded by multiple cameras. Each camera has unique characteristics such as color tone, contrast, brightness and so on. Thus, post-processing for each shot is essential to harmonize the whole content. Furthermore, it is usual to insert CGs and other visual effects into shots. However, re-captured videos are not edited by much post-processing. Only some of them are re-compressed or resized for convenience. Above three differences can affect PRNU of the original and re-captured video. The PRNU is pixel variation under illumination. It was proposed to identify the source digital camera by Lukas et al. [5]. Digital camera has a charge coupled device or complementary metal-oxide-semiconductor sensor, and the PRNU is caused by sensor imperfection which is introduced in sensor manufacturing process. Since the PRNU is unique for each sensor, it is considered as a fingerprint of a digital camera. Also, the PRNU can be used to identify source digital camcorders. Therefore, three differences between original and re-captured videos and the characteristics of the PRNU, we can infer some properties for the re-captured video detection as follows: Spcifically, the shot-based PRNU has low correlation with each other if we estimate them from original shots. First, the shots from analog films do not

4 4 Dae-Jin Jung et al. have their own PRNU because analog cameras do not include any digital sensor. Therefore, the PRNU estimated from alnalog shots cannot be used to identify source analog camera. Second, even though PRNU is estimated from digitally recorded shots, their source camcorders would vary and the estimated PRNU would be damaged from heavy post-processing. There might be several original shots which are recorded by digital camcorders and edited by little post-processing. It may give high correlation among shots. However, those shots are still not be correlated with other shots which are taken from other digital camcorders. Thus, those correlated shots will be grouped, consequently the number of groups will be greater than one. This factor would be an evidence that the given video is original. In contrast, the shot-based PRNU of re-captured videos is highly correlated with each other. All shots in the re-captured video are taken from the same digital camcorder and they are edited by little and same post-processing for each shot. These conditions let the PRNU from the re-captured video be correlated each other. By exploiting these properties, we can differentiate the re-captured videos from original videos. 3 Proposed method Fig. 2. An overview of proposed re-captured videos detection We propose a method that can discriminate re-captured videos from original videos. Fig. 2 depicts the proposed method. Once a suspected video is given, the shot change detection process is performed to find suitable shots for PRNU estimation. After dividing the given video into shots, we estimate PRNU from each shot. Then peak-to-correlation energy (PCE) values between PRNU is calculated as a measure to find out whether those shots are taken from the same digital camcorder or not. With results of PCE values, we decide whether the given video is a re-captured video or not. 3.1 Shot change detection We first divide a given video into numbers of shots. A shot can be defined as a continuous strip of motion picture film recorded with a single camera. Accurate

5 Detecting Re-captured Videos 5 shot change detector, which divides a given video into shots, is important since wrong shot change declaration can affect the result of re-captured video detection. If two or more shots are declared as a single shot by a shot change detector, PRNU estimated from that shot will be mixed PRNU from plural cameras so that the false positive rate in PRNU comparison will be increased. In addition, if one shot is declared as two or more shots by a shot change detector, it can also increase false positive rate in re-captured video detection. A histogram comparison method is used for shot change detection because it has good performance and it is relatively fast [6]. Let H i (j) denote a histogram value for ith frame, where j is one of G possible gray levels and SD i is the sum of absolute differences between ith frame and (i + 1)th frame. Then the sum of absolute differences, SD i, is given by the following formula: SD i = G H i (j) H i+1 (j) (1) j=1 To use SD i for shot change detection with any size of video, SD i is normalized by frame size. If the normalized SD i is larger than a given threshold, the shot change is declared. Note that the operations in the equations appeared in this paper are element-wise. 3.2 PRNU estimation To find out whether the shots are taken from the same digital camcorder or not, PRNU is estimated from those shots and compared each other. PRNU estimation method for digital camcorders are proposed by Chen et al. [7]. The PRNU for digital camcorders is modeled as follow: I = g γ [(1 + K)Y + Λ + Θ s + Θ r ] γ + Θ q (2) where I denotes the sensor output compromised by numerous in-camcorder processing, g does the color channel gain, γ is the gamma correction factor, K is PRNU multiplicative factor which can be used as a fingerprint of digital camcorder, Y is the light intensity, and Λ, Θ s, Θ r, Θ q denote dark current, shot noise, read-out noise, and quantization noise, respectively. Using first order Taylor expansion, simple form of this model can be obtained: I = I (0) + γi (0) K + Θ (3) Here, I (0) is the noise-free sensor output(frame) from one channel before demosaicing is applied. Θ is a noise component including above noises. We use simplified model in Eq (3) to estimate PRNU from each shot. To suppress the influence of the noise-free frame I (0), an estimate Î(0) of I (0) is subtracted from both sides of Eq (3). Î(0) can be estimated by using denoising filter which is a wavelet based filter [8]. W = I Î(0) = IK + (I (0) Î(0) ) + [(I (0) I)K] + Θ (4)

6 6 Dae-Jin Jung et al. PRNU factor K can be estimated by using Maximum Likelihood Estimation (MLE) method as N k=1 γ ˆK = WkÎ(0) k N (5) k=1 (Î(0) k )2 where W k is noise residual of kth frame. After MLE process, codec noise is removed by using denoising filter. Usually a video undergoes DPCM-block DCT transform which causes block artifacts [7]. The block artifact should be removed since it causes false correlations between uncorrelated PRNU. Wiener filter in frequency domain is used in our method to suppress the codec noise [9]. Then, PCE is calculated for a pair of PRNU to decide whether two shots are taken from same digital camcorder. To calculate PCE, we calculate normalized correlation first: (X X) (Y Y) NCC[X, Y] = (6) X X Y Y where X, Y are estimated PRNU, X is mean of X, X Y is dot product and X is the norm of X. Then PCE is calculated as follow [10]: P CE[X, Y] = NCC[X, Y](u 0, v 0 ) 2 E NCC[X,Y] (7) where (u 0, v 0 ) denotes the center location of correlation plane and E NCC[X,Y] is the correlation plane energy of NCC[X, Y]. If PCE of given two PRNU from two different shots is higher than certain threshold, then we decide that those two shots are taken from same source digital camcorder. 3.3 Detecting re-captured videos To decide whether a given video is re-captured or not, we investigate every PRNU from the video is related with each other. For this purpose, we use Warshalls algorithm which calculates the connectivity of a given graph [11]. Let the X i be the PRNU of selected shot, when i = 1,..., N. And we can consider the X i as a vertex. Then, a connection between two vertexes (X i, X j ) is decided by the value of P CE[X i, X j ]. If the P CE[X i, X j ] has greater value than predefined threshold T, X i and X j have connection to each other. In contrast, lower P CE[X i, X j ] value than threshold T implies X i and X j have no connection to each other. As a consequence a symmetric N N connection matrix is created after calculating the connectivity for every possible pair of PRNU. By using Warshalls algorithm, we can correct false negative connections. Fig. 3 depicts a simple case of the false negative connection correction by Warshall s algorithm. After processing Warshall s algorithm, we can decide the origin of a given video from the connection matrix. If the N N connection matrix has N 2 connections, we decide the given video is re-captured video because N 2 connections from N PRNU mean that the entire shots have same source digital

7 Detecting Re-captured Videos 7 (a) (b) (c) Fig. 3. A simple example of false negative connections correction : (a) Matrix with false negative connections, (b) Correcting false negative connections (c) Corrected false negative connections Resolution Main Camera(Digital/Analog) # x 720 Sony PMW-F3(Digital) # x 720 Sony PMW-F3(Digital) # x 720 Sony PMW-F3(Digital) # x 1080 Red One(Digital) # x 1080 Red One(Digital) # x 720 unknown(unknown) # x 1080 unknown(unknown) # x 720 Panavision camera(analog) # x 720 Panavision Panaflex Platinum(Analog) # x 720 unknown(unknown) Table 1. Information about original videos used in experiments(resolutions and their main cameras). camcorder. Otherwise, the given video is decided as original video since less than N 2 connections from N PRNU mean the given video has two or more source digital camcorders. 4 Experimental Results In this section, we examine the proposed re-captured video detection method. We used 4 digital camcorders (Samsung HMX-H205BD, Sony HDR-CX500, Sony HDR-CX550, and Sony HDR-SR10) to re-capture the original videos. We used 10 original videos and 5 of them were fully or partially recorded by digital video camcorders [12]. 40 videos were created by re-capturing 10 original videos with 4 digital camcorders. The resolution of the original videos varied from 1280x720 to 1920x1080 and the resolution of re-captured video was set as 1920x1080. Specific information about original videos used in experiments is in Table 1. To estimate PRNU, we divided each video into shots using the shot change

8 8 Dae-Jin Jung et al. detector. In shot change detection, frames are converted into gray frames to calculate the histogram differences. Before estimating PRNU from divided shots, some shots unsuitable for PRNU estimation are excluded. More specifically, shots constructed with small number of frames or dark frames are excluded since those shots can increase false negative rate in PRNU comparison. We extracted 200 successive frames from a shot in the PRNU estimation step. Fig. 4. PCE values in log scale of shots from the same camcorders shots from the different camcorders. Before testing proposed method, the threshold T for PRNU comparison needs to be set. To decide the adequate threshold T, 2400 pairs of PRNU from same camcorders and 2400 pairs of PRNU from different camcorders are prepared. Using PCE measurement, a scatter plot of PCE values in log scale for those pairs was obtained. As shown in Fig. 4, PCE values can be divided by simple straight line whose values is 80. Thus, we set 80 as a threshold T. 4.1 Re-captured video detection Experiment We tested re-captured video detection for 10 original videos and 10 videos for each digital camcorder, totally 40 re-captured videos. 20 shots are collected from each video(n = 20). Table 2 shows the result of re-captured video detection. Items in the table is the ratio of connections in N N connection matrix. Every video had at least 20 connections in diagonal line in connection matrix since each shot was correlated with itself. Original videos had lower number of connections than N 2 since it was not recorded by single digital camcorder. On the contrary,

9 Detecting Re-captured Videos 9 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Original movie Samsung HMX-H205BD Sony HDR-CX Sony HDR-CX Sony HDR-SR Table 2. Connection ratio for original videos and re-captured video (20 shots were used). every re-captured video had N 2 connections because it had only single source digital camcorder. In this experiment, the detection ratio of re-captured videos was 100% even before applying Warshall s algorithm. 4.2 Compression Experiment Fig. 5. Detection ratio for compressed videos with different quality factors. We also tested the robustness to compression. Re-captured videos were compressed with different quality factors (QFs) while the resolution was not changed. MPEG4 (AVC/H.264) was used in re-encoding. Fig. 5 shows the result for compressed re-captured videos. For QF , the proposed method showed 100% detection ratio. A few false negative connections appeared in QF 70, but all of false negative connections are corrected by using Warshall s algorithm. For QF 60, the detection ratio dropped to 35% because some shots which had no connection to other shots had appeared. Those shots were not able to be corrected by Warshall s algorithm. However, QF 60 is not commonly used in video compression due to severe quality degradation such as block artifacts.

10 10 Dae-Jin Jung et al. 4.3 Scaling Experiment Re-captured videos were scaled with various scale factors (SFs) while QF was set as 100. Since up-scaling is rare for videos, we only tested for SFs lower than 1. MPEG4 (AVC/H.264) was used for re-encoding. Fig. 6 shows the result for scaled re-captured videos. The proposed method showed low detection ratio for Fig. 6. Detection ratio for scaled videos with different scaling factors. SF 0.3 which is not parameter for common video resizing. However, the proposed method detected most of re-captured videos which were scaled with SF Combinational Experiment Combinational experiment was also conducted. Usually, re-captured videos are re-encoded before being redistributed. The common options for re-encoding are QFs higher than 80% and SFs higher than 0.5. Thus, we tested proposed method for re-captured videos which were re-encoded with parameters of QF 80 and SF 0.5. And the proposed method detected them 100%. This result is meaningful since those parameters are common for re-encoding videos. We did not conduct further geometric distortions such as affine transform because they are not common for videos. Even if any geometric distortion is proceeded for a re-captured video, every PRNU estimated from shots will be synchronized since all frames in the video are manipulated by the same distortion. Eventually, the re-captured video which has undergone any geometric distortion will be detected by the proposed method if the distortion does not ruin PRNU information severely. 5 Conclusion In this paper, we have investigated to detect the re-captured videos. The proposed method operates automatically for a given video and does not use any

11 Detecting Re-captured Videos 11 additive information such as watermarks. This proposed method is based on the photo-response non-uniformity (PRNU), which is unique fingerprint of digital image sensors. The proposed method consists of 3 steps. First, a given video is divided into shots. Then, PRNU is estimated from collected N shots. At last, an N N connection matrix is created by evaluating PCEs for each pair of N shots. Finally, we can decide the given video is re-captured or not with the result of the connection matrix. Experimental results show that proposed method performs excellent in detecting re-captured videos. The proposed method performs well even a given video is re-compressed and re-scaled. However, the proposed method is still weak against severe attacks. Therefore, our future work is to detect re-captured videos even they are re-compressed with low quality and scaling factors. Acknowledgments. This research project was supported by Ministry of Culture, Sports and Tourism(MCST) and from Korea Copyright Commission in References 1. Motion Picture Association Of America, 2. Lee, M.J., Kim, K.S., Lee, H.K.: Digital Cinema Watermarking for Estimating the position of the Pirate. In: Multimedia, IEEE transactions, pp (2010) 3. Cao,H., Kot, A.C.: Identification of recaptured photographs on LCD screens. In: Acoustics Speech and Signal Processing (ICASSP), pp IEEE press, Texas (2010) 4. Wang, W., Farid, H.: Detecting Re-Projected Video. In: International Workshop on Information Hiding, (2008) 5. Lukas, J., Fridrich, J., Goljan, M.: Digital camera identification from sensor pattern noise. In: IEEE Trans. Information Forensics and Security, pp (2006) 6. Zhang, H.J., Kankanhalli, A., Smoliar, S.W.:Automatic partitioning of full-motion video. Multimedia Syst. vol. 1, pp (1993) 7. Chen, M., Fridrich, J., Goljan, M., Lukas, J.: Source Digital Camcorder Identification Using Sensor Photo Response Non-Uniformity. In: IThe International Society for Optical Engineering (SPIE), (2008) 8. Mzhqak, M.K., Kozintsev, I., Ramchandran, K.: Spatially Adaptive Statistical Modeling of Wavelet Image Coefficients and its Application to Denoising. In: Acoustics, Speech, and Signal Processing (ICASSP), Vol. 6, pp IEE press, Arizona, (1999) 9. Chen, M., Fridrich, J., Goljan, M., Lukas, J.: Digital Imaging Sensor Identification (Further Study). In: The International Society for Optical Engineering (SPIE), (2007) 10. Kumar, B.V.K.V., Hassebrook, L.: Performance measures for correlation filters. In: Applied Optics, Vol. 29, pp (1990) 11. Warshall, S.: A theorem on boolean matrices. In: Journal of the ACM (JACM), Volume 9, pp (1962). 12. The Internet Movie Database,