Compressed Domain Video Fingerprinting Technique Using The Singular Value Decomposition

Transcription

1 Compressed Domain Video Fingerprinting Technique Using The Singular Value Decomposition ABBASS S. ABBASS ALIAA A. A. YOUSSIF ATEF Z. GHALWASH Information Technology Department Faculty of computers and information, Helwan University Egypt Abstract: - A vast amount of video data is generated around the world every day. Fast and efficient storage, indexing, browsing, and retrieval of video are necessary for the development of various multimedia database applications. Video fingerprinting is a proven and commercially available technique that can be used for content based copy detection. Fingerprints are compact content-based signatures that summarize a video signal or another media signal. The conventional video fingerprinting techniques have one resemblance, in which video decompression is still required for extracting the fingerprint from the compressed video. In practical, faster computational time can be achieved if the fingerprint is extracted directly from the compressed domain. This paper presents a spatio-temporal video fingerprinting technique that works directly in the compressed domain using the singular value decomposition of the macroblocks types. Experimental results show that the proposed fingerprint is highly robust against most signal processing transformations. Key-Words: - Video Fingerprinting- Compressed Domain-SVD- Perceptual Hash 1 Introduction Due to the advances in compression technology, the expansion of low cost storage media, and the growth of the internet, digitally available video quantity has grown enormously. Digital video has opened up the potential of using video sources in ways other than the traditional serial playback. Application comprises video copyright management, video content identification and content-based video retrieval [1]. Video fingerprinting is receiving an increased attention as an alternative to the watermarking approach for persistent identification of images and videos [2]. Copy detection is the application that motivated the development of video fingerprinting technology [3]. In 2010 two billion videos a day were watched on YouTube and hundreds of thousands of videos were uploaded daily, at a rate of 24 hours of content every minute [4]. Hosting services like YouTube can use video fingerprinting techniques to automatically check the copyrights of videos uploaded by users. Fingerprinting is an important tool for automated multimedia identification and has found applications in content filtering on usergenerated content (UGC) websites, automatic multimedia tagging [5] and multimedia database deduplication [4]. In comparison with watermarking, no data embedding is required since the content itself would be used to generate the video discriminative features [6]. Video is typically compressed to reduce the bandwidth requirements over networks. The compressed domain video processing techniques use the information extracted directly from compressed bitstream, and therefore are more advantageous than the uncompressed domain. It is shown that in MPEG video decompression; approximately 40% of the CPU time is spent in Inverse Discrete Cosine Transform (IDCT) calculations, even when using fast DCT algorithms [7]. This paper propose a video fingerprinting technique that work directly in the compressed domain at the stage of variable length decoding(figure[1], (see the appendix)), so it is computationally more efficient than uncompressed domain techniques and even the methods that utilize DCT coefficients that are partially decoded. The paper is organized as follows: literature survey and related work is given in section (2). Section (3) describes in detail the proposed technique. Experimental results, comparisons and discussion are given in section (4). Finally, Section (5) concludes the paper and highlights some directions for future work. ISBN:

2 2 Literature Survey Oostveen et al. [8] present the concept of video fingerprinting or hash function as a tool for video identification. They have proposed a spatiotemporal fingerprint based on the differential of luminance of partitioned grids in spatial and temporal regions. Coskun et al. [9] proposed a technique to extract a fingerprint based on 3Ddiscrete cosine transform (DCT) and a novel randomized basis set (RBT). Lee et al. designed a video fingerprinting method based on the centroid of gradient orientations [10]. Roover et al. proposed radial projection based hashing (RASH) [11] wherein they compute the variance of pixel values on a set of lines articulated around the center of the frame followed by a onedimensional DCT. Ordinal measures are used to extract video signatures for the spatial and the temporal dimensions of the video. And recently, the extended temporal ordinal measure is proposed which has more robustness than its predecessors [12, 13, and 14]. Ordinal measures are independent of absolute intensity scale and invariant to monotone transformations of intensity values. The 3- dimensional discrete wavelet transform is used in [15] to compute a hash of a group-of-frames from the spatiotemporal low-pass band of the transform. The local variances of the wavelet coefficients in the band are used to derive the hash. Joly et al. extracted local fingerprints around interest points in [16]. Zhao et al. extracted PCA- SHIFT descriptors for video matching in [17]. Malekesmaeili.M et al designed video fingerprint technique that carries both temporal and spatial information's [18]. The video is divided into frames and the weighted average of the frames is taken to obtain TIRI (Temporally Informative and Representative Images) then the DCT based hashing is applied to obtain the fingerprint [19]. So far, too fewer techniques are known to propose video fingerprinting that work directly in the compressed domain, one of them used DC coefficient to model the fingerprint [20]. Also, M. AlBaqir [21], proposed a video fingerprinting method, in which motion vectors were used to model the fingerprint. He utilized the motion vector to construct approximated motion field. 3 Proposed Technique In general, MPEG normally classify the video frames into I (intra) frame, P (predicted) frame and B (bi-directional) frame, and each frame is divided into macroblocks (MB). I frames can only have Intra blocks. P and B pictures can have different modes according to motion content. If intra coding is selected, the corresponding MB is encoded individually by exploiting the two dimensional discrete cosine transform (2D-DCT) coefficients. On the other hand, if inter coding is selected, the MB is then encoded using motion estimation-motion compensation (ME/MC) algorithms. In this paper the macroblock states are used to construct the proposed video fingerprint by accumulating in a graylevel-like image and then the resultant image is summarized by exploiting the local and global information within it. Figure [2] depict the proposed technique diagram. MPEG Video Variable Length Decoding Macroblocks Types 10-Bin Histogram Image Construction SVD Video Fingerprint Figure [2]: Proposed Technique Block Diagram The compressed MPEG video clip is parsed to extract the macroblocks information. Only VLC decoding and number of macroblocks times addition operation is necessary to obtain the MB data (see the rounded rectangle zone in figure [1] (see the appendix)). Then, 10-bin one dimensional histogram is constructed using the macroblocks types (Table [1]). The macroblocks types used to generate the macroblock histogram aggregate not only the simple types in the normal scenes but also the types that occurred in change-like scenes. So, this histogram carries intrinsic clues to the spatial and temporal ISBN:

3 content of the video because the prediction in normal scenes is performed in P and B frames. When there is a scene change, this prediction drops significantly, which lead to some macroblocks to be encoded in intra mode. Strictly speaking, if a frame is inside a shot, then the macroblocks should be predicted well from previous or next frames. However, when the frames are on the shot boundary, the frames cannot be predicted from the related macroblocks, and a high prediction error occurs [22]. This causes most of the macroblocks of the P frames to be intra coded instead of motion compensated. Table [1]: The used 10-bin histogram fingerprint description Bin Interpretation 1 Skipped Macroblocks 2 Intra Macroblocks in I frames 3 Intra Macroblocks in P frames 4 Intra Macroblocks in B frame 5 Non-motion Compensation Macroblocks in P frames 6 Non-motion Compensation Macroblocks in B frames 7 Forward motion Macroblocks in P frames 8 Forward motion Macroblocks in B frame 9 Backward motion Macroblocks 10 Bi-directionally Macroblocks Let call the generated histogram as (Macroblock Type Information Histogram): t X X X X t 1, 2, 3, (1) Such that: 10 t X i i 1 (2) Where η t is the total number of MB in frame t After the macroblock type information histogram is generated for each frame, then a two dimensional image is constructed from concatenating the before mentioned histograms in vertically-like fashion. Finally, to capture the intrinsic content of the synthesized image, the proposed technique employs the local filtering in addition to SVD. The aim of using the local filtering is to exploit the neighbourhood relations in the synthesized image, let call this technique as proposed technique-1. On the other hand the goal of using the SVD is to capture the global nature of the synthesized image since SVD is already known for its ability to derive a low-dimensional refined feature space from a high-dimensional raw feature space, and capturing the essential structure of a data set within a feature set, let call this as proposed technique-2. The experiments in the following section are conducted to determine if the local filtering or SVD is better in capturing the intrinsic content of the synthesized image of the histogram of the macroblocks types. In the local filtering approach, the proposed technique-1 uses three types of local filters: the entropy-based (E W ), the standard deviation-based (S W ), and the range-based (R W ) respectively. Let the synthesized image denoted as I, the local filter used as h, and the resultant fingerprint as F. The overall process can be described as convolution operation as follows: M N F I ( m, n) h( x m, y n) (3) MN g 0 m 0 n E p( I ( w), g).log[ p( I ( w), g)] (4) w Where p (,) is the probability of gray level g in I and W is the local window used to perform the local filtering. S w I ( w) I ( w) 2 (5) Where I (w) is the arithmetic mean of I in the a local window W Rw max I ( w) min I ( w) (6) In the proposed technique-2 the synthesized image is considered as matrix I, where the singular values represent the energy of matrix I projected on each subspace. The singular values and their distribution carry useful information about the contents of I. Since only a very few number of singular values is enough as features because a few larger singular values dominate the SVD decomposition, and yet all others close to small values. If I has r significant eigenvalues, then we can view σq to be effectively equal to zero for q > r: I =U V T I r r q 1 q u q v T q (7) (8) ISBN:

4 4 Experimental Results To evaluate the performance of the proposed technique, a test set of 200 videos all taken from the ocean [23] was used. Then attacks were individually mounted on the videos as Table [2] illustrates, so a new test set equals to 3600 videos was generated. Also, the hardware spec of the PC used for the experiments was an Intel dual-core running at 2 GHz with 3GB memory. However, all tests were ran using a single core. Finally, the baseline technique used to compare with is the motion field [21] technique which also operates in the compressed domain as the proposed one to be a fair comparison. Three types of experiments are conducted to investigate and validate the proposed work. The first set is concerned with comparing the proposed technique- 1(local filtering-based) against the motion field technique, in addition to investigate the performance of the different local filters (entropy-based, rangebased and standard deviation-based). The second set is devoted to study the proposed technique-2 (SVDbased). Finally, the last one is dedicated to compare the performance of the two proposed techniques. Table [2]: Distortions used in the study Index Distortion 1 Watermark 2 Mosaic 3 Embossment 4 Horizontal Flipping 5 Vertical Flipping 6 Adding Horizontal Lines 7 Adding Vertical Lines 8 Cropping (Big Window) 9 Cropping (Small Window) 10 Contrast Adjustment(negative image) 11 Contrast Adjustment(maximum contrast) 12 Brightness Adjustment (-50%) 13 Brightness Adjustment (-25%) 14 Brightness Adjustment (+50%) 15 Brightness Adjustment (+25%) 16 Different Bit Rate(512 Kbps) 17 Different Bit Rate(800 Kbps) Table [3]: Average retrieval rate across all distortions Technique Performance Motion Field Technique 74.12% 41.76% Using Entropy 72.35% Using Range 80.59% Using Standard Deviation Proposed Technique % As Table [3] shows the proposed technique-1 using the standard deviation as local filter outperforms the motion field technique. Also, the table shows that using the entropy gives poor performance against the range and the standard deviation. This can be attributed to blurred effect of the entropy as local filter against the synthesized image of the macroblocks types. Finally, the proposed technique- 2 (SVD-based) is better than the proposed technique-1 (local-filtering-based) and also it is better than the motion filed technique. For a detailed comparison, Figure [3] (see the appendix) clarifies the performance of the proposed work and the baseline technique distortion by distortion. It shows that the proposed work is better than the motion field technique especially at the flipping and adding the horizontal lines (distortion 5 and 6 respectively). An acceptable reasoning to the results is the motion field technique try to build the motion trajectories of the video content depending on using the available information in the compressed domain, but not all the macroblocks in the compressed domain carry motion information rather a lot of them are labelled as NO_MC or skipped macroblocks. On the other hand, the proposed technique alleviates this drawback by using the information associated with each macroblock in the compressed stream (the macroblocks types). Also, the global filtering operation represented by the SVD is better than the local filtering represented by the standard deviation. This can be attributed due the ability of the SVD to exploit the main characteristics of the synthesized image and in the same the time the local filtering smear the discriminated features that exist in the synthesized image. 5 Conclusion This paper proposed a video fingerprinting technique that availed of the macroblock states in the video compressed domain. Despite of its low computational overhead, the proposed work gave promising results. The proposed work indicated the importance of the macroblocks states in capturing the intrinsic information available in the compressed video in addition to the discriminative power of the SVD upon the cumulative image of the macroblocks types. One direction for future work will be combining the proposed technique with the compressed domain watermarking methods to design a robust broadcast monitoring fingerprinting system. ISBN:

5 References: [1] S.Cheung and A. Zakhor, Efficient video similarity measurement with video signature, In IEEE Trans. on Circuits and System for Video Technology, vol. 13, 2003,pp [2] S-A. Berrani, L. Amsaleg, and P. Gros, Robust content-based image searches for copyright protection, in Proc. of ACM Int. Workshop on Multimedia Databases,,2003, pp [3] S. Lee and C. Yoo, Robust video fingerprinting for content-based video identification, IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 7, 2008, pp [4] S. Paschalakis, K. Iwamoto, P. Brasnett, N. Sprljan, R. Oami, The MPEG-7 Video Signature Tools for Content Identification, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 7, 2012, pp [5] Perrott, A.J., Lindsay, A.T., Parkes, A.P., "Real-time multimedia tagging and contentbased retrieval for CCTV surveillance systems", In: Proc. of SPIE Internet Multimedia Management Systems, vol. 4862,2002, pp [6] Frederic Lefebvre, Bertrand Chupeau, Ayoub Massoudi and Eric Diehl, "Image and Video Fingerprinting: Forensic Applications", SPIE Conference on Media Forensics and Security, San Jose, CA, [7] S.-W. Lee, Y.-M. Kim, and S.W. Choi, "Fast Scene Change Detection using Direct Feature Extraction from MPEG Compressed Videos", IEEE Transactions on Multimedia, vol. 2, 2000, pp [8] J. Oostveen, T. Kalker, and J. Haitsma,"Feature extraction and a database strategy for video fingerprinting". The 5th International Conference on Recent Advances in Visual Information Systems, London, UK, Springer- Verlag, 2002, pp [9] B. Coskun, B. Sankur, and N. Memon, Spatiotemporal transform based video hashing, IEEE Transactions on Multimedia, vol. 8, no. 6, 2006,pp [10] Sunil Lee and Chang D. Yoo, "Video Fingerprinting Based on Centroids of Gradient Orientations", Proc. ICASSP, vol. 2, France, 2006, pp [11] C. D. Roover, C. D. Vleeschouwer, F. Lefebvre, and B. Macq, "Robust video hashing based on radial projection of key frames", IEEE Transactions on Signal Processing, vol. 53, no. 10,2005, pp [12] Dinkar N. Bhat, Shree K. Nayar, Ordinal Measures for Image Correspondence, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 4,1998, pp [13] Hua Xian-Sheng,Chen Xian,Zhang Hong- Jiung, "Robust video signature based on ordinal measure", Proceedings of the International Conference on Image Processing, China, 2004, pp [14] Heung Kyu Lee and June Kim, Extended Temporal Ordinal Measurement Using Spatially Normalized Mean for Video Copy Detection, ETRI Journal, vol.32, no.3, 2010, pp [15] Navajit Saikia, Prabin K. Bora, "Robust Video Hashing Using the 3D-DWT ", National Conference on Communications (NCC), 2011, ISBN: [16] A. Joly, C. Frelicot, and O. Buisson, "Robust content-based video copy identification in a large reference database", Proceedings of ACM International Conference on Image and Video Retrieval (CIVR), vol. 2728,, 2003, pp [17] W.-L. Zhao, C.-W. Ngo, H.-K. Tan, and X. Wu, "Near-duplicate Keyframe identification with interest point matching and pattern learning,", IEEE Transactions on Multimedia, vol. 9, 2007, pp [18] Malekesmaeili, M.,Fatourechi, Video Copy Detection Using Temporally Informative Representative Images, International Conference on Machine Learning and Applications,USA,2009, pp [19] E. Meenachi, G. Selva Vinayagam, C. Vinothini, "Enhancing Security in a Video Copy Detection System Using Content Based Fingerprinting", International Journal of Modern Engineering Research (IJMER),vol.2,no.4, July-Aug, 2012, pp [20] A. Mikhalev et. al., Video fingerprint structure, database construction and search algorithms, Direct Video & Audio Content Search Engine (DIVAS) project, Deliverable number D 4.2, February [21] M. AlBaqir, Video fingerprinting in compressed domain", MSc. Thesis, Delft university of technology, [22] B.-L. Yeo and B. Liu., Rapid Scene Analysis on Compressed Video, IEEE Transactions on Circuits and Systems for Video Technology, vol. 5, no. 6, 1995, pp [23] ReefVid: Free Reef Video Clip Database [Online].Available: ISBN:

6 Appendix (A) MPEG Parsing VLD Inverse Quantization IDCT Post Processing Motion Compensation Reference Frames Raw Figure [1]: Full decompression versus partial decompression of the compressed video (the bold segmented rectangle is the zone where the partially decoded-based techniques work and the rounded rectangle is where the proposed technique work). Distortion Different Bit Rate(800 Kbps) Different Bit Rate(512 Kbps) Brightness Adjustment (+25%) Brightness Adjustment (+50%) Brightness Adjustment (-25%) Brightness Adjustment (-50%) Contrast Adjustment(maximum Contrast Adjustment(negative image) Cropping (Small Window) Cropping (Big Window) Adding Vertical Lines Adding Horizontal Lines Vertical Flipping Horizontal Flipping Embossment Mosaic Watermark Average Retrieval Rate % Proposed Technique-2 Motion Field Technique Figure [3]: The performance of the proposed fingerprint against the baseline technique across all used distortions. ISBN: