Robust Image Identification without any Visible Information for Double-Compressed JPEG Images

Robust Image Identification without any Visible Information for Double-Compressed JPEG Images Kenta Iida and Hitoshi Kiya Tokyo Metropolitan University, Tokyo, Japan E-mail: iida-kenta1@ed.tmu.ac.jp, kiya@tmu.ac.jp Abstract A robust identification scheme for JPEG images is proposed in this paper. The aim is to robustly identify JPEG images generated from the same original image, under various compression conditions assumed for images uploaded to social networks. Conventional schemes are not applicable to identification of double-compressed images because they do not consider the errors caused by double-compression. In the proposed scheme, the use of new properties of DCT coefficients, in which the JPEG compression errors are considered, allows us to identify double-compressed images. In addition, the proposed one can carry out identification without using any visible information. This scheme is well-suited for the uploading images to social networks and for image retrieval and forensics. Experimental results demonstrate that the proposed scheme is effective in terms of the querying performance, even if images are doublecompressed. Index Terms Image identification, JPEG, social networks I. INTRODUCTION The growing popularity of social networks (SNs) like Twitter and Facebook has opened new perspectives in many research fields, including the emerging area of multimedia forensics. The huge amount of images uploaded to SNs are generally stored in a compressed format as JPEG images, after being re-compressed using compression parameters different from those used for the uploaded images [1] [3]. Several identification schemes and hash functions for compressed images have been proposed [4] [17]. They have been developed for the various purposes: producing evidence regarding image integrity, image retrieval and so on. However, those schemes do not consider performances for doublecompressed images. It is assumed that images stored in the servers of SNs are double-compressed. Therefore, identification of double-compressed images has to be considered for applications using images uploaded to SNs. In addition, considering that the features of uploaded images are leaked from a database, the features have to provide no visible information in terms of privacy concerns or copyright protection. The conventional schemes for identifying images can be broadly classified into two types: compression-methoddependent and compression-method-independent. Image hashing-based schemes [15] [17] correspond to the second type. Although they are robust against lossy compression, they did not evaluate querying performances for doublecompressed images. Moreover, they need to decompress images before carrying out identification. This paper focuses on the first type, which are generally strongly robust against differences in compression conditions. Conventional compression-method-dependent schemes [8], [9], [12], [13] are robust against differences in compression ratio, due to the use of the positive and negative signs of discrete cosine transform(dct) coefficients. However, they need to be combined with a security technique utilizing a secret key such as the fuzzy commitment scheme [12], [13], to securely protect the features that provide visible information. Moreover, they do not consider JPEG errors in double-compressed images. A robust scheme for identifying images compressed under the various compression conditions has been proposed [14]. This scheme does not require the protection of features because visually protected features are used. However, this scheme does not consider the errors caused in the process generating double-compressed images. Due to such situations, our proposed scheme can robustly identify JPEG images generated from the same original image, and not only single- but also double-compressed under various compression conditions assumed for images uploaded to SNs. In the proposed scheme, the use of new properties of DCT coefficients, in which three JPEG errors in double-compressed images are considered, can identify double-compressed images. It does this by using only the positions in which the DCT coefficients have zero values as features. In addition, these features do not provide visible information. Simulations demonstrate the effectiveness of the proposed scheme. It outperforms conventional ones in terms of query performances of identification of double-compressed images, without providing any visual information. A. JPEG Encoding II. PRELIMINARIES The JPEG standard is the most widely used image compression standard. The JPEG encoding procedure can be summarized as follows. 1) Perform color transformation from RGB space to YC b C r space and sub-sample C b and C r. 2) Divide an image into non-overlapping consecutive 8 8- blocks. 3) Apply DCT to each block to obtain 8 8 DCT coefficients S. 4) Quantize S using a quantization matrix Q. 5) Entropy code using Huffman coding.

In step 4), a quantization matrix Q with 8 8 components is used to obtain a matrix S q from S. For example, ( ) S(u, v) S q (u, v) = round, 0 u 7, 0 v 7 (1) Q(u, v) with round Q(u, v) = round ( Q0 (u,v) 5000 QF ( 100 Q0 (u,v) (200 2 QF ) 100 ), 1 QF < 50, ), 50 QF 100, (2) where S(u, v), Q(u, v), S q (u, v) and Q 0 (u, v) represent the (u, v)element of S, Q, S q and Q 0 respectively. The round(x) function is used to round a value x to the nearest integer value and x denotes the integer part of x. The quality factor QF (1 QF 100) parameter is used to control a matrix Q. The large QF results in a high quality image. All components of an initial quantization matrix Q 0 as well as QF are positive numbers. The data for quantization matrices are included in the header of the JPEG codestream. B. Notations and Terminologies The notations and terminologies used in the following sections are listed here. Single-compressed image is an image compressed by the JPEG standard one time. Double-compressed image is an image generated from a single-compressed image after decoding the singlecompressed one. X i represents a single-compressed image X i. X i can be Q for a query image Q and O for an original image O (all images have the same size). Q represents a single-compressed image and Q represents a double-compressed JPEG image generated from Q. A query image Q can be replaced with Q or Q. M represents the number of 8 8-blocks in an image. N represents the number of DCT coefficients used for identification in each block (0 < N 64). s(m, n) indicates nth DCT coefficient in mth block in an image (0 m < M, 0 n < N). X i (m, n), q(m, n), q (m, n) and q (m, n) indicate nth quantized DCT coefficient in mth block in images X i, Q, Q and Q respectively (0 m < M, 0 n < N). Q Xi,L, Q Q,L, Q Q,L and Q Q,L indicate the luminance quantization matrices used to generate images X i, Q, Q and Q respectively. Q Xi,L(n), Q Q,L (n), Q Q,L(n) and Q Q,L(n) indicate nth components of Q Xi,L, Q Q,L, Q Q,L and Q Q,L respectively (0 n < N). QF Xi, QF Q, QF Q and QF Q indicate quality factors used to generate X i, Q, Q and Q respectively. C. Image Identification Let us consider a situation in which there are two or more compressed images generated under different or the same coding conditions. They originated from the same image and were compressed using the various coding parameters including Fig. 1. Scenario initial quantization matrices and quality factors. We refer to the identification of those images as image identification. In other words, if the images did not originate from the same image, they are not identified. The requirement of the robustness is to robustly identify images against differences in coding conditions assumed in compression by SNs in this paper. 1) Scenario The scenario is considered in this paper, as shown in Fig. 1. In this scenario, a client/user identifies images by using an identification tool. When the client/user uploads JPEG images to a database server like Twitter, the features of these images are extracted and then stored in a client/user s database by the client/user. The uploaded images are re-compressed under different coding parameters and then are stored in a database server. Finally, the client/user carries out the identification after extracting the features from a query image i.e. an uploaded image. Images uploaded to SNs are generally compressed again in SNs, so that image identification schemes have to consider JPEG errors in double-compressed images. The proposed scheme considers the compression errors to identify images uploaded to SNs. In addition to re-compression, it is known that SN providers often resize uploaded images, if certain conditions are satisfied. For instance, when the filesize of images is larger than 3MB or the size of images is larger than 4096 4096, uploaded images will be resized in Twitter [3]. If both conditions are not satisfied, unresized images can be downloaded by adding orig to the URL for image view, even when displayed images are resized. The proposed scheme aims to identify images which have the same size. Moreover, the proposed scheme uses features without any visible information, so that unprotected features can be used for the scenario without any secret keys. Therefore, even when the features are leaked from a client/user s database, it is not required to protect the features. 2) Applications The proposed scheme aims at detecting images generated from the same original image as that of a query image. On social networks like Twitter, uploaded images are generally recompressed using coding parameters that are different from

double-compressed ones. Fig. 2. JPEG errors in single-/double-compression the uploaded ones. Therefore, the identification system is required to robustly identify images against differences in coding conditions. Target applications are: finding the original image of an uploaded image, detecting whether uploaded image has been altered, determining whether an uploaded image has been illegally distributed. Note that the proposed scheme does not aim to retrieve images visually similar to a query image. D. Errors in JPEG Encoding and Decoding The JPEG standard is generally used as a lossy compression method, so that there are some errors in pixel values between an original image and the image decoded after encoding the original one. Due to the errors, DCT coefficients of single-compressed images are different from those of doublecompressed ones, even if those images are generated from the same original image. In [18], the errors are classified into three types as below. Quantization error In the encoding process, quantizing and rounding DCT coefficients in step 4) generate this error. Rounding error In the decoding process, after applying inverse DCT (IDCT) to dequantized DCT coefficients, the obtained float numbers are converted to integer ones. This operation generates rounding error. Truncation error In the decoding process, after applying IDCT to dequantized DCT coefficients, the obtained float numbers need to belong to the range [0,255] in the spatial domain. The values that do not belong to [0,255] would be truncated to 0 or 255 respectively. This operation generates truncation error. These errors have to be considered for identification of JPEG images. Figure 2 shows the process for the generation of single-/double-compressed images, where e 1 represents the errors causing in the first decoding process. Three errors have to be considered for double-compressed images. Note that e 1 is not included in single-compressed images. III. PROPOSED IDENTIFICATION SCHEME Two new properties of DCT coefficients are proposed for identifying not only single-compressed images but also A. New Property of DCT Coefficients (Single Compression) It is verified from Eq.(1) that quantized DCT coefficients have the following property. It is assumed that two single-compressed images Q and X i are generated from the same original image O. When s(m, n) 0 and Q Xi,L(n) Q Q,L(n), the relation of DCT coefficients before rounding them is s(m, n) s(m, n) Q Q,L(n) Q Xi,L(n) In the other hand, if X i (m, n) = 0, i.e., ( ) s(m, n) X i (m, n) = round = 0, (4) Q Xi,L(n) the quantized DCT coefficient has to satisfy 0 From Eqs.(3) and (5), 0 (3) s(m, n) Q Xi,L(n) < 1 2. (5) s(m, n) s(m, n) Q Q,L(n) Q Xi,L(n) < 1 2. (6) In the case that the assumption s(m, n) 0 is replaced with s(m, n) 0, 1 2 s(m, n) s(m, n) < Q Xi,L(n) Q Q,L(n) 0 (7) is also satisfied. At this time, q (m, n) is calculated as below. ( ) s(m, n) q (m, n) = round. (8) Q Q,L(n) Therefore, X i (m, n) = 0 q (m, n) = 0 (9) is satisfied under Q Xi,L(n) Q Q,L(n). Similarly, when Q Xi,L(n) Q Q,L(n), q (m, n) = 0 X i (m, n) = 0. (10) B. New Property of DCT Coefficients (Double Compression) By extending the above discussion, a new property for identification of double-compressed images is also proposed here. Note that e 1 = 0 in Fig. 2 is assumed for the following discussion. When two single-compressed images Q and X i are generated from the same original image O, Q is generated from Q with Q Q,L(n) by following the equation. ( q q ) (m, n) Q Q,L(n) (m, n) = round. (11) Q Q,L(n) From this equation, is satisfied. q (m, n) = 0 q (m, n) = 0 (12)

Proceedings of APSIPA Annual Summit and Conference 2017 Here, the relation QXi,L (n) QQ,L (n)and QXi,L (n) QQ,L (n) is assumed. Under this assumption, Xi (m, n) = 0 q (m, n) = 0 and q (m, n) = 0 (13) is satisfied from Eqs.(9) and (12). Basing on this property under the relation QXi,L (n) QQ,L (n) and QXi,L (n) QQ,L (n), if Xi (m, n) = 0 q (m, n) = 0, (b) Image reconstructed from DCT signs Fig. 3. Visible information of DCT signs (14) it can be judged that Xi and Q have the different original images. In the case of identification of single-compressed images Xi and Q, the identification can be performed by replacing q (m, n) with q (m, n) in Eq.(14). Note that quality factor QFXi is required to be the highest value in QFXi, QFQ and QFQ to carry out identification, if images are compressed with the same initial quantization matrix Q0. C. Proposed Identification Scheme The proposed scheme uses the positions in which DCT coefficients have zero values as features. These features are used for identification, based on Eq. (14). Feature extraction and identification processes are explained, here. 1) Feature Extraction Process In order to extract and store features of image Xi, a client/user carries out the following steps. (a) Set values M and N. (b) Set m := 0 and n := 0. (c) Map a DCT coefficient Xi (m, n) into bxi (m, n) with 1 bit as { 0, Xi (m, n) = 0, bxi (m, n) = (15) 1, Xi (m, n) = 0, where bxi represents features of image Xi. (d) Set n := n + 1.If n < N, return to step (c). (e) Set n := 1 and m := m + 1.If m < M, return to step (c). Otherwise, store bxi as the features in the client/user s database. 2) Identification Process for Single-Compressed Images In order to compare single-compressed image Q with image Xi, a client/user extracts the features bq from Q as well as bxi. The client/user carries out the following steps. (a) Set values M and N. (b) Set m := 0 and n := 0. (c) Confirm whether Eq.(14) is satisfied by using bxi (m, n) and bq (m, n). If the equation is satisfied, the client/user judges that Xi and Q are generated from different original images and the process for image Xi is halted. (d) Set m := m + 1. If m < M, return to step (c). (e) Set n := n + 1 and m := 1. If n < N, return to step (c). Otherwise, the client/user judges that Xi and Q are generated from the same original image. 978-1-5386-1542-3@2017 APSIPA (a) Original image 3) Identification Process for Double-Compressed Images In the case of identification of double-compressed images, a client/user extracts the features bq from Q as well as that of single-compressed ones. The client/user carries out the following steps. (a) Set values M, N and th, where th is the threshold value of times that Eq.(14) is satisfied. (b) Set m := 0, n := 0 and count := 0, where count represents the number of times that Eq.(14) is satisfied. (c) Confirm whether Eq.(14) is satisfied by using bxi (m, n) and bq (m, n). If the equation is satisfied, count := count + 1. Otherwise, proceed to step(e). (d) If count > th, the client/user judges that Xi and Q are generated from different original images and the process for image Xi is halted. (e) Set m := m + 1. If m < M, return to step (c). (f) Set n := n + 1 and m := 1. If n < N, return to step (c). Otherwise, the client/user judges that Xi and Q are generated from the same original image. Actually, double-compressed images include truncation and rounding errors i.e, e1 = 0, although single-compressed ones do not. Therefore, in order to avoid the effect of e1, the threshold value th is used for identification of doublecompressed images. The conventional schemes which use the signs of coefficients as features [8], [9], [12], [13] have two important limitations. The first limitation is that these schemes do not consider identification of double-compressed images, and the second one is that the features have some visible information as shown in Fig.3 [19], [20]. IV. S IMULATION A number of simulations were conducted to evaluate the performance of the proposed scheme. We used the images in Head Pose Image Database (HPID) [21](see Fig. 4). HPID consists of face images of 15 persons and there are 186 images per person. We used 186 images of Person01. The images were compressed with several quality factors and the default initial quantization matrix in the encoder from IJG (Independent JPEG Group) [22]. In the simulations, the features were extracted from only Y components. The proposed scheme was compared with a secure featuresbased scheme [14], a state-of-art image hashing-based scheme [15] and a fuzzy commitment scheme (FCS)-based scheme APSIPA ASC 2017

TABLE II QUERYING PERFORMANCE FOR SINGLE-COMPRESSED IMAGES Fig. 4. Examples test images (384 288) TABLE I QUALITY FACTORS USED TO GENERATE JPEG IMAGES.DB 1, DB 2 AND DB 3 INDICATE DATABASES OF CLIENT/USER IN FIG.1 JPEG images Images stored as features in Quality factors DB 1 QF Xi = 90 DB 2 QF Xi = 92 DB 3 QF Xi = 95 Query images (Q ) QF Q = 85, 87, 90 Query images (Q ) QF Q = 85 [12], which consider the security as well as the proposed one. In the scheme [15], the hamming distances between the hash value of a query image and those of all images in each database were calculated, and then images that had the smallest distance were chosen as the images generated from the same original image as the query, after decompressing all images. A. Identification of Single-Compressed Images At first, querying performances in the case of identification of single-compressed images i.e., identification between X i and Q was evaluated. Table I summarizes the quality factors used to generate JPEG images, where DB 1, DB 2 and DB 3 indicate the databases of client/user in Fig.1. For instance, features stored in the database DB 1 were extracted from 186 images compressed with QF Xi = 90. Similarly, features in DB 2 were extracted from images with QF Xi = 92, and ones in DB 3 were extracted from images with QF Xi = 95. 558(=186 3) images compressed with QF Q = 85, 87, 90 were used as a query image for DB 1, DB 2 and DB 3. Therefore, identification was performed 186 558 times for each database generated from images to evaluate the proposed scheme. The proposed scheme identified images by using the algorithm shown in III-C 2). Table II shows the true positive rate (TPR) and false positive rate (FPR), defined by T P R = T P T P + F N, F P R = F P F P + T N, (16) where TP, TN, FP and FN represent the number of true positive, true negative, false positive and false negative matches respectively. Note that T P R = 100[%] means that there are no false negative matches and F P R 0[%] means that there are some false positive matches. It is confirmed that the proposed and secure features-based schemes did not produce any false negative matches for all databases, although the other schemes did. Besides, only the image hashing-based scheme produced false positive matches. scheme proposed secure features [14] FCS [12] image hashing [15] QF Q = database 85 87 90 TPR[%] FPR[%] TPR[%] FPR[%] TPR[%] FPR[%] DB 1 100 0 100 0 100 0 DB 2 100 0 100 0 100 0 DB 3 100 0 100 0 100 0 DB 1 100 0 100 0 100 0 DB 2 100 0 100 0 100 0 DB 3 100 0 100 0 100 0 DB 1 0 0 0 0 100 0 DB 2 0 0 0 0 0 0 DB 3 0 0 0 0 0 0 DB 1 98.92 0.02 98.39 0.02 100 0.01 DB 2 99.46 0.01 100 0.02 99.46 0.01 DB 3 98.92 0.01 98.39 0.01 100 0.01 TABLE III QUERYING PERFORMANCE FOR DOUBLE-COMPRESSED IMAGES scheme proposed (th = 0) proposed (th = 5) secure features [14] FCS [12] image hashing [15] QF Q = database 85 87 90 TPR[%] FPR[%] TPR[%] FPR[%] TPR[%] FPR[%] DB 1 40.86 0 46.24 0 89.78 0 DB 2 40.86 0 46.24 0 90.32 0 DB 3 49.46 0 49.46 0 93.01 0 DB 1 100 0 100 0 100 0 DB 2 100 0 100 0 100 0 DB 3 100 0 100 0 100 0 DB 1 40.86 0 46.24 0 89.78 0 DB 2 40.86 0 46.24 0 90.32 0 DB 3 49.46 0 49.46 0 93.01 0 DB 1 0 0 0 0 0.54 0 DB 2 0 0 0 0 0 0 DB 3 0 0 0 0 0 0 DB 1 98.92 0.02 99.46 0.02 100 0.01 DB 2 99.46 0.01 99.46 0.02 100 0.01 DB 3 98.92 0.01 99.46 0.01 100 0.01 B. Identification of Double-Compressed Images Identification performances for double-compressed images i.e., identification performances between X i and Q were investigated. Double-compressed images were generated with the quality factors shown in Tab. I. In order to generate query images, images compressed with QF Q = 85, 87, 90 were decoded once and compressed with QF Q = 85. The proposed scheme identified images by using the algorithm shown in III-C 3). Table III shows the performances of identification of doublecompressed images. Only the proposed scheme with th = 5 did not produce false negative matches, the other schemes and the proposed scheme with th = 0 did. V. CONCLUSION Our proposed identification scheme for JPEG images uses only the positions in which DCT coefficients have zero values as features. The identification can be performed without visible information under the various coding conditions assumed for images uploaded to SNs, including the initial quantization matrices and quality factors. In addition, the proposed scheme does not in principle produce any false negative matches for identification of single-compressed images. Moreover, doublecompressed images can be identified by considering three JPEG errors. The simulation results showed that the proposed scheme has high query performances for classification, even if images were double-compressed.

REFERENCES [1] R. Caldelli, R. Becarelli, and I. Amerini, Image origin classification based on social network provenance, IEEE Trans. Information Forensics and Security, vol. 12, no. 6, pp. 1299 1308, 2017. [2] M. Moltisanti, A. Paratore, S.Battiato, and L.Saravo, Image manipulation on facebook for forensics evidence, in Proc. Int l Conf. on Image Analysis and Processing, 2015, pp. 506 517. [3] T. Chuman, K. Iida, and H. Kiya, Image manipulation on social media for encryption-then-compression systems, in Proc. APSIPA Annual Summit and Conference, 2017(to be published). [4] C.-Y. Lin and S.-F. Chang, A robust image authentication method distinguishing jpeg compression from malicious manipulation, IEEE Trans. on Circuits and Systems for Video Technology, vol. 11, no. 2, pp. 153 168, 2001. [5] Z. Fan and R. L. de Queiroz, Identification of bitmap compression history: Jpeg detection and quantizer estimation, IEEE Trans. on Image Processing, vol. 12, no. 2, pp. 230 235, 2003. [6] D. Edmundson and G. Schaefer, An overview and evaluation of jpeg compressed domain retrieval techniques, in Proc. IEEE ELMAR, 2012, pp. 75 78. [7] K.O. Cheng, N.F. Law, and W.C. Siu, A fast approach for identifying similar features in retrieval of jpeg and jpeg2000 images, in Proc. APSIPA Annual Summit and Conference, 2009, pp. 258 261. [8] F. Arnia, I. Iizuka, M. Fujiyoshi, and H. Kiya, Fast and robust identification methods for jpeg images with various compression ratios, in Proc. IEEE Int l Conf. on Acoustics Speech and Signal Processing Proceedings, 2006, vol. 2, pp. II II. [9] H. Kobayashi, S. Imaizumi, and H. Kiya, A robust identification scheme for jpeg xr images with various compression ratios, in Proc. Pacific-Rim Symposium on Image and Video Technology, 2015, pp. 38 50. [10] O. Watanabe, T. Iida, T. Fukuhara, and H. Kiya, Identification of jpeg 2000 images in encrypted domain for digital cinema, in Proc. IEEE Int l Conf. on Image Processing, 2009, pp. 2065 2068. [11] K. Iida and H. Kiya, Codestream level secure identification for jpeg 2000 images under various compression ratios, in Proc. APSIPA Annual Summit and Conference, 2017, pp. 1 6. [12] K.Iida and H.Kiya, Fuzzy commitment scheme-based secure identification for jpeg images with various compression ratios, IEICE Trans. Fundamentals, vol. 99, no. 11, pp. 1962 1970, 2016. [13] K. Iida, H. Kobayashi, and H. Kiya, Secure identification based on fuzzy commitment scheme for jpeg xr images, in Proc. EURASIP European Signal Processing Conf., 2016, pp. 968 972. [14] K. Iida and H. Kiya, Robust image identification with secure features for jpeg images, in Proc. IEEE Int l Conf. on Image Processing, 2017(to be published). [15] Y. Li and P. Wang, Robust image hashing based on low-rank and sparse decomposition, in Proc. IEEE Int l Conf. on Acoustics, Speech and Signal Processing, 2016, pp. 2154 2158. [16] Y. N. Li, P. Wang, and Y. T. Su, Robust image hashing based on selective quaternion invariance, IEEE Signal Processing Letters, vol. 22, no. 12, pp. 2396 2400, 2015. [17] Z. Tang, X. Zhang, X. Li, and S. Zhang, Robust image hashing with ring partition and invariant vector distance, IEEE Trans. on Information Forensics and Security, vol. 11, no. 1, pp. 200 214, 2016. [18] F. Huang, J. Huang, and Y. Q. Shi, Detecting double jpeg compression with the same quantization matrix, IEEE Trans. on Information Forensics and Security, vol. 5, no. 4, pp. 848 856, Dec 2010. [19] I. Ito and H. Kiya, One-time key based phase scrambling for phaseonly correlation between visually protected images, EURASIP Journal on Information Security, vol. 2009, no. 1, 2009. [20] I.Ito and H.Kiya, A new class of image registration for guaranteeing secure data management, in Proc. IEEE Int l Conf. on Image Processing, 2008, pp. 269 272. [21] N. Gourier, D. Hall, and J. L Crowley, Estimating face orientation from robust detection of salient facial structures, in Proc. Int l Workshop on Visual Observation of Deictic Gestures, 2004, vol. 6. [22] The independent jpeg group software jpeg codec, http://www.ijg.org/.