Empirical mode decomposition based blind audio watermarking

Transcription

1 DOI /s Empirical mode decomposition based blind audio watermarking Zhaoyang Fu Peng Zhang Wei Huang Liang Wang Sabu Emmanuel Guang Chen Springer Science+Business Media New York 2014 Abstract Multiparty-multilevel digital rights management of audio requires blind detection of multiple watermarks. The proposed audio watermarking method offers copyright protection based on analysis filterbank decomposition, psychoacoustic model and empirical mode decomposition (EMD). The novel blind audio watermarking algorithm embeds the watermark bits in the final residue of the subbands in the transform domain. The watermarking system performance is optimized by selecting appropriate segment length for applying EMD process and by selecting the number of subbands for watermark embedding. Experimental This work is supported by the national grants No and No approved by the National Natural Science Foundation of China, as well as the doctoral program of High Education of China No approved by the Ministry of Education, China. Z. Fu School of Automation, Northwestern Polytechnical University, Xi an, China P. Zhang ( ) School of Computer Science, Northwestern Polytechnical University, Xi an, China zh0036ng@nwpu.edu.cn W. Huang ( ) School of Information Engineering, Nanchang University, Jiangxi, China huangwei@ncu.edu.cn L. Wang S. Emmanuel School of Computer Engineering, Nanyang Technological University, Singapore, Singapore L. Wang mail.liang.wang@gmail.com S. Emmanuel asemmanuel@ntu.edu.sg G. Chen Xi an Communications Institute, Xi an, China chenguang322@gmail.com

2 results show that the proposed scheme is robust against various common signal processing manipulations while multiple watermark messages can be embedded. Keywords Blind audio watermarking Multiple watermarks Empirical mode decomposition (EMD) Psychoacoustic model Analysis filterbank decomposition 1 Introduction Over the past decades, significant effort has been focused on the digital rights management (DRM) of the digital media (audio, image, and video). A promising solution to this problem is the addition of a watermark to the digitized media, where the special information (the watermark message) is hidden in the original data in an imperceptible manner [8, 9, 11, 13 15, 17, 20, 29, 34, 37]. Compared to embedding watermarks into images, audio watermarking is a more challenging task due to the fact that the human auditory system (HAS) is more sensitive to distortions than the human visual system (HVS) [8, 22, 24], and that inaudibility is much more difficult to achieve than invisibility for images [17]. Also, compared to the visual signals, audio signals are represented by much less number of samples per time interval, which limits the watermark capacity for the audio signals. Several techniques in audio watermarking have been proposed to address these challenges, including the echo coding [3], phase coding [19], patchwork coding [3], low-bit coding [11] and spread spectrum [24]. Among all the audio watermarking techniques, the watermark bits can be embedded either in the transform domain or in the spatial domain. In this research work, the transform domain watermark is studied because it is demonstrated to be more robust against various attacks [8]. During the past few years, manyof the developed audio watermarking algorithms [8, 11, 19, 30] took advantage of the perceptual properties of the HAS in order to increase the robustness of the watermark message by maximizing its strength while embedding it in a perceptually transparent manner. Therefore, the psychoacoustic model is adopted to make the audio watermark message as inaudible as possible. In order to efficiently use the psychoacoustic model in the transform domain audio data, a polyphase filterbank is used for the time to frequency mapping of the original input audio. Existing multimedia watermarking in either transform domain or temporal domain tends to embed the watermark bits directly into the coefficients. The empirical mode decomposition (EMD) [21] is proved useful while processing the nonlinear and non-stationary time series [5, 21, 25]. By using the EMD method, any multi-component signal is decomposed into a set of intrinsic mode functions (IMFs) and the final residue. This residue is proved to be highly robust under Gaussian noise attack and MP3 compression [5]. Thus it is possible to embed the watermark bits robustly into the residue rather than the subband audio signal itself. The traditional two-party digital rights management architecture only involves a seller and a buyer. However the present-day business models require multiparty (such as the owner, distributors, sub-distributors and consumers) for the content delivery[31]. In order to satisfy all the security needs of the copyright owner and the distributors, multiple watermarks are required whereas each serving a different need. In a multiparty DRM scenario, there should be more concern on the watermarking capacity if each party embeds its own watermark separately into the digital content. It should be noted that, while multiple watermarks are embedded into the host audio, all the watermarks should be embedded simultaneously and should not interfere.

3 This paper aims to propose a blind audio watermarking technique which can achieve high watermarking capacity and robustness while embedding multiple watermarks in an imperceptible manner. Seok and Hong [28] proposed a blind audio watermarking scheme which employs human auditory based embedding and prediction based detection. The watermarks were embedded into the Waveform Audio File Format (WAV) audio signal at a rate of 128 bits per 15s. Arnold et al. [1] proposed a scheme for blind detection of multiple audio watermarks. In their paper, 3 watermarks are embedded into the WAV audio signal. Each watermark bit is embedded over the audio data consisting of number of blocks of size 512. Bassia et al. [2] proposed an audio watermarking method that embedded the watermarks in the temporal domain segmented audio signal. Each watermark bit is embedded in each of the audio samples. In order to obtain a higher performance, the watermark bits are repeatedly embedded in the audio with the embedding length of 2 17 samples. However when embedded with multiple watermarks, the subjective quality evaluation could not provide promising results (the resulting watermarking system would produce a noticeable distortion while 4 watermark messages are embedded). Cvejic et al. [10] proposed a spread spectrum based audio watermarking scheme in temporal domain and claimed a watermarking capacity of 14.7bits persecondfor themonoaudiosignal.in theirlater research [12], a spread spectrum based audio watermarking scheme in spectral domain is proposed, with the watermarking capacity increased to 27.1 bits per second. Cheng et al. [6] proposed a spread spectrum watermarking of MPEG-2 AAC audio and could achieve robust embedding rate at around 30 bits per second. Liu and Inoue [26] proposed a spread spectrum based audio watermarking. The watermark was represented by sinusoidal patterns consisting of phase-modulated sinusoids. The inaudibility and robustness of the watermarking scheme was proved when 76 bits of watermark messages were embedded into a time frame of 30s of the host audio. Wu et al. [34] presented a self-synchronized audio data hiding technique. The synchronization codes are embedded into the low-frequency subband in DWT domain. Promising results was obtained where the estimated data payload was 172 bits per second. However in the multiparty DRM scenario more capacity might be required. The rest of the paper is organized as follows. Section 2 gives an overview of the polyphase filterbank analysis, the audio masking and the empirical mode decomposition. The novel audio watermark embedding procedure is proposed in Section 3. Section 4 introduces the watermark extraction process. The experimental setup is described in Section 5.1. Section 5.4 illustrates the experimental results for the quality evaluation. The experimental results of the watermarking capacity and robustness, as well as the performance against signal processing attacks are given in Section 5.5 And finally the paper concludes in Section 6. 2 Polyphase filterbank, audio masking and empirical mode decomposition 2.1 Polyphase filterbank The subband analysis and synthesis filters defined in [23] are used as the polyphase filter bank in the experiment. The analysisfilterbank is used to split the PCM audio inputsignalinto M equally spaced subbands and provides the primary frequency separation for the watermark embedding process. A detailed description of the analysis filterbank decomposition procedure can be found in Section 3 A. To determine the watermark strength for the transform domain audio signal, the signal-to-mask ratios (SMR) of all the subbands are calculated based on the

4 psychoacoustic model [16]. The SMR is defined as the distance between the level of of the masker and the masking threshold. After the watermark bits are embedded into the audio signal, the synthesis filterbank is used for the reconstruction. The analysis filterbank and the synthesis filterbank are perfect reconstruction filterbanks, which means that the output from the watermark embedding system is just a watermarked delayed version of the input. Also, it should be noted that these filterbanks are critically sampled (i.e. the number of samples in the analyzed domain is identical to the number of samples in the time domain). 2.2 Audio masking Audio masking is the effect that a faint but audible sound, i.e., the maskee, becomes inaudible in the presence of another louder sound, i.e., the masker [16]. Gold and Morgan [7, 16, 27] give a detailed introduction of the audio masking effect. The audio masking is usually distinguished in frequency masking and temporal masking. The frequency masking is also referred to as the simultaneous masking and it is occurred when the maskee and masker both appear at the same time. Temporal masking is the characteristic of the auditory system where the maskee and masker are heard at different time instances. Temporal masking can be either forward (postmasking) or backward (pre-masking). The post-masking can be in effect up to 200 ms while the pre-masking is relatively shorter and may last up to 20 ms. 2.3 Empirical mode decomposition A detailed mathematically formulated introduction for empirical mode decomposition (EMD) can be found in [21, 25]. By applying the EMD, any multi-component signal is decomposed into a set of intrinsic mode functions (IMFs) and the final residue. The IMF can be defined as a hidden oscillation mode that is embedded in the data series, and it is allowed to be non-stationary and either be amplitude or frequency modulated. Here we only briefly introduce the extraction procedure of all the IMFs and the final residue. AccordingtoHuang[21], the IMF is defined as a function satisfying the following conditions: (1) in the whole data set, the number of extrema and the number of zero-crossings must be either equal or differ at most by one; and (2) at any point, the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero. The first condition ensures that the local maxima of the data are always positive and the local minima are always negative. The second condition guarantees the physical meaning of the instantaneous frequency such that the instantaneous frequency will not have the unwanted fluctuations induced by asymmetric wave forms [21]. For an arbitrary signal P(t), EMD is performed and the signal can be decomposed as a sum of IMFs and a final residue [21]. Each IMF is calculated by an iteration process. If the order of the IMFs is denoted by the superscript and the number of iteration for calculating each IMF is denoted by the subscript, the calculation of the IMFs and the final residue can be expressed as follows. Extract all the local maxima and all the local minima in the original signal P(t).Calculate the upper envelop u 1 1 (t) and the lower envelop l1 1 (t) for the original signal P(t) with cubic spline function based on the local maxima and the local minima, respectively. Calculate the mean envelop m 1 1 (t) of the upper and lower envelop: m 1 1 (t) = u1 1 (t) + l1 1 (t) (1)

5 The detail h 1 1 (t) is extracted by subtracting the mean envelop from the original signal P(t): h 1 1 (t) = P(t) m1 1 (t) (2) Generally speaking, h 1 1 (t) is still not a stationary series therefore can not be represented as the first IMF, so the above procedure must be iterated k 1 times until the mean envelop approximates to zero, thus a stationary series h 1 (t) can be obtained: k 1 h 1 k 1 1 (t) m1 1 (t) = h1 k 1 (t) (3) If we rewrite h 1 k 1 (t) as c 1 (t), then the first IMF component c 1 (t) and its residue r 1 (t) can be expressed as: c 1 (t) = h 1 k 1 (t) (4) r 1 (t) = P(t) c 1 (t) (5) Overall, c 1 (t) contains the finest scale or the shortest period component of the signal [5, 21, 25]. Since the residue r 1 (t) still contains longer period variations in the data, it is treated as the new data and subjected to the same sifting process as described above. The sifting process is iterated for all the subsequent until r N (t) is less than a predetermined small value or is a monotonic function from which no more IMF can be extracted: r 1 (t) c 2 (t) = r 2 (t)... (6) r N 1 (t) c N (t) = r N (t) By summing equations (5)and(6), equation (7) can be obtained P(t)= N c n (t) + r N (t) (7) n=1 where c n (t) is the n-th IMF of the signal and r N (t) is the final residue. The completeness and orthogonality of IMFs are shown by Huang [21, 25]. It should be noted that, as the order of the mode increases, the time scale increases while the mean frequency of the mode decreases. The final residue is a monotonic function and is the coarsest component of the signal. It has been shown that the final residue behaves stable under the Gaussian noise and MPEG compression attack [5]. Thus we choose to embed watermark bits into the final residue obtained from the EMD process of the audio signal. An example of the empirical mode decomposition of an audio signal is shown in Fig. 1. It should be noted that, even for data with zero mean, the final residue can still be different from zero; for data with a trend, the final residue should be that trend Huang [21, 25]. In our experiment, the watermark message is embedded into the Waveform Audio File Format (WAV) audio signal where the bit stream is encoded with the Pulse Code Modulation (PCM) format. For the audio and speech processing, the PCM samples are stored and processed using floating point numbers which have the zero mean (or the mean value is sufficiently small compared with the amplitude of the signal) and varies in the interval [- 1.0, 1.0]. Thus, compared with the original audio signal, the amplitude of its final residue can be regarded sufficiently small (as shown in Fig. 1), which makes it possible to embed the watermarks in the final residue of the audio signal while the watermark messages are perceptually inaudible.

6 Fig. 1 The original PCM audio signal, its imf components (imf 1 - imf 8) and the final residue 3 Watermark embedding algorithm A novel blind audio watermarking embedding scheme is described in this section. The proposed embedding scheme (shown in Fig. 2) makes the use of the polyphase filterbank, psychoacoustic models and the empirical mode decomposition. 3.1 Analysis filterbank decomposition The analysis filterbank maps the PCM audio X(t) into M subbands. Here we let M = 32 to be consistent with the MPEG Audio Coding Standard defined in ISO/IEC :1995 [23]. Thus in each band, the subband audio signal can be calculated as: Watermark Embedding Domain Control S 0 (t) 0 Segmentation S 0, j (t) EMD for each segment j in band 0 IMFs c 0,j,n (t) Mean Trend r m,0,j (t) + + PCM Audio Input X(t) Analysis Filterbank (M-band) S 1 (t) 1 Segmentation S 1, j (t) EMD for each segment j in band 1 IMFs c 1,j,n (t) Mean Trend r m,1,j (t) + + Synthesis Filterbank Watermarked PCM Audio Output X w (t) S M-1 (t) M-1 Segmentation S M-1, j (t) EMD for each segment j in band M-1 IMFs c M-1,j,n (t) Mean Trend r m,m-1,j (t) + + FFT Masking Thresholds... Watermark Strength Adjustment Watermark Bits Embedded into Mean Trends Watermark Bits w(j) Watermark Generator Fig. 2 Block diagram of the watermark embedding procedure

7 S i (t) = 63 k=0 j=0 7 T (i)(k) (C(k + 64i) x(k + 64i)) (8) where i is the subband index and ranges from 0 to M 1(M = 32 in this case); S i (t) is the filter output for subband i at time t. C(k) is the pre-defined analysis window coefficients [23]; x(k) is the audio input sample; and the analysis matrix coefficients T (i)(k) is also defined as: T (i)(k) = cos((2i + 1)(k 16)π 64) (9) In order to increase the watermark capacity, the transform domain audio samples in each subband will be further divided into N S segments of J samples each. Let us denote by S i,j (t) the data samples belonging to the j-th segment. Then we have where t = 0, 1,..., J 1andj = 0, 1,..., N S Watermark embedding domain control S i,j (t) = S i (j J + t) (10) Often it is required to be able to embed the watermark into a certain part of the audio, or not to embed the watermark into a specific region, such as the silent region. Thus the watermark embedding domain control module is proposed, which makes it possible to embed the watermark bits in a more flexible manner. If we denote î as the appropriate bands and ĵ as the appropriate segments for the watermarking process, Ŝ i,j (t) := S i,j (t) i î,j ĵ is defined as the segmented, band-pass filtered audio stream that is suitable for the watermark embedding. 3.3 Empirical mode decomposition The EMD is applied to each of the segmented subband stream Ŝ i,j (t). Thus we have N i,j Ŝ i,j (t) = c i,j,n (t) + ri,j N (t) (11) n=1 where c i,j,n (t) are the IMFs of the segmented stream Ŝ i,j (t), N i,j is the number of IMFs for the segmented streams Ŝ i,j (t) and ri,j N (t) is the final residue of stream Ŝ i,j (t). It is worth noting that, the length of the segmented stream Ŝ i,j (t) may affect the watermarking system performance at a noticeable level. The longer the length of Ŝ i,j (t) provides more samples to perform the EMD, thus better performance can be expected. However in the proposed audio watermarking system, the watermarking capacity will be reduced, as more audio samples is used to watermark with same watermark information bit. 3.4 Watermark embedding In order to increase the watermarking capacity, the subband audio stream Ŝ i (t) is embedded with the watermark sequences W i (j) = w i,j, w i,j { 1, +1}, 0 j N S 1, 0 i

8 M 1. Since the watermarking robustness generally increases with the amplitude of the host audio signal, the signal-dependent watermark should be embedded in the host audio. Each watermark bit w i,j is embedded into the j-th segment in the i-th subband of the original audio signal by modifying its final residue r m,i,j (t). The segmented stream Ŝ i,j (t) after embedding watermark bit streams W i (j) are given by the following equations. For i î and j ĵ: For i î and j ĵ: Ŝi,j w (t) = Ŝ i,j (t) ri,j N (t) + α i,j W i (j) N i,j = c i,j,n (t) + α i,j W i (j) (12) n=1 Ŝ w i,j (t) = Ŝ i,j (t) (13) For the theoretical foundation of embedding strategy used in equation (12), it is inspired from the conducted study of [5], in which the possibilities to embed the watermarks in this manner have already been elaboratively explained. The substitution process is based on the complex EMD process. The α i,j in equation (12) is the weight for thresholding the amplitude of the watermark strength, which will be discussed in the following sub-section about its calculation. 3.5 Watermark strength adjustment In order to embed the watermark bits robustly while in an imperceptible manner, the psychoacoustic model [23] is used to determine the maximum possible power of the watermark message. By calculation of the signal-to-mask ratio (SMR) for each segment in each subband, the total maximum possible watermark strength can also be obtained. If we denote the signal-to-mask ratio for the segment j in subband i as SMR i,j,thenwe should have J 1 31 ri,j N α i,j W i (j) SMR i,j (14) (i î,j ĵ) t=0 where α i,j is the weight for thresholding the amplitude of the watermark strength, and α i,j should be proportional to the signal strength S i,j (t) since the masking threshold is also proportional to the signal strength. It should be noted that, in [5] where the EMD is adopted in the image watermarking, two parameters are used to adjust the watermark strength for better performance (in terms of both the robustness and the invisibility). Tuning these two parameters carefully to acquire the suitable watermark strength would be an very time-consuming task because these parameters are also content dependent. As an improvement, these two parameters are fixed in [5] to set a balancing point of PSNR for visual imperceptibility and robustness against various attacks. In the proposed work, since the total watermark strength can be determined once the signal-mask-ratio is calculated, all possible weights α i,j of embedding strengths can be obtained as the watermarks have been normalized to ensure the same strength before they are embedded into the different subbands. Therefore the proposed i=0

9 strategy can determine the strength of multiple watermarks easily as well as guarantee the inaudibility by adopting the psychoacoustic model. 3.6 Watermarked audio For any 0 i M 1, the watermarked streams Ŝi,j w (t)(0 j N s 1,j ĵ) and the non-watermarked streams Ŝi,j w (t)(0 j N s 1,j ĵ) are combined into the subband signals Ŝi,j w (t), and the synthesis filterbank [23] is used for the reconstruction. The watermarked PCM audio signal can be denoted as X w (t). 3.7 Embedding watermark messages into multiple bands In the proposed work, the embedding of multiple watermark messages is accomplished by embedding one message in each subband, up to a theoretical maximum of M messages. It should be noted that fully utilize of all the subbands may not be possible due to the overlapping effect of the adjacent filter bands. However in Section 5.5, we will show that the proposed watermarking scheme is robust while embedding multiple watermarks in an imperceptible manner. 4 Watermark extraction algorithm With a watermarked PCM audio signal in hand, one can extract the watermark bits by following the procedure shown in Fig Analysis filterbank decomposition Apply the same analysis filterbank used in the watermark embedding procedure to decompose the watermarked PCM audio X w (t), AfterwhichM subband signals Si,j w (t), 0 i M 1, are obtained. Watermark Embedding Domain Control Band 0 Segmentation S w 0, j (t) EMD for each segment j in band 0 Mean Trend r w m,0,j (t) J-1 t=0 r w m,0,j (t) Watermarked PCM Audio Output X w (t) Analysis Filterbank (M-band) Band 1 Segmentation S w 1, j(t) EMD for each segment j in band 1 Mean Trend r w m,1,j (t) J-1 t=0 r w m,1,j (t) Watermark Bits Calculation Extracted Watermarks Band M-1 EMD Segmentation S w M-1, j (t) for each segment j in band M-1 Mean Trend r w m,m-1,j (t) J-1 t=0 r w m,m-1,j (t) Fig. 3 Block diagram of the watermark extraction procedure

10 4.2 Watermarked embedding domain control The same watermark embedding domain defined in the previous section is used for the watermark extraction process. For the watermark extraction process, the watermark message should be extracted from Ŝi,j w (t),wherei î and j ĵ. 4.3 Empirical mode decomposition It has been proved by previous research work that the mean trend of the sifting process is the coarsest component of the signal [5, 21]. This indicates that the embedded watermark bits would affect the mean trend at the most significant level. Apply EMD to each of the watermarked segment Ŝi,j w (t), one can obtain Ŝ w i,j Nw i,j (t) = ci,j,n w (t) + rn,w i,j (t) (15) n=1 where ci,j,n w are IMFs of the watermarked signal Ŝw i,j (t) and rn,w i,j (t) are the final residue of the watermark signal. 4.4 Watermark bits calculation and watermark message extraction The final residue value r N,w i,j (t) is used to determine the embedded watermark bits. The watermark bit wi,j can be calculated by using the following formula: or J 1 wi,j = 1,if r N,w i,j (t) 0 (16) t=0 J 1 wi,j = 1,if r N,w i,j (t) < 0 (17) t=0 Thus, for each of the watermarked subband i, wherei î, the corresponding watermark message can be extracted as: Wi (j) = w i,j,wherew i,j { 1, +1}, 0 j N S 1, 0 i M 1. 5 Experiment and discussion A total number of 10 testing audio clips are used for the evaluation. The testing audio clips are chosen from Rock music, Classic music, Jazz music and Electrical music. Both the fast paced and slow paced music are included in the testing audio clips. All audio files are 16- bits mono audio sampled at 44.1 khz (CD quality) ranged from 1-5 min. It should be noted that, the watermarking capacity can be varied according to the number of subbands and the length of the segments used for the EMD process in each of the band. The performance of a typical audio watermarking system can be evaluated on the inaudibility, the watermarking capacity, and the robustness.

11 5.1 Inaudibility For the proposed audio watermarking system based on filterbank analysis and empirical mode decomposition, by using the psychoacoustic model and with the properties of the final residue discussed in Section 2, the inaudibility of the watermark embedding is guaranteed. 5.2 Capacity For a certain audio signal to be watermarked, the watermarking capacity is determined by two parameters: (1) the number of bands (i, wherei î) used for the watermarking; and (2) the length of the segment (J ) used for the EMD process. Given a host audio to be watermarked, the number of maximum possible watermark bits in each of the subband i(0 i M 1, and M = 32 in our case) that can be embedded is determined by the length of the segment (J ) usedfortheemdprocess.for1minofthe mono audio signal with the sampling rate of 44.1 khz, the maximum number of watermark bits that can be embedded in each subband can be calculated as: 5.3 Robustness (32 J) = J (18) The number of bands (M) used for the watermarking and the length of the segment (J ) used for the EMD process can also affect the robustness of the proposed audio watermarking system. The adjacent filter bands have a major frequency overlap [23], thus watermark embedding at a single subband can affect two adjacent filter-bank outputs. Therefore in the proposed audio watermarking system, no adjacent subbands would be watermarked at the same time. Based on our previous discussion on the final residue of the EMD process in Section 2, for the WAV audio signal encoded with the PCM format where the mean is close to zero, the final residue of the original audio signal is also close to zero. This property is used to embed the watermarks in the final residue in an imperceptible manner. However, once the EMD process is applied to the segmented audio signal, since the mean value of the PCM formatted audio for that particular audio segment may no longer be zero, the amplitude of the final residue for that audio segment may no longer be zero as well. Since the total modification of the final residue is bounded by the psychoacoustic model, once the final residue is increased, the watermark strength will be decreased. And the decreasing of the watermark strength will result in the poorer robustness. Sections 5.4 and 5.5 discuss experimental results of the inaudibility, the watermark system capacity and the watermark robustness of the proposed audio watermarking system. Part of the experimental results in Sections 5.4 and 5.5 were submitted to IEEE ICME 2010 [33]. 5.4 Quality evaluation Subjective quality evaluation of the watermarking scheme was conducted by the listening tests. There are altogether 20 listeners participate the listening test. None of the participant was trained for the listening test and all of them were only music listeners. All participants were

12 Table 1 The grading scale used in the listening test Impairment Grade SDG= Grade watermarked description Grade reference Imperceptible Perceptible, but not annoying Slightly annoying Annoying Very annoying given the instruction of the listening test just before the test began and they all used their own headsets. In the first part of the quality evaluation, participants were given the non-watermarked and watermarked audio files in random order, and they had to identify the watermarked ones blindly. For each of the audio file, the listeners could make their choices as one of the three options: (1) non-watermarked, (2) watermarked, or (3) can not tell the difference. While most of the listeners could not tell the difference, this indicates that the non-watermarked and watermarked audio can not be discriminated. In the second part of the evaluation, with the prior knowledge of the non-watermarked and watermarked (been watermarked with one watermark message only) audio files, the listeners were asked to report the dissimilarities between the two signals, using the so called Subjective Difference Grades (SDG) [11] as described in Table 1. The audio quality of a watermarking system can be linked to the perceived difference (impairment) between the watermarked audio signal and the original audio signal. To facilitate data analysis, subjective difference grade (SDG) is calculated as the difference of the grades between the watermarked signal and the original signal. In the subjective listening test, the average SDG score was This indicates that the proposed EMD based watermarking scheme cause almost no perceptible distortion to the watermarked signal. When 8 subbands are randomly chosen to be watermarked, the SDG score was still Watermarking capacity and robustness experimental results The inaudibility of the proposed watermarking system is discussed in the previous section. This section evaluates the watermarking capacity and robustness of the EMD and filterbank analysis based watermarking system. As discussed in Section 5, for a given host audio signal, both the audio watermarking capacity and the watermarking robustness are affected by the number of bands (i, where i î) used for the watermarking and the length of the segment (J ) used for the EMD process. Though the number of bands and the length of the segment can affect the system performance simultaneously, the impact of both factors will be examined separately The impact of changing the segment length To evaluate the impact of changing the segment length, we at first fix the subbands (i,where i î) that are used for watermarking, and then vary the length of the segment in order to

13 Total watermark bits/second The length of segment for EMD process Fig. 4 The watermarking capacity per second versus the segment length (watermarks embedded into 4 subbands) optimize the performance of the watermarking system. In order to avoid the overlapping effect of the adjacent filter bands, subband 3, subband 7, subband 11 and subband 15 are used to embed watermark messages. We evaluate the watermarking system performance by varying the segment length from 32 samples to 1,024 samples with the increment of 2. We have discussed in Section 5, that the mean value for a segment of the PCM audio signal may no longer be zero, thus the final residue of the EMD for that segment would be increased and watermark strength would be increased as well. In the watermark embedding procedure, since the total modification of the final residue is bounded by the masking threshold calculated from the psychoacoustic model. The accuracy of the watermark extraction will decrease when the required watermark strength is increased because stronger watermark bits would make the watermark signal easier to be identified. Figure 4 shows the relationship between the watermarking capacity and the segment length. If 4 subbands are chosen to be embedded with watermark messages, the maximum number of watermark bits can be achieved with the shortest segment length. In our experiment, the maximum number of watermark bits per second equals to 172 when the segment length equals to 32. Since 4 subbands are embedded with the watermark messages, we should have 172/4 = 43 bits/sec embedded into each of the subband. For a typical audio file with 5 min duration, the watermarking capacity for each of the subband would be 12,900 bits. If the segment length is set to 1,024, then the total watermark bits can be embedded into each subband would approximately equals to 1.35 bits/sec. For a typical audio file with 5 min duration, the watermarking capacity for each of the subband would be 404 bits. Figure 5 shows the relationship between the misdectected watermark bits and the segment length. As expected, when the segment length is reduced, the number of misdectected watermark bits will be increased. When the EMD process is applied to the segment with the length of 32 samples, the average misdectected watermark bits in one second is The minimum number of misdectected watermark bits in one second can be achieved is bits. When the segment length is longer than 600, the misdectected watermark bits in one second would be less than 0.1 bits.

14 Miss detected watermark bits/sec The length of segment for EMD process Fig. 5 The overall average misdectected watermark bits per second versus the segment length (watermarks embedded into 4 subbands) It should be noted that, Fig. 5 only shows the average misdectected watermark bits in one second. The actual misdectected watermarked bits should be varied for different audio files, since the corresponding masking thresholds are different as well. Figure 6a and b show the misdectected watermark bits per second for two audio signals. Though the values for the misdectected watermark bits per second are different, their distribution follows a similar pattern - the number of misdectected watermark bits per second decreases as the length of the segment for EMD process increases. Figure 7a and b shows the bit error rates (BER) for these two audio signals. The bit error rate is the number of misdectected watermark bits divided by the total number of transferred watermark bits during a certain time interval [18]. Compared with Fig. 6a and b, the patterns of the distribution of the misdectected watermark bits per second are different. Thus in order to interpolate the relationship between the bit error rate and the length of segment for the a b Miss detected watermark bits/sec Miss detected watermark bits/sec The length of segment for the EMD process The average misdectected watermark bits per second versus the segment length (watermarks embedded into 4 subbands) for one of the electrical music The length of segment for the EMD process The average misdectected watermark bits per second versus the segment length (watermarks embedded into 4 subbands) for one of the classical music. Fig. 6 The average misdectected watermark bits per second versus the segment length (watermarks embedded into 4 subbands), for individual audio clip

15 a 0.04 b 9 x Bit error rate Bit error rate The length of segment for the EMD process Thebiterrorrate versus the segment length (watermarks embedded into 4 subbands) for one of the electrical music The length of segment for the EMD process The bit error rate versus the segment length (watermarks embedded into 4 subbands) for one ofthe classical music. Fig. 7 The bit error rate versus the segment length (watermarks embedded into 4 subbands) EMD process, one should rely on the averaged bit error rate for all the testing audio signals (shown in Fig. 8). Figure 8 shows the relationship between the average bit error rate and the length of segment for EMD process when there are 4 watermark messages embedded into subband 3, subband 7, subband 11 and subband 15. The minimum bit error rate can be achieved is 0 when 4 watermark messages are embedded; while the maximum bit error rate is when 4 watermark messages are embedded. For the length of segment for EMD process varied from 32 to 1,024 with the increment of 2, the average bit error rate is From Fig. 8, no direct conclusion can be made. This is because in the proposed audio watermarking system, the system performance (in terms of the bit error rate) is highly related with the watermark strength, which depends on the masking threshold for the particular audio segment. However, the bit error rate varies in a limited region and shows no significant change within that region. Considering the cryptographical security of the proposed watermark system, which requires the length of segment for the EMD process should be as short as possible, one can choose the length of segment equals to the smallest value Bit error rate The length of segment for EMD process Fig. 8 The bit error rate versus the segment length (watermarks embedded into 4 subbands)

16 Bit error rate Band to be watermarked Fig. 9 The bit error rate of embedding the watermark message into one of the subbands (with the length of segment for EMD process be 32) Since the process of increasing the cryptographic security means that more watermark bits need to be embedded which is equivalent to reducing the length of segment for EMD process. Therefore in our experiment, we choose 32 instead of other shorter values (e.g. 16), to perform EMD, because shorter segments would become meaningless due to the limitation of number of points The impact of changing the subband for the embedding of watermark messages Having the length of segment for EMD process fixed, one can embed the watermark message into each of the subband separately, to examine the impact of changing the subband for the embedding of watermark messages. In our experiment, the audio signal is decomposed into 32 subbands. Thus by embedding the watermark message into each of the 32 subbands, the bit error rate can be obtained as the result of the evaluation (shown in Fig. 9). ItcanbeshowninFig.9 that the proposed watermarking system can achieve low bit error rate for the lower frequency subbands, while as the subband order increases the bit error rate will increase as well, e.g., while the watermark message is embedded into the 3 rd subband, 13 th subband and 23 rd subband separately, the bit error rates are 5.64e-4, 3.10e-3 and 8.32e- 2, respectively. This indicates that the subbands with lower order are more suitable for the proposed watermark scheme. To explain the reason for this, it should be noted that for a typical music signal, most of the energy is distributed in the lower frequency bands and for the frequency range over 15kHz the amplitudes have only very small values [1, 7, 16, 27]. Since the watermark strength is related with the energy of the host audio signal, higher energy of the host audio signal will result in a more robust watermark performance. Also in the lossy compression of the audio signal, information in high frequency bands are normally coded with decreased accuracy or not coded at all [23]. Thus the lower order subbands of the host audio signal are chosen for embedding the watermark messages.

17 Table 2 Robustness comparison between proposed work and other schemes according to different types of attacks Types of attack BER Proposed work Bhat 2010 Wu 2005 Yang 2010 Xiang 2007 [4] [34] [33] [35] All-pass filtering 2.31e e e e e 02 Low-pass filtering 4.13e e e e e 02 Amplitude compression 3.79e e e e e 02 MP3 compression (128 kbps) 1.08e e e e e 03 MP3 compression (96 kbps) 3.63e e e e e 02 MP3 compression (64 kbps) 9.77e e e e e 02 Re-sampling 4.39e e e e e 02 Re-quantization 6.57e e e e e 02 Adding Gaussian 8.78e e e e e 02 noise (SNR=30dB) PayLoad (bits/sec) Robustness analysis Based on the experimental results in Sections and 5.5.2, we choose to embed 7 watermark messages into subbands 1, 3, 5, 7, 9, 11 and 13 with one watermark message embedded into each of the subband; and to use the segment of 32 samples for the EMD process. Since each subband has a watermarking capacity of 43 bits/sec, the total watermarking capacity is 301 bits/sec (43 7). With this configuration, the average bit error rate is 8.52e-3 which is very small. We can employ error-correction coding technique to the watermark information bits before embedding which would help to restore the watermarking bits even if the miss detection exits. Table 2 shows the average bit error rates for all the testing audio clips with all-pass filtering, low-pass filtering, amplitude compression, MP3 compression with various compression rates, re-sampling, re-quantization and adding Gaussian Noise attacks. To implement the low-pass filtering, a finite impulse response (FIR) low pass filter with cutoff frequency at 22, 050 Hz, passband ripple at 0.15dB and stopband attenuation 40dB is used. It should be noted that, the cutoff frequency is chosen at 22, 050 Hz in order to avoid audible distortion. For the amplitude compression, the compression ratios are defined as: 8 : 1 for amplitude > 25 db, 2 : 1for 25 db > amplitude > 45 db and 1.2 : 1 for amplitude < 45 db. For the lossy compression, the bit rates were set to 128 kbps, 96 kbps and 64 kbps respectively. For re-sampling, the watermarked audio signal was firstly subsampled down to khz and then interpolated back to 44.1 khz. Re-quantization of the watermarked audio signal includes re-quantize the audio at 8 bits and then back to 16 bits. It can be seen from Table 2 that, the proposed audio watermarking scheme is robust against MP3 compression and Gaussian noise attacks. Especially under the Gaussian noise attack, bit error rate is still very low. This is because as the EMD decomposition proceeds, the time scale increases while the mean frequency of the modes decreases. Thus the IMFs are extracted with the finest scale from the signal and the remainder final residue is the coarsest component of

18 the signal [7]. Therefore for the zero mean Gaussian noise, it is sifted in the lower order of IMFs and the final residue (the mean trend) remains un-influenced. Geometrical transforms such as random cropping, random removal, random duplication, time shifting, and time scale modification are common operations on audio signals. It is also challenging to design a robust blind audio watermarking scheme against such geometric distortion attacks. Due to the geometrical structure of our polyphase filterbank and empirical mode decomposition based watermarking scheme, and also the high watermarking capacity, the bit error rate under certain geometric attacks could be noticeable. We noticed that there are plenty of audio watermarking algorithms such as in [32, 34] where the robustness against geometric distortion can be achieved. One possible solution is to embed the synchronization codes into the original audio signal together with the watermarks and also apply the errorcorrection coding technique. Table 2 also compares the proposed work with other recent audio watermarking schemes. Bhat et. al [4] proposed a watermarking scheme that applies singular value decomposition (SVD) in order to make the embedding more robust. Wu et. al [34] proposed a selfsynchronization algorithms for audio watermarking, which embedded the synchronization codes and hidden information into the lower frequency coefficients in DWT domain. The watermarking scheme proposed by Yang et. al [36] was focused on the quantized parameters in the wavelet domain. Xiang s scheme [35] also showed promising results by using the spread spectrum embedding. It can be shown that, the proposed audio watermarking approach can provide higher watermarking capacity, while achieving a promising robustness performance. 6Conclusion In this paper, a novel blind audio watermarking system based on the psychoacoustic model, the polyphase filterbank analysis and the empirical mode decomposition is proposed. In our approach, the analysis filterbank is used to decompose the host audio signal into multiple subbands, and each of the subbands can be embedded with a unique watermark message. Within each of the subbands, the signal is firstly segmented and the empirical mode decomposition (EMD) is applied to each of the segments. The watermark bits are embedded into the final residue extracted by the EMD process. The inaudibility of the watermarks is guaranteed with the use of the psychoacoustic model. The watermark extraction procedure does not use the original audio signal. The proposed blind audio watermarking scheme is proved to be robust against MP3 compression and adding Gaussian noise attacks. However this method may not be robust to some other attacks such as geometrical transforms. Our future works include that how to increase the robustness of the proposed watermarking scheme against such attacks. Another consideration of future work is to apply the error-correction coding technique to improve the robustness of the watermarking system. References 1. Arnold M, Huang ZW (2001) Blind detection of multiple audio watermarks. In: Proceedings of IEEE international conference on WEB delivering of music, pp Bassia P, Pitas I, Nikolaidis N (2001) Robust audio watermarking in the time domain. IEEE Trans Multimedia 3(2): Bender W, Gruhl D, Morimoto N, Lu A (1996) Techniques for data hiding. IBM Syst J 35(3/4):

19 4. Bhat V, Sengupta K, Das A (2011) An audio watermarking scheme using singular value decomposition and dither-modulation quantization. Multimed Tools Appl J 52(2 3): Springer 5. Bi N, Sun Q, Huang D, Yang Z, Huang J (2007) Robust image watermarking based on multiband wavelets and empirical mode decomposition. IEEE Trans Image Process 16(8): Cheng S, Yu H, Xiong Z (2002) Enhanced spread spectrum watermarking of MPEG-2 AAC audio. In: Proceedings of IEEE international conference on acoustic, speech, and signal processing, pp Chou W, Juang BH (2003) Pattern recognition in speech and language processing. CRC Press 8. Cox IJ, Miller ML (2002) Digital watermarking. Morgan Kaufmann 9. Cox I, Miller M, Bloom J, Fridrich J, Kalker T (2007) Digital watermarking and steganography. Morgan Kaufmann 10. Cvejic N, Keskinarkaus A, Seppanen T (2001) Audio watermarking using m-sequences and temporal masking. In: Proceedings of IEEE workshop on the applications of signal processing on audio and acoustics, pp Cvejic N (2004) Algorithm for audio watermarking and steganography. Department of Electrical and Information Engineering, University of Oulu 12. Cvejic N, Seppanen T (2004) Spread spectrum audio watermarking using frequency hopping and attack characterization. Signal Process 84(1): Cvejic N, Seppanen T (2008) Digital audio watermarking techniques and technologies: applications and benchmarks. Information Science Reference 14. El Hamdouni N, Adib A, Larbi SD, Turki M (2010) Hybrid embedding strategy for a blind audio watermarking system using EMD and ISA techniques. In: 4th international symposium on communications, control and signal processing (ISCCSP), pp El Hamdouni N, Adib A, Larbi SD, Turki M (2013) A blind digital audio watermarking scheme based on EMD and UISA techniques. Multimed Tool Appl 64(3): Gold B, Morgan N (1999) Speech and audio signal processing: processing and perception of speech and music. Wiley 17. Hartung F, Kutter M (1999) Multimediawatermarkingtechniques. Proc IEEE 87(7) 18. Haykin S (1998) Digital communications. Wiley 19. He X, Lliev A, Scordilis M (2004) A high capacity watermarking technique for stereo audio. Proc IEEE Int Conf Acoust Speech Signal Process 5: He X (2008) Watermarking in audio: key techniques and techonologies. Cambria Press 21. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, Yen NC, Tung CC, Liu HH (1998) The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Roy Soc A Math Phys Eng Sci 454: Huang W, Gao Y, Chan K (2010) A review of region-based image retrieval. J Signal Process Syst Signal Image Video Techno 59(2): ISO/IEC Int l Standard IS Information technology-coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbits/s-Part 3: Audio 24. Kirovski D, Malvar HS (2003) Spread-spectrum watermarking of audio signals. IEEE Trans Signal Process Spec Issue Data Hiding 51(4): Liang H, Bressler SL, Desimone R, Fries P (2005) Empirical mode decomposition: a method for analyzing neural data. Comput Neurosci Trends Res 65 66: Liu Z, Inoue A (2003) Audio watermarking techniques using sinusoidal patterns based on pseudorandom sequences. IEEE Trans Circ Syst Video Tech 13: Pressnitzer D, Cheveigne A, McAdams S, Collet L (2004) Auditory signal processing: physiology, psychoacoustics, and models. Springer 28. Seok JW, Hong JW (2001) Audio watermarking for copyright protection of digital audio data. IEEE Electron Lett 37: Shih FY (2008) Digital watermarking and steganography. CRC Press 30. Swanson MD, Zhu B, Tewfik AH, Boney L (1998) Robust audio watermarking using perceptual masking, signal processing. Spec Issue Copyr Prot Access Control 66(3): Thomas T, Emmanuel S, Subramanyam AV, Kankanhalli M (2009) Joint watermarking scheme for multiarty multilevel DRM architecure. IEEE Trans Inf Forensic Sec 4(4): Wang Y, Wu SQ, Huang JW (2007) Audio watermarking robust to geometrical distortions based on dyadic wavelet transform. In: Proceedings of SPIE security, steganography, and watermarking of multimedia contents IX 33. Wang L, Emmanuel S, Kankanhalli M (2010) EMD and psychoacoustic model based watermarking for audio. In: Proceedings of IEEE international conference on multimedia and expo 34. Wu S, Huang J, Huang D, Shi YQ (2005) Efficiently self-synchronized audio watermarking for assured audio data transmission. IEEE Trans Broadcast 51(1):69 76

20 35. Xiang S (2007) Histogram based audio watermarking against time scale modification and cropping attacks. IEEE Trans Multimed 9(7): Yang H, Bao D, Wang X, Niu P (2010) A robust content based audio watermarking using UDWT and invariant histogram. Multimedia tools and applications. Springer 37. Zaman ANK, Khalilullah KMI, Islam MW, Molla MKI (2010) A robust digital audio watermarking algorithm using empirical mode decomposition. In: 23rd Canadian conference on electrical and computer engineering (CCECE), pp 1 4 Dr. Fu Zhaoyang received the B.E., M.S. and Ph.D. degrees in electrical engineering from Northwestern Polytechnical University, Xi an, China, in 2004, 2007 and 2010 respectively. He is now the assistant professor of school of automation, northwestern polytechnical university. Dr. Fu is the IEEE member, and his research interests include singal processing, intelligent control theory, information security and fault diagnosis. Dr. Peng Zhang received the B.E. degree from the Xian Jiaotong University, China in He recieved his PhD from Nanyang Technological University, Singapore in He is now an associate professor in School of Copmuter Science, Northwestern Polytechnical University, China. His current research interests include signal processing, multimedia security and pattern recognition. He is a member of IEEE.

21 Dr. Wei Huang obtained his B.Eng and M.Eng degrees from Harbin Institute of Technology in 2004 and 2006, respectively. He obtained his Ph.D degree from Nanyang Technological University in Before joining Nanchang University as an Associate Professor, he worked in University of California San Diego as well as Agency for Science Technology and Research as Research Associate and Research Fellow, respectively. Dr Huang s recent research interests mainly include but not limited to computer vision, medical image computing, information security, and singal processing. Liang Wang received the B.E. degree in Information Engineering from Jiao Tong University, Xi an, China, in 2001, the M.E. degree and the Ph.D degrees in Electrical Engineering and Telecommunications from the University of New South Wales, Australia in 2004 and 2009, respectively. He is now the research fellow in Nanyang Technological University, Singapore. His research interests include multimedia watermarking, speech and audio forensics, language recognition, speaker recognition and speech recognition.

22 Dr. Sabu Emmanuel is currently an Assistant Professor in the School of Computer Engineering, Nanyang Technological University, Singapore. He received his BE andme from REC and IISc in 1988 and 1998 respectively. He received his PhD from the National University of Singapore, Singapore in His current research interests are in media content and software protection techniques and surveillance media processing. He is now a member of IEEE. Guang Chen received B.E. degree from the Nanjing Institute of Communication Engineering, China in He received his M.E.degree from the Xidian University, China in He is now a faculty in Xi an Communications Institute. His current research interests include software engineering, information forensics and signal processing.