DESIGN OF HIGH SPEED STEGOSYSTEM BASED ON TRELLIS CODES JOINTLY WITH GENERALISED VITERBI ALGORITHM *

Transcription

1 International Journal of Computer Science and Applications, Technomathematics Research Foundation Vol. 3, No. 2, pp. 5, 26 DESIGN OF HIGH SPEED STEGOSYSTEM BASED ON TRELLIS CODES JOINTLY WITH GENERALISED VITERBI ALGORITHM * VALERY KORZHIK State University of Telecommunications, St. Petersburg, Russia val-korzhik@yandex.ru GUILLERMO MORALES-LUNA Computer Science, CINVESTAV-IPN, Mexico City, Mexico gmorales@cs.cinvestav.mx IVAN FEDYANIN State University of Telecommunications, St. Petersburg, Russia ivan.a.fedyanin@gmail.com One of the most promising direction in the design of high speed stegosystem was project HUGO based on the use of trellis codes (TC) and Viterbi Algorithm(VA) in the embedding procedure. However the optimality of VA was kept still unclear because a generic purpose of VA is to correct errors.we prove that optimality of embedding can be achieved with the use of generalised Viterbi algorithm(gva) proposed by one of the authors of this paper about 3 years ago. Optimisation of the trellis code check matrix based on the use of GVA is also considered. The second goal of the current paper is to consider to use TC jointly with GVA in order to provide both security of a stegosystem and simultaneously their ability to correct errors which can be induced by an attacker in order to remove embedded information even in the case if the attacker is unable to detect a presence of hidden information (blind attack). We consider also a trade off between security and error correction. Experimental results show that it is possible to build a stegosystem with acceptable errorcorrection capabilities together with minimization of the embedding impact. Keywords: Error correction, generalised Viterbi algorithm, HUGO project, images, stegosystem, trellis codes.. Introduction Steganography (SG) is the information hiding technique that embeds the hidden information into an innocent cover object (CO) under conditions that CO is not corrupted significantly and that a presence of the additional information in CO may not be detected. This goal entails an obvious requirement: the embedding impact has to be minimised from the steganalysis point of view. Moreover, it does not mean that the number of

2 2 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin changes into CO just after embedding should be minimized because the change weights may differ, hence a minimisation of changes into CO does not necessary results in a minimisation of SG detectability. Let X n n 2 be a grey scale cover image and let Y n n 2 be the resulting image after embedding using some stego-algorithm. Let D(X,Y ) be the distortion of the stegoimage and the cover image, in the sense of ability of the SPAM stegoalgorithm to distinguish X and Y. Up to an enumeration of the pixel arrays, the images can be regarded as linear arrays of length n = n n 2. The additive distortion function chosen for Highly Undetectable stego (HUGO) algorithm [Pevný et al. (2)] is () where, ρ i + {+ } is the weight coefficient (the cost of changing the i-th pixel), x i,y i are the values of the i-th pixel at X and Y respectively. All changes are restricted to ± increments, so that the following inequality holds after ±-embeddings in LSB: The additive form of () means that detectability of SG does not depend on the correlation between the embedded bits. That assumption holds when the changed pixels are located sufficiently far from each other, which in turn holds when the embedding rate is relatively low. Three problems arise in the design of effective SG: () How to choose adequately the weights i= n? (2) What is the best coding method for changing pixels according to their weights? (3) How to correct errors caused by blind attack jointly with effective SG detecting? In order to find the pixel weights to be used at (), for each i {,,n}, let Y (i) be the image X with the i-th pixel changed. Then, j {,,n} : x j - y j (i). Let us pick ρ i = D(X,Y (i) ). The weight D(X,Y ) can be calculated as proposed in [Pevny et al. (2)], as the addition of two sums: where the map C dd2 k is calculated, in line with the SPAM features, as the Markov transition probabilities for the eight directions at U V, with In particular, for the case of the horizontal direction ( ), for any integers d,d 2 [-T,T]:

3 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 3 (3) where for i n and j n 2 -, D i,j = X i,j - X i,j+. As in [Pevny et al. (2)], we consider a weight function of the form: for some optimised parameters σ > and γ >. It is well known [Fridrich (29)] that in order to minimise the number of changes among pixels of the CO, a syndrome coding may be used: (5) where y is the block of stego-image bits of length n = n n 2, m is a block of information bits of length k that should be embedded into y and H is a (k n)-matrix chosen for encoding. Next, among all the solutions y, for given m and H, it is necessary to select those providing minimum number of changes between y and its CO block x. At [Fridrich (29)], a technique to solve this problem was presented, and it is especially simple for the Hamming code with a specific check matrix H. But in our case it is necessary to minimise not the number of changes in CO but the distortion function D : (X,Y ) D(X,Y ) given by (). It seems to be similar to a transition from hard decoding to soft decoding in communications where trellis codes using Viterbi algorithm for decoding can be more favourable than block codes, using some algebraic decoding algorithms. Nevertheless there exists a significant difference in the solution of the matrix equation (5) with respect to y given m with minimising the distortion function D given x as a CO and the solution of the error correction procedure equation Hm = y, given some distance between y and x. Fortunately, there exists the so called Generalised Viterbi Algorithm (GVA) proposed in 984 [Korzhik (984), Korzhik and Lopato (987)] that is able to describe both error correction and also steganographic embedding problem in analytic form without execution on trellis graphs. This method of stegosystem design based on trellis codes and GVA becomes clearer and gets a strict justification. We describe GVA and its application to steganographic problem design in the Section 2. Results of matrix optimisation, which are obtained by simulation with the use of GVA, are presented in Section 3. In Section 4 it is considered a combination of TC and GVA in order to minimise both SG detectability and simultaneously to correct errors caused by blind attack. Section 5 concludes the paper. 2. Application of GVA to the SG system design problem Let us initially state a problem that results in a natural solution within GVA [Korzhik (984), Korzhik and Lopato (987)]. Problem. Find where (4) (6)

4 4 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin ν N -, each entry x j is in the set X, card = r, and λ is a real function defined on the set of finite length real sequences. Here, the goal at (6) is to minimise the function, but it can be used to maximise the function, as well. The stated problem can be solved through the so called GVA introduced at [Korzhik and Lopato (987)]: () Find = argmin x Λ ν+ (x,x 2,,x ν+ ). (2) Find 2 = argmin xs Λ ν+2 (,x 2,,x ν+2 ). (3) (4) Find s = argmin xs Λ ν+2 (,, s-,x s,,x ν+s ). (5) (6) Find ( N-ν,, N) = argmin (xn-ν,,xn)λ ν+2 ( N), where N = (,, N-ν-,x N-ν,,x ν+s ) It is easy to see that for all steps, except the last one, the number of calculations for every argument is at most r ν+ and the last step requires at most r ν operations. We note that the conventional VA is a particular case of GVA with ν =. Then: (7) If we assume now that each x k in (7) is the state of trellis on the k-th step and λ(x k,x k+ ) is the length of branch from the state x k to the state x k+, then the decoding problem for convolutional code presented by trellis diagram is equivalent to a minimisation of (6). But fortunately, GVA can be used also for a situation of embedding problem for SG given by (5) that seems to be at a single glance completely different than correction of errors by convolutional codes. Let us consider a matrix H in (5) that has the step-wise sliding form based on a submatrix of order t w pictured at Fig.. Fig.. H as a step-wise sliding matrix..

5 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 5 In order to simplify further the description, let us consider the particular case of t = 2, w = 2, and = i,j 2. Let us assume also k >, and n = 2k. Then equation (5) can be written as follows: (8) with y = T, m = T, determining the equation system (9) It is possible to apply GVA to solve the system (9) given the column vector m and x providing a minimisation of the weight This algorithm can be performed through the following steps: () Build the Table of variants (ŷ,ŷ 2 ) for all possible tuples (y 3,y 4 ) satisfying equation (9), for j = 2, and minimise the function where ρ(x i,y i ) = ρ i x i - y i. (2) Build the Table of variants (ŷ 3,ŷ 4 ) for all possible tuples (y 5,y 6 ) satisfying equation (9), for j = 3, and minimise the function where ρ(x i,y i ) = ρ i x i - y i. Proceed similarly up to the last equation at (9). Example. Let k = 3, ρ i = for i =,,6, x = and m =. Then the matrix H is

6 6 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin In line with (8) we have () () In order to satisfy the first equation in (9) we get two possibilities (y,y 2 ) = (,), and (y,y 2 ) = (,). (2) For all possible pairs (y 3,y 4 ) and (y,y 2 ) which are obtained in the Step ) we get the possibilities to satisfy the second equation in (9) shown at Table. Table. Illustration for step y y 2 y 3 y (*) 4 * * * * (*) Left side of the second equation in (9) Asterisk means that left side of the second equation of (9) coincides with m 2 =. Table. Values of Λ. In Table there are presented also the calculation results of the values Λ for survived strings (with an asterisk). Next in Table 2 we present all possible pairs (y 3,y 4 ) satisfying to the corresponding equation in (9) for every pair (y 5,y 6 ). Table 2. Admissible tuples (y,y 2,y 3,y 4 ). y 3 y 4 y 5 y 6

7 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 7 By combining the Tables and 2, we get Table 3 presenting all possible(y,y 2,y 3,y 4 ) tuples for every pair (y 5,y 6 ) and corresponding to them, the values Λ 3. Table 3. Values of Λ 3. y y2 y3 y4 y 5 y Now it is possible to minimise Λ 3 by selecting a pair (y 5,y 6 ). This gives two optimal tuples y = and y =. It is easy to see that each of these tuples requires one change of the tuple x and satisfies the equation (). This approach can be extended to any step-wise matrix generated by shifting the (t w)- submatrix. Then the complexity for GVA will be of the order O(kwt2 wt ) with respect to operations. 3. Optimisation of by simulation with GVA Let us consider initially the case with ρ i =, for i =,,n. This means that we try to minimise the number of changes into the image pixels after embedding against the sizes t and w of a randomly chosen submatrix. The results of simulation for randomly chosen CO are shown in Fig. 2 where relative changes are presented on the vertical axis.

8 8 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin (a) (b) Fig. 2. Relative changes of randomly chosen binary sequence of length 5 as CO after embedding based on trellis codes. (a) Against the matrix parameter t given w, and (b) against the matrix parameter w given t. From Fig. 2 it can be seen that it is sufficient to bound the sizes of the submatrix 5, w, at least for the case ρ i =, for i =,,n. as t But it is more interesting to investigate the undetectability of SG based on trellis embedding into real images under the condition that the SG detection is performed by blind SVM-based steganalysis with the use of SPAM features (see (2)-(4)). The experiment has been arranged as follows: The image base was taken similar to those considered in [Bas et al. (2)]. Since the embedding procedure is very time-consuming the images were reduced to lesser sizes. The weight coefficients ρ ij in () were calculated by (2), (3) with the weight function (4) after optimisation of parameters σ and γ. At the

9 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 9 training SVM stage both 5 images with and without embedding were used. During the testing SVM stage, there were executed 5 images with and without embedding. This HUGO-based algorithm was compared with conventional ± embedding algorithm taken with the same embedding rate R =, where k is the number of embedded bits and n is the number of image pixels. As a criterion of stegosystem undetectability, it was used (in line with a recommendations [Pevný et al. (2)]) the averaged error probability P e = where P m is the probability of SG missing and P fa is the probability of SG false alarm. The value P e has been minimised at the cost of SVM threshold optimisation. The results of simulations are shown in Tables 4 and 5. Table 4. The probabilities of incorrect ± SG system detection (P e ) for different image sizes and embedding rates R. Image size P e Embedding rate k / n 6x x x x x Table 5. The probabilities of incorrect HUGO SG system SPAM-based detection (P e ) for different image sizes and embedding rates R Image size Embedding SPAM parameters P e rate k / n T 6x x x x We can see from these Tables that the use of the HUGO trellis code-based SG offers some advantages in undetectabilities against a conventional ± in LSB embedding SG. It is worth to note that the undetectability of SG, even in the case of a trellis code-based embedding (used with the HUGO project), depends significantly on the image texturing. We have found that the greater is the degree of texture, the most frequent undetectability of the corresponding image. Qualitatively, image texturing means that the image has a presence of precise contours, while not texture images have sliding luminance changing and noisy background. Numerically image texture can be estimated by the parameter [Pevný et al. (2)] () where B ij k is a (2 2)-pixel block with (i,j)-coordinates and k is the k-th pixel of this block. As it can be seen by (), in order to calculate a measure of image texture t n it is

10 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin necessary to divide the image on disjoint (2 2)-blocks and next to calculate for every block the difference between maximum and minimum pixel luminance of this block. In Table 6 there are presented the averaged error detecting probabilities for HUGO-based SG after embedding of messages into the images with different texturing. For both SVM training and testing phases 5 images from different image sets were used. Table 6. The error detecting probabilities for HUGO-based SG after embedding of messages into the images with different texturing. Image t Embedding SPAM parameters n P e size rate k / n T 64x64 < x64 > We can see from this Table that, in fact, the level of image texturing affects very significantly on SG system detectability. It seems to be recommended to select for SG embedding such images, which have large texture level. But on the other side it can be looking suspiciously with point of steganographic usage view. Maybe it is better to embed the amount of secret bits depending on the level of image texture. 4. Trellis codes for SG embedding with additional ability to correct errors As we can see from the previous material, the TC with the use of GVA can be effectively used in order to minimise the detectability of SG. However this approach has a significant defect. Even if a single error occurs on the SG it results in corruption of up to t information bits. Moreover these errors can be generated by an attacker who even may ignore whether hidden information is presented or absent on the testing SG but he tries to remove it at extreme case (like a proverbial watch handle rotation at US custom service during the Second World War). Thus it would be interesting to correct some errors while keeping an ability to minimise stegosystem detection. Let us try to design encoding and decoding procedures for TC which would be able to solve this problem. The trellis code in our case can be described by a sequence of states and by a transition function depending on the embedding symbols: where i {,,k}, s i is the i-th TC state, Y i = [y (i-)n+,,y in ], where N =, n is the number of cover object (CO) bits, m i is the i-th embedding bit, and k is the total number of information bits to be embedded. In order to explain the requirements for function g we collect them in the Table 7 depending on the goals of encoding. Table 7. Main requirements for function g. (2)

11 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm Encoding goals Ordinary embedding by trellis code for error correcting only. Embedding of information bit in CO with minimisation of weight. A combination of both embedding with minimal weight and correction of some errors. Main requirements for g For each information bit m i there comes only one path Y i from each state. All other paths are prohibited. For each transition path on the trellis code there exist a set of transition paths Y i. No path is prohibited. For each state of the trellis code, there exists a set of allowed transitions but a part of the paths is prohibited. In general it is possible to devise a code that has correcting capabilities as well as the ability to optimise embedding of weighted cost based on any state diagram that satisfy the four conditions given later in this section. We consider a state diagram that is constructed from submatrix using the following equation: where s i, s i- are the binary representation of the integer numbers from to 2 t -, r is the operation of reversing the vectors, T is the transposition operation and LB is the function of the last bit keeping for vector argument. All operations in (3) are performed as bitwise modulo 2. (Such choice of state diagram is in line with the geometrical definition of the TC given in [Filler et al. (23)].) In order to get a trellis code which allows to minimise a detectability of SG and simultaneously to correct some errors, the assignment Y i = g(s i-,m i ) must satisfy the following conditions: () At least two paths corresponding to embedding and should steam from each state. (2) It should be more than one path on average steaming from each state for embedding of the same information bit. (3) At least one path is coming into each state. (4) Some vectors Y i should be prohibited. (Let us denote by P dis the ratio of the number of allowed vectors to the number of all possible vectors Y i.) The TC, satisfying the above conditions can be constructed as follows: Consider the TC with the use of the step-wise sliding matrix based on the submatrix as described above. Remove a part of transitions randomly with probability P dis. Check whether the conditions () and (3) are satisfied or not. If it occurs that the conditions are not satisfied, then repeat randomly the removal of transitions with probability P dis. We define the full weight of corruption after embedding into CO as: (3) (4) where X i = [x (i-)n+,,x in ], N =, P i = [ρ (i-)n+,,ρ in ], Y (k) = i= k and T is the symbol of transposition. (We note that X i denotes the string of CO bits in which is embedded the information bit m i.) By substituting (2) into (4),

12 2 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin (5) with s (k) = i= k-. Evidently, every term in (5) depends only on the preceding one. Thus we can use GVA for Λ k minimisation as follows: Step. Find all those Y and corresponding states s which allow to embed the first bit m, in line with the given function: Y = g(s,m ), s = g (s,m ) and Y 2 = g(s,m 2 ). Find Ŷ (Y 2 ) = arg min Y (Y 2) Λ 2 where Λ 2 is given as (4), with k = 2. Step 2. Find all those Ŷ 2 (Y 3 ) which allow to embed the second bit m 2, and select thereafter those ones minimising Ŷ 2 (Y 3 ) = arg min Y 2(Y 3) Λ 3 where Λ 3 is given as (4), with k = 3. Step k -. Find all those Ŷ k- (Y k ) which allow to embed the (k-)-th bit m k-, and select thereafter those ones minimising Ŷ k- (Y k ) = arg min Y k-(y k) Λ k where Λ k is given as (4). Step k. Find all those Ŷ k which allow to embed the k-th bit m k, and select thereafter those ones minimising Ŷ k = arg min Y k Λ k. The decoding procedure with error correction can be performed as conventional VA: Let us introduce two functions which can be considered as the inverse functions in (2) Let us denote by Ŷ i the vector Y i after passing through the channel with errors. Then it is necessary to minimise the following function during decoding: (6) (7) where is the Hamming distance: the number of nonzero elements in a vector. Then by rewriting (7) using (6) the following equation results: (8) From (8), a VA can be used to find a sequence of TC states, 2,, k minimising the function Λ k D (s,s 2,,s k ). Next, the sequence of the embedded bits m, m 2,, m k can be obtained given the sequence of TC states, 2,, k. In Fig. 3 there are presented dependencies of the distortion function D(X,Y ) given by () after different types of embedding algorithms into randomly generated binary sequence X of length n and weights ρ i, randomly chosen as absolute values of Gaussian distributions. We can see from these dependencies that in line with theory the largest distortion function (that is the best detectability) occurs for a scenario without the use of TC. The

13 Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 3 minimal distortion provides the use of TC without error correction, while joint minimisation of distortion and error correction occupies intermediate position. Fig. 3. Distortion function D after different embedding methods into randomly generated binary sequence X of length n. In Fig. 4 there are presented the dependencies of error probabilities after decoding for three different embedding algorithms also into randomly generated binary sequence X of length n and randomly chosen weights ρ i against channel symbol probability P. We can see at Fig. 4 that the use of TC for distortion function minimisation only results in significant error expansion just at the channel error probability P., while additional error correction by TC allows to keep acceptable reliability after decoding up to channel error probability P.6. (It is worth to note that by introducing large channel error probabilities, an attack results in a degradation of the CO value, hence it is largely unfit for the attacker.)

14 4 Valery Korzhik, Guillermo Morales-Luna, Ivan Fedyanin Fig. 4. Error probability after decoding for three different embedding algorithms. (TC without error correction has w = 5, t = 2, TC with correction has w = 5,t=2, P dis =.4). By changing the probability P dis on a stage of TC generation, it can be traded off between a minimisation of distortion function (hence a minimisation of SG detection) and the maximisation of error correction after decoding. 5. Conclusion A new generation of stegosystems with the use of trellis code-based embedding is very promising because this approach minimises an embedding impact with the point of view blind SVM-SPAM based steganalysis. This is the so called HUGO project developed recently in [Pevny et al. (2)]. But our main contribution into this direction is to make more clear and to prove rigorously that the use of generalised Viterbi Algorithm is in fact the optimal embedding procedure jointly with trellis codes. Another contribution of ours is an investigation of scenario where TC are used not only for minimisation of SG detectability by SPAM-based steganalysis but also for channel error correction. We proposed generation algorithm for such TC and corresponding to them encoding and decoding algorithms that allow to make a trade off between SG detectability and channel error correcting. It is very important if there exists a threat of blind attack.

15 References Design of high speed stegosystem based on trellis codes jointly with generalised Viterbi algorithm 5 Bas, P., T. Filler, and T. Pevný (2). Break our steganographic system : the ins and outs of organizing BOSS. In Proceedings of the 3th international conference on Information hiding, IH, Berlin, Heidelberg, pp Springer-Verlag. Filler, T., J. Judas, and J. Fridrich (2). Minimizing embedding impact in steganography using trellis-coded quantization. In Media Forensics and Security II, part of the IS&T-SPIE Electronic Imaging Symposium, San Jose, CA, USA, pp SPIE Proceedings. Fridrich, J. (29). Steganography in Digital Media: Principles, Algorithms, and Applications (st ed.). New York, NY, USA: Cambridge University Press. Ker, A. D., P. Bas, R. Böhme, R. Cogranne, S. Craver, T. Filler, J. Fridrich, and T. Pevný (23). Moving steganography and steganalysis from the laboratory into the real world. In Proceedings of the First ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec 3, New York, NY, USA, pp ACM. Korzhik, V. (984). A generalization of Viterbi algorithm on the case of channel model described by additive markov channel. In Proceedings of the VI International Symposium on Information Theory, Part II, Moscow-Tashkent, pp. 9. Korzhik, V. and Y. Lopato (987). Optimal decoding of convolutional codes in channels with additive Markov chain model. Problems of Information Transmission XXIII(6), pp Pevný, T., P. Bas, and J. Fridrich (2, June). Steganalysis by subtractive pixel adjacency matrix. Information Forensics and Security, IEEE Transactions on 5(2), Pevný, T., T. Filler, and P. Bas (2). Using high-dimensional image models to perform highly undetectable steganography. In Proceedings of the 2th International Conference on Information Hiding, IH, Berlin, Heidelberg, pp Springer-Verlag.