Deep learning is a good steganalysis tool when embedding key is reused for different images, even if there is a cover source-mismatch

Transcription

1 Deep learnng s a good steganalyss tool when embeddng key s reused for dfferent mages, even f there s a cover source-msmatch Lonel PIBRE 2,3, Jérôme PASQUET 2,3, Dno IENCO 2,3, Marc CHAUMONT 1,2,3 (1) Unversty of Nîmes, France (2) Unversty Montpeller, France (3) CNRS, Montpeller, France March 3, 2016 Meda Watermarkng, Securty, and Forenscs, IS&T Int. Symp. on Electronc Imagng, SF, Calforna, USA, Feb Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

2 The bg promse of CNN... Superlatves: lots of enthusasm, fresh deas, amazng results,... RM+EC CNN FNN Max 24.93% 7.94% 8.92% Mn 24.21% 7.01% 8.44% Varance Average 24.67% 7.4% 8.66% Table: Steganalyss results (P E ) wth S-UNIWARD, 0.4 bpp, clarvoyant scenaro, for RM+EC, CNN, and FNN But... expermental setup was artfcal... Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

3 Outlne 1 CNN 2 Story 3 Experences 4 Concluson Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

4 An example of Convoluton Neural Network Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Inspred from Krzhevsky et al Network, Detecton percentage only 3% to 4% lower than EC + RM. A. Krzhevsky, I. Sutskever, and G. E. Hnton, ImageNet Classfcaton wth Deep Convolutonal Neural Networks, n Advances n Neural Informaton Processng Systems 25, NIPS Ynlong Qan, Jng Dong, We Wang, and Tenu Tan, Deep Learnng for Steganalyss va Convolutonal Neural Networks, n Proceedngs of SPIE Meda Watermarkng, Securty, and Forenscs Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

5 Convoluton Neural Network: Prelmnary flter F (0) = CNNs converge much slower wthout ths prelmnary hgh-pass flterng. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

6 Convoluton Neural Network: Layers Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Insde one layer; successve steps: a convoluton step, the applcaton of an actvaton functon, a step, a normalzaton step. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

7 Convoluton Neural Network: Convolutons Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Frst layer: Other layers: Ĩ (1) k = I (0) F (1) k. (1) Ĩ (l) =K (l 1) k = =1 I (l 1) F (l) k,, (2) Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

8 Convoluton Neural Network: Actvaton Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Possble actvaton functons: absolute functon f (x) = x, sne functon f (x) = snus(x), functon as n the Qan et al. network f (x) = e x2 σ 2, (for Rectfed Lnear Unts): f (x) = max(0, x) as n our work, Hyperbolc tangent: f (x) = tanh(x) Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

9 Convoluton Neural Network: Poolng Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Poolng s a local operaton computed on a neghborhood: local average (preserve the sgnal), or local maxmum (translaton nvarance). + a sub-samplng operaton. For our artfcal experments, the was not necessary. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

10 Convoluton Neural Network: Normalzaton Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Case where normalzaton s done across the maps: norm(i (1) k (x, y)) = ( 1 + α sze I (1) k (x, y) k =mn(k,k sze/2 +sze) k =max(0,k sze/2 ) (I (1) k (x, y)) 2 ) β Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

11 Convoluton Neural Network: Fully Connected Network Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. three layers. a softmax functon normalzes values between [0, 1]. the network delvers a value for cover (resp. for stego). Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

12 Our CNN Convolutonal layers Classfcaton neurons neurons 256 Kerrnel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 16 feature maps 16 kernels 124 x 124 Layer 2 16 feature maps 16 kernels 61 x 61 Layer 3 16 feature maps 16 kernels 29 x 29 Layer 4 16 kernels 16 feature maps Layer 5 16 feature maps 13 x kernels 4 x 4 Fully connected layers Softmax Fgure: Qan et al. Convolutonal Neural Network. Convolutonal layers Classfcaton neurons neurons 256 Kernel 256 mage F (0) 252 Fltered 252 mage Label=0/1 Layer 1 64 feature maps 64 kernels 127 x 127 7x7 strde 2 Layer 2 16 feature maps 16 kernels 127 x 127 Fully connected layers Softmax Fgure: Our Convolutonal Neural Network. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

13 Outlne 1 CNN 2 Story 3 Experences 4 Concluson Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

14 The story of that paper... Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

15 Outlne 1 CNN 2 Story 3 Experences 4 Concluson Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

16 Experences mages of sze from BOSSBase, S-UNIWARD at 0.4 bts per pxels, Same embeddng key and use of the smulator, learnng on mages, Why usng the same key? We dd not want to do that... Documentaton error n the C++ S-UNIWARD software, Qan et al have also msled. We dscovered ths key problem the 23th of December Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

17 About the smulator Probabltes for modfyng a pxel x wth {1...n} are: where p ( ) p (0) p (+) = exp( λρ( ) ) Z = exp( λρ(0) ) Z = exp( λρ(+) ) Z, for a 1 modfcaton,, for no modfcaton,, for a +1 modfcaton, {ρ ( ) }, {ρ (0) }, and {ρ (+) } are the changng costs, λ s obtaned n order to respect the payload constrant, Z = exp( λρ ( ) ) + exp( λρ (0) ) + exp( λρ (+) ). Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

18 Usng the same key... Probabltes for modfyng pxel x : p ( ) = exp( λρ( ) ) Z, p (0) = exp( λρ(0) ) Z What happen when usng the same key..., and p (+) = exp( λρ(+) ). Z The embeddng key ntalze the Pseudo-Random-Number-Sequence Generator, Whatever the mage, the Pseudo Random Number Sequence [0, 1] n s the same, The sequence s used to sample the dstrbuton (see probabltes), Whatever the mage, some poston wll be most of the tme always modfed, and always wth the same polarty (-1 or +1)... Ths stuaton s artfcal!!! Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

19 Illustraton on the cropped BOSSBase database. Fgure: Probablty of change. In whte the most probable stes and n black the less probable ones. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

20 Our best CNN n that artfcal scenaro mages of sze from BOSSBase, S-UNIWARD at 0.4 bts, Same embeddng key and use of the smulator, learnng on mages, RM+EC CNN FNN Max 24.93% 7.94% 8.92% Mn 24.21% 7.01% 8.44% Varance Average 24.67% 7.4% 8.66% Table: Steganalyss results (P E ) wth S-UNIWARD, 0.4 bpp, clarvoyant scenaro, for RM+EC, CNN, and FNN But... expermental setup was artfcal... Note that wth dfferent embeddng keys, the same CNN structure n 2 layers, and wth more neurons, the probablty of error s 38.1%... There s stll hope! Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

21 Concluson on that story Be careful to the software s mplementatons! Be careful to use dfferent keys for embeddng! Be careful: the smulator only does a smulaton (dfferent from STC), Rch Models are under-effcent to detect the spatal phenomenons, You wll also fnd n the paper: Explanaton/dscusson on CNN, behavor of a CNN, A dscusson on embeddng keys, The presentaton of the LIRMMBase. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

22 End of talk CNN s not dead there s stll thngs to do... Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23

23 About LIRMMBase LIRMMBase: A database bult from a mx of Columba, Dresden, Photex, and Rase databases, and whose mages do not come from the same cameras as the BOSSBase database., L. Pbre, J. Pasquet, D. Ienco, and M. Chaumont, LIRMM Laboratory, Montpeller, France, June 2015, Webste: chaumont/lirmmbase.html. Marc CHAUMONT About Deep Learnng and other thngs... March 3, / 23