REVVIND REVerse engineering of audio VIsual content Data

Transcription

1 REVVIND REVerse engineering of Newsletter Issue n.3, November 2013 REWIND Page 1 of 12

2 1.1 The REWIND is a FET (Future and Emerging Technology) research project funded by the European Commission within the ICT area of the Seventh Framework Program (FP7). The project is devoted to the analysis of multimedia content in order to trace back its processing history with potential impact in several areas ranging from forensics and law enforcement to quality assessment. The project started on May and will end on April This newsletter is addressed to subjects that might be potentially interested in the outcomes of the project, including companies, public and private research institutions or individual researchers working in this field. This is the third issue of the REWIND newsletter. The first and second issues can be downloaded at The REWIND activities are following the established plans On May 2013 REWIND entered its third year of life. As established at the beginning of the project, during the first two years the activities have been oriented toward the mathematical modeling of footprints left by specific operators on digital content and on the development of detectors able to identify a single operator applied to the considered data. In particular acquisition/coding/editing based footprint detectors have been developed and tested. At the same time, datasets have been created to verify the efficiency and the effectiveness of the proposed tools. A framework for the automatic test of footprint detectors has been also realized. All the obtained results are documented in scientific papers published on international journals and conferences. All these papers can be downloaded from the project website ( At this moment about fifty papers are available on the website. A series of tools and datasets are also downloadable for research purposes. In the third year the project activities will focus on the analysis of complex processing chains and on the creation of phylogenies associated to a set of contents, for example available on the web, and associated to the same event. The phylogeny describes how contents are obtained one from the others through editing, mixing, re coding, and so on. November 5, 2013 Page 2 of 12

3 1.3 Dissemination of the project results Dissemination is a key issue for the REWIND project. It is pursued through the publication of the research results on well reputed journals and conferences, but also through demos and special sessions organized at important conferences. Here is a list of demos and special sessions to which REWIND participated in the last months: Demo WIAMIS 2013 (Paris, July 3 5, 2013), telecom.fr/ Demo MMSP 2013 (Pula, Sardinia, Sept. 30 Oct. 2, 2013), Special MMSP 2013 on Video Forensics REWIND has also sponsored the best paper award at MMSP 2013 and it is one of the sponsors of the challenge on image tampering detection/localization organized by the IEEE Technical Committee on Information Security and Forensics (see the project website for further details). REWIND will participate to the ICT Event 2013 organized in Vilnius on November 6 8 by the European Commission ( agenda/en/ict 2013). REWIND will be present at ICT Event 2013 with a booth (booth id 5A1). The booth visitors will have the opportunity to interact with applications that show how tampering with digital content is today an extremely simple yet powerful operation. On the other hand they will see how REWIND technologies can November 5, 2013 Page 3 of 12

4 be used to build completely automatic systems able to detect multimedia content tampering. Visitors are also invited to use an interactive system in order to become a "forensics analyst", thus identifying which part of the considered digital content has been modified. On November 12 REWIND will organize, in Milan, a workshop on Digital Doctoring: How to tell the real from the fake. The workshop will take place from 10 A.M to 5 P.M. at the premises of the Università degli Studi di Milano, Sala Napoleonica di Palazzo Greppi, Via Sant'Antonio 12, Milano with the following program: Opening Prof. Stefano Tubaro, Politecnico di Milano, Coordinator of the REWIND Project Introduction to Multimedia Forensics technologies Prof. Mauro Barni, Università di Siena Investigative strategies for multimedia data: the law enforcement viewpoint Dott.sa Elvira D Amato, Mail and Computer Fraud Police Service, Italian Ministry of the Interior, Roma Magg. Claudio Ciampini, Scientific Investigation Group, Arm of Carabineers, Roma A software product for forensic investigations Martino Jerian, AMPED, Trieste Investigative strategies for multimedia data: the judicial viewpoint Dott. Francesco Caiani, Deputy Public Prosecutor of the Milano Court, Computer Crime Team Lunch & Demo Session results of the REWIND project are showcased through some demos User generated multimedia content as a witnessing tool for human rights violation Prof. Giovanni Ziccardi, Università di Milano Multimedia Forensics techniques: how discover the history of a digital content excerpt Prof. Pier Luigi Dragotti, Imperial College, UK Round Table on Participative Journalism Moderator Giorgio Mennella, AgoraVox Partcipants will be defined shortly Closing Prof. Stefano Tubaro, Politecnico di Milano, Coordinator of the REWIND Project Further information is available at: November 5, 2013 Page 4 of 12

5 1.4 Short descriptions of the REWIND demos and tools presented at the ICT Event Video tampering detection Nowadays, manipulating videos has become a relatively easy task thanks to the increasing number of dedicated user friendly tools. These software solutions allow people to easily edit a video sequence in a way that is realistic, hence believable. Although creating a doctored video is more time consuming than forging an image, there is an urgent necessity for forgery detection algorithms that can automatically prove or disprove the authenticity of a video. One of the most common video tampering operations is object insertion or removal. However, this kind of attack can be operated in many different ways (e.g., pasting the image representing the inserted object over a set of consecutive frames, using inpanting algorithms to fill the hole left by a removed object, covering an object with background regions taken from other frames belonging to the same sequence, etc.). This demo based on the work in [1] allows a user to detect and localize object insertion or removal operated on a video according to two possible strategies: i) image copy paste; ii) video copy move. The first attack consists in replacing a spatial region of a frame with a still image (e.g., an object to insert), and repeating such forgery in time. The second attack consists in copying a space time region from a video (e.g., representing the background), and pasting it onto another space time region in the same video (e.g., an object to hide). In order to detect such kind of forgeries, the demo interface allows a user to load a video, and run a test on either a preselected region or on the full sequence. If a forgery is detected, the user is warned and the tampered area is shown. Figure 1 shows a preview of the demo s interface. Figure 1. Interface of the video tampering detection demo. [1] P. Bestagini, S. Milani, M. Tagliasacchi, S. Tubaro, Local tampering detection in video sequences, IEEE International Workshop on Multimedia Signal Processing (MMSP), Pula, Italy, 2013 November 5, 2013 Page 5 of 12

6 1.4.2 Video Recapture Detection With the increasing availability of small, inexpensive video recording devices, casual movie making is now within everyone's reach. This represents a great leap forward for budding directors; however it poses a significant threat in terms of copyrighted video material that can be covertly recorded and distributed. Moreover, illegitimate video material circulating on the net may not be limited to the case of recaptured copyrighted footage; another possibility is to recapture a fraudulent or otherwise doctored video in order to conceal traces of forgery, claiming its authenticity. In this demo, we present two techniques for automatic recapture detection. The first uses scene jitter as a cue for classification [2]. During recapture, jitter is introduced due to either physiological hand tremor, or environmental factors. When recapturing planar surfaces approximately parallel to the imaging plane (such as a cinema screen captured by a handheld camera), any added motion due to jitter will result in approximately uniform high frequency 2D motion fields. The second tool improves on the original formulation by considering the scenario where a video is displayed on a Liquid Crystal Display (LCD) monitor and recaptured by a camcorder perfectly aligned with the monitor and kept fixed on a tripod, with little or no jitter [3]. The main idea is to detect whether a sequence has been recaptured, exploiting a characteristic ghosting artifact that is generated as a consequence of the lack of synchronization between the camera and the monitor. It is important to stress that there is no easy way to control the synchronization between camera and monitor, making ghosting artifacts a robust footprint for recapture detection. Figure 2. Screenshots from the video recapture tools. [2] M. Visentini Scarzanella, P. L. Dragotti, Video Jitter Analysis for Automatic Bootleg Detection, IEEE International Workshop on Multimedia Signal Processing (MMSP), Banff, Canada, [3] P. Bestagini, M. Visentini Scarzanella, M. Tagliasacchi, P. L. Dragotti, S. Tubaro Video Recapture Detection Based on Ghosting Artifact Analysis, IEEE International Conference on Image Processing (ICIP), Melbourne, Australia, Sept November 5, 2013 Page 6 of 12

7 1.4.3 IPTV Source Identification The spectacular growth of Internet Protocol (IP) networks in the last decade has changed our way of watching TV. Both for Video on Demand (where the user asks for the content to be delivered) and for multicast (where the content is scheduled for delivering, and the user applies for being included in the destination list) scenarios, IPTV has shown to be a powerful distribution tool for both time shifted playing and live streaming distribution. These new video distribution models have advantages from the point of view of the user (e.g., huge selection of contents; pause, play, rewind and forward features; paying just for those contents which are really consumed, unlike most of the previously used cable/satellite systems) and the content distributor (e.g., almost unlimited potential market; cheap infrastructures, well known distribution technology). Nevertheless, the streaming of contents by providers that do not own the corresponding rights causes huge losses to the legitimate content right owners, who indeed are interested in tracing back the distribution channel undergone by the content for arriving to the server of the unauthorized provider. This demo illustrates the possibility of classifying the origin of one streamed program between terrestrial and satellital, both for time shifted and live streaming. Figure 3 contains a screenshot of the developed application; the current interface allows the user to watch on real time the video in the considered stream, as well as the instant rate variability. Figure 3. A screenshot of the IPTV source identification demonstrator. [4] Masciopinto, M.; Comesaña, P.;, "ITPV streaming source classification," IEEE International Workshop on Information Forensics and Security (WIFS), pp , December November 5, 2013 Page 7 of 12

8 1.4.4 Multi clue image tampering detection The advent of technologies easily enables modification and manipulation of digital images, so that we are no longer confident that what we are seeing in a photo is a true representation of what really happened: the value of photography as a record of events must be carefully evaluated. This interactive demonstrator turns the user into a Forensic Analyst (FA), and allows him to analyze the integrity of images. This is done exploiting some of the state of the art results obtained in the Image Forensic research field [5 7]. The demo provides a friendly graphical interface (Figure 4) that helps the FA in viewing the image and its metadata (EXIF), selecting arbitrarily shaped suspect regions or performing fullyautomatic forgery localization, and viewing the output of forensic tools in a friendly way. In particular, the demo embeds a decision fusion framework tailored to the Image Forensic scenario (published in IEEE TIFS [8]), which allows the user to have a unique final decision (as fusion of different algorithm outputs) on the integrity of the analysed image region. Figure 4. A screenshot of the proposed demonstrator, during the selection of suspect regions. [5] Bianchi, T.; Piva, A.;, "Detection of non aligned double JPEG compression with estimation of primary compression parameters," IEEE International Conference on Image Processing (ICIP), 2011, pp , 2011 [6] Bianchi, T.; De Rosa, A.; Piva, A.;, "Improved DCT coefficient analysis for forgery localization in JPEG images," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp , 2011 [7] Farid, H.;, "Exposing Digital Forgeries From JPEG Ghosts," Information Forensics and Security, IEEE Transactions on, vol.4, no.1, pp , 2009 [8] Fontani, M.; Bianchi, T.; De Rosa, A.; Piva, A.; Barni, M.;, "A Framework for Decision Fusion in Image Forensics Based on Dempster Shafer Theory of Evidence," IEEE Transactions on Information Forensics and Security (TIFS), vol.8, no.4, pp.593,607, April 2013 November 5, 2013 Page 8 of 12

9 1.4.5 Music Plagiarism Analyzer Music plagiarism, i.e., the use of another work while presenting it as one's own original music, has always been a topic of public interest. Typically, if music plagiarism cases are brought to court, independent music experts, often musicologists, are asked to analyze the similarities between two songs, and the judges rely on their opinion. In order to provide technical support for such difficult cases, the Music Plagiarism Analyzer software facilitates the retrieval of samples of one song which are contained in another song, thus helping to prove or disprove suspected music plagiarism. It enables the user to select an excerpt of a song, which is then located and removed from the suspected plagiarism content, retaining other music elements. It can thereby also deal with the fact that the original sample may have undergone manipulations such as timestretching and pitch shifting. Further details of the approach are described in [9]. The software has been successfully applied to disprove alleged music plagiarism in case of the German entry to the 2013 Eurovision Song Contest and was covered in related press articles. Figure 5. Outline of the GUI for the music plagiarism analyzer. [9] C. Dittmar et al. Audio Forensics meets Music Information Retrieval A Toolbox for Inspection of Music Plagiarism Proceedings of the 20th European Signal Processing Conference (EUSIPCO), Bucharest, Romania, 2012 November 5, 2013 Page 9 of 12

10 1.4.6 Audio Tampering Detection based on ENF analysis Thanks to lowered cost and increased availability of devices and global networks, it is now easier than ever to record and distribute, but also to manipulate audio visual material. Considering the growing importance of such material for news coverage, investigations, and many other domains, it is crucial to be able to assess whether content is authentic or not. ENF (Electrical Network Frequency) represents one interesting approach for audio tampering detection: It fluctuates slightly and randomly over time around a defined frequency (typically 50 Hz in Europe), and it does so across the entire electrical network, typically spanning huge geographic areas. Via electromagnetic induction, many digital recordings pick up the ENF, which leads to an extra low frequency component in the recorded audio signal. The demonstrated software allows the user to visualize the ENF and to extract it from the audio file (see Figure 6). Once extracted, it can be used for further analysis, including the following (see also [10]): 1. ENF temporal pattern matching: It is possible to compare the ENF signature of an audio file against a reference database, in order to determine the time of the recording of the audio file. 2. Tampering detection based on ENF phase analysis: ENF fluctuations are continuous oscillations. If the ENF and its phase extracted from an audio file are both continuous, it can be assumed that the recording has not been modified. In contrast, if the ENF or its phase shows discontinuities, this can indicate possible editing of the original recording. Figure 6. Screenshots from the ENF Analysis Tool [10] S. Mann, L. Cuccovillo, P. Aichroth, and C. Dittmar, Combining ENF Phase Discontinuity Checking and Temporal Pattern Matching for Audio Tampering Detection, Workshop for applied signal processing (WASP), Koblenz, Germany, September 2013 November 5, 2013 Page 10 of 12

11 1.4.7 Automatic testing framework and inverse audio decoder Evaluation activities within REWIND included the development of a proof of concept framework for storage of content and annotations, and for automatic testing, using existing partner background components. The goal of this activity was to support the R&D process within REWIND, to support dataset sharing, and for more objective and unified evaluation of components. More specifically, the framework includes: XML schemas for the fine grain annotations (describing operations and content as results of the operations), chains, detector interfaces, and test cases A Web service (SOAP + REST) to store, browse, visualize and retrieve datasets (content and associated fine grain XML annotations); to semantically link annotations to detectors and use case evaluation definitions, and to perform automatic testing based on this A client application to perform the testing process in a decentralized manner, i.e., the evaluated components can stay within the developer s domain. The demo will show the functionality of this proof of concept, calling the inverse decoder detector implementation as an evaluation example, which has also been developed within REWIND. The inverse decoder builds on previous work by Fraunhofer IIS ( home of MP3 ) and is able to recognize traces of previous audio compression in decompressed audio data, by detecting the framing grid and bitrate that had been used to encode the material. The inverse decoder is implemented as a command line tool that analyses uncompressed audio files, detecting various audio codecs (MP3, MP3PRO, AAC, HE AAC) and respective encoding parameters in various content segments. It can e.g. be used to detect bitrate fraud or audio tampering detection, based on framing grid discontinuities and codec changes within a content item. Figure 6. Testing process overview and screenshots from the Testing Service November 5, 2013 Page 11 of 12

12 For further information Project Public WebSite Project Contact Point Prof. Stefano Tubaro Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano Piazza Leonardo da Vinci Milano Italy Tel/Fax: / stefano.tubaro@polimi.it November 5, 2013 Page 12 of 12