REVVIND REVerse engineering of audio-visual content Data

Transcription

1 REVVIND REVerse engineering of Newsletter Issue n.1, October 2012 REWIND Page 1 of 12

2 1.1 The REWIND is a FET (Future and Emerging Technology) research project funded by the European Commission within the ICT area of the Seventh Framework Program (FP7). The project is devoted to the analysis of multimedia content in order to trace back its processing history with potential impact in several areas ranging from forensics and law enforcement to quality assessment. This newsletter is addressed to subjects that might be potentially interested in the outcomes of the project, including companies, public and private research institutions or individual researchers working this field. 1.2 Project context and objectives With the rapid proliferation of inexpensive hardware devices that enable the acquisition of audio-visual data, new types of multimedia digital objects (audio, images and videos) can be readily created, stored, transmitted, modified and tampered with. The need of methods and tools that enable REVerse engineering of is therefore becoming more and more of an urgent necessity. The REWIND project is aimed at synergistically combining principles of signal processing, machine learning and information theory to answer relevant questions on the past history of such objects. As an example, Figure 1 illustrates a simple processing chain that corresponds to a video clip originally acquired with a webcam and then downloaded from a video sharing web application such as YouTube. First, the original content is acquired in a digital format and subsequently compressed, typically using a lossy coding algorithm. The user might use off-the-shelf commercial software to alter either the audio or the video track, or both, before re-encoding the content in order to be transmitted and stored on a remote server. Upon a download request, the video clip is transmitted over a noisy network, decoded and finally played out by the receiver device. Figure 1. Example of a simple processing chain of multimedia content. The REWIND project relies on the fact that each one of these processing steps is bound to leave a characteristic footprint that can be detected and analyzed to trace back the past history of the considered multimedia object. This operation, in principle, can be done in a blind fashion, i.e. without needing to access the content at any earlier point of the processing chain. Notice that these processing steps often introduce undesired and irreversible distortions in the original multimedia object, e.g. due to insufficient sampling rate in the acquisition phase, coding artifacts in lossy compression, or imperfect error concealment strategies. Conventionally, these footprints are deemed as misprints of the multimedia processing chain, as they indicate that something went wrong during one of the processing steps of Figure October 30, 2012 Page 2 of 12

3 1. As a consequence, considerable effort in multimedia processing research has been directed towards preventing these artifacts through, for instance, sophisticated sampling and interpolation methods, errorresilience mechanisms or techniques to avoid undesired or unauthorized modifications of the content streams. In the REWIND project, this perspective is completely reversed: footprints are here considered as an asset, i.e., as a precious source of information on the history of the multimedia object, which can be leveraged to reconstruct the processing chain. 1.3 Potential impact and use Content production and access technologies for the multimedia market have undergone a radical transformation over the past decade. Indeed, a wide range of novel networked services have brought a formidable amount of multimedia content at the user s reach. We have also witnessed a radical change in the role of the Citizen, who has been elevated from that of a mere end-user to that of a contributor, thanks to the multitude of powerful audio-visual authoring tools that are today at hand. This, in fact, has promoted the production of an enormous amount of user-generated content (UGC), whose dissemination is greatly favored by a wide variety of novel and effective distribution channels, such as social networks or media sharing websites (e.g. YouTube, Google Videos). In this context the REWIND project intends to play a key role by providing tools and methodologies for assessing and understanding the processing chains that account for the history of multimedia objects. The impact of these methodologies and tools will be significant on a wide range of application areas. If the tracing of the history can be done with a high degree of reliability, relevant applications can be envisioned in fields where stakes are high, such as forensics and law enforcement. Here the information extracted can be used, for example, for authenticating content without the need of prior modifications of the considered content; for tracing content (associating it with the devices used for acquiring/editing it); or for detecting a malicious concealment of footprints. There is, however, a wide range of low-risk applications where the tracing process is narrowed down to a decision on whether or not to trust or to accept the digital object. These include, for example: the evaluation of the face validity of a digital object (i.e. the assessment of its trustworthiness at face value, an essential ingredient for the emerging paradigms of information gathering, such as Citizen Journalism); the quality assessment of a multimedia experience, in all cases in which the reference content is not available; the detection of pirated (or illegally distributed) content. As such evaluations are needed on a regular basis, it is important to perform them automatically and with little or no interaction at all. October 30, 2012 Page 3 of 12

4 1.4 List of the project partners, project public website, project contact point Partner # Partner Name Acronym Country P1 Politecnico di Milano (Coordinator) PoliMi Italy P2 CNIT (Consorzio Interuniversitario per le Telecomunicazioni) CNIT Italy P3 Imperial College of Science, Technology & Medicine Imperial UK P4 University of Vigo UVigo Spain P5 Fraunhofer - Institute for Digital Media Technology IDMT Fraunhofer Germany Project Public WebSite Project Contact Point Prof. Stefano Tubaro Dipartimento di Elettronica e Informazione Politecnico di Milano Piazza Leonardo da Vinci Milano Italy Tel/Fax: / stefano.tubaro@polimi.it 1.5 Some facts about REWIND The REWIND project started on May 1, 2011 and will end on April 30, 2014, for a total duration of 36 months. As of October 2012, the date of this first issue of the REWIND newsletter a series of significant results have been obtained, which were published on top-ranked scientific journals and on international conferences (see the project websiste, for the complete list, including references to publicly available papers). In addition, the project has also realized a set of prototypes demonstrating on real data what can be obtained by using the developed techniques, thus indicating the potential impact in practical applications. In the following, a selection of these prototypes is briefly illustrated. October 30, 2012 Page 4 of 12

5 1.5.1 Image integrity verification The advent of technologies easily enables modification and manipulation of digital visual data, so that we are no longer confident that what we are seeing in a photo is a true representation of what really happened: the value of photography as a record of events must be carefully evaluated. This interactive demonstrator turns the user into a Forensic Analyst (FA), and allows him to analyze the integrity of images. This is done exploiting some of the state-of-the-art results obtained in the Image Forensic research field [1-3]. The demo provides a friendly graphical interface (Figure 2) that helps the FA in viewing the image and its metadata (EXIF), selecting arbitrarily shaped suspect regions or performing fullyautomatic forgery localization, and viewing the output of forensic tools in a friendly way. In particular, the demo embeds a decision fusion framework tailored to the Image Forensic scenario (presented at WIFS11 [4]), that allows the user to have a unique final decision (as fusion of different algorithm outputs) on the integrity of the analysed image region. This software will run on a laptop, it is fully interactive and several realistic splicings (see Figure 2) can be demonstrated, so that users may play with the demo. Figure 2. Outline of the GUI for the image integrity verification tool, during the selection of suspect regions. October 30, 2012 Page 5 of 12

6 1.5.2 Multiple JPEG compression detector The analysis of double-compressed images is a problem largely studied by the multimedia forensics community, as it might be exploited, e.g., for tampering localization or source device identification. In many practical scenarios, e.g., photos uploaded on blogs or sharing web sites, images might be compressed several times. However, the identification of the number of compression stages applied to an image remains an open issue, as many authors address only the problem of double compression. With this demo we show that we are able to estimate the number of JPEG compression steps applied to an image, if it has been compressed up to three times. This is a valuable hint in determining the image authenticity. The algorithm we use is based on the work in [5] and it relies on the analysis of the distribution of the first significant digits of DCT coefficients, which is modeled according to Benford s law. At this purpose, the software we present (see Figure 2) allows a user to load an uncompressed image, compress it from one to three times with random quality factors within a range, and run the detector. The detector output is the estimated number of compression steps. This software can be shown on a laptop, and no other equipment is required. Figure 2. Outline of the GUI for the multiple JPEG compression detector, that shows the estimated number of IPEG compressions applied to the considered image. October 30, 2012 Page 6 of 12

7 1.5.3 Music Plagiarism Analyzer Music plagiarism, i.e., the use of another work while presenting it as one s own original music, has always been a topic of public interest. Typically, if music plagiarism cases are brought to court, independent music experts, often musicologists, are asked to analyze the similarities between two songs, and the judges rely on their opinion. With this demo we show that we are able to help music experts to inspect cases of suspected samplingplagiarism (see Figure 3). The software Music Plagiarism Analyzer facilitates retrieval of samples of one song in another song and thus helps to prove or disprove suspected sampling-plagiarism. The original sample thereby may have undergone manipulations such as time-stretching and pitch-shifting. Commonly it is also superimposed with additional music sources, such as drums or bass inside the new song. The algorithm we use is based on [6] and it relies on spectrogram decomposition of both the original and suspected plagiarism by means of Non-Negative Matrix Factorization. For the suspected plagiarism, the basis vectors of the original spectrogram are used as priors and retained during decomposition. The software enables the user to select an excerpt of a song and have the software erase this part from the suspected plagiarism. Further details can be found in [7]. Figure 3. Outline of the GUI for the music plagiarism analyzer. October 30, 2012 Page 7 of 12

8 1.5.4 Automatic video recapture detection With the increasing availability of small, inexpensive video recording devices, casual movie making is now within everyone's reach. This represents a great leap forward for budding directors; however it poses a significant threat in terms of copyrighted video material that can be covertly recorded and distributed. Moreover, illegitimate video material circulating on the net may not be limited to the case of recaptured copyrighted footage; another possibility is to recapture a fraudulent or otherwise doctored video in order to conceal traces of forgery, claiming its authenticity. Automatic detection of recaptured videos has not been thoroughly examined in the literature, and constitutes a useful tool to automatically flag media for potential copyright infringement or tampering concealment. Our technique [8] uses scene jitter as a cue for classification. During recapture, jitter is introduced due to either physiological hand tremor, or environmental factors such as projector flicker or floor vibrations. When recapturing planar surfaces approximately parallel to the imaging plane (such as a cinema screen captured by a handheld camera), any added motion due to jitter will result in approximately uniform high-frequency 2D motion fields. In our work the high-frequency motion components are analyzed through stages of feature tracking, wavelet decomposition and normalized cross correlation in order to obtain a distribution of the correlation coefficients between the high frequency motion components. The obtained distributions are then used directly for classification. To show the validity of our approach, our demo allows the user to either load a video from our dataset (either original or recaptured) or to load a video and synthetically add recapture jitter. An outline of the user interface is shown in Figure 1. Visual features are then tracked and shown in real time to the user, together with their trajectory before and after the wavelet decomposition. Finally, both the motion correlation matrix and the correlation distribution are shown together with the classification result, in order for the user to visually appreciate the strong influence of even very modest jitter on the correlation coefficients. Figure 4. Outline of the GUI for the video recapture detection software October 30, 2012 Page 8 of 12

9 1.5.5 Automatic video recapture detection The spectacular growth of Internet Protocol (IP) networks in the last decade, as well as the generalization of their use, have changed our way of interacting with people and learning, our consumption habits, and even our entertainment preferences. Concerning the latter, probably the first example are the widespread online games, although in the last years the online consumption of multimedia contents has also experienced a quantum leap. Due to its bandwidth characteristics and its added value, special attention is deserved by the IP distribution of video contents. Typically, two ways of video distribution can be found: Video on Demand (VoD): the user asks for the content to be delivered. Multicast: the content is scheduled for delivering and the user applies for being included in the destination list. Generally VoD streams contents (videos or TV shows) which are stored at the streaming server (a.k.a. delayed or time-shifted playing), as movies or TV shows, while multicast deals with the streaming of live events, typically sports. These new video distribution models have advantages from the point of view of the user: ease of use, possibility of choosing the screened contents among a huge selection, pause, play, rewind and forward features, paying just for those contents which are really consumed (unlike most of the previously used cable/satellite systems), enjoy their favorite programs/movies even when the users are not at home, and open a number of possibilities to the content distributor: almost unlimited potential market, cheap infrastructures, well-known distribution technology, leading to a new video distribution business model. Nevertheless, new technological and legal problems must be also faced, probably being the most important of them the streaming of contents by providers not owning the corresponding rights. This misuse of the contents causes huge losses to the legitimate content right owners, who indeed are interested in tracing back the distribution channel undergone by the content for arriving to the server of the unauthorized provider. October 30, 2012 Page 9 of 12

10 Since terrestrial and satellital channels are two of the most important ways of broadcasting video programs, this demo illustrates the possibility of classifying the origin of one streamed program between terrestrial and satellital, both for time-shifted and live streaming. In the considered framework, the streamed video will have high quality (specifically, no video or audio transcoding will be performed) and it will be streamed according to the Internet Protocol Television (IPTV) standard. Figure 5 contains a screenshot of the developed application; the current interface allows the user to watch on real-time the video in the considered stream, as well as the instant rate variability. The length of the considered video sequence can be determined by the used, although a minimum length around 30 seconds should be considered in order to provide reliable results. Figure 5. Screenshot of ClasifIPTV October 30, 2012 Page 10 of 12

11 1.5.6 References [1] Bianchi, T.; Piva, A.; "Detection of non-aligned double JPEG compression with estimation of primary compression parameters", IEEE International Conference on Image Processing (ICIP), 2011, pp , 2011 [2] Bianchi, T.; De Rosa, A.; Piva, A.; "Improved DCT coefficient analysis for forgery localization in JPEG images," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp , 2011 [3] Farid, H.; "Exposing Digital Forgeries From JPEG Ghosts", IEEE Transactions on Information Forensics and Security, vol.4, no.1, pp , 2009 [4] Fontani, M.; Bianchi, T.; De Rosa, A.; Piva, A.; Barni, M.;, "A Dempster-Shafer framework for decision fusion in image forensics" IEEE International Workshop on Information Forensics and Security (WIFS), pp.1-6, 2011 [5] Milani S., Tagliasacchi M., Tubaro S., Discriminating multiple JPEG compression using first digit features, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan, March 2012 [6] P. Smaragdis, From learning music to learning to separate, Invited paper at Forum Acusticum, Budapest, Hungary, 2005 [7] C. Dittmar et al. Audio Forensics meets Music Information Retrieval A Toolbox for Inspection of Music Plagiarism to appear in Proceedings of the 20 th European Signal Processing Conference (EUSIPCO), Bucharest, Romania, 2012 [1] M. Visentini-Scarzanella, P. L. Dragotti, Video Jitter Analysis for Automatic Bootleg Detection, to appear in Proceedings of IEEE International Workshop on Multimedia Signal Processing (MMSP), Banff (Canada), 2012 October 30, 2012 Page 11 of 12

12 1.6 Upcoming events REWIND supports a series of events in which it will disseminate its outcomes, in the form of effective methods for performing reverse engineering of multimedia content. These are the upcoming events: IEEE International Workshop on Information Forensics and Security (WIFS 2012), Tenerife, Spain, December 2-5, REWIND is a sponsor of the workshop and supports a Best Paper Awards on Multimedia Forensics and Security. Moreover REWIND will organize a tutorial on Uncovering the processing history of video content and a set of REWIND demonstrators will be presented during the workshop. IEEE International Workshop on Multimedia Signal Processing (MMSP 2013), Pula (Sardinia), Italy, September 30 - October 2, REWIND will be one of the workshop sponsors and a series of activities will be organized during the events to showcase the results of the project. REWIND Workshop, Milan November The event will be specifically organized to present the results of the projects to companies, organizations and people potentially interested in using tools for discovering the past history of multimedia objects. The exact date of the event will be fixed soon. October 30, 2012 Page 12 of 12