CHAPTER 1: INTRODUCTION

Transcription

1 CHAPTER 1: INTRODUCTION In the modern age, digital images and videos have become the main information carriers to disseminate knowledge and establish the bridge among several sources. Visual information has a capability to convey the broader spectrum of information because of its ease in acquisition, distribution, and storage to express knowledge, thoughts, evidences, etc. and represents one of the effective means for communication since eras. The advancements in visual (video) technologies, viz. compression, editing, retrieval, transmission, etc. have facilitated the society at several domains such as social, technological, educational, medical, entertainment, commercial, etc. beside to video surveillance and legal evidence. In the socio-economic knowledge and scientific development, images and videos available at various video sharing and social networking websites (viz. YouTube, TacherTube, Google Videos, facebook, etc.) are becoming modern need and signify their unprecedented role in the current context [1-3]. Apart from several useful applications in diversified domains, there are some darker sides of visual (video) information, viz. misuse or the wrong projection of information through videos. One of them is video tampering, where a forger can intentionally manipulate actual or original videos to create tampered or doctored videos for malpractice [4-10] viz. video based forgery, doctored video evidences, criminal activities, defamation, etc. However, video tampering is relatively new area; image doctoring is as old as the art of photography itself where numerous incidences of serious cases of fake photographs have been reported in literature [10-16]. Some of the famous manipulations with visual information which includes photographs, videos, and audios have been displayed in Figure 1.1. The easy availability of many sophisticated video editing tools provides a platform for forger to manipulate original videos and create perceptually indistinguishable fake videos. Every malicious manipulation with videos is not equally consequential [17], but in many serious scenarios viz. video surveillance, defamation, etc. where video is submitted as evidence during court trials and 1

2 one of the parties involved in the litigation disagree about the authenticity of the video in evidence, the authenticity or originality of videos are necessary to be examined [18][19]. (b) Many fake videos were aired, claiming the video is (a) Mikhail G. Delyagin was digitally erased from a video original and captured during WTO building collapse captured during Russian Talk Show in 2007 (c) Joseph Goebbels was erased from the photo which was captured in 1937 during the meet of Hitler and Leni Riefenstahl. First photo is the doctored photo, whereas second one is the original photo of that meet. (d) In order to conceal that second missile from the right (the second photo which is the original photo) had not fired during Iranian Missile Test in 2008, it was decided to paste a successful launch (first photo, which is doctored photo) in its place. (e) An ongoing controversy in India, which is under litigation and yet to prove or disprove the authenticity of the evidence (in form of conversation between the persons involved in litigation). Source of Figures Figure 1.1(a) Figure 1.1(b) Figure 1.1(c) and (d) Figure 1.1(e) Figure 1.1: Some of the famous manipulation with visual information 2 Accessed on 26 Jan 2014

3 Here, video forensic laboratories and experts play a key role to examine the authenticity of videos as well as detection of location of tampering in videos either through visual examination [20] or by detecting the traces or clue or fingerprints (if any) of tampering left by forger while creation of doctored videos [21]. The success or failure of experts to verify the authenticity of videos depend upon how intelligently tampering has been carried out by the forger. It is difficult for forensic experts to verify the authenticity of a video and/or detect the location of tampering in videos, if there are no (or little) traces left by forger while tampering. Although tampering with visual information is not new, digital video forensic (a branch of multimedia security) is still an emerging research area and scientific community is involved in the development of established methodologies to examine the authenticity and credibility of videos as well as detection of location of tampering (i.e. detection of tampering) in an automatic manner [4][5][22-26]. On the other hand, anti-forensics is the branch of multimedia security which facilitates the forger to counter the existing forensic techniques and procedures. Although some anti-forensic techniques are available in literature [27-30] which provide guidance to the forger, but the primary expectation from the forger is to perform such manipulation which is perceptually indistinguishable and deceive casual eye [4]. It is evident from Figure 1.1 (c) that due to casual manipulation a perceptually visible clue / trace is left by forger, where the removal of Joseph Goebbels from the original photo resulted into the quality degradation in surrounding pixels of tampered photo, whereas there is no perceptually visible clue left by forger in the photograph of Figure 1.1 (d). Here, the video quality assessment methods [31-36] may helpful for forger to assess the quality degradation in tampered video with reference to original video and accordingly manipulations can be performed by the forger. However, in literature, a little contribution have been made over this aspect of anti-forensic [37-40], one of the project involving quality assessment for multimedia security is under development at University of Surrey [41]. 3

4 The involvement of digital videos in our day to day life, its prevalence in criminal activities in recent years, and lack of established methodologies as a preventive measure for forensic science and counter measure for anti-forensics seeks immediate attention of the scientific community and our motivation to carry out the research in the domain of multimedia security. In the thesis, our contribution is majorly focused toward development of efficient schemes for followings. (a) Detection of tampering with videos, and (b) Quality assessment in tampered videos The forger can manipulate videos in numerous ways to create tampered videos. A single scheme may not detect all types of tampering; therefore we developed the schemes focused to a specific form of tampering which cannot be applied singlehandedly to detect all types of video tampering. Similarly, the scheme developed for quality assessment in tampered videos cannot be applied to assess the quality in all types of tampered videos. In the following sections, we provide basics of digital videos and different ways to tamper videos, modes of tampering detection, and fundamentals of video quality assessment along with our assumptions and considerations. Section 1.1 presents the tampering with videos. Section 1.2 focuses over the detection of tampering with videos. Fundamentals of quality assessment are presented in Section 1.3. Organization of the thesis is outlined in Section Tampering with videos There can be two types of videos, viz. analog video and digital video, but throughout in the thesis, the word video is used to state the digital video. Besides the video basics, this section presents the tampering domains, levels of tampering and common tampering with videos. As illustrated in Figure 1.2, a video consists of a time-ordered sequence of orthogonal bitmap images (often called as frames), where each frame consists of w h pixels (w and h are the width and height of a frame respectively), i.e. a video is represented in three dimensions involving two domains, viz. temporal domain (i.e. the time-ordered set of frames) and spatial domain (i.e. the pixel domain of w h pixels). In other words, pixels are the smallest unit in spatial domain or in an image, whereas frames are the smallest unit in temporal domain. 4

5 w h P 1 1 P P 1 w P 2 1 P P 2 w Timeordered frames V(1) V(2) V(m 1) V(m) P h 1 P h P h w (a) A video V is a time-ordered sequence of frames. Here, V(i), represents i th frame in V (b) Each frame consists of w h pixels, where, w and h are the width and height of a frame. Figure 1.2: Video as a sequence of frames Each frame is captured at a constant rate and the number of frames captured per second (fps) depends upon the hardware specifications of video capturing device. Since, these devices capture several frames per seconds (usually 25 to 30 fps) it is quiet often to have temporally redundant frames in raw or uncompressed videos (where each frame consists redundant pixels, i.e. spatial redundancy) resulting into high bit rate and large volume of video data [42]. Temporal variation or redundancy depends on various parameters, viz. camera motion (still or motion), object motion (still or motion), sensor noise, illumination and reflectance change, etc [43]. However, these parameters are important to characterize a video; we have not considered them as input parameters for the schemes developed in this thesis. Further, using various compression techniques [42][43], video encoders reduce the spatial and temporal redundancies and compress these raw videos for efficient storage. Videos may be available in compressed form but to process a video, the conventional approach is to uncompress the video, perform the processing with decoded video, and re-encode the video [44][45]. Videos, which are publicly available in compressed or uncompressed form, are the major source for creating tampered videos. These tampered videos can be created involving either a single video, i.e. single source, or multiple videos, i.e. many sources [5]. In this thesis, our aim is to develop schemes for detection of tampering in such tampered videos which were created from single source video. The next section presents the domains in which videos can be manipulated Tampering Domain Depending upon the domain in which manipulation is performed, there can be following types of video tampering [46-51] 5

6 (a) Tampering in spatial domain (i.e. Spatial Tampering) (b) Tampering in temporal domain (i.e. Temporal Tampering) (c) Tampering in spatio-temporal domain (i.e. Spatio-Temporal Tampering) A forger can tamper source videos spatially (i.e. spatial tampering) by manipulating pixel bits within a video frame or across the video frames (i.e. set of adjacent frames). Figure 1.3 (b) presents a spatially tampered video created from the actual video of Figure 1.3 (a). Further, as presented in Figure 1.3 (c), forger can tamper source videos by disturbing the frame sequence (i.e. temporal tampering) through frames replacement, frames addition, and by the removal of video frames and thus create temporally tampered videos. P 11 P P h w P 11 ' P 12 ' - - P P 21 P P P P - - P (a) Actual or Original Video V(1) V(5) V(4) V(3) V(2) V(6) P 21 ' P 22 ' - - P P P - - P (b) Manipulation with pixels V(1) V(5) V(4) V(3) V(2) V(6) Frames P P - - P P V(6) P - - P V(5 ) = V(3) V(4 ) = V(3) V(3) P P - - P V(2) V(1) (c) Manipulation with the frame sequence Frames w h P 11 ' P 12 ' - - P P 21 ' P 22 ' - - P V(6) V(5 ) = V(3) V(4 ) = V(3) V(3) P P - - P V(2) V(1) (d) Manipulation with pixels as well as frame sequence An example of (a) Actual video (b) Spatially tampered video (c) Temporally tampered video and (d) Spatio-temporal tampered video. Where V(i) represents the i frame and P is the pixel intensity. V(i ) is the manipulated i frame and P is the manipulated pixel intensity. Figure 1.3: Video Tampering Domains 6

7 Lastly, referring Figure 1.3 (d), forger can tamper videos in combination of both spatial and temporal domain (i.e. spatio-temporal tampering) by manipulating pixel bits within a video frame or across the video frames (i.e. set of adjacent frames) as well as disturb the frame sequence and thus create spatio-temporally tampered videos. Over the past few years, significant contributions have been made for detection of spatial tampering or image tampering [4][22], whereas, relatively little contributions have been made for detection of temporal tampering. Thus, this thesis aims to detect the temporal tampering (i.e. tampering with frame sequence) in temporally or spatio-temporally tampered videos. Proceeding section describes the level at which temporal tampering is possible Levels of Temporal Tampering Videos can be manipulated or tampered by a forger at following levels [46-48][52-54] (a) Frame level (b) Scene level (c) Video level In temporal domain, frame is the smallest unit, whereas a video scene is comprised of semantically related sequence of video frames [55]. Based on the accumulative changes in successive frames, scenes in a video are often bounded by either, abrupt change between successive frames in a video or gradual change [56][57]. In this thesis, our consideration about a scene is the sequence of gradually changing successive frames in a video and we will call change of scene when there is abrupt change between two successive frames. Depending upon above discussion, videos can be manipulated at frame level viz. frames of a video (irrespective of scene) can be deleted, or intermediate frames of a video scene can be removed. At the scene level, an entire scene of a video is manipulated viz. deletion of a video scene (i.e. scene deletion), copying of a video scene to another place, etc. Finally, there can be manipulations at the video level viz. making copy of a video. Although, videos may be temporally tampered at the video level, some temporal tampering may not feasible at the video level viz. deletion of frames at video level will be resulted into deletion 7

8 of entire video. Thus, this thesis aims to detect temporal tampering in such videos which were created by manipulating video frames at frame and scene level. Next section presents commonly used temporal tampering with videos Common Temporal Tampering with Videos However, one can temporally manipulate videos in several ways, mostly addressed temporal tampering in literature are mentioned below [46-50][52][53][58-77] (a) Frame Drop or Frame Deletion or Frame Removal (b) Frame Swapping or Reordering of frame sequences (c) Frame Copy or Frame Addition While tampering with original or source video, forger can drop or delete frames of his/her choice, resulting into tampered or doctored videos with reduced frame count. These frames can be the intermediate frames of a video scene or can be the set of frames spread into two scenes. Although, the count of frames get reduced due to tampering of frame drop, but the order or sequence of frames remain unchanged, i.e. if i frame appears before the j frame in the original video and if both frames are present in tampered video (i.e. non dropped frames), then their order will remain same in the tampered video. Figure 1.4 (b) illustrates an example of frame drop at the frame level, where V T is the tampered video created by deleting frames in the original video V O, presented in Figure 1.4 (a). Further, deletion can also be performed at the scene level, i.e. an entire scene is removed or dropped by deleting all frames (i.e. i frame to j frame, where i < j) in that scene. Unlike frame drop, frame count remains unchanged while swapping the video frames (or reordering the frame sequences) to create a tampered or doctored video from original or source video. At the frame level, these swapped frames can be intermediate video frames of one/two scene(s), whereas, there will be swapping of entire scene at the scene level, i.e. all frames of one scene may be swapped with all frames of another scene. Although, the number of frames in tampered video remains same as in original video, but the tampering of frame swapping will be resulted into disordering of frames, i.e. if, the i frame appears before the j frame in the 8

9 original video then the swapping of i and j frames to create a tampered video will disorder the frame sequence in tampered video. Figure 1.4 (c) presents an example depicting tampering of the frame swapping at the frame level. Drop of frames V O (3) and V O (4) V O (1) V O (2) V O (3) V O (4) V O (5) V O (6) V T (1) V O (1) V T (2) V O (2) V O (3) V O (4) V T (3) V O (5) V T (4) V O (6) Temporal domain (a) An example of original video V with 6 video frames Swapping of frames V O (2) and V O (5) (b) Temporal tampering of frame drop. V is a temporally tampered video created by dropping frames in V Copying of frames V O (2) and V O (3) V T (1) V O (1) V T (2) V O (5) V T (3) V O (3) V T (4) V O (4) V T (5) V O (2) V T (6) V O (6) V T (1) V O (1) V T (2) V O (2) V T (3) V O (3) V T (4) V O (4) V T (5) V O (5) V T (6) V O (2) V T (7) V O (3) V T (8) V O (6) Temporal domain Temporal domain (c) Temporal tampering of frame swapping. V is a temporally tampered video created by swapping frames in V (d) Temporal tampering of frame copy. V is a temporally tampered video created by copying or adding frames in V Figure 1.4: Examples of common temporal tampering Finally, frame count will be increased if a source video is manipulated by copying the video frames and pasting to some other location in the source video. At the frame level, these copied frames may be the intermediate frames of a video scene, or at the scene level, an entire scene can be copied and pasted after another scene. Copying can also be at the video level, i.e. all frames of a video can be copied and pasted to create the copy of a source video. Like tampering of frame swapping, tampering of frame copying (at frame level or scene level) will also be resulted into disordering of frame sequence in the tampered video. Figure 1.4 (d) presents an example depicting tampering of frame copying at the frame level. 9

10 Based on the discussion of common temporal tampering, we briefly present two different aspects of temporal tampering as follows (a) Count of frames remain unchanged or get changed in tampered videos, viz. in the tampering of frame swapping number of frames in tampered video remains same as in original or source video, whereas it gets changed due to the tampering of frame drop and frame copy. (b) Order of frames remain same or it gets disordered in tampered videos, viz. in the tampering of frame drop the sequence of frames in tampered video remains same as in original or source video, whereas it gets disordered due to the tampering of frame swapping and frame copy. Based on the presented aspects of temporal tampering, this thesis focuses on the detection of temporal tampering in tampered videos, if created by frame drop (at frame level), frame swapping (at frame level), and frame copying (at scene level). The following section presents the basics of video tampering detection. 1.2 Detection of Tampering with Videos As discussed, the authenticity of video evidences which are submitted during court trials need to be examined before using these videos as evidence. Forensic experts and laboratories contribute to examine the authenticity of video evidences either through blind investigation or investigation with some prior information, which include availability of original or source video, information about video capturing devices viz. CCTV, digital camera, etc., information about some additional information viz. watermarks and digital signature, etc. In literature, these investigation approaches have been categorized under the heads, Active Tamper Detection Techniques, ATDT and Passive Tamper Detection Techniques PTDT [6][50][59-62][65][77-79]. The PTDT are used for blind investigation, whereas the ATDT are based on the availability of some prior information. Inspirit of the categorization used for video quality assessment in the literature [36], we breakup the investigation approaches for detection of tampering with videos into following three categories 10

11 (a) Full Reference (FR) Mode: In this mode of tampering detection, the original or source video which was used for the creation of tampered video is available to conduct the investigation. This is applicable under such circumstances where, the tampering has been performed with the copy of the original video. Hence, the original video is available as reference to verify the authenticity of the tampered video. (b) Reduced Reference (RR) Mode: In this mode of tampering detection, the original or source video is not available but some additional information (as discussed above, excluding the availability of original video) is available to examine the authenticity of video evidence. Generally in this mode, the authenticity of video evidences are examined either by retrieving the embedded information (viz. watermark and digital signature), or verifying the video characteristics as per the details of video capturing device, viz. CCTV. (c) No Reference (NR) Mode: In this mode, no additional information available to forensic experts about the video whose authenticity is to be examined, unlike FR and RR. In the absence of any additional information, the tampering detection under NR mode (or PTDT) is usually performed for identification of any abnormalities or inconsistencies either, within and across video frames (for tracing the spatial tampering), or in the expected temporal redundancy (for tracing the temporal tampering), or combination of both for tracing the spatio-temporal tampering. This thesis aims to detect the tampering under full reference (FR) and no reference (NR) modes for the types of tampering considered to be detected in Section Now we present another aspect of the thesis related with tampered videos, i.e. video quality assessment in the proceeding section Video Quality Assessment As discussed earlier, assessment of quality degradation in tampered video may play a counter measure for forger to create perceptually indistinguishable fake videos. This section briefly presents the overview of the methodologies used in literature for assessing the quality of a video. 11

12 Quality of a video is assessed in two ways, viz. subjective assessment and objective assessment [80][81], where video quality are generally assessed in three modes viz. quality assessment under full reference (FR) mode, reduced reference (RR) mode, and no reference (NR) mode [36][82]. Subjective quality assessment involves human subjects to assess the video quality (in all the three modes i.e. FR, RR, and NR), whereas, objective quality assessment involves various objective quality metrics viz. Mean Squared Error (MSE), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), etc. to measure the video quality automatically [83][84]. Broadly, objective quality metrics are classified in two categories viz. statistical metrics and perceptual metrics such as PSNR and SSIM respectively [36][83][84]. Statistical metrics employ the application of various mathematical functions like calculating pixel-by-pixel weighted differences (viz. MSE and PSNR) between reference and distorted video (in context of video quality assessment, original videos and tampered videos are referred as reference videos and distorted videos respectively), whereas perceptual metrics use Human Visual Systems (HVS) characteristics [35][36]. Subjective assessments are the most reliable way to assess the video quality but it is an exhaustive and costly procedure, whereas in objective quality assessment, challenges lie to design objective quality metrics whose assessment is the true representation of the perceived quality. Distorted videos which are involved in the quality assessment (either in FR or RR or NR modes of quality assessment) may be either the temporally distorted or tampered videos or spatially distorted videos or spatio-temporally distorted videos. In thesis, we aim to assess the quality of spatio-temporally distorted or tampered videos under full reference mode, where the temporal distortion or tampering is due to frame drop. Next section presents the organization of the thesis, which includes the introductory chapter, literature review, and contributory chapters Thesis Organization Apart from introduction, presented in Chapter 1, the remaining chapters of the thesis are organized as follows: 12

13 Chapter 2 presents the explored literature along with limitations of the existing techniques for the detection of tampering with videos. It also presents the reviewed literature for video quality assessment, viz. subjective and objective assessment. Based on the limitations with existing schemes, this chapter presents the explored research issues and accordingly set the objectives in the thesis. Chapter 3 proposes full reference (FR) schemes for detection of tampering of frame drop considering that the tampered video is either a temporally tampered video or spatio-temporally tampered video. This chapter also presents the data parallelization of the scheme proposed for spatio-temporally tampered videos. Chapter 4 proposes learning based no reference (NR) scheme, which detects the tampering of frame drop in the temporally tampered video and accordingly verify the authenticity of the video. Chapter 5 describes the limitations of the learning based scheme proposed in the chapter 4 and proposes a threshold based NR scheme to detect the tampering of frame drop (at frame level) in the temporally tampered video and accordingly verify the authenticity of the video. Chapter 6 proposes a threshold based NR scheme to detect the tampering of frame copy (at the scene level) in the temporally tampered video and accordingly verify the authenticity of the video. Chapter 7 proposes a threshold based NR scheme to detect the tampering of frame swapping (at frame level) in the temporally tampered video and accordingly verify the authenticity of the video. Chapter 8 describes the conducted subjective experiments and presents the obtained mean opinion score (MOS) for spatio-temporal tampered videos (where frame drop was used as the temporal distortion or tampering). Further, the MOS has been used to analyze the performance of two objective quality metrics viz. PSNR and SSIM. Finally, we have proposed a FR dropped frames video quality metrics to assess the quality degradation in the spatio-temporally tampered videos. 13

14