Searching Video Collections:Part I

Size: px
Start display at page:

Download "Searching Video Collections:Part I"

Transcription

1 Searching Video Collections:Part I Introduction to Multimedia Information Retrieval Multimedia Representation Visual Features (Still Images and Image Sequences) Color Texture Shape Edges Objects, Motion Multimedia Indexing Video Segmentation Shot-Boundary Detection Effects Detection Beyond Basic Visual Features: Text, Face 1

2 Video Indexing Analysis of Still Image Features: Color, Texture, Shape Distance Metrics Analysis of Image Sequence Segmentation Cut Detection Motion Vectors Shot Transitions Camera Operations Scene Analysis Selection of Keyframes Shot Similarity video scenes shots frames 2

3 Camera Motion Descriptors Camera track, boom, and dolly motion modes, Camera pan, tilt and roll motion modes. 3

4 Video Indexing Multilayered Hierarchical Structure of a Video Clip Copyright by J. Hunter 2001, Dublin Core and MPEG-7 Metadata for Video 4

5 Video Indexing Semantic Units (Hierarchy) Object, Regions, Frames Shot: continuous sequence of frames captured from one camera Scene: one or more shots presenting different views of the same event (time or space related) Segment: one or more related scenes Transitions Cut - an abrupt shot change that occurs in a single frame Dissolves continuous transition, progressive linear combination Fade - a slow change in brightness usually resulting in or starting with a solid black frame Wipes pixels from the second shot replace those of the first shot in a regular pattern Others special effects, editing tools can offer up to 200 effects 5

6 Video Indexing Example Shots Scenes Description Formats Description Formats Text Text Text Text Camera Distance Controlled Vocabulary Script Text Camera Angle Controlled Vocabulary Transcript Text Camera Motion Controlled Vocabulary Edit List Text Duration secs, frames Duration secs, frames Start Time secs, frame #, SMPTE Start Time secs, frame #, SMPTE End Time secs, frame #, SMPTE End Time secs, frame #, SMPTE KeyFrame GIF, JPEG KeyFrame GIF, JPEG Lighting Controlled Vocabulary Locale Text Open Trans Controlled Vocabulary Cast Text Close Trans Controlled Vocabulary Object Text Dublin Core Metadata 6

7 Reliable Shot Detection The three most commonly used transition types are: Abrupt Cut, Hard Cuts Fades Dissolves 7

8 Cut Detection Time Cut: Sudden Change of Image Content between continuous shots Cut Detection: Separate Video into Shots and calculate Features for Shots separately. 8

9 Shot Transitions Fade In change of image content from monochrome color to image Fade Out example: fade from white/black change of image content from image to monochrome color example: fade to white/black Time 9

10 What is Dissolve? Dissolve: Shot Transition with Image Overlays Time 10

11 Types of Dissolve Cross dissolve Additive dissolve 11

12 Shot Boundary Detection Pixel Differences Statistical Differences Histograms Compression Differences Edge Tracking Motion Vectors SMPTE 00:12:45:20 12

13 Pixel Differences: Basic Idea Compute total number of pixels that change in value more than a threshold t If this total is greater than a second T b threshold then a shot boundary is detected Drawbacks Sensitive to camera motion (pan, zoom) Sensitive to object motion 13

14 Pixel Differences: Improvements Basic method plus the use of a 3x3 averaging filter before the comparison [Zhang93] Divide image in 12 regions and find the best match for each region in a neighborhood around the region in the other image. Difference is the sum of the region differences. [Shahraray95] Chromatic images: Change in gray level in 2 nd image Relatively constant for dissolves and fades Still sensitive to camera and object motion 14

15 Histogram Differences Use color/gray-scale histograms of pixels as a feature to detect shot boundaries Assumption: for the same background and same objects, there is very little change in the histogram th Let H ( j) be the histogram for the j bin of the th i frame, then difference is given by i CHD i j i+ Hi ( j) H 1( j) If the difference exceeds a threshold A shot boundary is detected = CHD i > T b 15

16 Histograms: Example Cut 16

17 Histograms: Difference Graph Cuts Threshold 17

18 Histogram-Based Cut Detection Different images can have same histograms Same Histogram Obvious example Not so obvious example Same Histogram 18

19 Histogram-Based Cut Detection: Challenges Different images can have similar histograms Color values of subsequent images change significantly without a cut occurring explosions change of scene illumination fast movement of large objects Performance of histogram-based cut detection between 90 and even 98 (in some cases) 19

20 Histogram Differences: improvements A coarse quantization is good enough. Typically, 6-bit code: 2 higher order bits or R, G and B channels. This leads to 64-bin histograms. Good trade-off between accuracy and speed for shot boundary detection Threshold selection is crucial. Threshold T b depends very much on the content Gradual transitions: use two thresholds instead of one global threshold, one for abrupt cuts and one for special effects 20

21 Histogram Comparison Talk Show Sequence Copyright Philips (MPEG-7 contribution) Frame Number Similarity Measure

22 Histograms Differences: Twin-Comparison Method CHD i Compute for all frames in video Mark camera breaks where CHD i > T b Mark potential gradual transitions subsequences GT = {[ F, F ]} wherever CHD i > T s e s For each gradual transitions [ F, F ],accumulate s e frame-to-frame difference: If AC > T b, then declare [ F, F ] s e as a gradual transition This algorithm works well and is widely used 22

23 IBM s CueVideo Shot Boundary Detection SMPTE 00:12:45:20 Detects cuts, dissolves, fades and other gradual changes Compare multiple pairs of frames: 1, 3 and 7 frames apart Processes decoded frames Supports MPEG, QT, AVI, live feed, No user-tuned parameters - allows batch processing Detection of flashes, bad frames One pass - allows live video processing Copyright IBM Almaden 23

24 CueVideo Histogram Example: 24

25 Edge Change Ratio (ECR) Properties edge pixel in image i and (i-1): s i and s i-1 Eout: pixel in image (i-1) is edge pixel, pixel in image i is not an edge pixel E in : pixel in image (i-1) is not an edge pixel, pixel in image i is edge pixel use of broad edges (noise independence) edge change ratio between images i and (i-1) Ein ECR = i 1 max, si 1 E s out i 25

26 Computation of ECR: Example AND Image (i-1) Edge Image (i-1) EC out i-1 ECR AND Image i Edge Image i Inverted Images EC ECR-Images i in 26

27 ECR Cut Detection D D D Time Time Inside Shot Cut Fade Out Time D D Fade In Time Dissolve Time 27

28 ECR Cut Detection: Cuts if ECR i is edge change ratio between frames i and (i-1) a cut is detected if where T is a threshold ECR i T Fast object and camera motion leads to high ECR-values without cuts Cuts 28

29 ECR Cut Detection Fade In, Fade Out Fade out: number of edge pixels zero after last frame of sequence Fade in: number of edge pixels zero before first frame of sequence Fade In Fade Out 29

30 ECR Cut Detection: Problems Fast object or camera motion Explosions Fades and dissolves soft transitions are difficult to detect other effects: wipe detection unreliable Performance typically between 90 and 95 percent 30

31 Shot-Boundary Detection: Conclusions Histogram-based technique are good to recognize cuts Standard deviation techniques good to recognize fades Dissolves are the more challenging Problems Ground truth: experimental data must be analyzed manually Database? Benchmarks? Definition of a fade/dissolve 31

32 Searching Video Collections:Part I Introduction to Multimedia Information Retrieval Multimedia Representation Visual Features (Still Images and Image Sequences) Color Texture Shape Edges Objects, Motion Multimedia Indexing Video Segmentation Shot-Boundary Detection Effects Detection Beyond Basic Visual Features: Text, Face 32

33 Text Detection: Applications Annotation and search of image and video libraries TV, movie studios, advertising, and surveillance Automatic identification and logging of the beginning and end of key events based on captions Video Summarization Ticker Tape analysis Commercial Detection Sports Programs indexing 33

34 Text Detection: Design Decisions What kind of text occurrences? Scene text Overlay text With what style attributes? Font size Font type Text color any In what kind of media data? Image-based Video-based both What should be achieved? Localization Segmentation Recognition How will the results be used? Indexing Object-based video encoding 34

35 Example: MPEG-4 Text Extraction Locate text of any size at any position in images, web pages and videos Segment and recognize text Encode extracted text as rigid foreground object in MPEG4 (with Yen- Kuang Chen) 27.5 PSNR Y Signle VOP KBits/sec Multiple VOP 35

36 Example: OCR result: Dec

37 Text Detection Example - Latin Script 37

38 Text Detection: Korean Script Example 38

39 Text Extracted from Video 39

40 Searching Video Collections:Part I Introduction to Multimedia Information Retrieval Multimedia Representation Visual Features (Still Images and Image Sequences) Color Texture Shape Edges Objects, Motion Multimedia Indexing Video Segmentation Shot-Boundary Detection Effects Detection Beyond Basic Visual Features: Text, Face 40

41 Face Detection 41

42 Pool of Features => ~ features for 24x24 window 42

43 Rapid Computation x y y Rainer Lienhart,Jochen Maydt. An Extended Set of Haar-like Features x for Rapid Object Detection. IEEE ICIP 2002, pp , Sep

44 Cascade of Classifiers Premise Input Pattern Size of feature pool (>100000) exceeds what any reasonable classifier can handle Cascade of classifiers (special kind of decision tree) can outperform a single stage classifier because it can use more features at the same computational complexity Use Boosting (Discrete/Real/ Gentle Adaboost, LogitBoost) P(x o) =.998 Stage 1 Stage 2 P(x o) = =.996 Stage N P(x o) =.998 N ~.90 Object P(x o)=.5 P(x o) =.002 P(x o)=.5 2 P(x o) =.004 P(x o)=.5 N P(x o) ~.1 44

45 Cascade Concept Background removal in stage 3 Background removal in stage 4 Background removal in stage 1 Target Concept Background removal in stage 5 Background removal in stage 2 Background removal in stage 3 45

46 Face Recognition: Eigenfaces 46

47 Gracias por su Atencion 47

48 Searching Video Collections: Part I Overview Introduction to Multimedia Information Retrieval Multimedia Representation Multimedia Indexing Part II Audio Analysis Speech Indexing Query Formulation Multimedia Retrieval Part III Browsing Distribution/Streaming Evaluation Multimedia IR Applications Conclusions 48

49 Edge Detection Basic Idea: 1st and 2nd derivative of an edge position of the edge can be estimated with the maximum of the 1st derivative or with the zero-crossing of the 2nd derivative Generalize technique to calculate the derivative of a two-dimensional image 49

50 Canny Edge Detector designed to be an optimal edge detector (according to particular criteria) It takes as input a gray scale image as output an image showing the positions of tracked intensity discontinuities. 50

51 Canny Edge Detector Multi-stage process Image Smoothed by Gaussian Convolution Simple 2-D first derivative operator to highlight regions of the image with high first spatial derivatives tracks along the top of these ridges and sets to zero all pixels that are not actually on the ridge top non-maximal suppression The tracking process exhibits hysteresis 51

Chapter 3 Image Registration. Chapter 3 Image Registration

Chapter 3 Image Registration. Chapter 3 Image Registration Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation

More information

Analysis of Image and Video Using Color, Texture and Shape Features for Object Identification

Analysis of Image and Video Using Color, Texture and Shape Features for Object Identification IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 16, Issue 6, Ver. VI (Nov Dec. 2014), PP 29-33 Analysis of Image and Video Using Color, Texture and Shape Features

More information

A Robust Wipe Detection Algorithm

A Robust Wipe Detection Algorithm A Robust Wipe Detection Algorithm C. W. Ngo, T. C. Pong & R. T. Chin Department of Computer Science The Hong Kong University of Science & Technology Clear Water Bay, Kowloon, Hong Kong Email: fcwngo, tcpong,

More information

CHAPTER 3 SHOT DETECTION AND KEY FRAME EXTRACTION

CHAPTER 3 SHOT DETECTION AND KEY FRAME EXTRACTION 33 CHAPTER 3 SHOT DETECTION AND KEY FRAME EXTRACTION 3.1 INTRODUCTION The twenty-first century is an age of information explosion. We are witnessing a huge growth in digital data. The trend of increasing

More information

Motion in 2D image sequences

Motion in 2D image sequences Motion in 2D image sequences Definitely used in human vision Object detection and tracking Navigation and obstacle avoidance Analysis of actions or activities Segmentation and understanding of video sequences

More information

Video shot segmentation using late fusion technique

Video shot segmentation using late fusion technique Video shot segmentation using late fusion technique by C. Krishna Mohan, N. Dhananjaya, B.Yegnanarayana in Proc. Seventh International Conference on Machine Learning and Applications, 2008, San Diego,

More information

Tamil Video Retrieval Based on Categorization in Cloud

Tamil Video Retrieval Based on Categorization in Cloud Tamil Video Retrieval Based on Categorization in Cloud V.Akila, Dr.T.Mala Department of Information Science and Technology, College of Engineering, Guindy, Anna University, Chennai veeakila@gmail.com,

More information

Multimedia Databases. Wolf-Tilo Balke Younès Ghammad Institut für Informationssysteme Technische Universität Braunschweig

Multimedia Databases. Wolf-Tilo Balke Younès Ghammad Institut für Informationssysteme Technische Universität Braunschweig Multimedia Databases Wolf-Tilo Balke Younès Ghammad Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Previous Lecture Audio Retrieval - Query by Humming

More information

Multimedia Databases. 9 Video Retrieval. 9.1 Hidden Markov Model. 9.1 Hidden Markov Model. 9.1 Evaluation. 9.1 HMM Example 12/18/2009

Multimedia Databases. 9 Video Retrieval. 9.1 Hidden Markov Model. 9.1 Hidden Markov Model. 9.1 Evaluation. 9.1 HMM Example 12/18/2009 9 Video Retrieval Multimedia Databases 9 Video Retrieval 9.1 Hidden Markov Models (continued from last lecture) 9.2 Introduction into Video Retrieval Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme

More information

AUTOMATIC VIDEO INDEXING

AUTOMATIC VIDEO INDEXING AUTOMATIC VIDEO INDEXING Itxaso Bustos Maite Frutos TABLE OF CONTENTS Introduction Methods Key-frame extraction Automatic visual indexing Shot boundary detection Video OCR Index in motion Image processing

More information

Video search requires efficient annotation of video content To some extent this can be done automatically

Video search requires efficient annotation of video content To some extent this can be done automatically VIDEO ANNOTATION Market Trends Broadband doubling over next 3-5 years Video enabled devices are emerging rapidly Emergence of mass internet audience Mainstream media moving to the Web What do we search

More information

5. Hampapur, A., Jain, R., and Weymouth, T., Digital Video Segmentation, Proc. ACM Multimedia 94, San Francisco, CA, October, 1994, pp

5. Hampapur, A., Jain, R., and Weymouth, T., Digital Video Segmentation, Proc. ACM Multimedia 94, San Francisco, CA, October, 1994, pp 5. Hampapur, A., Jain, R., and Weymouth, T., Digital Video Segmentation, Proc. ACM Multimedia 94, San Francisco, CA, October, 1994, pp. 357-364. 6. Kasturi, R. and Jain R., Dynamic Vision, in Computer

More information

Digital Image Processing COSC 6380/4393

Digital Image Processing COSC 6380/4393 Digital Image Processing COSC 6380/4393 Lecture 21 Nov 16 th, 2017 Pranav Mantini Ack: Shah. M Image Processing Geometric Transformation Point Operations Filtering (spatial, Frequency) Input Restoration/

More information

Video De-interlacing with Scene Change Detection Based on 3D Wavelet Transform

Video De-interlacing with Scene Change Detection Based on 3D Wavelet Transform Video De-interlacing with Scene Change Detection Based on 3D Wavelet Transform M. Nancy Regina 1, S. Caroline 2 PG Scholar, ECE, St. Xavier s Catholic College of Engineering, Nagercoil, India 1 Assistant

More information

Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong)

Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) References: [1] http://homepages.inf.ed.ac.uk/rbf/hipr2/index.htm [2] http://www.cs.wisc.edu/~dyer/cs540/notes/vision.html

More information

Optical Flow-Based Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.

Optical Flow-Based Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. Optical Flow-Based Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object

More information

Multimedia Database Systems. Retrieval by Content

Multimedia Database Systems. Retrieval by Content Multimedia Database Systems Retrieval by Content MIR Motivation Large volumes of data world-wide are not only based on text: Satellite images (oil spill), deep space images (NASA) Medical images (X-rays,

More information

Lesson 11. Media Retrieval. Information Retrieval. Image Retrieval. Video Retrieval. Audio Retrieval

Lesson 11. Media Retrieval. Information Retrieval. Image Retrieval. Video Retrieval. Audio Retrieval Lesson 11 Media Retrieval Information Retrieval Image Retrieval Video Retrieval Audio Retrieval Information Retrieval Retrieval = Query + Search Informational Retrieval: Get required information from database/web

More information

Shot Detection using Pixel wise Difference with Adaptive Threshold and Color Histogram Method in Compressed and Uncompressed Video

Shot Detection using Pixel wise Difference with Adaptive Threshold and Color Histogram Method in Compressed and Uncompressed Video Shot Detection using Pixel wise Difference with Adaptive Threshold and Color Histogram Method in Compressed and Uncompressed Video Upesh Patel Department of Electronics & Communication Engg, CHARUSAT University,

More information

CORRELATION BASED CAR NUMBER PLATE EXTRACTION SYSTEM

CORRELATION BASED CAR NUMBER PLATE EXTRACTION SYSTEM CORRELATION BASED CAR NUMBER PLATE EXTRACTION SYSTEM 1 PHYO THET KHIN, 2 LAI LAI WIN KYI 1,2 Department of Information Technology, Mandalay Technological University The Republic of the Union of Myanmar

More information

Video Key-Frame Extraction using Entropy value as Global and Local Feature

Video Key-Frame Extraction using Entropy value as Global and Local Feature Video Key-Frame Extraction using Entropy value as Global and Local Feature Siddu. P Algur #1, Vivek. R *2 # Department of Information Science Engineering, B.V. Bhoomraddi College of Engineering and Technology

More information

Automatic Video Caption Detection and Extraction in the DCT Compressed Domain

Automatic Video Caption Detection and Extraction in the DCT Compressed Domain Automatic Video Caption Detection and Extraction in the DCT Compressed Domain Chin-Fu Tsao 1, Yu-Hao Chen 1, Jin-Hau Kuo 1, Chia-wei Lin 1, and Ja-Ling Wu 1,2 1 Communication and Multimedia Laboratory,

More information

Semantic Movie Scene Segmentation Using Bag-of-Words Representation THESIS

Semantic Movie Scene Segmentation Using Bag-of-Words Representation THESIS Semantic Movie Scene Segmentation Using Bag-of-Words Representation THESIS Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State

More information

Content Based Image Retrieval Using Color Quantizes, EDBTC and LBP Features

Content Based Image Retrieval Using Color Quantizes, EDBTC and LBP Features Content Based Image Retrieval Using Color Quantizes, EDBTC and LBP Features 1 Kum Sharanamma, 2 Krishnapriya Sharma 1,2 SIR MVIT Abstract- To describe the image features the Local binary pattern (LBP)

More information

Scene Change Detection Based on Twice Difference of Luminance Histograms

Scene Change Detection Based on Twice Difference of Luminance Histograms Scene Change Detection Based on Twice Difference of Luminance Histograms Xinying Wang 1, K.N.Plataniotis 2, A. N. Venetsanopoulos 1 1 Department of Electrical & Computer Engineering University of Toronto

More information

Shot segmentation and edit effects

Shot segmentation and edit effects Video segmentation Video segmentation Segmentation is the process of breaking out a video in its constituent basic elements, the shots, and in their higher-level aggregates, like episodes or scenes. A

More information

Ulrik Söderström 16 Feb Image Processing. Segmentation

Ulrik Söderström 16 Feb Image Processing. Segmentation Ulrik Söderström ulrik.soderstrom@tfe.umu.se 16 Feb 2011 Image Processing Segmentation What is Image Segmentation? To be able to extract information from an image it is common to subdivide it into background

More information

Lecture 7: Most Common Edge Detectors

Lecture 7: Most Common Edge Detectors #1 Lecture 7: Most Common Edge Detectors Saad Bedros sbedros@umn.edu Edge Detection Goal: Identify sudden changes (discontinuities) in an image Intuitively, most semantic and shape information from the

More information

Edge Detection CSC 767

Edge Detection CSC 767 Edge Detection CSC 767 Edge detection Goal: Identify sudden changes (discontinuities) in an image Most semantic and shape information from the image can be encoded in the edges More compact than pixels

More information

Real-Time Content-Based Adaptive Streaming of Sports Videos

Real-Time Content-Based Adaptive Streaming of Sports Videos Real-Time Content-Based Adaptive Streaming of Sports Videos Shih-Fu Chang, Di Zhong, and Raj Kumar Digital Video and Multimedia Group ADVENT University/Industry Consortium Columbia University December

More information

Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems

Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems Differential Compression and Optimal Caching Methods for Content-Based Image Search Systems Di Zhong a, Shih-Fu Chang a, John R. Smith b a Department of Electrical Engineering, Columbia University, NY,

More information

Edge detection. Winter in Kraków photographed by Marcin Ryczek

Edge detection. Winter in Kraków photographed by Marcin Ryczek Edge detection Winter in Kraków photographed by Marcin Ryczek Edge detection Goal: Identify sudden changes (discontinuities) in an image Intuitively, most semantic and shape information from the image

More information

Semantic Video Indexing

Semantic Video Indexing Semantic Video Indexing T-61.6030 Multimedia Retrieval Stevan Keraudy stevan.keraudy@tkk.fi Helsinki University of Technology March 14, 2008 What is it? Query by keyword or tag is common Semantic Video

More information

Search Engines. Information Retrieval in Practice

Search Engines. Information Retrieval in Practice Search Engines Information Retrieval in Practice All slides Addison Wesley, 2008 Beyond Bag of Words Bag of Words a document is considered to be an unordered collection of words with no relationships Extending

More information

Introduzione alle Biblioteche Digitali Audio/Video

Introduzione alle Biblioteche Digitali Audio/Video Introduzione alle Biblioteche Digitali Audio/Video Biblioteche Digitali 1 Gestione del video Perchè è importante poter gestire biblioteche digitali di audiovisivi Caratteristiche specifiche dell audio/video

More information

Lecture 6: Edge Detection

Lecture 6: Edge Detection #1 Lecture 6: Edge Detection Saad J Bedros sbedros@umn.edu Review From Last Lecture Options for Image Representation Introduced the concept of different representation or transformation Fourier Transform

More information

8.5 Application Examples

8.5 Application Examples 8.5 Application Examples 8.5.1 Genre Recognition Goal Assign a genre to a given video, e.g., movie, newscast, commercial, music clip, etc.) Technology Combine many parameters of the physical level to compute

More information

Region-based Segmentation

Region-based Segmentation Region-based Segmentation Image Segmentation Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation. Applications: Finding tumors, veins, etc.

More information

Comparison between Various Edge Detection Methods on Satellite Image

Comparison between Various Edge Detection Methods on Satellite Image Comparison between Various Edge Detection Methods on Satellite Image H.S. Bhadauria 1, Annapurna Singh 2, Anuj Kumar 3 Govind Ballabh Pant Engineering College ( Pauri garhwal),computer Science and Engineering

More information

AIIA shot boundary detection at TRECVID 2006

AIIA shot boundary detection at TRECVID 2006 AIIA shot boundary detection at TRECVID 6 Z. Černeková, N. Nikolaidis and I. Pitas Artificial Intelligence and Information Analysis Laboratory Department of Informatics Aristotle University of Thessaloniki

More information

Motion Detection. Final project by. Neta Sokolovsky

Motion Detection. Final project by. Neta Sokolovsky Motion Detection Final project by Neta Sokolovsky Introduction The goal of this project is to recognize a motion of objects found in the two given images. This functionality is useful in the video processing

More information

Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors

Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors Ajay Divakaran, Kadir A. Peker, Regunathan Radhakrishnan, Ziyou Xiong and Romain Cabasson Presented by Giulia Fanti 1 Overview Motivation

More information

MULTIVIEW REPRESENTATION OF 3D OBJECTS OF A SCENE USING VIDEO SEQUENCES

MULTIVIEW REPRESENTATION OF 3D OBJECTS OF A SCENE USING VIDEO SEQUENCES MULTIVIEW REPRESENTATION OF 3D OBJECTS OF A SCENE USING VIDEO SEQUENCES Mehran Yazdi and André Zaccarin CVSL, Dept. of Electrical and Computer Engineering, Laval University Ste-Foy, Québec GK 7P4, Canada

More information

5. Feature Extraction from Images

5. Feature Extraction from Images 5. Feature Extraction from Images Aim of this Chapter: Learn the Basic Feature Extraction Methods for Images Main features: Color Texture Edges Wie funktioniert ein Mustererkennungssystem Test Data x i

More information

Edge detection. Goal: Identify sudden. an image. Ideal: artist s line drawing. object-level knowledge)

Edge detection. Goal: Identify sudden. an image. Ideal: artist s line drawing. object-level knowledge) Edge detection Goal: Identify sudden changes (discontinuities) in an image Intuitively, most semantic and shape information from the image can be encoded in the edges More compact than pixels Ideal: artist

More information

Object detection using Region Proposals (RCNN) Ernest Cheung COMP Presentation

Object detection using Region Proposals (RCNN) Ernest Cheung COMP Presentation Object detection using Region Proposals (RCNN) Ernest Cheung COMP790-125 Presentation 1 2 Problem to solve Object detection Input: Image Output: Bounding box of the object 3 Object detection using CNN

More information

Multi-Camera Calibration, Object Tracking and Query Generation

Multi-Camera Calibration, Object Tracking and Query Generation MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Multi-Camera Calibration, Object Tracking and Query Generation Porikli, F.; Divakaran, A. TR2003-100 August 2003 Abstract An automatic object

More information

HIERARCHICAL VISUAL DESCRIPTION SCHEMES FOR STILL IMAGES AND VIDEO SEQUENCES

HIERARCHICAL VISUAL DESCRIPTION SCHEMES FOR STILL IMAGES AND VIDEO SEQUENCES HIERARCHICAL VISUAL DESCRIPTION SCHEMES FOR STILL IMAGES AND VIDEO SEQUENCES Universitat Politècnica de Catalunya Barcelona, SPAIN philippe@gps.tsc.upc.es P. Salembier, N. O Connor 2, P. Correia 3 and

More information

Edge and Texture. CS 554 Computer Vision Pinar Duygulu Bilkent University

Edge and Texture. CS 554 Computer Vision Pinar Duygulu Bilkent University Edge and Texture CS 554 Computer Vision Pinar Duygulu Bilkent University Filters for features Previously, thinking of filtering as a way to remove or reduce noise Now, consider how filters will allow us

More information

Multimedia Technology CHAPTER 4. Video and Animation

Multimedia Technology CHAPTER 4. Video and Animation CHAPTER 4 Video and Animation - Both video and animation give us a sense of motion. They exploit some properties of human eye s ability of viewing pictures. - Motion video is the element of multimedia

More information

Designing Applications that See Lecture 7: Object Recognition

Designing Applications that See Lecture 7: Object Recognition stanford hci group / cs377s Designing Applications that See Lecture 7: Object Recognition Dan Maynes-Aminzade 29 January 2008 Designing Applications that See http://cs377s.stanford.edu Reminders Pick up

More information

Text Information Extraction And Analysis From Images Using Digital Image Processing Techniques

Text Information Extraction And Analysis From Images Using Digital Image Processing Techniques Text Information Extraction And Analysis From Images Using Digital Image Processing Techniques Partha Sarathi Giri Department of Electronics and Communication, M.E.M.S, Balasore, Odisha Abstract Text data

More information

A feature-based algorithm for detecting and classifying production effects

A feature-based algorithm for detecting and classifying production effects Multimedia Systems 7: 119 128 (1999) Multimedia Systems c Springer-Verlag 1999 A feature-based algorithm for detecting and classifying production effects Ramin Zabih, Justin Miller, Kevin Mai Computer

More information

Digital Image Processing. Image Enhancement - Filtering

Digital Image Processing. Image Enhancement - Filtering Digital Image Processing Image Enhancement - Filtering Derivative Derivative is defined as a rate of change. Discrete Derivative Finite Distance Example Derivatives in 2-dimension Derivatives of Images

More information

Edge detection. Winter in Kraków photographed by Marcin Ryczek

Edge detection. Winter in Kraków photographed by Marcin Ryczek Edge detection Winter in Kraków photographed by Marcin Ryczek Edge detection Goal: Identify sudden changes (discontinuities) in an image Intuitively, edges carry most of the semantic and shape information

More information

Recall precision graph

Recall precision graph VIDEO SHOT BOUNDARY DETECTION USING SINGULAR VALUE DECOMPOSITION Λ Z.»CERNEKOVÁ, C. KOTROPOULOS AND I. PITAS Aristotle University of Thessaloniki Box 451, Thessaloniki 541 24, GREECE E-mail: (zuzana, costas,

More information

Adobe Premiere Pro CC 2018

Adobe Premiere Pro CC 2018 Course Outline Adobe Premiere Pro CC 2018 1 TOURING ADOBE PREMIERE PRO CC Performing nonlinear editing in Premiere Pro Expanding the workflow Touring the Premiere Pro interface Keyboard shortcuts 2 SETTING

More information

Feature Detectors - Canny Edge Detector

Feature Detectors - Canny Edge Detector Feature Detectors - Canny Edge Detector 04/12/2006 07:00 PM Canny Edge Detector Common Names: Canny edge detector Brief Description The Canny operator was designed to be an optimal edge detector (according

More information

Hierarchical Segmentation of Videos into Shots and Scenes using Visual Content

Hierarchical Segmentation of Videos into Shots and Scenes using Visual Content Hierarchical Segmentation of Videos into Shots and Scenes using Visual Content prepared by Andrew Thompson supervised by Robert Laganière and Pierre Payeur Thesis submitted to the Faculty of Graduate and

More information

Topic 4 Image Segmentation

Topic 4 Image Segmentation Topic 4 Image Segmentation What is Segmentation? Why? Segmentation important contributing factor to the success of an automated image analysis process What is Image Analysis: Processing images to derive

More information

Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation

Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical

More information

CHAPTER 8 Multimedia Information Retrieval

CHAPTER 8 Multimedia Information Retrieval CHAPTER 8 Multimedia Information Retrieval Introduction Text has been the predominant medium for the communication of information. With the availability of better computing capabilities such as availability

More information

Edge and corner detection

Edge and corner detection Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements

More information

Multimedia Systems Video II (Video Coding) Mahdi Amiri April 2012 Sharif University of Technology

Multimedia Systems Video II (Video Coding) Mahdi Amiri April 2012 Sharif University of Technology Course Presentation Multimedia Systems Video II (Video Coding) Mahdi Amiri April 2012 Sharif University of Technology Video Coding Correlation in Video Sequence Spatial correlation Similar pixels seem

More information

A threshold decision of the object image by using the smart tag

A threshold decision of the object image by using the smart tag A threshold decision of the object image by using the smart tag Chang-Jun Im, Jin-Young Kim, Kwan Young Joung, Ho-Gil Lee Sensing & Perception Research Group Korea Institute of Industrial Technology (

More information

APPLICATION OF SAD ALGORITHM IN IMAGE PROCESSIG FOR MOTION DETECTION AND SIMULINK BLOCKSETS FOR OBJECT TRACKING

APPLICATION OF SAD ALGORITHM IN IMAGE PROCESSIG FOR MOTION DETECTION AND SIMULINK BLOCKSETS FOR OBJECT TRACKING APPLICATION OF SAD ALGORITHM IN IMAGE PROCESSIG FOR MOTION DETECTION AND SIMULINK BLOCKSETS FOR OBJECT TRACKING Menakshi Bhat 1, Pragati Kapoor 2, B.L.Raina 3 1 Assistant Professor, School of Electronics

More information

An Algorithm for Blurred Thermal image edge enhancement for security by image processing technique

An Algorithm for Blurred Thermal image edge enhancement for security by image processing technique An Algorithm for Blurred Thermal image edge enhancement for security by image processing technique Vinay Negi 1, Dr.K.P.Mishra 2 1 ECE (PhD Research scholar), Monad University, India, Hapur 2 ECE, KIET,

More information

Rushes Video Segmentation Using Semantic Features

Rushes Video Segmentation Using Semantic Features Rushes Video Segmentation Using Semantic Features Athina Pappa, Vasileios Chasanis, and Antonis Ioannidis Department of Computer Science and Engineering, University of Ioannina, GR 45110, Ioannina, Greece

More information

Real-time Monitoring System for TV Commercials Using Video Features

Real-time Monitoring System for TV Commercials Using Video Features Real-time Monitoring System for TV Commercials Using Video Features Sung Hwan Lee, Won Young Yoo, and Young-Suk Yoon Electronics and Telecommunications Research Institute (ETRI), 11 Gajeong-dong, Yuseong-gu,

More information

Multimedia Computing: Algorithms, Systems, and Applications: Edge Detection

Multimedia Computing: Algorithms, Systems, and Applications: Edge Detection Multimedia Computing: Algorithms, Systems, and Applications: Edge Detection By Dr. Yu Cao Department of Computer Science The University of Massachusetts Lowell Lowell, MA 01854, USA Part of the slides

More information

Detection of a Single Hand Shape in the Foreground of Still Images

Detection of a Single Hand Shape in the Foreground of Still Images CS229 Project Final Report Detection of a Single Hand Shape in the Foreground of Still Images Toan Tran (dtoan@stanford.edu) 1. Introduction This paper is about an image detection system that can detect

More information

CS 664 Segmentation. Daniel Huttenlocher

CS 664 Segmentation. Daniel Huttenlocher CS 664 Segmentation Daniel Huttenlocher Grouping Perceptual Organization Structural relationships between tokens Parallelism, symmetry, alignment Similarity of token properties Often strong psychophysical

More information

Motion Tracking and Event Understanding in Video Sequences

Motion Tracking and Event Understanding in Video Sequences Motion Tracking and Event Understanding in Video Sequences Isaac Cohen Elaine Kang, Jinman Kang Institute for Robotics and Intelligent Systems University of Southern California Los Angeles, CA Objectives!

More information

Editing and Finishing in DaVinci Resolve 12

Editing and Finishing in DaVinci Resolve 12 Editing and Finishing in DaVinci Resolve 12 1. Introduction Resolve vs. Resolve Studio Working in the Project Manager Setting up a Multi User Login Accessing the Database Manager Understanding Database

More information

Object detection using non-redundant local Binary Patterns

Object detection using non-redundant local Binary Patterns University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2010 Object detection using non-redundant local Binary Patterns Duc Thanh

More information

CS 231A Computer Vision (Fall 2012) Problem Set 3

CS 231A Computer Vision (Fall 2012) Problem Set 3 CS 231A Computer Vision (Fall 2012) Problem Set 3 Due: Nov. 13 th, 2012 (2:15pm) 1 Probabilistic Recursion for Tracking (20 points) In this problem you will derive a method for tracking a point of interest

More information

CS 223B Computer Vision Problem Set 3

CS 223B Computer Vision Problem Set 3 CS 223B Computer Vision Problem Set 3 Due: Feb. 22 nd, 2011 1 Probabilistic Recursion for Tracking In this problem you will derive a method for tracking a point of interest through a sequence of images.

More information

Learning to Recognize Faces in Realistic Conditions

Learning to Recognize Faces in Realistic Conditions 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050

More information

Lecture 12: Video Representation, Summarisation, and Query

Lecture 12: Video Representation, Summarisation, and Query Lecture 12: Video Representation, Summarisation, and Query Dr Jing Chen NICTA & CSE UNSW CS9519 Multimedia Systems S2 2006 jchen@cse.unsw.edu.au Last week Structure of video Frame Shot Scene Story Why

More information

About MPEG Compression. More About Long-GOP Video

About MPEG Compression. More About Long-GOP Video About MPEG Compression HD video requires significantly more data than SD video. A single HD video frame can require up to six times more data than an SD frame. To record such large images with such a low

More information

PixSO: A System for Video Shot Detection

PixSO: A System for Video Shot Detection PixSO: A System for Video Shot Detection Chengcui Zhang 1, Shu-Ching Chen 1, Mei-Ling Shyu 2 1 School of Computer Science, Florida International University, Miami, FL 33199, USA 2 Department of Electrical

More information

AN EFFICIENT VIDEO WATERMARKING USING COLOR HISTOGRAM ANALYSIS AND BITPLANE IMAGE ARRAYS

AN EFFICIENT VIDEO WATERMARKING USING COLOR HISTOGRAM ANALYSIS AND BITPLANE IMAGE ARRAYS AN EFFICIENT VIDEO WATERMARKING USING COLOR HISTOGRAM ANALYSIS AND BITPLANE IMAGE ARRAYS G Prakash 1,TVS Gowtham Prasad 2, T.Ravi Kumar Naidu 3 1MTech(DECS) student, Department of ECE, sree vidyanikethan

More information

Selective Search for Object Recognition

Selective Search for Object Recognition Selective Search for Object Recognition Uijlings et al. Schuyler Smith Overview Introduction Object Recognition Selective Search Similarity Metrics Results Object Recognition Kitten Goal: Problem: Where

More information

Text Area Detection from Video Frames

Text Area Detection from Video Frames Text Area Detection from Video Frames 1 Text Area Detection from Video Frames Xiangrong Chen, Hongjiang Zhang Microsoft Research China chxr@yahoo.com, hjzhang@microsoft.com Abstract. Text area detection

More information

Image Segmentation. Segmentation is the process of partitioning an image into regions

Image Segmentation. Segmentation is the process of partitioning an image into regions Image Segmentation Segmentation is the process of partitioning an image into regions region: group of connected pixels with similar properties properties: gray levels, colors, textures, motion characteristics

More information

What Are Edges? Lecture 5: Gradients and Edge Detection. Boundaries of objects. Boundaries of Lighting. Types of Edges (1D Profiles)

What Are Edges? Lecture 5: Gradients and Edge Detection. Boundaries of objects. Boundaries of Lighting. Types of Edges (1D Profiles) What Are Edges? Simple answer: discontinuities in intensity. Lecture 5: Gradients and Edge Detection Reading: T&V Section 4.1 and 4. Boundaries of objects Boundaries of Material Properties D.Jacobs, U.Maryland

More information

Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation

Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation M. Blauth, E. Kraft, F. Hirschenberger, M. Böhm Fraunhofer Institute for Industrial Mathematics, Fraunhofer-Platz 1,

More information

Highlights Extraction from Unscripted Video

Highlights Extraction from Unscripted Video Highlights Extraction from Unscripted Video T 61.6030, Multimedia Retrieval Seminar presentation 04.04.2008 Harrison Mfula Helsinki University of Technology Department of Computer Science, Espoo, Finland

More information

Bluray (

Bluray ( Bluray (http://www.blu-ray.com/faq) MPEG-2 - enhanced for HD, also used for playback of DVDs and HDTV recordings MPEG-4 AVC - part of the MPEG-4 standard also known as H.264 (High Profile and Main Profile)

More information

DATA and signal modeling for images and video sequences. Region-Based Representations of Image and Video: Segmentation Tools for Multimedia Services

DATA and signal modeling for images and video sequences. Region-Based Representations of Image and Video: Segmentation Tools for Multimedia Services IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 9, NO. 8, DECEMBER 1999 1147 Region-Based Representations of Image and Video: Segmentation Tools for Multimedia Services P. Salembier,

More information

CS 4495 Computer Vision. Linear Filtering 2: Templates, Edges. Aaron Bobick. School of Interactive Computing. Templates/Edges

CS 4495 Computer Vision. Linear Filtering 2: Templates, Edges. Aaron Bobick. School of Interactive Computing. Templates/Edges CS 4495 Computer Vision Linear Filtering 2: Templates, Edges Aaron Bobick School of Interactive Computing Last time: Convolution Convolution: Flip the filter in both dimensions (right to left, bottom to

More information

Edge detection. Convert a 2D image into a set of curves. Extracts salient features of the scene More compact than pixels

Edge detection. Convert a 2D image into a set of curves. Extracts salient features of the scene More compact than pixels Edge Detection Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin of Edges surface normal discontinuity depth discontinuity surface

More information

Elimination of Duplicate Videos in Video Sharing Sites

Elimination of Duplicate Videos in Video Sharing Sites Elimination of Duplicate Videos in Video Sharing Sites Narendra Kumar S, Murugan S, Krishnaveni R Abstract - In some social video networking sites such as YouTube, there exists large numbers of duplicate

More information

MAXIMIZING BANDWIDTH EFFICIENCY

MAXIMIZING BANDWIDTH EFFICIENCY MAXIMIZING BANDWIDTH EFFICIENCY Benefits of Mezzanine Encoding Rev PA1 Ericsson AB 2016 1 (19) 1 Motivation 1.1 Consumption of Available Bandwidth Pressure on available fiber bandwidth continues to outpace

More information

convolution shift invariant linear system Fourier Transform Aliasing and sampling scale representation edge detection corner detection

convolution shift invariant linear system Fourier Transform Aliasing and sampling scale representation edge detection corner detection COS 429: COMPUTER VISON Linear Filters and Edge Detection convolution shift invariant linear system Fourier Transform Aliasing and sampling scale representation edge detection corner detection Reading:

More information

Histograms. h(r k ) = n k. p(r k )= n k /NM. Histogram: number of times intensity level rk appears in the image

Histograms. h(r k ) = n k. p(r k )= n k /NM. Histogram: number of times intensity level rk appears in the image Histograms h(r k ) = n k Histogram: number of times intensity level rk appears in the image p(r k )= n k /NM normalized histogram also a probability of occurence 1 Histogram of Image Intensities Create

More information

Review of Filtering. Filtering in frequency domain

Review of Filtering. Filtering in frequency domain Review of Filtering Filtering in frequency domain Can be faster than filtering in spatial domain (for large filters) Can help understand effect of filter Algorithm: 1. Convert image and filter to fft (fft2

More information

Image Processing, Analysis and Machine Vision

Image Processing, Analysis and Machine Vision Image Processing, Analysis and Machine Vision Milan Sonka PhD University of Iowa Iowa City, USA Vaclav Hlavac PhD Czech Technical University Prague, Czech Republic and Roger Boyle DPhil, MBCS, CEng University

More information

Lecture 3 Image and Video (MPEG) Coding

Lecture 3 Image and Video (MPEG) Coding CS 598KN Advanced Multimedia Systems Design Lecture 3 Image and Video (MPEG) Coding Klara Nahrstedt Fall 2017 Overview JPEG Compression MPEG Basics MPEG-4 MPEG-7 JPEG COMPRESSION JPEG Compression 8x8 blocks

More information

Local Image preprocessing (cont d)

Local Image preprocessing (cont d) Local Image preprocessing (cont d) 1 Outline - Edge detectors - Corner detectors - Reading: textbook 5.3.1-5.3.5 and 5.3.10 2 What are edges? Edges correspond to relevant features in the image. An edge

More information