Scene Detection Media Mining I
|
|
- Milo Gallagher
- 6 years ago
- Views:
Transcription
1 Scene Detection Media Mining I Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uni-augsburg.de
2 Overview Hierarchical structure of video sequence Fames Shots = Sequence taken by a single camera during one ON/OFF cycle Scenes (Story units) = Single dramatic event taken by a small number of related cameras A Musical Analogy Notes Shots Chords Scenes Musical Phrases Themes/Acts Common Approach Shot detection prerequisite Scene detection Abstracted as a question of symbolic pattern segmentation such as ABABCDCD 2
3 Scene Transition Graph
4 Time Constrained Clustering (1) Step 1: Assign to each shot s i a label L i which provides a description of the content of the shot Difficult: Automatic semantic labels such as news anchor, news room, man behind table are difficult Instead use low-level feature such as color histograms Time-constrained clustering: Let C i be the ith cluster, and w, x, y the elements (video shots) in a cluster Clustering criteria: for all x in C i Visual dissimilarity between shots Temporal distance between shots Max. visual separation between 2 shots in cluster C i Max. Temporal distance between start/stop of earlier/later shot in C i 4
5 Time Constrained Clustering (2) Frame Dissimilarity Normalized color histogram difference (8r*8g*4b=256 bins): D(ƒ i, ƒ j ) = b h ib h jb /N h ib = a given bin b in color histogram of frame i N = total # of pixels in each frame L1-Norm normalized to [0,1] Shot Dissimilarity Minimum dissimilarity between any two frames of each shot D(S i,s j ) = min k,l D(ƒ ki, ƒ lj ) ƒ ki = frame k of shot i ƒ lj = frame l of shot j Not a metric: violates triangle inequality and positive definite req. 5
6 Time Constrained Clustering (3) Perform complete-link(-age) hierarchical clustering = merge in each step the two clusters whose merger has the smallest diameter (or: the two clusters with the smallest maximum pairwise distance) NOT Single-link(-age) hierarchical clustering = merge in each step the two clusters whose two closest members have the smallest distance (or: the two clusters with the smallest minimum pairwise distance). Worst case time complexity: O(n^2 log n) One O(n^2 log n) algorithm: 1. Compute the n^2 distance metric 2. Sort the distances for each data point (overall time: O(n^2 log n)). 3. After each merge iteration, the distance metric can be updated in O(n). We pick the next pair to merge by finding the smallest distance that is still eligible for merging. If we do this by traversing the n sorted lists of distances, then, by the end of clustering, we will have done n^2 traversal steps. 4. Adding all this up gives you O(n^2 log n). 6
7 Complete-link Clustering 1. Start with n clusters C 1,...,C n, where C i = x i 2. Minimize cost function c(c i,c j ) in order to find the two best clusters to merge.. 3. Replace C i and C j by C i C j 4. Repeat 2. and 3. until all clusters are merged into one Complete-link c( C i, C j ) max x C, y i C j d( x, y) 7
8 Time Constrained Clustering (4) Result of time-constrained clustering: Video partition into C 1, C 2,, C N. Each C j consists of one or more shots Each shot s i in C j is assigned label L j Label sequence {L j } Example: A video sequence: ABABCDCDABAB 8
9 Scene Transition Graph (1) Step 1: Derive story units / scenes last(a, b) = max i b:l i ==A i ; i in set of shot indices = the shot index of the last occurrence of label A starting at shot s b [Algorithm Detect Story Unit] 1) l := m, e := last (L l,m) 2) While l <= e do if ( last (L l,m) > e) e := last (L l,m) l++; 3) Shots s m, s m+1,, s e constitute a story unit 9
10 Scene Transition Graph (2) Example 1: Short video sequence ABABCDCDABAB The graph is: A B D C Cyclic with no cut edge. Example 2: Long video sequence ABABCDCDABAB Becomes: ABABCDCDEFEF The graph is: A B C D E F Two cut edges. Example 3: Video sequence ABABCDCD 8 shots, 4 camera positions The graph is: A B C D The cut edge is from (the last) B to (the first) C. Less than T apart More than T apart 10
11 Scene Transition Graph (3) 11
12 Scene Transition Graph (4) 12
13 13
14 Experimental Results 14
15 Scene Determination based on Video and Audio Features
16 System Overview 16
17 Audio Feature Audio sequence := sequence of shots in which very similar audio signal re-occur. Pre-Filter: Background elimination Get rid of unimportant sound pieces Threshold (a) signal power or (b) perceptual loudness measure Audio Cuts = time instances which delimit periods of similar sound => audio shots Power spectrum Distance Measure Each video shots has at least one associated audio shot Disaggregated set representation of audio shots Minimal Euclidean distance found between any two feature vectors of two audio shots 17
18 Video Features Color Coherence Vectors (CCV) Developed by Pass, Zabih, and Miller Orientation Correlograms Shot Dissimilarity Measure Dissimilarity between 2 shots with respect to their color/orientation content is measured based on the disaggregated set representation of the shots, using the minimum distance between the most common feature values Each shot S i is described by a set of feature values {f 1i,, f n i } derived from each of its frames and compared with shot S j = {f 1j,, f n j } by dist(s i,s j ) = min {d(f i, f j ) f i in S i, f j in S j } 18
19 Color Coherence Vector Shots with very similar color content usually belong to a common setting because they share a common background. The color content changes more dramatically from setting to setting than within a single setting. Color content is usually measured by some sort of refined color histogram technique such as the color coherence vector (CCV) CCV makes use of spatial coherence and discriminates much better than the basic color histogram. Instead of counting only the number of pixels of a certain color, the CCV additionally distinguishes between coherent and incoherent pixels within each color class j depending on the size of the color region to which they belong. If the region (i.e., the connected 8-neighbor component of that color) is larger than a threshold t ccv, a pixel is regarded as coherent, otherwise as incoherent. Thus, there are two values associated with each color j: α j : the number of coherent pixels of color j and β j : the number of incoherent pixels of color j Two CCVs CCV 1 and CCV 2 are compared by 19
20 Orientation Correlogram (1) Local orientation := image structure in which the gray or color values change only in exactly one direction, but remain static in the orthogonal direction. Does not distinguish between direction x and (x+180) (unlike direction) Inertia Tensor: Gradient Gaussian Filter I(x,y) = grayscale image D x and D y = first derivative in x and y direction 20
21 Orientation Correlogram (2) measures the angle of orientation of the image structure around the location (x, y). The relation of the eigenvalues to each other can be used as a certainty measure of the estimated orientation. 3 cases can be distinguished: 1. λ 1» λ 2 : There is an orientation in direction Φ 2. λ1 λ2 : The gray-scale values change similarly in all directions, and thus, the structure is isotropic. 3. λ1,λ2 0 : The local environment has a constant gray-scale value. 21
22 Orientation Correlogram (3) Capture local orientation of an image independently of translations and small or middle scalings Orientation correlogram := a table indexed by an orientation pair <i,j>. The kth entry γ i,jk of an orientation pair <i,j> specifies the probability that within a distance of k of an orientation i the orientation j can be found in the image. Describes how the spatial correlation of orientation pairs changes with distance Orientation Φ is discretized in n classes K i : 22
23 Dialog Detection (1) Frontal Face Detection Neural network detector Proposed by Rowley, Baluja and Kanade 23
24 Dialog Detection (2) Dialog Detection A sequence of contiguous shots of a minimum length of three shots where at least one face-based class is present in each shot and the Eigenface-based shot-overlapping relations between face-based classes interlink shots by crossings. 24
25 Scene Determination (1) Observation: Same features need not to be present in each shot Human similarity judgement is very adaptive Our Approach: shot cluster := all shots between two shots which are no further apart than the lookahead (e.g., 3 shots) and their distance is below a predefined (absolute) threshold. Overlapping shot clusters are grouped into scenes Merging overlapping shot clusters from different domains 25
26 Scene Determination (2) 26
27 Ground Truth Determined manually for each kind of scene: Dialog, Locale, Audio Scene, Semantic Scene by classifying each shot boundary whether the two involved shots belong together 1 if two associated shots belong to the same scene and 0 otherwise. Sometimes intensive discussion at critical points necessary Tool to support creation of ground truth shot boundary being labeled shot n Future shots help to decide shot n+1 shot n+2 n++ Play 2 seconds preceding and following the shot boundary from n to n+1 27
28 Experiments Test Set Groundhog Day Forrest Gump Methodology Hit rate h = (# of correctly eliminated shot boundaries + correctly detected scene boundaries) / (total # of shot boundaries) Miss rate m = (# of missed scene boundaries) / (# of shot boundaries) False hit rate f = (# of falsely detected scene boundaries) / (# of shot boundaries) = 1 h - m 28
29 Quality of Audio Sequence Determination 85 % 10 % threshold 29
30 Quality of Video Setting Determination 80 % 10 % threshold 30
31 Quality of Dialog Determination Groundhog Day Forrest Gump # shots # dialogs Hit rate 35% 60% False hits 5% 118% 31
32 Examples of Detected Scenes 32
33 Conclusion Four type of features Clustered into audio sequences, settings and dialogs Hit rate rates 66% - 90% (except dialogs) False hit rates 5% - 28% (except dialogs) 33
34 Unified Shot & Scene Detection
35 temporal Video Structure non-temporal 35
36 Core Idea Use short term visual memory buffer Common models in psychology: sensory buffers are leaky integrators Encode extend to which the present frame reminds the viewer of a prior one Shot Detection Encode extend to which the present shot reminds the viewer of a prior one Scene Detection A B C B C A B A B C X Y B Z Good coherence bad coherence 36
37 Basic Notations (Shot Detection) D(F i,f j ) = symmetric frame dissimilarity measure between frame F i at time i and frame F j at time j, i<j, e.g., L 1 norm on color histograms (see Time Constrained Clustering (2)) e -(j-i)/b = probability a frame F j recalls a (prior) frame F i (of dissimilarity D(F i,f j ) expected frame-to-frame dissimilarity: D(F i,f j ) e -(j-i)/b If F j is similar/dissimilar to frame F i, it is also very likely similar/dissimilar to those frames immediately preceding F i. Recall(i,j)= i B=size of short term memory - D(F t,f j ) e -(j-t)/b dt // D made suitably continuous aggregated expected dissimilarity experienced at frame F j due to all frames occurring before frame F i Total amount of dissimilarity induced at the interframe gap just after frame F i : Dis(i) = i Recall(i,j)dj= i i- D(F t,f j ) e -(j-t)/b dt dj Simply a double convolution of pair-wise Features: frame dissimilarities with an exponential Due to integration this method is more robust than simple adjacent kernel frame comparison modeling the memory buffer loss a peak at shot boundaries a local, but more rounded maximum at dissolve boundaries a local minimum at very representative key frames 37
38 Dissolve Model (1/2) Without loss of generality we make the following assumptions: Unit of time = buffer size B normalized equations Edits are center at t=0 ranging from w to w Edit duration = 2Bw d = dissimilarity between frame just before and after dissolve Cross Dissolve dissimilarity between frames at time i and j within dissolve (i.e., -w i j w) is given by d(j-i)/(2w) Before/after edit the shot is almost constant in its dissimilarity measure, i.e., D(F i, F j ) 0 Recall(i,j) and Dis(i) can be computed 38
39 Dissolve Model (2/2) 3 cases to be considered Case 2: -w i w j Recall: Recent past of F j consists of frames before, during, and after the dissolve For i>0 Before: D(i,j) d During: D(i,j) d(j-i)/(2w) After: D(i,j) 0 Recall( i, j) w de ( j t) dt Dis(i) = i Recall(i,j)dj = i w w t d e 2w // linear dissolve ( j t) dt 39
40 Examples (1/3) Dis(0)=d/w(1-exp(-w)) Theoretic response of Dis(i) to ideal dissolves of size w = 0 to w = 5 with frame distances measured in units of buffer size. An ideal dissolve with w = 0 is a cut, and shows the impulse response 40
41 Examples (2/3) Response of model to a section of a video Cut at frame # 4869 Dissolve at frame # 4936 with width 26 Small dissimilarities in the middle of the graph due to a pan Do not affect segmentation results 41
42 Examples (3/3) Fade out centered at frame # 2255 with width 19 Fade in centered at frame # 2293 with width 20 42
43 Parameter Settings The large buffer size B, the more frames are recalled, and the video tends to be perceived as being smoother The smaller buffer size B, the more unlikely that incoming frames have similar frames to recall, and local frame differences predominate in the measure Shot Detection B = 4, K = 120 (+-60; extend of exp. loss filter) Maxima detection in dis(i): dis(i) >dis(j), j [i-n, i+n] & monotonic decrease; N=6 43
44 Scene Detection Scene Boundary = Shot boundary with a small total coherence recall Same approach as for shot detection, but Buffer size B significantly larger e.g., 8 time the average shot length (~1000 frames) Local maximum search = maximum dissimilarity at shot boundaries, i.e, search only at shot boundaries e.g., N=3 44
Motion Tracking and Event Understanding in Video Sequences
Motion Tracking and Event Understanding in Video Sequences Isaac Cohen Elaine Kang, Jinman Kang Institute for Robotics and Intelligent Systems University of Southern California Los Angeles, CA Objectives!
More information8 Conclusion and Outlook
[8] M. Gorkani and R. W. Picard. Texture Orientation for Sorting Photos "at a Glance". Proc. International Conference on Pattern Recognition, Jerusalem, Vol. I, pp. 459-464, 1995. [9] A.G. Hauptmann, and
More informationMotion in 2D image sequences
Motion in 2D image sequences Definitely used in human vision Object detection and tracking Navigation and obstacle avoidance Analysis of actions or activities Segmentation and understanding of video sequences
More informationOptical Flow-Based Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.
Optical Flow-Based Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object
More informationVideo Syntax Analysis
1 Video Syntax Analysis Wei-Ta Chu 2008/10/9 Outline 2 Scene boundary detection Key frame selection 3 Announcement of HW #1 Shot Change Detection Goal: automatic shot change detection Requirements 1. Write
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationSegmentation and Grouping
Segmentation and Grouping How and what do we see? Fundamental Problems ' Focus of attention, or grouping ' What subsets of pixels do we consider as possible objects? ' All connected subsets? ' Representation
More informationImage Segmentation. Selim Aksoy. Bilkent University
Image Segmentation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Examples of grouping in vision [http://poseidon.csd.auth.gr/lab_research/latest/imgs/s peakdepvidindex_img2.jpg]
More informationImage Segmentation. Selim Aksoy. Bilkent University
Image Segmentation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Examples of grouping in vision [http://poseidon.csd.auth.gr/lab_research/latest/imgs/s peakdepvidindex_img2.jpg]
More informationAccepted for publication in Kluwer Multimedia Tools and Applications
REIHE INFORMATIK 5/97 VisualGREP: A Systematic Method to Compare and Retrieve Video Sequences Rainer Lienhart Universität Mannheim Praktische Informatik IV L5,6 D-683 Mannheim revised Mai 998 2 VisualGREP:
More informationNormalized cuts and image segmentation
Normalized cuts and image segmentation Department of EE University of Washington Yeping Su Xiaodan Song Normalized Cuts and Image Segmentation, IEEE Trans. PAMI, August 2000 5/20/2003 1 Outline 1. Image
More informationSearching Video Collections:Part I
Searching Video Collections:Part I Introduction to Multimedia Information Retrieval Multimedia Representation Visual Features (Still Images and Image Sequences) Color Texture Shape Edges Objects, Motion
More informationNon-exhaustive, Overlapping k-means
Non-exhaustive, Overlapping k-means J. J. Whang, I. S. Dhilon, and D. F. Gleich Teresa Lebair University of Maryland, Baltimore County October 29th, 2015 Teresa Lebair UMBC 1/38 Outline Introduction NEO-K-Means
More informationELEC Dr Reji Mathew Electrical Engineering UNSW
ELEC 4622 Dr Reji Mathew Electrical Engineering UNSW Review of Motion Modelling and Estimation Introduction to Motion Modelling & Estimation Forward Motion Backward Motion Block Motion Estimation Motion
More informationA Robust Wipe Detection Algorithm
A Robust Wipe Detection Algorithm C. W. Ngo, T. C. Pong & R. T. Chin Department of Computer Science The Hong Kong University of Science & Technology Clear Water Bay, Kowloon, Hong Kong Email: fcwngo, tcpong,
More informationApplications. Foreground / background segmentation Finding skin-colored regions. Finding the moving objects. Intelligent scissors
Segmentation I Goal Separate image into coherent regions Berkeley segmentation database: http://www.eecs.berkeley.edu/research/projects/cs/vision/grouping/segbench/ Slide by L. Lazebnik Applications Intelligent
More informationRegion-based Segmentation
Region-based Segmentation Image Segmentation Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation. Applications: Finding tumors, veins, etc.
More informationEdge and local feature detection - 2. Importance of edge detection in computer vision
Edge and local feature detection Gradient based edge detection Edge detection by function fitting Second derivative edge detectors Edge linking and the construction of the chain graph Edge and local feature
More informationGene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
More informationDetecting and Identifying Moving Objects in Real-Time
Chapter 9 Detecting and Identifying Moving Objects in Real-Time For surveillance applications or for human-computer interaction, the automated real-time tracking of moving objects in images from a stationary
More informationPattern recognition. Classification/Clustering GW Chapter 12 (some concepts) Textures
Pattern recognition Classification/Clustering GW Chapter 12 (some concepts) Textures Patterns and pattern classes Pattern: arrangement of descriptors Descriptors: features Patten class: family of patterns
More informationAIIA shot boundary detection at TRECVID 2006
AIIA shot boundary detection at TRECVID 6 Z. Černeková, N. Nikolaidis and I. Pitas Artificial Intelligence and Information Analysis Laboratory Department of Informatics Aristotle University of Thessaloniki
More informationComputer Vision 2. SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung. Computer Vision 2 Dr. Benjamin Guthier
Computer Vision 2 SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung Computer Vision 2 Dr. Benjamin Guthier 1. IMAGE PROCESSING Computer Vision 2 Dr. Benjamin Guthier Content of this Chapter Non-linear
More informationCS 664 Slides #11 Image Segmentation. Prof. Dan Huttenlocher Fall 2003
CS 664 Slides #11 Image Segmentation Prof. Dan Huttenlocher Fall 2003 Image Segmentation Find regions of image that are coherent Dual of edge detection Regions vs. boundaries Related to clustering problems
More informationAnalysis of Image and Video Using Color, Texture and Shape Features for Object Identification
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 16, Issue 6, Ver. VI (Nov Dec. 2014), PP 29-33 Analysis of Image and Video Using Color, Texture and Shape Features
More informationThe Detection of Faces in Color Images: EE368 Project Report
The Detection of Faces in Color Images: EE368 Project Report Angela Chau, Ezinne Oji, Jeff Walters Dept. of Electrical Engineering Stanford University Stanford, CA 9435 angichau,ezinne,jwalt@stanford.edu
More informationMachine learning Pattern recognition. Classification/Clustering GW Chapter 12 (some concepts) Textures
Machine learning Pattern recognition Classification/Clustering GW Chapter 12 (some concepts) Textures Patterns and pattern classes Pattern: arrangement of descriptors Descriptors: features Patten class:
More informationCS 534: Computer Vision Segmentation and Perceptual Grouping
CS 534: Computer Vision Segmentation and Perceptual Grouping Ahmed Elgammal Dept of Computer Science CS 534 Segmentation - 1 Outlines Mid-level vision What is segmentation Perceptual Grouping Segmentation
More informationVisual Tracking. Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania.
Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania 1 What is visual tracking? estimation of the target location over time 2 applications Six main areas:
More informationHow and what do we see? Segmentation and Grouping. Fundamental Problems. Polyhedral objects. Reducing the combinatorics of pose estimation
Segmentation and Grouping Fundamental Problems ' Focus of attention, or grouping ' What subsets of piels do we consider as possible objects? ' All connected subsets? ' Representation ' How do we model
More informationImage Analysis - Lecture 5
Texture Segmentation Clustering Review Image Analysis - Lecture 5 Texture and Segmentation Magnus Oskarsson Lecture 5 Texture Segmentation Clustering Review Contents Texture Textons Filter Banks Gabor
More informationCluster Analysis. Angela Montanari and Laura Anderlucci
Cluster Analysis Angela Montanari and Laura Anderlucci 1 Introduction Clustering a set of n objects into k groups is usually moved by the aim of identifying internally homogenous groups according to a
More informationComputer Vision I - Filtering and Feature detection
Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
More informationCHAPTER 6 PERCEPTUAL ORGANIZATION BASED ON TEMPORAL DYNAMICS
CHAPTER 6 PERCEPTUAL ORGANIZATION BASED ON TEMPORAL DYNAMICS This chapter presents a computational model for perceptual organization. A figure-ground segregation network is proposed based on a novel boundary
More informationFunction approximation using RBF network. 10 basis functions and 25 data points.
1 Function approximation using RBF network F (x j ) = m 1 w i ϕ( x j t i ) i=1 j = 1... N, m 1 = 10, N = 25 10 basis functions and 25 data points. Basis function centers are plotted with circles and data
More informationMarkov Networks in Computer Vision
Markov Networks in Computer Vision Sargur Srihari srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Some applications: 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction
More informationINF4820. Clustering. Erik Velldal. Nov. 17, University of Oslo. Erik Velldal INF / 22
INF4820 Clustering Erik Velldal University of Oslo Nov. 17, 2009 Erik Velldal INF4820 1 / 22 Topics for Today More on unsupervised machine learning for data-driven categorization: clustering. The task
More informationEE368 Project: Visual Code Marker Detection
EE368 Project: Visual Code Marker Detection Kahye Song Group Number: 42 Email: kahye@stanford.edu Abstract A visual marker detection algorithm has been implemented and tested with twelve training images.
More informationCHAPTER 3 SHOT DETECTION AND KEY FRAME EXTRACTION
33 CHAPTER 3 SHOT DETECTION AND KEY FRAME EXTRACTION 3.1 INTRODUCTION The twenty-first century is an age of information explosion. We are witnessing a huge growth in digital data. The trend of increasing
More informationMarkov Networks in Computer Vision. Sargur Srihari
Markov Networks in Computer Vision Sargur srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Important application area for MNs 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction
More informationSupervised texture detection in images
Supervised texture detection in images Branislav Mičušík and Allan Hanbury Pattern Recognition and Image Processing Group, Institute of Computer Aided Automation, Vienna University of Technology Favoritenstraße
More informationUniversity of Florida CISE department Gator Engineering. Clustering Part 5
Clustering Part 5 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville SNN Approach to Clustering Ordinary distance measures have problems Euclidean
More informationLecture 8: Fitting. Tuesday, Sept 25
Lecture 8: Fitting Tuesday, Sept 25 Announcements, schedule Grad student extensions Due end of term Data sets, suggestions Reminder: Midterm Tuesday 10/9 Problem set 2 out Thursday, due 10/11 Outline Review
More informationWhat is Computer Vision?
Perceptual Grouping in Computer Vision Gérard Medioni University of Southern California What is Computer Vision? Computer Vision Attempt to emulate Human Visual System Perceive visual stimuli with cameras
More informationMultidimensional Image Registered Scanner using MDPSO (Multi-objective Discrete Particle Swarm Optimization)
Multidimensional Image Registered Scanner using MDPSO (Multi-objective Discrete Particle Swarm Optimization) Rishiganesh V 1, Swaruba P 2 PG Scholar M.Tech-Multimedia Technology, Department of CSE, K.S.R.
More informationMotion Estimation and Optical Flow Tracking
Image Matching Image Retrieval Object Recognition Motion Estimation and Optical Flow Tracking Example: Mosiacing (Panorama) M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 Example 3D Reconstruction
More informationBiometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong)
Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) References: [1] http://homepages.inf.ed.ac.uk/rbf/hipr2/index.htm [2] http://www.cs.wisc.edu/~dyer/cs540/notes/vision.html
More informationRecall precision graph
VIDEO SHOT BOUNDARY DETECTION USING SINGULAR VALUE DECOMPOSITION Λ Z.»CERNEKOVÁ, C. KOTROPOULOS AND I. PITAS Aristotle University of Thessaloniki Box 451, Thessaloniki 541 24, GREECE E-mail: (zuzana, costas,
More informationImage Segmentation. Shengnan Wang
Image Segmentation Shengnan Wang shengnan@cs.wisc.edu Contents I. Introduction to Segmentation II. Mean Shift Theory 1. What is Mean Shift? 2. Density Estimation Methods 3. Deriving the Mean Shift 4. Mean
More informationObject Extraction Using Image Segmentation and Adaptive Constraint Propagation
Object Extraction Using Image Segmentation and Adaptive Constraint Propagation 1 Rajeshwary Patel, 2 Swarndeep Saket 1 Student, 2 Assistant Professor 1 2 Department of Computer Engineering, 1 2 L. J. Institutes
More informationImage Segmentation. Segmentation is the process of partitioning an image into regions
Image Segmentation Segmentation is the process of partitioning an image into regions region: group of connected pixels with similar properties properties: gray levels, colors, textures, motion characteristics
More informationMULTIVIEW REPRESENTATION OF 3D OBJECTS OF A SCENE USING VIDEO SEQUENCES
MULTIVIEW REPRESENTATION OF 3D OBJECTS OF A SCENE USING VIDEO SEQUENCES Mehran Yazdi and André Zaccarin CVSL, Dept. of Electrical and Computer Engineering, Laval University Ste-Foy, Québec GK 7P4, Canada
More informationVideo Key-Frame Extraction using Entropy value as Global and Local Feature
Video Key-Frame Extraction using Entropy value as Global and Local Feature Siddu. P Algur #1, Vivek. R *2 # Department of Information Science Engineering, B.V. Bhoomraddi College of Engineering and Technology
More information82 REGISTRATION OF RETINOGRAPHIES
82 REGISTRATION OF RETINOGRAPHIES 3.3 Our method Our method resembles the human approach to image matching in the sense that we also employ as guidelines features common to both images. It seems natural
More informationMULTI ORIENTATION PERFORMANCE OF FEATURE EXTRACTION FOR HUMAN HEAD RECOGNITION
MULTI ORIENTATION PERFORMANCE OF FEATURE EXTRACTION FOR HUMAN HEAD RECOGNITION Panca Mudjirahardjo, Rahmadwati, Nanang Sulistiyanto and R. Arief Setyawan Department of Electrical Engineering, Faculty of
More informationVideo shot segmentation using late fusion technique
Video shot segmentation using late fusion technique by C. Krishna Mohan, N. Dhananjaya, B.Yegnanarayana in Proc. Seventh International Conference on Machine Learning and Applications, 2008, San Diego,
More informationPattern recognition. Classification/Clustering GW Chapter 12 (some concepts) Textures
Pattern recognition Classification/Clustering GW Chapter 12 (some concepts) Textures Patterns and pattern classes Pattern: arrangement of descriptors Descriptors: features Patten class: family of patterns
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationColor Image Segmentation
Color Image Segmentation Yining Deng, B. S. Manjunath and Hyundoo Shin* Department of Electrical and Computer Engineering University of California, Santa Barbara, CA 93106-9560 *Samsung Electronics Inc.
More informationOperators-Based on Second Derivative double derivative Laplacian operator Laplacian Operator Laplacian Of Gaussian (LOG) Operator LOG
Operators-Based on Second Derivative The principle of edge detection based on double derivative is to detect only those points as edge points which possess local maxima in the gradient values. Laplacian
More informationIntroduction to Medical Imaging (5XSA0) Module 5
Introduction to Medical Imaging (5XSA0) Module 5 Segmentation Jungong Han, Dirk Farin, Sveta Zinger ( s.zinger@tue.nl ) 1 Outline Introduction Color Segmentation region-growing region-merging watershed
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationThe Pre-Image Problem in Kernel Methods
The Pre-Image Problem in Kernel Methods James Kwok Ivor Tsang Department of Computer Science Hong Kong University of Science and Technology Hong Kong The Pre-Image Problem in Kernel Methods ICML-2003 1
More informationAutomatic Movie Abstracting and Its Presentation on an HTML-Page ABSTRACT
Automatic Movie Abstracting and Its Presentation on an HTML-Page Rainer Lienhart, Silvia Pfeiffer, and Stephan Fischer University of Mannheim, Praktische Informatik IV, 68131 Mannheim, Germany {lienhart,
More informationThe goals of segmentation
Image segmentation The goals of segmentation Group together similar-looking pixels for efficiency of further processing Bottom-up process Unsupervised superpixels X. Ren and J. Malik. Learning a classification
More informationOCCLUSION BOUNDARIES ESTIMATION FROM A HIGH-RESOLUTION SAR IMAGE
OCCLUSION BOUNDARIES ESTIMATION FROM A HIGH-RESOLUTION SAR IMAGE Wenju He, Marc Jäger, and Olaf Hellwich Berlin University of Technology FR3-1, Franklinstr. 28, 10587 Berlin, Germany {wenjuhe, jaeger,
More informationUlrik Söderström 16 Feb Image Processing. Segmentation
Ulrik Söderström ulrik.soderstrom@tfe.umu.se 16 Feb 2011 Image Processing Segmentation What is Image Segmentation? To be able to extract information from an image it is common to subdivide it into background
More informationCS 664 Segmentation. Daniel Huttenlocher
CS 664 Segmentation Daniel Huttenlocher Grouping Perceptual Organization Structural relationships between tokens Parallelism, symmetry, alignment Similarity of token properties Often strong psychophysical
More informationFeature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking
Feature descriptors Alain Pagani Prof. Didier Stricker Computer Vision: Object and People Tracking 1 Overview Previous lectures: Feature extraction Today: Gradiant/edge Points (Kanade-Tomasi + Harris)
More informationComputer Vision I - Basics of Image Processing Part 1
Computer Vision I - Basics of Image Processing Part 1 Carsten Rother 28/10/2014 Computer Vision I: Basics of Image Processing Link to lectures Computer Vision I: Basics of Image Processing 28/10/2014 2
More informationUsing the Kolmogorov-Smirnov Test for Image Segmentation
Using the Kolmogorov-Smirnov Test for Image Segmentation Yong Jae Lee CS395T Computational Statistics Final Project Report May 6th, 2009 I. INTRODUCTION Image segmentation is a fundamental task in computer
More informationImage Processing. Bilkent University. CS554 Computer Vision Pinar Duygulu
Image Processing CS 554 Computer Vision Pinar Duygulu Bilkent University Today Image Formation Point and Blob Processing Binary Image Processing Readings: Gonzalez & Woods, Ch. 3 Slides are adapted from
More informationDense Image-based Motion Estimation Algorithms & Optical Flow
Dense mage-based Motion Estimation Algorithms & Optical Flow Video A video is a sequence of frames captured at different times The video data is a function of v time (t) v space (x,y) ntroduction to motion
More informationClustering Part 4 DBSCAN
Clustering Part 4 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville DBSCAN DBSCAN is a density based clustering algorithm Density = number of
More informationPeripheral drift illusion
Peripheral drift illusion Does it work on other animals? Computer Vision Motion and Optical Flow Many slides adapted from J. Hays, S. Seitz, R. Szeliski, M. Pollefeys, K. Grauman and others Video A video
More informationSegmentation Computer Vision Spring 2018, Lecture 27
Segmentation http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 218, Lecture 27 Course announcements Homework 7 is due on Sunday 6 th. - Any questions about homework 7? - How many of you have
More informationINF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationRobust Shape Retrieval Using Maximum Likelihood Theory
Robust Shape Retrieval Using Maximum Likelihood Theory Naif Alajlan 1, Paul Fieguth 2, and Mohamed Kamel 1 1 PAMI Lab, E & CE Dept., UW, Waterloo, ON, N2L 3G1, Canada. naif, mkamel@pami.uwaterloo.ca 2
More informationModern Medical Image Analysis 8DC00 Exam
Parts of answers are inside square brackets [... ]. These parts are optional. Answers can be written in Dutch or in English, as you prefer. You can use drawings and diagrams to support your textual answers.
More informationMulti-Camera Calibration, Object Tracking and Query Generation
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Multi-Camera Calibration, Object Tracking and Query Generation Porikli, F.; Divakaran, A. TR2003-100 August 2003 Abstract An automatic object
More informationAn Introduction to Content Based Image Retrieval
CHAPTER -1 An Introduction to Content Based Image Retrieval 1.1 Introduction With the advancement in internet and multimedia technologies, a huge amount of multimedia data in the form of audio, video and
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 WRI C225 Lecture 04 130131 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Histogram Equalization Image Filtering Linear
More informationIdea. Found boundaries between regions (edges) Didn t return the actual region
Region Segmentation Idea Edge detection Found boundaries between regions (edges) Didn t return the actual region Segmentation Partition image into regions find regions based on similar pixel intensities,
More informationRapid Natural Scene Text Segmentation
Rapid Natural Scene Text Segmentation Ben Newhouse, Stanford University December 10, 2009 1 Abstract A new algorithm was developed to segment text from an image by classifying images according to the gradient
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 11 140311 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Motion Analysis Motivation Differential Motion Optical
More informationSYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
More informationOther Linear Filters CS 211A
Other Linear Filters CS 211A Slides from Cornelia Fermüller and Marc Pollefeys Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin
More informationFundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F
Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental
More informationAnalysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009
Analysis: TextonBoost and Semantic Texton Forests Daniel Munoz 16-721 Februrary 9, 2009 Papers [shotton-eccv-06] J. Shotton, J. Winn, C. Rother, A. Criminisi, TextonBoost: Joint Appearance, Shape and Context
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationCS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
More informationObtaining Feature Correspondences
Obtaining Feature Correspondences Neill Campbell May 9, 2008 A state-of-the-art system for finding objects in images has recently been developed by David Lowe. The algorithm is termed the Scale-Invariant
More informationCS 4495 Computer Vision Motion and Optic Flow
CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS4 is out, due Sunday Oct 27 th. All relevant lectures posted Details about Problem Set: You may *not* use built in Harris
More informationFingerprint Classification Using Orientation Field Flow Curves
Fingerprint Classification Using Orientation Field Flow Curves Sarat C. Dass Michigan State University sdass@msu.edu Anil K. Jain Michigan State University ain@msu.edu Abstract Manual fingerprint classification
More informationCS 534: Computer Vision Segmentation and Perceptual Grouping
CS 534: Computer Vision Segmentation and Perceptual Grouping Spring 2005 Ahmed Elgammal Dept of Computer Science CS 534 Segmentation - 1 Where are we? Image Formation Human vision Cameras Geometric Camera
More informationFeature extraction. Bi-Histogram Binarization Entropy. What is texture Texture primitives. Filter banks 2D Fourier Transform Wavlet maxima points
Feature extraction Bi-Histogram Binarization Entropy What is texture Texture primitives Filter banks 2D Fourier Transform Wavlet maxima points Edge detection Image gradient Mask operators Feature space
More informationSegmentation (continued)
Segmentation (continued) Lecture 05 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr Mubarak Shah Professor, University of Central Florida The Robotics
More informationFiltering Images. Contents
Image Processing and Data Visualization with MATLAB Filtering Images Hansrudi Noser June 8-9, 010 UZH, Multimedia and Robotics Summer School Noise Smoothing Filters Sigmoid Filters Gradient Filters Contents
More informationEdge detection. Convert a 2D image into a set of curves. Extracts salient features of the scene More compact than pixels
Edge Detection Edge detection Convert a 2D image into a set of curves Extracts salient features of the scene More compact than pixels Origin of Edges surface normal discontinuity depth discontinuity surface
More information