INTERACTIVE SEARCH FOR IMAGE CATEGORIES BY MENTAL MATCHING

Size: px
Start display at page:

Download "INTERACTIVE SEARCH FOR IMAGE CATEGORIES BY MENTAL MATCHING"

Transcription

1 INTERACTIVE SEARCH FOR IMAGE CATEGORIES BY MENTAL MATCHING Donald GEMAN Dept. of Applied Mathematics and Statistics Center for Imaging Science Johns Hopkins University and INRIA, France

2 Collaborators Yuchun FANG (former postdoc, INRIA) Marin FERECATU (postdoc, INRIA) 11/14/2007 2

3 Outline Standard Image Retrieval Mental Matching A Mathematical Framework Modeling Human Behavior Experiments 11/14/2007 3

4 Query-by-Example (QBE) Start from an query image in a database. Find other images which are close or closest in overall color or texture or shape, or in some spatial region, or in a semantic sense, or Matching is performed by the system. Good results in limited domains, e.g., comparing paintings, plants and landscapes. 11/14/2007 4

5 Random I IKONA Search Engine (INRIA) 11/14/2007 5

6 RetrieveI.1 QBE Alinari database images 11/14/2007 6

7 Retrieve I.2 QBE Alinari database images 11/14/2007 7

8 Relevance Feedback (RF) Find the images in a database which satisfy a particular theme. An iterative learning process. An active user, providing positive and negative examples. Requires a starting point (some examples) Better adapted to semantics than QBE. 11/14/2007 8

9 Web Database, images Target theme: interior car design QBE: semantic gap Relevance Feedback 11/14/2007 9

10 Outline Standard Image Retrieval Mental Matching A Mathematical Framework Modeling Human Behavior Experiments 11/14/

11 Page Zero Problem QBE and RF require a starting point: A query image for QBE Positive and negative examples for RF Dilemma: Random sampling a large database is too slow in practice. 11/14/

12 External Images Mental Image: The user has a picture in mind, e.g., a face or painting or house. Viewed Image: The user is looking at a picture, e.g., in a magazine or on the web. Physical Object: The user is holding an object. 11/14/

13 Who is that person? Beckham?? Steve McQueen?? zizou?? 11/14/

14 Mental Category Search Assume this external query is represented in our database, either by a version of the same image (e.g., same person), or variations on a theme, i.e., a category of images (e.g., similar houses). Objective: Find an efficient way to display this version or representatives of this category. Solution: Small database: direct inspection Large database: interactive search 11/14/

15 Small database: direct search Large database:?? 11/14/

16 Potential Applications Image retrieval ( page zero ) Web browsing Security Art management E-Commerce Multimedia content providers Blah blah blah 11/14/

17 Simplifications Single target search is a special case of category search with singleton categories. Hence, Target: the object of the search, whether variations on a single image or on a theme. Assume the user always recognizes an instance of his target. 11/14/

18 Interactive Search At each iteration, some images are displayed, typically two to sixteen. The user responds by signaling his target if present; otherwise by selecting some as relevant and not relevant, or choosing the one deemed closest, or 11/14/

19 Interactive Search (cont) Based on this feedback, the system chooses another set of images to display. Goal: Minimize the number of iterations until an exemplar of the target is displayed. Then display other examples ( page zero ) for specialization and refinement. 11/14/

20 Scenarios The user may be naïve or primed (about the nature of the image representations). The target may or may not be constantly displayed. The database may or may not be structured into categories. 11/14/

21 Mental Face Retrieval Interface 11/14/

22 Category Search Interface 11/14/

23 Complications Mental matching involves human memory, perception and opinions. People are semantically oriented. However, images are not indexed by semantic content, but rather by low-level features ( semantic gap ). Large databases, order 10,000 to 1,000, /14/

24 Outline Standard Image Retrieval Mental Matching A Mathematical Framework Modeling Human Behavior Experiments 11/14/

25 Structured Database Ω= N k I, I,..., I : Database of images { } 1 2 Ω=Ω Ω 1 { } N K : Division into K categories : The number of images in category k Y 1,2,...,K : The user's category 11/14/

26 Features and Metrics f ( I ), f ( I ),..., f ( I ) :"features" in R { } 1 2 [ ] M M d : R R 0,1 : Metric on features f d ( i, j) : "Metric" on categories: N 1 1 d ( i, j) = d f ( f ( I), f ( I ')) N N I Ω I Ω i j i ' j M 11/14/

27 Metrics vs. Semantics First two principal components (PCA): Red: Meadow category, 100 images Green: 100 random images Blue: Whole database Distance histogram: Red: Meadow category, 100 images Blue: 100 random images 11/14/

28 Metrics vs. Semantics (cont) First two principal components (PCA): Red: Waterfall category, 100 images Green: 100 random images Blue: Whole database Distance histogram: Red: Waterfall category, 100 images Blue: 100 random images 11/14/

29 Display D 1, 2,..., K : A set of L distinct categories. { } (The actual display is Limages, one per category.) Dt ( ) : The categories displayed at time t= 1,2,... X D : The response of the user to D. For Y D, X = i means i is "closest" to D Y, in the opinion of the user and for the category Y in the mind of the user. 11/14/

30 Bayesian Framework Prior Distribution: A probability on categories (on individual images for single target search). Answer Model: The distribution of the user s response given the target. Display Model: An algorithm for choosing the display. Posterior Distribution (at time t): The likelihood after iteration t of each category being the target. 11/14/

31 Bayesian Framework (cont) Prior model: p ( k) = P( Y = k), k = 1,..., K 0 Answer ("data") model: P( X = i Y = k), k D D History ("evidence") after t steps: {,..., } H = X = a X = a t D (1) 1 D ( t) t Posterior distribution at step t : p ( k) = P( Y = k H ) t t 11/14/

32 The Posterior Distribution Updating p for t 0 requires the joint t distribution of Y and { X,..., X } D(1) D( t+ 1) Basic assumption: Given Y, the answers X for different Ds ' are independent random variables Consequently, p ( k) = P( Y = k H, X = a ) t+ 1 t D( t+ 1) t+ 1 PX ( = a Y= kp ) ( k) Dt ( + 1) t+ 1 t D 11/14/

33 Display Criterion D( t + 1) = arg max I( X ; Y H ) = D D t arg min H ( Y H, X ) D t D Due to conditional independence, H ( Y H, X ) is determined by p and X Y t t D D 11/14/

34 Recall: I( X ; Y ) = H ( Y ) H ( Y X ) = H ( X ) H ( X Y ) where and H ( Y ) = P( Y = y) log P( Y = y) y H ( Y X ) = P( X = x) x P ( Y = y X = x) log P( Y = y X = x) y 11/14/

35 Outline Standard Image Retrieval Mental Matching A Mathematical Framework Modeling Human Behavior Experiments 11/14/

36 Taking Stock In our framework, category search by mental matching reduces to two difficult tasks: An optimization problem: Discover approximations to the optimal display. A modeling problem: Discover answer models which match human behavior. 11/14/

37 Ideal User For k D : P( X = i Y = k ) = 1 D if d ( i, k ) < d ( j, k ) for all i, j D, j i. Since Y determines X : D ( t + 1) arg max I ( X ; Y H ) = arg m ax H ( X H ), w hich m otivates the follow ing: D D D t D D t 11/14/

38 Display Model: Heuristics Select the category k with the maximum posterior mass as the first element of D(t) Based on distances to k, collect categories until the total mass is close to 1/L. Remove this cluster from consideration. Repeat until L categories are selected. 11/14/

39 Optimal Display: The Voronoi Cells Have Equal Mass /14/

40 Real Answers φ( dik (, )) P( X D = i Y = k) = φ( d( j, k)) ( φ decreasing on [0,1]) Ex: Random response: φ 1 Ex: Optimistic choices: j D φ( d) = 1 d, φ( d ) = 1 / d Ex: M ore realistic choice (where θ1 and θ 2 are estimated from data): 0 0 θ /14/ θ 2 φ

41 Parameters Interpretation: θ 2 controls coherence with the distance, e.g.,the mass on a near-perfect match; θ 1 represents a no preference threshold (between two categories this far away from Y) Estimated from real data by maximum likelihood assuming independent decisions. 11/14/

42 Outline Standard Image Retrieval Mental Matching A Mathematical Framework Modeling Human Behavior Experiments 11/14/

43 Measures of Performance T: number of iterations until Y is displayed. P(T<t): The probability distribution some population of users. E(T): The mean of this population. Coherence: The probability that the user selects the i th closest category to his target, i=1,,l. 11/14/

44 Baseline Means: Random Search Parameters: N: # images in database K: # of classes L: # displayed per iteration Structured database, no category re-displayed: E(T) K/(2L) Unstructured database: E(T) N/(L(N*+1)), where N* = # images satisfying the user s theme. 11/14/

45 Experiment I: Mental Face Retrieval Feret Database 1199 images of distinct faces User memorizes one face Image Descriptors Preprocessing: lighting adjustment, alignment Region-based subspace methods, e.g., PCA, LDA, ICA, KPCA, KLDA. 11/14/

46 Experimental Conditions Web-based interface FERET balanced database (512 subjects, 1 image/subject) N=8 images per display θ 1 ~.1, θ 2 ~ complete searches (seven researchers at INRIA) representing 665 total decisions. 11/14/

47 Search Time Distribution: One Face 1 Probability distribution of T Number of iterations φ (d)=1-d, E(T)=10.5 S im ulatio n,, E (T )=9.7 R e al T e s t, E (T )=1 3.9 Random Response, E(T)=32 11/14/

48 Experiment II: Category Search Test Database 246 categories 9 images per category High intra-cluster semantic coherence Image Descriptors Global color, texture and shape, equally weighted 120 dimensions 11/14/

49 Category to Category Diversity 11/14/

50 Four Themes 11/14/

51 Experimental Conditions Web-based interface Target displayed alongside n=8 images per display θ 1 ~.27, θ 2 ~ complete searches (10 researchers at INRIA) representing 874 total decisions 11/14/

52 Coherence with System Metric 11/14/

53 Search Time Distribution: Categories 11/14/

54 Experiment III: Unstructured Database Ω= I, I,..., I : Database of images { } 1 2 S Ω : The category in the user's mind, a random set. Define Y = 1 if k S and k Y = 0 if k S, k=1,...,n k N Maintain a separate Bayesian system for each k. 11/14/

55 11/14/ Answer Models + + = = = D x j i k i D j k x d k x d Y x X P )), ( ( )), ( ( 1) ( φ φ = = = D x j i k i D j k x d k x d Y x X P )), ( ( )), ( ( 0) ( φ φ Positive model Negative model

56 Parameter Estimation 652 data items collected from 12 users:( Si, Di, xi) L + ( θ 1, θ 2 ) = Pi ( Si, Di, xi ) P ( S i i, D i, x i i ) = φ ( d( x + x j D i + i, S φ ( d( x j i )), S i )) φ φ + φ θ1 θ /14/

57 Semantic Ground Truth Sample from three semantic categories: - Monument Valley (left) - Pedigree dogs (middle) - Waterfalls (right) Other classes: African antelope, Butterfly, Doors of Paris, Fireworks, Deep forest, Molecule, Owl. 11/14/

58 Coherence with System Metric The probability that the user selects the m-th closest image to the target. 11/14/

59 The User Interface 11/14/

60 Alinari Database Madonna and Child (top rows) and Horse and Rider (bottom rows). 11/14/

61 Search Time Distribution: N=20,000 Cumulative distribution of the search time for real, ideal and random users. 11/14/

62 Conclusions Rich possibilities for mathematical modeling in building efficient man-machine interfaces. Mixes geometry, probability, optimization and information theory. Solving the vision problem is probably not around the corner. Hence extending to databases of order 1,000,000 remains a challenge. 11/14/

Bayesian Approaches to Content-based Image Retrieval

Bayesian Approaches to Content-based Image Retrieval Bayesian Approaches to Content-based Image Retrieval Simon Wilson Georgios Stefanou Department of Statistics Trinity College Dublin Background Content-based Image Retrieval Problem: searching for images

More information

CS 223B Computer Vision Problem Set 3

CS 223B Computer Vision Problem Set 3 CS 223B Computer Vision Problem Set 3 Due: Feb. 22 nd, 2011 1 Probabilistic Recursion for Tracking In this problem you will derive a method for tracking a point of interest through a sequence of images.

More information

Bayesian analysis of genetic population structure using BAPS: Exercises

Bayesian analysis of genetic population structure using BAPS: Exercises Bayesian analysis of genetic population structure using BAPS: Exercises p S u k S u p u,s S, Jukka Corander Department of Mathematics, Åbo Akademi University, Finland Exercise 1: Clustering of groups of

More information

CS 231A Computer Vision (Fall 2012) Problem Set 3

CS 231A Computer Vision (Fall 2012) Problem Set 3 CS 231A Computer Vision (Fall 2012) Problem Set 3 Due: Nov. 13 th, 2012 (2:15pm) 1 Probabilistic Recursion for Tracking (20 points) In this problem you will derive a method for tracking a point of interest

More information

Computer Vision. Exercise Session 10 Image Categorization

Computer Vision. Exercise Session 10 Image Categorization Computer Vision Exercise Session 10 Image Categorization Object Categorization Task Description Given a small number of training images of a category, recognize a-priori unknown instances of that category

More information

10701 Machine Learning. Clustering

10701 Machine Learning. Clustering 171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among

More information

IMAGE SEGMENTATION. Václav Hlaváč

IMAGE SEGMENTATION. Václav Hlaváč IMAGE SEGMENTATION Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception http://cmp.felk.cvut.cz/ hlavac, hlavac@fel.cvut.cz

More information

Sensor Tasking and Control

Sensor Tasking and Control Sensor Tasking and Control Outline Task-Driven Sensing Roles of Sensor Nodes and Utilities Information-Based Sensor Tasking Joint Routing and Information Aggregation Summary Introduction To efficiently

More information

Gene Clustering & Classification

Gene Clustering & Classification BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering

More information

Object Recognition. Lecture 11, April 21 st, Lexing Xie. EE4830 Digital Image Processing

Object Recognition. Lecture 11, April 21 st, Lexing Xie. EE4830 Digital Image Processing Object Recognition Lecture 11, April 21 st, 2008 Lexing Xie EE4830 Digital Image Processing http://www.ee.columbia.edu/~xlx/ee4830/ 1 Announcements 2 HW#5 due today HW#6 last HW of the semester Due May

More information

Object Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision

Object Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision Object Recognition Using Pictorial Structures Daniel Huttenlocher Computer Science Department Joint work with Pedro Felzenszwalb, MIT AI Lab In This Talk Object recognition in computer vision Brief definition

More information

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality

More information

Automatic Categorization of Image Regions using Dominant Color based Vector Quantization

Automatic Categorization of Image Regions using Dominant Color based Vector Quantization Automatic Categorization of Image Regions using Dominant Color based Vector Quantization Md Monirul Islam, Dengsheng Zhang, Guojun Lu Gippsland School of Information Technology, Monash University Churchill

More information

Learning Spatial Context: Using Stuff to Find Things

Learning Spatial Context: Using Stuff to Find Things Learning Spatial Context: Using Stuff to Find Things Wei-Cheng Su Motivation 2 Leverage contextual information to enhance detection Some context objects are non-rigid and are more naturally classified

More information

ECLT 5810 Clustering

ECLT 5810 Clustering ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping

More information

ECLT 5810 Clustering

ECLT 5810 Clustering ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping

More information

Learning to Recognize Faces in Realistic Conditions

Learning to Recognize Faces in Realistic Conditions 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050

More information

A Content Based Image Retrieval System Based on Color Features

A Content Based Image Retrieval System Based on Color Features A Content Based Image Retrieval System Based on Features Irena Valova, University of Rousse Angel Kanchev, Department of Computer Systems and Technologies, Rousse, Bulgaria, Irena@ecs.ru.acad.bg Boris

More information

Multimedia Data Management M

Multimedia Data Management M ALMA MATER STUDIORUM - UNIVERSITÀ DI BOLOGNA Multimedia Data Management M Second cycle degree programme (LM) in Computer Engineering University of Bologna Semantic Multimedia Data Annotation Home page:

More information

Supervised Learning for Image Segmentation

Supervised Learning for Image Segmentation Supervised Learning for Image Segmentation Raphael Meier 06.10.2016 Raphael Meier MIA 2016 06.10.2016 1 / 52 References A. Ng, Machine Learning lecture, Stanford University. A. Criminisi, J. Shotton, E.

More information

Extracting Layers and Recognizing Features for Automatic Map Understanding. Yao-Yi Chiang

Extracting Layers and Recognizing Features for Automatic Map Understanding. Yao-Yi Chiang Extracting Layers and Recognizing Features for Automatic Map Understanding Yao-Yi Chiang 0 Outline Introduction/ Problem Motivation Map Processing Overview Map Decomposition Feature Recognition Discussion

More information

Clustering and The Expectation-Maximization Algorithm

Clustering and The Expectation-Maximization Algorithm Clustering and The Expectation-Maximization Algorithm Unsupervised Learning Marek Petrik 3/7 Some of the figures in this presentation are taken from An Introduction to Statistical Learning, with applications

More information

K Means Clustering Using Localized Histogram Analysis and Multiple Assignment. Michael Bryson 4/18/2007

K Means Clustering Using Localized Histogram Analysis and Multiple Assignment. Michael Bryson 4/18/2007 1 K Means Clustering Using Localized Histogram Analysis and Multiple Assignment Michael Bryson 4/18/2007 2 Outline Introduction Redefining Distance Preliminary Results Multiple Assignment Discussion 3

More information

Mesh segmentation. Florent Lafarge Inria Sophia Antipolis - Mediterranee

Mesh segmentation. Florent Lafarge Inria Sophia Antipolis - Mediterranee Mesh segmentation Florent Lafarge Inria Sophia Antipolis - Mediterranee Outline What is mesh segmentation? M = {V,E,F} is a mesh S is either V, E or F (usually F) A Segmentation is a set of sub-meshes

More information

Optimizing feature representation for speaker diarization using PCA and LDA

Optimizing feature representation for speaker diarization using PCA and LDA Optimizing feature representation for speaker diarization using PCA and LDA itsikv@netvision.net.il Jean-Francois Bonastre jean-francois.bonastre@univ-avignon.fr Outline Speaker Diarization what is it?

More information

PSU Student Research Symposium 2017 Bayesian Optimization for Refining Object Proposals, with an Application to Pedestrian Detection Anthony D.

PSU Student Research Symposium 2017 Bayesian Optimization for Refining Object Proposals, with an Application to Pedestrian Detection Anthony D. PSU Student Research Symposium 2017 Bayesian Optimization for Refining Object Proposals, with an Application to Pedestrian Detection Anthony D. Rhodes 5/10/17 What is Machine Learning? Machine learning

More information

Clustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford

Clustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically

More information

Computer vision: models, learning and inference. Chapter 13 Image preprocessing and feature extraction

Computer vision: models, learning and inference. Chapter 13 Image preprocessing and feature extraction Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction Preprocessing The goal of pre-processing is to try to reduce unwanted variation in image due to lighting,

More information

Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University

Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate

More information

Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval

Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval Kien A. Hua, Ning Yu, Danzhou Liu School of Electrical Engineering and Computer Science

More information

Image Processing (IP)

Image Processing (IP) Image Processing Pattern Recognition Computer Vision Xiaojun Qi Utah State University Image Processing (IP) Manipulate and analyze digital images (pictorial information) by computer. Applications: The

More information

FUSION OF MULTITEMPORAL AND MULTIRESOLUTION REMOTE SENSING DATA AND APPLICATION TO NATURAL DISASTERS

FUSION OF MULTITEMPORAL AND MULTIRESOLUTION REMOTE SENSING DATA AND APPLICATION TO NATURAL DISASTERS FUSION OF MULTITEMPORAL AND MULTIRESOLUTION REMOTE SENSING DATA AND APPLICATION TO NATURAL DISASTERS Ihsen HEDHLI, Josiane ZERUBIA INRIA Sophia Antipolis Méditerranée (France), Ayin team, in collaboration

More information

Energy Minimization for Segmentation in Computer Vision

Energy Minimization for Segmentation in Computer Vision S * = arg S min E(S) Energy Minimization for Segmentation in Computer Vision Meng Tang, Dmitrii Marin, Ismail Ben Ayed, Yuri Boykov Outline Clustering/segmentation methods K-means, GrabCut, Normalized

More information

Video Google faces. Josef Sivic, Mark Everingham, Andrew Zisserman. Visual Geometry Group University of Oxford

Video Google faces. Josef Sivic, Mark Everingham, Andrew Zisserman. Visual Geometry Group University of Oxford Video Google faces Josef Sivic, Mark Everingham, Andrew Zisserman Visual Geometry Group University of Oxford The objective Retrieve all shots in a video, e.g. a feature length film, containing a particular

More information

Collection Guiding: Multimedia Collection Browsing and Visualization. Outline. Context. Searching for data

Collection Guiding: Multimedia Collection Browsing and Visualization. Outline. Context. Searching for data Collection Guiding: Multimedia Collection Browsing and Visualization Stéphane Marchand-Maillet Viper CVML University of Geneva marchand@cui.unige.ch http://viper.unige.ch Outline Multimedia data context

More information

Object perception by primates

Object perception by primates Object perception by primates How does the visual system put together the fragments to form meaningful objects? The Gestalt approach The whole differs from the sum of its parts and is a result of perceptual

More information

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify

More information

8/3/2017. Contour Assessment for Quality Assurance and Data Mining. Objective. Outline. Tom Purdie, PhD, MCCPM

8/3/2017. Contour Assessment for Quality Assurance and Data Mining. Objective. Outline. Tom Purdie, PhD, MCCPM Contour Assessment for Quality Assurance and Data Mining Tom Purdie, PhD, MCCPM Objective Understand the state-of-the-art in contour assessment for quality assurance including data mining-based techniques

More information

Lecture on Modeling Tools for Clustering & Regression

Lecture on Modeling Tools for Clustering & Regression Lecture on Modeling Tools for Clustering & Regression CS 590.21 Analysis and Modeling of Brain Networks Department of Computer Science University of Crete Data Clustering Overview Organizing data into

More information

Expectation Maximization!

Expectation Maximization! Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University and http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Steps in Clustering Select Features

More information

Using the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection

Using the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection Using the Deformable Part Model with Autoencoded Feature Descriptors for Object Detection Hyunghoon Cho and David Wu December 10, 2010 1 Introduction Given its performance in recent years' PASCAL Visual

More information

Last week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints

Last week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints Last week Multi-Frame Structure from Motion: Multi-View Stereo Unknown camera viewpoints Last week PCA Today Recognition Today Recognition Recognition problems What is it? Object detection Who is it? Recognizing

More information

CHAPTER 6 PROPOSED HYBRID MEDICAL IMAGE RETRIEVAL SYSTEM USING SEMANTIC AND VISUAL FEATURES

CHAPTER 6 PROPOSED HYBRID MEDICAL IMAGE RETRIEVAL SYSTEM USING SEMANTIC AND VISUAL FEATURES 188 CHAPTER 6 PROPOSED HYBRID MEDICAL IMAGE RETRIEVAL SYSTEM USING SEMANTIC AND VISUAL FEATURES 6.1 INTRODUCTION Image representation schemes designed for image retrieval systems are categorized into two

More information

SYDE Winter 2011 Introduction to Pattern Recognition. Clustering

SYDE Winter 2011 Introduction to Pattern Recognition. Clustering SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned

More information

Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network. Nathan Sun CIS601

Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network. Nathan Sun CIS601 Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network Nathan Sun CIS601 Introduction Face ID is complicated by alterations to an individual s appearance Beard,

More information

A NOVEL FEATURE EXTRACTION METHOD BASED ON SEGMENTATION OVER EDGE FIELD FOR MULTIMEDIA INDEXING AND RETRIEVAL

A NOVEL FEATURE EXTRACTION METHOD BASED ON SEGMENTATION OVER EDGE FIELD FOR MULTIMEDIA INDEXING AND RETRIEVAL A NOVEL FEATURE EXTRACTION METHOD BASED ON SEGMENTATION OVER EDGE FIELD FOR MULTIMEDIA INDEXING AND RETRIEVAL Serkan Kiranyaz, Miguel Ferreira and Moncef Gabbouj Institute of Signal Processing, Tampere

More information

Applications. Foreground / background segmentation Finding skin-colored regions. Finding the moving objects. Intelligent scissors

Applications. Foreground / background segmentation Finding skin-colored regions. Finding the moving objects. Intelligent scissors Segmentation I Goal Separate image into coherent regions Berkeley segmentation database: http://www.eecs.berkeley.edu/research/projects/cs/vision/grouping/segbench/ Slide by L. Lazebnik Applications Intelligent

More information

Analysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009

Analysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009 Analysis: TextonBoost and Semantic Texton Forests Daniel Munoz 16-721 Februrary 9, 2009 Papers [shotton-eccv-06] J. Shotton, J. Winn, C. Rother, A. Criminisi, TextonBoost: Joint Appearance, Shape and Context

More information

An Overview of Mathematics 6

An Overview of Mathematics 6 An Overview of Mathematics 6 Number (N) read, write, represent, and describe numbers greater than one million and less than one-thousandth using symbols, expressions, expanded notation, decimal notation,

More information

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped

More information

Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information

Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information Probabilistic Facial Feature Extraction Using Joint Distribution of Location and Texture Information Mustafa Berkay Yilmaz, Hakan Erdogan, Mustafa Unel Sabanci University, Faculty of Engineering and Natural

More information

Consistent Line Clusters for Building Recognition in CBIR

Consistent Line Clusters for Building Recognition in CBIR Consistent Line Clusters for Building Recognition in CBIR Yi Li and Linda G. Shapiro Department of Computer Science and Engineering University of Washington Seattle, WA 98195-250 shapiro,yi @cs.washington.edu

More information

Texton Clustering for Local Classification using Scene-Context Scale

Texton Clustering for Local Classification using Scene-Context Scale Texton Clustering for Local Classification using Scene-Context Scale Yousun Kang Tokyo Polytechnic University Atsugi, Kanakawa, Japan 243-0297 Email: yskang@cs.t-kougei.ac.jp Sugimoto Akihiro National

More information

Rough Feature Selection for CBIR. Outline

Rough Feature Selection for CBIR. Outline Rough Feature Selection for CBIR Instructor:Dr. Wojciech Ziarko presenter :Aifen Ye 19th Nov., 2008 Outline Motivation Rough Feature Selection Image Retrieval Image Retrieval with Rough Feature Selection

More information

Contents III. 1 Introduction 1

Contents III. 1 Introduction 1 III Contents 1 Introduction 1 2 The Parametric Distributional Clustering Model 5 2.1 The Data Acquisition Process.................... 5 2.2 The Generative Model........................ 8 2.3 The Likelihood

More information

Efficient Content Based Image Retrieval System with Metadata Processing

Efficient Content Based Image Retrieval System with Metadata Processing IJIRST International Journal for Innovative Research in Science & Technology Volume 1 Issue 10 March 2015 ISSN (online): 2349-6010 Efficient Content Based Image Retrieval System with Metadata Processing

More information

Computer vision: models, learning and inference. Chapter 10 Graphical Models

Computer vision: models, learning and inference. Chapter 10 Graphical Models Computer vision: models, learning and inference Chapter 10 Graphical Models Independence Two variables x 1 and x 2 are independent if their joint probability distribution factorizes as Pr(x 1, x 2 )=Pr(x

More information

Segmentation: Clustering, Graph Cut and EM

Segmentation: Clustering, Graph Cut and EM Segmentation: Clustering, Graph Cut and EM Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL 60208 yingwu@northwestern.edu http://www.eecs.northwestern.edu/~yingwu

More information

Lecture 7: Decision Trees

Lecture 7: Decision Trees Lecture 7: Decision Trees Instructor: Outline 1 Geometric Perspective of Classification 2 Decision Trees Geometric Perspective of Classification Perspective of Classification Algorithmic Geometric Probabilistic...

More information

Automatically Improving 3D Neuron Segmentations for Expansion Microscopy Connectomics. by Albert Gerovitch

Automatically Improving 3D Neuron Segmentations for Expansion Microscopy Connectomics. by Albert Gerovitch Automatically Improving 3D Neuron Segmentations for Expansion Microscopy Connectomics by Albert Gerovitch 1 Abstract Understanding the geometry of neurons and their connections is key to comprehending

More information

Snakes, level sets and graphcuts. (Deformable models)

Snakes, level sets and graphcuts. (Deformable models) INSTITUTE OF INFORMATION AND COMMUNICATION TECHNOLOGIES BULGARIAN ACADEMY OF SCIENCE Snakes, level sets and graphcuts (Deformable models) Centro de Visión por Computador, Departament de Matemàtica Aplicada

More information

8. Automatic Content Analysis

8. Automatic Content Analysis 8. Automatic Content Analysis 8.1 Statistics for Multimedia Content Analysis 8.2 Basic Parameters for Video Analysis 8.3 Deriving Video Semantics 8.4 Basic Parameters for Audio Analysis 8.5 Deriving Audio

More information

ELL 788 Computational Perception & Cognition July November 2015

ELL 788 Computational Perception & Cognition July November 2015 ELL 788 Computational Perception & Cognition July November 2015 Module 6 Role of context in object detection Objects and cognition Ambiguous objects Unfavorable viewing condition Context helps in object

More information

CS Introduction to Data Mining Instructor: Abdullah Mueen

CS Introduction to Data Mining Instructor: Abdullah Mueen CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts

More information

Clustering CE-324: Modern Information Retrieval Sharif University of Technology

Clustering CE-324: Modern Information Retrieval Sharif University of Technology Clustering CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2014 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276, Stanford) Ch. 16 What

More information

Computer Vision Patch-based Object Recognition (2)

Computer Vision Patch-based Object Recognition (2) Computer Vision Patch-based Object Recognition (2) Contents Papers on patch-based object recognition Previous class: basic idea Bayes Theorem: probability background Papers in this class Hierarchy recognition

More information

EE 701 ROBOT VISION. Segmentation

EE 701 ROBOT VISION. Segmentation EE 701 ROBOT VISION Regions and Image Segmentation Histogram-based Segmentation Automatic Thresholding K-means Clustering Spatial Coherence Merging and Splitting Graph Theoretic Segmentation Region Growing

More information

Generative and discriminative classification techniques

Generative and discriminative classification techniques Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15

More information

Grade 6 Middle School Math Solution Alignment to Oklahoma Academic Standards

Grade 6 Middle School Math Solution Alignment to Oklahoma Academic Standards 6.N.1 Read, write, and represent integers and rational numbers expressed as fractions, decimals, percents, and ratios; write positive integers as products of factors; use these representations in real-world

More information

Feature LDA: a Supervised Topic Model for Automatic Detection of Web API Documentations from the Web

Feature LDA: a Supervised Topic Model for Automatic Detection of Web API Documentations from the Web Feature LDA: a Supervised Topic Model for Automatic Detection of Web API Documentations from the Web Chenghua Lin, Yulan He, Carlos Pedrinaci, and John Domingue Knowledge Media Institute, The Open University

More information

Matrix Co-factorization for Recommendation with Rich Side Information HetRec 2011 and Implicit 1 / Feedb 23

Matrix Co-factorization for Recommendation with Rich Side Information HetRec 2011 and Implicit 1 / Feedb 23 Matrix Co-factorization for Recommendation with Rich Side Information and Implicit Feedback Yi Fang and Luo Si Department of Computer Science Purdue University West Lafayette, IN 47906, USA fangy@cs.purdue.edu

More information

Image segmentation. Václav Hlaváč. Czech Technical University in Prague

Image segmentation. Václav Hlaváč. Czech Technical University in Prague Image segmentation Václav Hlaváč Czech Technical University in Prague Center for Machine Perception (bridging groups of the) Czech Institute of Informatics, Robotics and Cybernetics and Faculty of Electrical

More information

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Zhen Qin (University of California, Riverside) Peter van Beek & Xu Chen (SHARP Labs of America, Camas, WA) 2015/8/30

More information

Image Classification. RS Image Classification. Present by: Dr.Weerakaset Suanpaga

Image Classification. RS Image Classification. Present by: Dr.Weerakaset Suanpaga Image Classification Present by: Dr.Weerakaset Suanpaga D.Eng(RS&GIS) 6.1 Concept of Classification Objectives of Classification Advantages of Multi-Spectral data for Classification Variation of Multi-Spectra

More information

Lecture 6: Multimedia Information Retrieval Dr. Jian Zhang

Lecture 6: Multimedia Information Retrieval Dr. Jian Zhang Lecture 6: Multimedia Information Retrieval Dr. Jian Zhang NICTA & CSE UNSW COMP9314 Advanced Database S1 2007 jzhang@cse.unsw.edu.au Reference Papers and Resources Papers: Colour spaces-perceptual, historical

More information

IMPROVING THE PERFORMANCE OF CONTENT-BASED IMAGE RETRIEVAL SYSTEMS WITH COLOR IMAGE PROCESSING TOOLS

IMPROVING THE PERFORMANCE OF CONTENT-BASED IMAGE RETRIEVAL SYSTEMS WITH COLOR IMAGE PROCESSING TOOLS IMPROVING THE PERFORMANCE OF CONTENT-BASED IMAGE RETRIEVAL SYSTEMS WITH COLOR IMAGE PROCESSING TOOLS Fabio Costa Advanced Technology & Strategy (CGISS) Motorola 8000 West Sunrise Blvd. Plantation, FL 33322

More information

Data Clustering Hierarchical Clustering, Density based clustering Grid based clustering

Data Clustering Hierarchical Clustering, Density based clustering Grid based clustering Data Clustering Hierarchical Clustering, Density based clustering Grid based clustering Team 2 Prof. Anita Wasilewska CSE 634 Data Mining All Sources Used for the Presentation Olson CF. Parallel algorithms

More information

Linear combinations of simple classifiers for the PASCAL challenge

Linear combinations of simple classifiers for the PASCAL challenge Linear combinations of simple classifiers for the PASCAL challenge Nik A. Melchior and David Lee 16 721 Advanced Perception The Robotics Institute Carnegie Mellon University Email: melchior@cmu.edu, dlee1@andrew.cmu.edu

More information

AS STORAGE and bandwidth capacities increase, digital

AS STORAGE and bandwidth capacities increase, digital 974 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 13, NO. 7, JULY 2004 Concept-Oriented Indexing of Video Databases: Toward Semantic Sensitive Retrieval and Browsing Jianping Fan, Hangzai Luo, and Ahmed

More information

Introduction to digital image classification

Introduction to digital image classification Introduction to digital image classification Dr. Norman Kerle, Wan Bakx MSc a.o. INTERNATIONAL INSTITUTE FOR GEO-INFORMATION SCIENCE AND EARTH OBSERVATION Purpose of lecture Main lecture topics Review

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Pierre Gaillard ENS Paris September 28, 2018 1 Supervised vs unsupervised learning Two main categories of machine learning algorithms: - Supervised learning: predict output Y from

More information

CHAPTER 6 QUANTITATIVE PERFORMANCE ANALYSIS OF THE PROPOSED COLOR TEXTURE SEGMENTATION ALGORITHMS

CHAPTER 6 QUANTITATIVE PERFORMANCE ANALYSIS OF THE PROPOSED COLOR TEXTURE SEGMENTATION ALGORITHMS 145 CHAPTER 6 QUANTITATIVE PERFORMANCE ANALYSIS OF THE PROPOSED COLOR TEXTURE SEGMENTATION ALGORITHMS 6.1 INTRODUCTION This chapter analyzes the performance of the three proposed colortexture segmentation

More information

Markov Networks in Computer Vision

Markov Networks in Computer Vision Markov Networks in Computer Vision Sargur Srihari srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Some applications: 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction

More information

IJREAT International Journal of Research in Engineering & Advanced Technology, Volume 1, Issue 5, Oct-Nov, 2013 ISSN:

IJREAT International Journal of Research in Engineering & Advanced Technology, Volume 1, Issue 5, Oct-Nov, 2013 ISSN: Semi Automatic Annotation Exploitation Similarity of Pics in i Personal Photo Albums P. Subashree Kasi Thangam 1 and R. Rosy Angel 2 1 Assistant Professor, Department of Computer Science Engineering College,

More information

Robust Shape Retrieval Using Maximum Likelihood Theory

Robust Shape Retrieval Using Maximum Likelihood Theory Robust Shape Retrieval Using Maximum Likelihood Theory Naif Alajlan 1, Paul Fieguth 2, and Mohamed Kamel 1 1 PAMI Lab, E & CE Dept., UW, Waterloo, ON, N2L 3G1, Canada. naif, mkamel@pami.uwaterloo.ca 2

More information

Flat Clustering. Slides are mostly from Hinrich Schütze. March 27, 2017

Flat Clustering. Slides are mostly from Hinrich Schütze. March 27, 2017 Flat Clustering Slides are mostly from Hinrich Schütze March 7, 07 / 79 Overview Recap Clustering: Introduction 3 Clustering in IR 4 K-means 5 Evaluation 6 How many clusters? / 79 Outline Recap Clustering:

More information

Topological Mapping. Discrete Bayes Filter

Topological Mapping. Discrete Bayes Filter Topological Mapping Discrete Bayes Filter Vision Based Localization Given a image(s) acquired by moving camera determine the robot s location and pose? Towards localization without odometry What can be

More information

Computer Vision I - Filtering and Feature detection

Computer Vision I - Filtering and Feature detection Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image

More information

An Enhanced Image Retrieval Using K-Mean Clustering Algorithm in Integrating Text and Visual Features

An Enhanced Image Retrieval Using K-Mean Clustering Algorithm in Integrating Text and Visual Features An Enhanced Image Retrieval Using K-Mean Clustering Algorithm in Integrating Text and Visual Features S.Najimun Nisha 1, Mrs.K.A.Mehar Ban 2, 1 PG Student, SVCET, Puliangudi. najimunnisha@yahoo.com 2 AP/CSE,

More information

Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University

Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate

More information

INF 4300 Classification III Anne Solberg The agenda today:

INF 4300 Classification III Anne Solberg The agenda today: INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15

More information

CS443: Digital Imaging and Multimedia Binary Image Analysis. Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University

CS443: Digital Imaging and Multimedia Binary Image Analysis. Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University CS443: Digital Imaging and Multimedia Binary Image Analysis Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines A Simple Machine Vision System Image segmentation by thresholding

More information

Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA

Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 08 Feature Aggregation II: Fisher Vector, Super Vector and AKULA Zhu Li Dept of CSEE,

More information

Markov Networks in Computer Vision. Sargur Srihari

Markov Networks in Computer Vision. Sargur Srihari Markov Networks in Computer Vision Sargur srihari@cedar.buffalo.edu 1 Markov Networks for Computer Vision Important application area for MNs 1. Image segmentation 2. Removal of blur/noise 3. Stereo reconstruction

More information

Content-Based Image Retrieval Readings: Chapter 8:

Content-Based Image Retrieval Readings: Chapter 8: Content-Based Image Retrieval Readings: Chapter 8: 8.1-8.4 Queries Commercial Systems Retrieval Features Indexing in the FIDS System Lead-in to Object Recognition 1 Content-based Image Retrieval (CBIR)

More information

Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University

Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University Kinds of Clustering Sequential Fast Cost Optimization Fixed number of clusters Hierarchical

More information

Efficient Acquisition of Human Existence Priors from Motion Trajectories

Efficient Acquisition of Human Existence Priors from Motion Trajectories Efficient Acquisition of Human Existence Priors from Motion Trajectories Hitoshi Habe Hidehito Nakagawa Masatsugu Kidode Graduate School of Information Science, Nara Institute of Science and Technology

More information

Clustering Relational Data using the Infinite Relational Model

Clustering Relational Data using the Infinite Relational Model Clustering Relational Data using the Infinite Relational Model Ana Daglis Supervised by: Matthew Ludkin September 4, 2015 Ana Daglis Clustering Data using the Infinite Relational Model September 4, 2015

More information

Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors

Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors Ajay Divakaran, Kadir A. Peker, Regunathan Radhakrishnan, Ziyou Xiong and Romain Cabasson Presented by Giulia Fanti 1 Overview Motivation

More information

Quantitative Assessment of Composition in Art

Quantitative Assessment of Composition in Art NICOGRAPH International 202, pp. 80-85 Quantitative Assessment of Composition in Art Sachi URANO Junichi HOSHINO University of Tsukuba s0853 (at) u.tsukuba.ac.jp Abstract We present a new method to evaluate

More information