Pattern Recognition: Theory and Applications
|
|
- Eugenia Wade
- 5 years ago
- Views:
Transcription
1 Pattern Recognition: Theory and Applications Saroj K. Meher Systems Science and Informatics Unit, Indian Statistical Institute, Bangalore Centre.
2 2/134 Contents: Pattern recognition Introduction Definitions and framework Advantages and Applications areas Discrimination functions Learning methodologies Some examples of classification and Clustering methods Semisupervised and co-training algorithm Multiple Classification Systems Contd
3 3/134 Pattern Recognition (PR): Motivation Pattern recognition stems from the need for automated machine recognition of objects, signals or images, or the need for automated decision-making based on a given set of parameters.
4 4/134 PR Defined PR is a domain of machine learning. It is the act of taking in raw data and make an action based on it. The assignment of a physical object or event to one of several prespecified categories Duda and Hart A problem of estimating density functions in a high-dimensional space and dividing the space into the regions of categories or classes Fukunaga The science that concerns the description or classification (recognition) of measurements Schalkoff The process of giving names ω to observations x, Schürmann PR is concerned with answering the question What is this? Morse PR PLAYS THE ROLE WHEN A DECISION IS ABOUT TO TAKE
5 5/134 Hybrid Systems Knowledge-based Systems Probabilistic reasoning Approximate reasoning Case based reasoning Fuzzy logic Rough sets Neuro-fuzzy Genetic neural Fuzzy genetic Fuzzy Rough Neuro Rough Machine Intelligence Non-linear Dynamics Data Driven Systems Neural network system Evolutionary computing Pattern recognition and learning Chaos theory Rescaled range analysis (wavelet) Fractal analysis Machine Intelligence: A core concept for grouping various advanced technologies with Pattern Recognition and Learning
6 6/134 Pattern Recognition System (PRS) Measurement Feature Decision Space Space Space
7 7/134 What is a pattern? A pattern is the opposite of a chaos; it is an entity vaguely defined, that could be given a name. WHAT ABOUT TEXTURE?
8 8/134 What is a pattern? A pattern is an abstract object, or a set of measurements describing a physical object.
9 9/134 Examples of Patterns Cristal Patterns: atómic or molecular Their structures are represented by 3D graphs and can be described by deterministic grammars or formal languages
10 10/134 Examples of Patterns Patterns of Constellations Patterns of constellations are represented by 2D planar graphs Human perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high.
11 11/134 Examples of Patterns Biological Patterns --- Landmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants, Points cannot be registered crossing instances. Applications: Biometrics, computacional anatomy, brain mapping,
12 12/134 Examples of Patterns Biological Patterns Landmarks are identified from biologic forms and these patterns are then represented by a list of points.
13 13/134 Examples of Patterns Music Patterns Ravel Symphony?
14 14/134 Examples of Patterns People Recognition Funny, Funny Patterns Behavior?
15 15/134 Examples of Patterns Discovery and Association of Patterns Statistics show connections between the shape of one s face (adults) and his/her Character. There is also evidence that the outline of children s face is related to alcohol abuse during pregnancy.
16 16/134 Examples of Patterns Discovery and Association of Patterns What are the features? Statistics show connections between the shape of one s face (adults) and his/her Character.
17 17/134 Examples of Patterns Patterns of Brain Activity We may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviors
18 18/134 Examples of Patterns Variation Patterns: 1. Expression geometric deformation 2. illumination--- Photometric deformation 3. Transformation 2D pose 3D 4. Noise and Occlusion
19 19/134 Examples of Patterns A broad range of texture patterns are generated by stochastic processes.
20 20/134 Examples of Patterns. How are these patterns represented in human mind?
21 21/134 Examples of Patterns Speech signals and Hidden Markov models
22 22/134 Examples of Patterns Natural Language and stochastic grammar..
23 23/134 Examples of Patterns Object Recognition Patterns everywhere?
24 24/134 Examples of Patterns Maps Recognition Patterns of Global Warming?
25 25/134 Examples of Patterns Finacial Series Pattern Recognition
26 26/134 Examples of Patterns How to Trade Chart Patterns?
27 27/134 Examples of Patterns Pattern Recognition in Medical Diagnosis
28 28/134 Examples of Patterns Optical Character Recognition
29 29/134 Examples of Patterns Graphic Arts Escher, who else?
30 30/134 Examples of Patterns Human Genome Beautiful Patterns!
31 31/134 Approaches Statistical PR: based on underlying statistical model of patterns and pattern classes. Neural networks: classifier is represented as a network of cells modeling neurons of the human brain (connectionist approach). Structural (or syntactic) PR: pattern classes represented by means of formal structures as grammars, automata, strings, etc.
32 32/134 What is a pattern class? A pattern class (or category) is a set of patterns sharing common attributes. A collection of similar (not necessarily identical) objects. During recognition given objects are assigned to prescribed classes.
33 33/134 What is pattern recognition? Theory, Algorithms, Systems to put Patterns into Categories Relate Perceived Pattern to Previously Perceived Patterns Learn to distinguish patterns of interest from their background
34 34/134 Human Perception Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g., Recognizing a face. Understanding spoken words. Reading handwriting. Distinguishing fresh food from its smell. We would like to give similar capabilities to machines.
35 35/134 Examples of applications Optical Character Recognition (OCR) Handwritten: sorting letters by postal code. Printed texts: reading machines for blind people, digitalization of text documents. Biometrics Face recognition, verification, retrieval. Finger prints recognition. Speech recognition. Diagnostic systems Medical diagnosis: X-Ray, EKG (ElectroCardioGraph) analysis. Military applications Automated Target Recognition (ATR). Image segmentation and analysis (recognition from aerial or satelite photographs).
36 36/134 THE STATISTICAL APPROACH
37 37/134 Examples of applications Grid by Grid Comparison A A B Grid by Grid Comparison
38 38/134 Examples of applications Grid by Grid Comparison A A B No of Mismatch= 3 38
39 39/134 Examples of applications Grid by Grid Comparison A A B Grid by Grid Comparison
40 40/134 Examples of applications Grid by Grid Comparison A A B No of Mismatch=
41 41/134 Problem with Grid by Grid Comparison Time to recognize a pattern - Proportional to the number of stored patterns ( Too costly with the increase of number of patterns stored ) A-Z a-z 0-9 Solution Artificial Intelligence
42 42/134 Human and Machine Perception We are often influenced by the knowledge of how patterns are modeled and recognized in nature when we develop pattern recognition algorithms. Research on machine perception also helps us gain deeper understanding and appreciation for pattern recognition systems in nature. Yet, we also apply many techniques that are purely numerical and do not have any correspondence in natural systems.
43 43/134 Pattern Recognition Two Phase : Learning and Detection. Time to learn is higher. Driving a car Difficult to learn but once learnt it becomes natural. Can use AI learning methodologies such as: Neural Network. Machine Learning.
44 44/134 Learning How can machine learn the rule from data? Supervised learning: a teacher provides a category label or cost for each pattern in the training set. Here samples from information classes (training data) are used for learning and then classifying unknown data points/patterns. Unsupervised learning: the system forms clusters or natural groupings of the input patterns. Semi-Supervised: A small training samples from information classes are used for initial learning. The model is further built using unlabeled samples for classifying unknown patterns
45 Types of Prediction Problems (1/2) Classification The PR problem of assigning an object to a class The output of the PR system is an integer label e.g. classifying a product as good or bad in a quality control test Regression A generalization of a classification task The output of the PR system is a real-valued number e.g. predicting the share value of a firm based on past performance and stock market indicators
46 Types of Prediction Problems (2/2) Clustering The problem of organizing objects into meaningful groups The system returns a (sometimes hierarchical) grouping of objects e.g. organizing life forms into a taxonomy of species Description The problem of representing an object in terms of a series of primitives The PR system produces a structural or linguistic description e.g. labeling an ECG signal in terms of P, QRS and T complexes
47 47/134 Classification vs. Clustering Classification (known categories) Clustering (creation of new categories) Category A Category B Classification (Supervised Classification) Clustering (Unsupervised Classification)
48 48/134
49 49/134 Basic concepts (Classification) y Hidden state y Y - Cannot be directly measured. x1 Feature vector x2 = x - x is a point in feature space X. x n - Patterns with equal hidden state belong to the same class. Task Pattern - To design a classifer (decision rule) x X - A vector of observations (measurements). d : X Y which decides about a hidden state based on an onbservation.
50 50/134 Dimension The dimension of a space or object is informally defined as the minimum number of coordinates needed to specify any point within it, e.g., A line has a dimension of one because only one coordinate is needed to specify a point on it (for example, the point at 5 on a number line). A surface such as a plane or the surface of a cylinder or sphere has a dimension of two because two coordinates are needed to specify a point on it (for example, to locate a point on the surface of a sphere you need both its latitude and its longitude). The inside of a cube, a cylinder or a sphere is three-dimensional because three coordinates are needed to locate a point within these spaces. From left to right, the square, the cube, and the tesseract. The square is bounded by 1-dimensional lines, the cube by 2-dimensional areas, and the tesseract by 3-dimensional volumes.
51 51/134 Feature extraction Task: to extract features which are good for classification. Good features: Objects from the same class have similar feature values. Objects from different classes have different values. Good features Bad features
52 52/134 Dimensionality Reduction
53 53/134 Dimensionality Reduction A limited yet salient feature set simplifies both pattern representation and classifier design. Pattern representation is easy for 2D and 3D features. How to make pattern with high dimensional features viewable?
54 54/134 Dimensionality Reduction How? Feature Extraction Create new features based on the original feature set Transforms are usually involved Feature Selection Select the best subset from a given feature set.
55 55/134 Main Issues in Dimensionality Reduction The choice of a criterion function Commonly used criterion: classification error The determination of the appropriate dimensionality Correlated with the intrinsic dimensionality of data
56 56/134 Dimensionality Reduction Feature Extraction
57 57/134 Feature Extractor x i Feature Extractor y i ( x, x 2,, x ) T ( y, y, y ) T i1 i id i i2, im m d, usually 1
58 58/134 Some Important Methods Principal Component Analysis (PCA) or Karhunen-Loeve Expansion Project Pursuit Independent Component Analysis (ICA) Factor Analysis Discriminate Analysis Kernel PCA Multidimensional Scaling (MDS) Feed-Forward Neural Networks Self-Organizing Map Linear Approaches Nonlinear Approaches Neural Networks
59 59/134 Dimensionality Reduction Feature Selection
60 60/134 Feature Selector d # possible Selections m x x Feature Selector ( x, x, x ) T 2, ( x, x,, x ) T 1 d 1 ' 2' m ' m d, usually
61 61/134 The problem Given a set of d features, select a subset of size m that leads to the smallest classification error. No nonexhaustive sequential feature selection procedure can be guaranteed to produce the optimal subset.
62 62/134 Optimal Methods Exhaustive Search Evaluate all possible subsets Branch-and-Bound The monotonicity property of the criterion function has to be held.
63 63/134 Suboptimal Methods Best Individual Features Sequential Forward Selection (SFS) Sequential Backward Selection (SBS) Plus l-take away r Selection Sequential Forward Floating Search and Sequential Backward Floating Search
64 64/134 m k m m 2 1 x n x x 2 1 φ 1 φ 2 φ n m k m m m x n x x 2 1 Feature extraction Feature selection Feature extraction methods
65 65/134 Example: 1 height Task: jockey-hoopster recognition. weight x x 1 2 = x The set of hidden state is The feature space is X Y = { H, J} 2 = R Linear classifier: Training examples x, y ),,( x l, y )} x 2 {( 1 1 l y = H d( x) = H J if if ( w x) + b ( w x) + b < 0 0 y = J ( w x) + b = 0 x 1
66 66/134 Classifier A classifier partitions feature space X into class-labeled regions such that X = X X X and X X X {0} 1 2 Y 1 2 Y = X 1 X 3 X 1 X 1 X 2 X 2 X 3 The classification consists of determining to which region a feature vector x belongs to. Borders between decision boundaries are called decision regions.
67 67/134 Representation of classifier A classifier is typically represented as a set of discriminant functions f i ( x) : X R, i = 1,, Y i > j j i The classifier assigns a feature vector x to the i-th class if f ( x) f ( x) f 1 ( x) x Feature vector f 2 ( x) max y Class identifier f Y ( x) Discriminant function
68 68/134 Pattern Discrimination Revisited Decision Regions and Functions The feature space is x = [ x, x ] R 1 Linear decision function : d( x) = w x1 + w2 x2 + w = 0 Linear classifier: d( x) = o x if if ( w x) + b ( w x) + b < 0 0
69 69/134 Case Study Fish Classification: Sea Bass / Salmon. Salmon Problem: Sorting incoming fish on a conveyor belt according to species. Sea-bass
70 70/134 Case Study (Cont.) What can cause problems during sensing? Lighting conditions. Position of fish on the conveyor belt. Camera noise. etc What are the steps in the process? 1. Capture image. 2. Isolate fish 3. Take measurements 4. Make decision
71 71/134 Case Study (Cont.) Pre-processing Feature Extraction Classification Sea Bass Salmon
72 72/134 Case Study (Cont.) Pre-Processing: Image enhancement Separating touching or occluding fish. Finding the boundary of the fish.
73 73/134 Case Study (Cont.) How to separate sea bass from salmon? Possible features to be used: Length Lightness Width Number and shape of fins Position of the mouth Etc Assume a fisherman told us that a sea bass is generally longer than a salmon. Even though sea bass is longer than salmon on the average, there are many examples of fish where this observation does not hold.
74 74/134 Feature Selection: Good/Bad Good features Bad features
75 75/134 Case Study (Cont.) How to separate sea bass from salmon? To improve recognition, we might have to use more than one feature at a time. Single features might not yield the best performance. Combinations of features might yield better performance. x x 1 2 x x 1 2 : lightness : width
76 76/134 Features and Distributions The length/width is a poor feature alone! Select the lightness as a possible feature.
77 77/134 Decision/classification Boundaries
78 78/134 Decision/classification Boundaries
79 79/134 Decision/classification Boundaries We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such noisy features Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure (next slide):
80 80/134 Decision/classification Boundaries
81 81/134 Decision/classification Boundaries However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input Issue of generalization!
82 82/134 Decision/classification Boundaries
83 83/134 Occam s Razor Entities are not to be multiplied without necessity William of Occam ( )
84 84/134 Mapping Data to a New Space Fourier transform Wavelet transform Two Sine Waves Two Sine Waves + Noise Frequency April 16,
85 85/134 What Is Wavelet Transform? Decomposes a signal into different frequency subbands Applicable to n-dimensional signals Data are transformed to preserve relative distance between objects at different levels of resolution Allow natural clusters to become more distinguishable Used for image compression April 16,
86 86/134 What Is Wavelet Transform? Discrete wavelet transform (DWT) for linear signal processing, multiresolution analysis Compressed approximation: store only a small fraction of the strongest of the wavelet coefficients Similar to discrete Fourier transform (DFT), but better lossy compression, localized in space Method: Length, L, must be an integer power of 2 (padding with 0 s, when necessary) Each transform has 2 functions: smoothing, difference Applies to pairs of data, resulting in two set of data of length L/2 Applies two functions recursively, until reaches the desired length April 16,
87 87/134 What Decomposition Wavelets: A math tool for space-efficient hierarchical decomposition of functions S = [2, 2, 0, 2, 3, 5, 4, 4] can be transformed to S^ = [2 3 / 4, -1 1 / 4, 1 / 2, 0, 0, -1, -1, 0] Compression: many small detail coefficients can be replaced by 0 s, and only the significant coefficients are retained April 16,
88 88/134 Why Wavelet Transform? Use hat-shape filters Emphasize region where points cluster Suppress weaker information in their boundaries Effective removal of outliers Insensitive to noise, insensitive to input order Multi-resolution Detect arbitrary shaped clusters at different scales Efficient Complexity O(N) April 16,
89 89/134 Principal Component Analysis (PCA) Find a projection that captures the largest amount of variation in data The original data are projected onto a much smaller space, resulting in dimensionality reduction. We find the eigenvectors of the covariance matrix, and these eigenvectors define the new space x 2 e x 1 April 16,
90 90/134 Principal Component Analysis (Steps) Given N data vectors from n-dimensions, find k n orthogonal vectors (principal components) that can be best used to represent data Normalize input data: Each attribute falls within the same range Compute k orthonormal (unit) vectors, i.e., principal components Each input data (vector) is a linear combination of the k principal component vectors The principal components are sorted in order of decreasing significance or strength Since the components are sorted, the size of the data can be reduced by eliminating the weak components, i.e., those with low variance (i.e., using the strongest principal components, it is possible to reconstruct a good approximation of the original data) Works for numeric data only April 16,
91 91/134 Feature Subset Selection: revisited Another way to reduce dimensionality of data Redundant attributes Duplicate much or all of the information contained in one or more other attributes E.g., purchase price of a product and the amount of sales tax paid Irrelevant attributes Contain no information that is useful for the data mining task at hand E.g., students' ID is often irrelevant to the task of predicting students' GPA April 16,
92 92/134 Feature Subset Selection Techniques: Brute-force approch: Try all possible feature subsets as input to data mining algorithm Embedded approaches: Feature selection occurs naturally as part of the data mining algorithm Filter approaches: Features are selected before data mining algorithm is run Wrapper approaches: Use the data mining algorithm as a black box to find best subset of attributes
93 93/134 Example: Classification and Clustering
94 94/134 Validation of models
95 95/134 Semisupervised and Co-Training
96 96/134 Multiple Classification Systems
97 97/134 Granular Computing
98 98/134 GrC practical
99 99/134 Pattern Recognition System (PRS) Measurement Feature Decision Space Space Space Uncertainties arise from deficiencies of information available from a situation Deficiencies may result from incomplete, imprecise, illdefined, not fully reliable, vague, contradictory information in various stages of a PRS
CS 521 Data Mining Techniques Instructor: Abdullah Mueen
CS 521 Data Mining Techniques Instructor: Abdullah Mueen LECTURE 2: DATA TRANSFORMATION AND DIMENSIONALITY REDUCTION Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major Tasks
More information2. Basic Task of Pattern Classification
2. Basic Task of Pattern Classification Definition of the Task Informal Definition: Telling things apart 3 Definition: http://www.webopedia.com/term/p/pattern_recognition.html pattern recognition Last
More informationData Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining
Data Mining: Data Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach, Kumar Data Preprocessing Aggregation Sampling Dimensionality Reduction Feature subset selection Feature creation
More informationData Preprocessing. Data Preprocessing
Data Preprocessing Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville ranka@cise.ufl.edu Data Preprocessing What preprocessing step can or should
More informationUniversity of Florida CISE department Gator Engineering. Data Preprocessing. Dr. Sanjay Ranka
Data Preprocessing Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville ranka@cise.ufl.edu Data Preprocessing What preprocessing step can or should
More informationCS 584 Data Mining. Classification 1
CS 584 Data Mining Classification 1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class. Find a model for
More informationUNIT 2 Data Preprocessing
UNIT 2 Data Preprocessing Lecture Topic ********************************************** Lecture 13 Why preprocess the data? Lecture 14 Lecture 15 Lecture 16 Lecture 17 Data cleaning Data integration and
More informationData Preprocessing. Data Mining 1
Data Preprocessing Today s real-world databases are highly susceptible to noisy, missing, and inconsistent data due to their typically huge size and their likely origin from multiple, heterogenous sources.
More informationApplying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
More informationData Preprocessing. Why Data Preprocessing? MIT-652 Data Mining Applications. Chapter 3: Data Preprocessing. Multi-Dimensional Measure of Data Quality
Why Data Preprocessing? Data in the real world is dirty incomplete: lacking attribute values, lacking certain attributes of interest, or containing only aggregate data e.g., occupation = noisy: containing
More informationCS570: Introduction to Data Mining
CS570: Introduction to Data Mining Fall 2013 Reading: Chapter 3 Han, Chapter 2 Tan Anca Doloc-Mihu, Ph.D. Some slides courtesy of Li Xiong, Ph.D. and 2011 Han, Kamber & Pei. Data Mining. Morgan Kaufmann.
More information3. Data Preprocessing. 3.1 Introduction
3. Data Preprocessing Contents of this Chapter 3.1 Introduction 3.2 Data cleaning 3.3 Data integration 3.4 Data transformation 3.5 Data reduction SFU, CMPT 740, 03-3, Martin Ester 84 3.1 Introduction Motivation
More informationFeature Selection. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Feature Selection CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Dimensionality reduction Feature selection vs. feature extraction Filter univariate
More information2. Data Preprocessing
2. Data Preprocessing Contents of this Chapter 2.1 Introduction 2.2 Data cleaning 2.3 Data integration 2.4 Data transformation 2.5 Data reduction Reference: [Han and Kamber 2006, Chapter 2] SFU, CMPT 459
More informationData Mining Classification: Bayesian Decision Theory
Data Mining Classification: Bayesian Decision Theory Lecture Notes for Chapter 2 R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification, 2nd ed. New York: Wiley, 2001. Lecture Notes for Chapter
More information3 Feature Selection & Feature Extraction
3 Feature Selection & Feature Extraction Overview: 3.1 Introduction 3.2 Feature Extraction 3.3 Feature Selection 3.3.1 Max-Dependency, Max-Relevance, Min-Redundancy 3.3.2 Relevance Filter 3.3.3 Redundancy
More informationBy Mahesh R. Sanghavi Associate professor, SNJB s KBJ CoE, Chandwad
By Mahesh R. Sanghavi Associate professor, SNJB s KBJ CoE, Chandwad Data Analytics life cycle Discovery Data preparation Preprocessing requirements data cleaning, data integration, data reduction, data
More informationMachine Learning Feature Creation and Selection
Machine Learning Feature Creation and Selection Jeff Howbert Introduction to Machine Learning Winter 2012 1 Feature creation Well-conceived new features can sometimes capture the important information
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationSegmentation of Images
Segmentation of Images SEGMENTATION If an image has been preprocessed appropriately to remove noise and artifacts, segmentation is often the key step in interpreting the image. Image segmentation is a
More informationBiometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong)
Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) References: [1] http://homepages.inf.ed.ac.uk/rbf/hipr2/index.htm [2] http://www.cs.wisc.edu/~dyer/cs540/notes/vision.html
More informationDI TRANSFORM. The regressive analyses. identify relationships
July 2, 2015 DI TRANSFORM MVstats TM Algorithm Overview Summary The DI Transform Multivariate Statistics (MVstats TM ) package includes five algorithm options that operate on most types of geologic, geophysical,
More informationFeature Selection for Image Retrieval and Object Recognition
Feature Selection for Image Retrieval and Object Recognition Nuno Vasconcelos et al. Statistical Visual Computing Lab ECE, UCSD Presented by Dashan Gao Scalable Discriminant Feature Selection for Image
More informationPATTERN RECOGNITION USING NEURAL NETWORKS
PATTERN RECOGNITION USING NEURAL NETWORKS Santaji Ghorpade 1, Jayshree Ghorpade 2 and Shamla Mantri 3 1 Department of Information Technology Engineering, Pune University, India santaji_11jan@yahoo.co.in,
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationK236: Basis of Data Science
Schedule of K236 K236: Basis of Data Science Lecture 6: Data Preprocessing Lecturer: Tu Bao Ho and Hieu Chi Dam TA: Moharasan Gandhimathi and Nuttapong Sanglerdsinlapachai 1. Introduction to data science
More informationData Preprocessing. Slides by: Shree Jaswal
Data Preprocessing Slides by: Shree Jaswal Topics to be covered Why Preprocessing? Data Cleaning; Data Integration; Data Reduction: Attribute subset selection, Histograms, Clustering and Sampling; Data
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationData Mining: Data. What is Data? Lecture Notes for Chapter 2. Introduction to Data Mining. Properties of Attribute Values. Types of Attributes
0 Data Mining: Data What is Data? Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach, Kumar Collection of data objects and their attributes An attribute is a property or characteristic
More informationData Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining
10 Data Mining: Data Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 What is Data? Collection of data objects
More informationTable Of Contents: xix Foreword to Second Edition
Data Mining : Concepts and Techniques Table Of Contents: Foreword xix Foreword to Second Edition xxi Preface xxiii Acknowledgments xxxi About the Authors xxxv Chapter 1 Introduction 1 (38) 1.1 Why Data
More informationLinear Methods for Regression and Shrinkage Methods
Linear Methods for Regression and Shrinkage Methods Reference: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, J. Friedman, Springer 1 Linear Regression Models Least Squares Input vectors
More informationName of the lecturer Doç. Dr. Selma Ayşe ÖZEL
Y.L. CENG-541 Information Retrieval Systems MASTER Doç. Dr. Selma Ayşe ÖZEL Information retrieval strategies: vector space model, probabilistic retrieval, language models, inference networks, extended
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationMachine Learning Techniques for Data Mining
Machine Learning Techniques for Data Mining Eibe Frank University of Waikato New Zealand 10/25/2000 1 PART VII Moving on: Engineering the input and output 10/25/2000 2 Applying a learner is not all Already
More informationData Preprocessing Yudho Giri Sucahyo y, Ph.D , CISA
Obj ti Objectives Motivation: Why preprocess the Data? Data Preprocessing Techniques Data Cleaning Data Integration and Transformation Data Reduction Data Preprocessing Lecture 3/DMBI/IKI83403T/MTI/UI
More informationData Mining. Ryan Benton Center for Advanced Computer Studies University of Louisiana at Lafayette Lafayette, La., USA.
Data Mining Ryan Benton Center for Advanced Computer Studies University of Louisiana at Lafayette Lafayette, La., USA January 13, 2011 Important Note! This presentation was obtained from Dr. Vijay Raghavan
More informationSGN (4 cr) Chapter 10
SGN-41006 (4 cr) Chapter 10 Feature Selection and Extraction Jussi Tohka & Jari Niemi Department of Signal Processing Tampere University of Technology February 18, 2014 J. Tohka & J. Niemi (TUT-SGN) SGN-41006
More informationSlides for Data Mining by I. H. Witten and E. Frank
Slides for Data Mining by I. H. Witten and E. Frank 7 Engineering the input and output Attribute selection Scheme-independent, scheme-specific Attribute discretization Unsupervised, supervised, error-
More informationData Exploration and Preparation Data Mining and Text Mining (UIC Politecnico di Milano)
Data Exploration and Preparation Data Mining and Text Mining (UIC 583 @ Politecnico di Milano) References Jiawei Han and Micheline Kamber, "Data Mining, : Concepts and Techniques", The Morgan Kaufmann
More informationAnnouncements. Recognition. Recognition. Recognition. Recognition. Homework 3 is due May 18, 11:59 PM Reading: Computer Vision I CSE 152 Lecture 14
Announcements Computer Vision I CSE 152 Lecture 14 Homework 3 is due May 18, 11:59 PM Reading: Chapter 15: Learning to Classify Chapter 16: Classifying Images Chapter 17: Detecting Objects in Images Given
More informationImage Analysis, Classification and Change Detection in Remote Sensing
Image Analysis, Classification and Change Detection in Remote Sensing WITH ALGORITHMS FOR ENVI/IDL Morton J. Canty Taylor &. Francis Taylor & Francis Group Boca Raton London New York CRC is an imprint
More information3D Models and Matching
3D Models and Matching representations for 3D object models particular matching techniques alignment-based systems appearance-based systems GC model of a screwdriver 1 3D Models Many different representations
More informationData Mining. Part 2. Data Understanding and Preparation. 2.4 Data Transformation. Spring Instructor: Dr. Masoud Yaghini. Data Transformation
Data Mining Part 2. Data Understanding and Preparation 2.4 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Normalization Attribute Construction Aggregation Attribute Subset Selection Discretization
More informationData Mining. Jeff M. Phillips. January 7, 2019 CS 5140 / CS 6140
Data Mining CS 5140 / CS 6140 Jeff M. Phillips January 7, 2019 What is Data Mining? What is Data Mining? Finding structure in data? Machine learning on large data? Unsupervised learning? Large scale computational
More informationData Preprocessing. Komate AMPHAWAN
Data Preprocessing Komate AMPHAWAN 1 Data cleaning (data cleansing) Attempt to fill in missing values, smooth out noise while identifying outliers, and correct inconsistencies in the data. 2 Missing value
More informationFacial Expression Recognition Using Non-negative Matrix Factorization
Facial Expression Recognition Using Non-negative Matrix Factorization Symeon Nikitidis, Anastasios Tefas and Ioannis Pitas Artificial Intelligence & Information Analysis Lab Department of Informatics Aristotle,
More informationAnalytical model A structure and process for analyzing a dataset. For example, a decision tree is a model for the classification of a dataset.
Glossary of data mining terms: Accuracy Accuracy is an important factor in assessing the success of data mining. When applied to data, accuracy refers to the rate of correct values in the data. When applied
More informationRecognition: Face Recognition. Linda Shapiro EE/CSE 576
Recognition: Face Recognition Linda Shapiro EE/CSE 576 1 Face recognition: once you ve detected and cropped a face, try to recognize it Detection Recognition Sally 2 Face recognition: overview Typical
More informationMachine Learning (CSMML16) (Autumn term, ) Xia Hong
Machine Learning (CSMML16) (Autumn term, 28-29) Xia Hong 1 Useful books: 1. C. M. Bishop: Pattern Recognition and Machine Learning (2007) Springer. 2. S. Haykin: Neural Networks (1999) Prentice Hall. 3.
More informationContents. Foreword to Second Edition. Acknowledgments About the Authors
Contents Foreword xix Foreword to Second Edition xxi Preface xxiii Acknowledgments About the Authors xxxi xxxv Chapter 1 Introduction 1 1.1 Why Data Mining? 1 1.1.1 Moving toward the Information Age 1
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationData Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Exploratory data analysis tasks Examine the data, in search of structures
More informationWhat is machine learning?
Machine learning, pattern recognition and statistical data modelling Lecture 12. The last lecture Coryn Bailer-Jones 1 What is machine learning? Data description and interpretation finding simpler relationship
More informationInternational Journal of Scientific Research & Engineering Trends Volume 4, Issue 6, Nov-Dec-2018, ISSN (Online): X
Analysis about Classification Techniques on Categorical Data in Data Mining Assistant Professor P. Meena Department of Computer Science Adhiyaman Arts and Science College for Women Uthangarai, Krishnagiri,
More informationSupervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationCHAPTER 4 WAVELET TRANSFORM-GENETIC ALGORITHM DENOISING TECHNIQUE
102 CHAPTER 4 WAVELET TRANSFORM-GENETIC ALGORITHM DENOISING TECHNIQUE 4.1 INTRODUCTION This chapter introduces an effective combination of genetic algorithm and wavelet transform scheme for the denoising
More informationCSE 158. Web Mining and Recommender Systems. Midterm recap
CSE 158 Web Mining and Recommender Systems Midterm recap Midterm on Wednesday! 5:10 pm 6:10 pm Closed book but I ll provide a similar level of basic info as in the last page of previous midterms CSE 158
More informationData Preprocessing. S1 Teknik Informatika Fakultas Teknologi Informasi Universitas Kristen Maranatha
Data Preprocessing S1 Teknik Informatika Fakultas Teknologi Informasi Universitas Kristen Maranatha 1 Why Data Preprocessing? Data in the real world is dirty incomplete: lacking attribute values, lacking
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationFACE RECOGNITION USING INDEPENDENT COMPONENT
Chapter 5 FACE RECOGNITION USING INDEPENDENT COMPONENT ANALYSIS OF GABORJET (GABORJET-ICA) 5.1 INTRODUCTION PCA is probably the most widely used subspace projection technique for face recognition. A major
More informationRapid growth of massive datasets
Overview Rapid growth of massive datasets E.g., Online activity, Science, Sensor networks Data Distributed Clusters are Pervasive Data Distributed Computing Mature Methods for Common Problems e.g., classification,
More informationCSIS Introduction to Pattern Classification. CSIS Introduction to Pattern Classification 1
CSIS8502 1. Introduction to Pattern Classification CSIS8502 1. Introduction to Pattern Classification 1 Introduction to Pattern Classification What is Pattern Classification? 1 Input: Pattern description
More informationCHAPTER 3 DIFFERENT DOMAINS OF WATERMARKING. domain. In spatial domain the watermark bits directly added to the pixels of the cover
38 CHAPTER 3 DIFFERENT DOMAINS OF WATERMARKING Digital image watermarking can be done in both spatial domain and transform domain. In spatial domain the watermark bits directly added to the pixels of the
More informationUnsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
More informationDimension reduction : PCA and Clustering
Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental
More informationGene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
More informationData Mining. Data preprocessing. Hamid Beigy. Sharif University of Technology. Fall 1395
Data Mining Data preprocessing Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 15 Table of contents 1 Introduction 2 Data preprocessing
More informationEfficient Image Compression of Medical Images Using the Wavelet Transform and Fuzzy c-means Clustering on Regions of Interest.
Efficient Image Compression of Medical Images Using the Wavelet Transform and Fuzzy c-means Clustering on Regions of Interest. D.A. Karras, S.A. Karkanis and D. E. Maroulis University of Piraeus, Dept.
More informationUnsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationCPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2018
CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2018 Last Time: Multi-Dimensional Scaling Multi-dimensional scaling (MDS): Non-parametric visualization: directly optimize the z i locations.
More information10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
More informationImage Processing. Image Features
Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching
More informationData Preprocessing. Erwin M. Bakker & Stefan Manegold. https://homepages.cwi.nl/~manegold/dbdm/
Data Preprocessing Erwin M. Bakker & Stefan Manegold https://homepages.cwi.nl/~manegold/dbdm/ http://liacs.leidenuniv.nl/~bakkerem2/dbdm/ s.manegold@liacs.leidenuniv.nl e.m.bakker@liacs.leidenuniv.nl 9/26/17
More informationResearch on the New Image De-Noising Methodology Based on Neural Network and HMM-Hidden Markov Models
Research on the New Image De-Noising Methodology Based on Neural Network and HMM-Hidden Markov Models Wenzhun Huang 1, a and Xinxin Xie 1, b 1 School of Information Engineering, Xijing University, Xi an
More informationSummary of Last Chapter. Course Content. Chapter 3 Objectives. Chapter 3: Data Preprocessing. Dr. Osmar R. Zaïane. University of Alberta 4
Principles of Knowledge Discovery in Data Fall 2004 Chapter 3: Data Preprocessing Dr. Osmar R. Zaïane University of Alberta Summary of Last Chapter What is a data warehouse and what is it for? What is
More informationCOSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
More informationData Mining: Concepts and Techniques. (3 rd ed.) Chapter 3. Chapter 3: Data Preprocessing. Major Tasks in Data Preprocessing
Data Mining: Concepts and Techniques (3 rd ed.) Chapter 3 1 Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major Tasks in Data Preprocessing Data Cleaning Data Integration Data
More informationCHAPTER 8 COMPOUND CHARACTER RECOGNITION USING VARIOUS MODELS
CHAPTER 8 COMPOUND CHARACTER RECOGNITION USING VARIOUS MODELS 8.1 Introduction The recognition systems developed so far were for simple characters comprising of consonants and vowels. But there is one
More informationAn Introduction to Pattern Recognition
An Introduction to Pattern Recognition Speaker : Wei lun Chao Advisor : Prof. Jian-jiun Ding DISP Lab Graduate Institute of Communication Engineering 1 Abstract Not a new research field Wide range included
More information(Refer Slide Time 00:17) Welcome to the course on Digital Image Processing. (Refer Slide Time 00:22)
Digital Image Processing Prof. P. K. Biswas Department of Electronics and Electrical Communications Engineering Indian Institute of Technology, Kharagpur Module Number 01 Lecture Number 02 Application
More informationNominal Data. May not have a numerical representation Distance measures might not make sense. PR and ANN
NonMetric Data Nominal Data So far we consider patterns to be represented by feature vectors of real or integer values Easy to come up with a distance (similarity) measure by using a variety of mathematical
More informationIMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING
SECOND EDITION IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING ith Algorithms for ENVI/IDL Morton J. Canty с*' Q\ CRC Press Taylor &. Francis Group Boca Raton London New York CRC
More informationClustering. Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationCHAPTER 1 INTRODUCTION
CHAPTER 1 INTRODUCTION 1.1 Introduction Pattern recognition is a set of mathematical, statistical and heuristic techniques used in executing `man-like' tasks on computers. Pattern recognition plays an
More informationDiscriminate Analysis
Discriminate Analysis Outline Introduction Linear Discriminant Analysis Examples 1 Introduction What is Discriminant Analysis? Statistical technique to classify objects into mutually exclusive and exhaustive
More informationContents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
More informationINTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY
INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK UNSUPERVISED SEGMENTATION OF TEXTURE IMAGES USING A COMBINATION OF GABOR AND WAVELET
More informationCOMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
More informationCS325 Artificial Intelligence Ch. 20 Unsupervised Machine Learning
CS325 Artificial Intelligence Cengiz Spring 2013 Unsupervised Learning Missing teacher No labels, y Just input data, x What can you learn with it? Unsupervised Learning Missing teacher No labels, y Just
More informationObject Recognition. Lecture 11, April 21 st, Lexing Xie. EE4830 Digital Image Processing
Object Recognition Lecture 11, April 21 st, 2008 Lexing Xie EE4830 Digital Image Processing http://www.ee.columbia.edu/~xlx/ee4830/ 1 Announcements 2 HW#5 due today HW#6 last HW of the semester Due May
More informationCMPUT 391 Database Management Systems. Data Mining. Textbook: Chapter (without 17.10)
CMPUT 391 Database Management Systems Data Mining Textbook: Chapter 17.7-17.11 (without 17.10) University of Alberta 1 Overview Motivation KDD and Data Mining Association Rules Clustering Classification
More informationData Mining. Data preprocessing. Hamid Beigy. Sharif University of Technology. Fall 1394
Data Mining Data preprocessing Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1394 1 / 15 Table of contents 1 Introduction 2 Data preprocessing
More information10/14/2017. Dejan Sarka. Anomaly Detection. Sponsors
Dejan Sarka Anomaly Detection Sponsors About me SQL Server MVP (17 years) and MCT (20 years) 25 years working with SQL Server Authoring 16 th book Authoring many courses, articles Agenda Introduction Simple
More informationSelection of Location, Frequency and Orientation Parameters of 2D Gabor Wavelets for Face Recognition
Selection of Location, Frequency and Orientation Parameters of 2D Gabor Wavelets for Face Recognition Berk Gökberk, M.O. İrfanoğlu, Lale Akarun, and Ethem Alpaydın Boğaziçi University, Department of Computer
More informationPattern Recognition in Image Analysis
Pattern Recognition in Image Analysis Image Analysis: Et Extraction ti of fknowledge ld from image data. dt Pattern Recognition: Detection and extraction of patterns from data. Pattern: A subset of data
More informationPattern Classification Algorithms for Face Recognition
Chapter 7 Pattern Classification Algorithms for Face Recognition 7.1 Introduction The best pattern recognizers in most instances are human beings. Yet we do not completely understand how the brain recognize
More information