Pattern Recognition in Image Analysis

Transcription

1 Pattern Recognition in Image Analysis Image Analysis: Et Extraction ti of fknowledge ld from image data. dt Pattern Recognition: Detection and extraction of patterns from data. Pattern: A subset of data that may be described by some well-defined set of rules. Patterns may constitute the smallest entity in the data that represent knowledge. pixel this is a pattern PRIA01: Introduction, Klaus D. Toennies 1

2 Applications of Pattern Recognition automatic analysis of medical images automatic inspection of parts on an assembly line human speech recognition by computers classification of seismic signals (e.g., for oil and mineral exploration) selection of tax returns to audit, stocks to buy, people p to insure identification of people from fingerprints, retinal scans, handwriting,... automatic inspection of printed circuits, printed characters, handwriting recognition automatic analysis of satellite pictures (e.g., weather condition, water reserves, mineral prospects,...) Pattern recognition (in general as well as applied to images) is mainly a classification task. Images often constitute the data source for pattern recognition. PRIA01: Introduction, Klaus D. Toennies 2

3 Pattern Recognition and Objects Patterns in images represent (attributes of) objects Pattern recognition tasks are Object detection tasks to find an unknown number of instances of a known kind of an object in the image Object recognition tasks to recognise a detected object as one of a specific kind PRIA01: Introduction, Klaus D. Toennies 3

4 Detection and Recognition Dt Detectionti techniques: segmentation, object matching, searching techniques Recognition techniques: feature computation/ reduction, classification, clustering PRIA01: Introduction, Klaus D. Toennies 4

5 Detection and Recognition May be used independently of each other In combination as process chain hi Detection (and segmentation) separates object detail of objects of some broad class (e.g. cars ) Recognition separates detected objects in sub-classes (e.g. sports cars, limousines, ) Some problems need an integrated solution (e.g. separate different car types directly in an image ) Integrated t solution ismore human-lokevision ii but much more difficult Potentially every pixel contributes to the solution Features are not guaranteed to belong to an object to be classified PRIA01: Introduction, Klaus D. Toennies 5

6 Classical Solution Pattern recognition as a processing pipeline (Segmentation) Feature computation Classification PRIA01: Introduction, Klaus D. Toennies 6

7 Pattern Recognition as a Classification Task Features: (f 1, f 2,..., f n ) Objects to be classified are described by features. Features are evaluated to separate objects into different classes. Features: (f 1, f 2,..., f n ) PRIA01: Introduction, Klaus D. Toennies 7

8 Feature Detection An important prerequisite for feature detection in images is the extraction of structures that have common feature values Segmentation. Why? Often, features are not computed from single pixels but from pixel sets. Their computation is erroneous if feature values change over the set. Image Segmentation Feature Computation Classification Pattern Recognition Segmentation and Pattern Recognition: Knowledge on the features to be evaluated greatly enhances the segmentation success. PRIA01: Introduction, Klaus D. Toennies 8

9 Classification Classification: grouping patterns (samples) according to their features into different classes. How do I decide this? Decide which features are relevant to the problem and decide on a way to compute them. Decide on a general classification i technique (based on the type of features) Train a classifier based on samples of the images to be analysed: - Find out a differentiation between different meanings based on feature characteristics. - Find a suitable generalisation of the feature values based on the training set Estimate the error of the classifier using an independent, d representative test data set. Apply the classifier to images of the problem domain. PRIA01: Introduction, Klaus D. Toennies 9

10 Classification Techniques Statistical pattern recognition: Assume that the pattern is a sample of from a number of known distributions and assign it to the one class to which it most likely belongs to. Requires a feature vector as input information. Solutions are, e.g., knn-classifier, backpropagation networks, support vector machines Semantical pattern recognition: Assume that the features of a pattern follows known laws / rules for constructing the pattern and assign it to the class whose rules are most closely followed. Requires a feature structure to reflect the rules Slti Solutions are grammars (not often used din image analysis) i) and graph matching techniques PRIA01: Introduction, Klaus D. Toennies 10

11 Is Pattern Recognition Equivalent to Image Analysis? Task: Detecting windows and doors in image. Doors are rotated with respect to camera Windows are partially occluded Scene is 3-d but features are from 2-d picture. Feature-based pattern recognition from 3-d scenes: Features are those of the object and not that of its projection! Pattern recognition may require 3-d surface reconstruction prior to analysis. PRIA01: Introduction, Klaus D. Toennies 11

12 Useful and not so Useful Features This should have similar appearances! Useful features pertain to object Useless features are those pertaining to imaging the scene (e.g., highlights...) PRIA01: Introduction, Klaus D. Toennies 12

13 Finding the Features Reconstruct 3-d surfaces (= 3-d computer vision) Induce locations of light sources Synthesise influences from light sources, surface curvature, camera position Extract object features after accounting for synthesised influences This is much too complicated in real-world applications! Solutions: looking at 2-d scenes only for feature-based pattern recognition using reconstructed 3-d geometric features (fitting methods) PRIA01: Introduction, Klaus D. Toennies 13

14 2-d Scenes and Almost 2-d Scenes Medical images such as CT, MRI, etc. satellite images and aerial photos drawings and text t images some images from computer aided manufacturing PRIA01: Introduction, Klaus D. Toennies 14

15 What, if it is truly 3-d? These cars belong to the same class while this one does not PRIA01: Introduction, Klaus D. Toennies 15

16 The not-so-classical way: Fitting a Model Generate models of the class and decide on quality of fit. Model: geometrical, 3-d model: fit edges to model edges. Picture 2-d model (set of 2-d pictures): fit to interpolation from pictures. Why? Human Vision: There must be some fast classification that may be later refined. PRIA01: Introduction, Klaus D. Toennies 16

17 3-d Model Fitting Rough classification of an object 3-d model may aid to segmentation in 3-d being pre-requisite to PR techniques 2-d model may allow for segmentation free classification Problem: scene may contain more than just the object (i.e., segmentation before segmentation) and in may be distorted by artefacts / noise. Find the important features in an image (saliency of features) PRIA01: Introduction, Klaus D. Toennies 17

18 Fitting 2-d to 2-d Top-Down-Approach Alternative to segmentation-tofeatures-to-object Does not requires segmentation Requires appropriate model Capturing the essence of members of an object class Capturing acceptable variability Simple road model PRIA01: Introduction, Klaus D. Toennies 18

19 Top-Down vs. Bottom-Up Bottom-Up = Classical Pattern Recognition Generation of (potentially) ti meaningful features Conceptual distance between features and semantics is large (shape features, texture features) Requires elaborated classification techniques but feature generation is (largely) domain independent Top-Down = Model Fitting Generation of a meaning model of object s appearance in an image Conceptual distance between model dland semantics is small Simple classification in low-dimensional feature space but domain-dependent PRIA01: Introduction, Klaus D. Toennies 19

20 What this course will be about... Segmentation Review segmentation techniques, deformable templates, texture segmentation Features Feature detection, feature representation, feature reduction Pattern Recognition Techniques statistical, semantic, and neural network pattern recognition... combined with a project on texture segmentation Aim of the course is to give an overview on techniques and applications in pattern recognition as applied to image analysis. PRIA01: Introduction, Klaus D. Toennies 20

21 Background / Literature Requirements Basic knowledge of image processing (what is a filter, a gradient, what is segmentation, etc.) Literature R.O.Duda, P.E.Hart, D.G.Stork, Pattern Classification, 2nd edition, 2001, Wiley and Sons. E. Gose, R. Johnsbaugh, S. Jost, Pattern Recognition and Image Analysis, Prentice Hall, 1996 R.Jain, R.Kasturi, B.G.Schunck, Machine Vision, McGraw-Hill, 1995 ERD E.R.Davies, Machine Vision i Theory, Algorithms, Practicalities, 3nd edition, Academic Press, Powerpoint slides of the course (on the net) PRIA01: Introduction, Klaus D. Toennies 21