Image Processing (IP)

Size: px

Start display at page:

Download "Image Processing (IP)"

Eugene Curtis
6 years ago
Views:

Image Processing Pattern Recognition Computer Vision Xiaojun Qi Utah State University Image

Applications: The applications applied to almost every area of human activities Biological

Medical Diagnostic Imaging, Photography, Astronomy, Image Database Retrieval and etc. 1.

tissue, and cell analysis (counting and classification) Analysis, classification, and matching

Defense/Intelligence Automatic interpretation of earth satellite imagery Recognize and track

Document Processing Scanning, archiving, and transmission of documents Automatic detection and

Factory Automation Visual inspection and assembly Industrial Inspection 5.

1 Image Processing Pattern Recognition Computer Vision Xiaojun Qi Utah State University Image Processing (IP) Manipulate and analyze digital images (pictorial information) by computer. Applications: The applications applied to almost every area of human activities Biological Research, Defense/Intelligence, Document Processing, Factory Automation, Law Enforcement, Medical Diagnostic Imaging, Photography, Astronomy, Image Database Retrieval and etc. 1. Biological Research Automatic analysis of a biological example (specimen analysis) Bone, tissue, and cell analysis (counting and classification) Analysis, classification, and matching of DNA material 2. Defense/Intelligence Automatic interpretation of earth satellite imagery Recognize and track targets in real time Security and surveillance 3. Document Processing Scanning, archiving, and transmission of documents Automatic detection and recognition of printed characters 4. Factory Automation Visual inspection and assembly Industrial Inspection 5. Law Enforcement Fingerprint feature extraction, classification, and identification DNA Matching 6. Medical Diagnostic Imaging Digital Angiography Skin Cancer Detection Computed Tomography Brain Tumor Mammography (Breast Cancer) 7. Photography Add/Subtract objects to and from a scene Special effects (Morphing, Warping)

8. Astronomy Separating stars from galaxies Galaxy classification 9.

produced by an image database system Pattern Recognition Classify what inside of the image Applications: Speech Recognition/Speaker Identification Fingerprint/Face Identification

Focus on view analysis using techniques from IP, PR and artificial intelligence (AI).

2 8. Astronomy Separating stars from galaxies Galaxy classification 9. Image Database Retrieval Shape Retrieval Color Retrieval Texture Retrieval Content-based Image Retrieval Image query by example: Query Image (left), and two most similar images produced by an image database system Pattern Recognition Classify what inside of the image Applications: Speech Recognition/Speaker Identification Fingerprint/Face Identification Signature Verification Character Recognition Biomedical: DNA Sequence Identification Remote Sensing Meteorology Industrial Inspection Robot Vision Linear Classifier Computer Vision Focus on view analysis using techniques from IP, PR and artificial intelligence (AI). It is the area of AI concerned with modeling and replicating human vision using computer software and hardware. Applications: Robotics Traffic Monitoring Face Identification 3D Modeling in Medical Imaging Current Research -- Content-based Image Retrieval and Annotation System The driving forces Internet Storage devices Computing power Two approaches Text-based approach Content-based approach

Text-Based Approach Input keywords descriptions Text-Based Approach Index images using keywords Advantages: (Google, Lycos, etc.

a large DB Manual annotation is not accurate A picture is worth a thousand words Surrounding text may not describe the image How to describe this image?

and shape information, of images are extracted automatically Similarities of images are based on the distances between features Diagram for content-based retrieval

3 Text-Based Approach Input keywords descriptions Text-Based Approach Index images using keywords Advantages: (Google, Lycos, etc.) Easy to implement Fast retrieval Web image search (surrounding text) Disadvantages: Manual annotation is not always available Manual annotation is impossible for a large DB Manual annotation is not accurate A picture is worth a thousand words Surrounding text may not describe the image How to describe this image? Content-Based Approach Index images using low-level features Content-Based Approach Index images using images Advantages Visual features, such as color, texture, and shape information, of images are extracted automatically Similarities of images are based on the distances between features Diagram for content-based retrieval system user Query Formation Image Database Visual Content Description Visual Content Description Feature Vectors Feature Database Relevance Feedback Similarity Comparison Indexing & Retrieval output Retrieval results

A Data Flow Diagram CBIR is a highly interdisciplinary research area CBIR Applications Commerce (fashion, catalogue, ) Biomedicine (X-ray, CT, ) Crime prevention (security filtering, ) Cultural (art

Sunset, mountain, dogs, Semantic gap Discrepancy between low-level features and highlevel concepts High feature similarity may not always correspond to semantic similarity Different users at

4 A Data Flow Diagram CBIR is a highly interdisciplinary research area CBIR Applications Commerce (fashion, catalogue, ) Biomedicine (X-ray, CT, ) Crime prevention (security filtering, ) Cultural (art galleries, museums, ) Military (radar, aerial, ) Entertainment (personal album, ) Open Problems Nature of digital images: arrays of numbers Descriptions of images: high-level concepts. Sunset, mountain, dogs, Semantic gap Discrepancy between low-level features and highlevel concepts High feature similarity may not always correspond to semantic similarity Different users at different time may give different interpretations for the same image. Image Categorization -- High-Level Concepts What is image categorization To label images into one or several predefined categories (e.g., Dinosaur, Elephant, Horse, Bus, Building, etc.) To map low-level visual features to high level semantics. Challenges faced by automatic image categorization Various imaging condition. Complex and hard-to-describe objects. Highly textured background. Occlusions. Common Techniques for Categorization General used techniques Statistics Support Vector Machines (SVMs) Neural Networks Multiple-Instance Learning (MIL) Our Approach: Expand SVMs to multi- Category SVMs

5 Our Categorization Approach -- Feature Extraction Only global features are used to avoid the problems of inaccurate image segmentation Features include global color histogram and edge histogram HSV color space is used for the color histogram, which is one of the MPEG-7 color descriptors. MPEG-7 also defines the edge histogram descriptor (EHD), which captures the edge distribution in 16 non-overlapping sub-images. Our Categorization Approach -- Feature Extraction Based on the original EHD, we construct global EHD (gehd) and semi-global EHD (sehd). gehd represent the edge distribution of the whole image. sehd can be constructed as follows: R 1 R 2 R 3 C 1 C 2 C C 4 R 4 4 Our Categorization Approach -- Multi-Category SVMs Radial Basis Function kernel is used 3-fold cross-validation and grid-search algorithm are used to decide the parameters C and γ. Pairwise coupling approach is used to handle the multiple category case. The output of the SVMs is also mapped to the probability so we can assign confidence to each labeled keywords. Average Accuracy Our Categorization Results Prop.1 Prop.2 ALIP HistSVM Category ID Our Categorization Results Horse: 96% Food: 1% Building: 92% Beach: 3% Our system can classify images by a set of confidence values for each automatically labeled keywords. Our Retrieval Results -- using both global and regional features (7) Beach: 47% Mountain: 40% Vehicle: 31% Building: 25% (A) 10 matches out of 11, 18 matches out of 20

6 Our Retrieval Results Our Retrieval Results (5) (3) (B) 10 matches out of 11, 17 matches out of (C) 9 matches out of 11, 14 matches out of 20 Our Retrieval Results Our Retrieval Results (6) Average Precision Prop. NFECR HisC (D) 10 matches out of 11, 19 matches out of Category ID Average retrieval precision for 20 returned images Our Retrieval Results Image Semantics Average Precision 0.75 Prop. UFM 0.70 NFECR 0.65 HisC Number of Returned Images Image semantics may be related to objects in the image Semantically similar images may contain semantically similar objects Can a computer program learn semantic concepts about images based on objects? Average retrieval precision for different number of returned images

Our Image Segmentation Approach Sample Segmentation Results Original Image 2 Regions 3 Regions 4 Regions 5 Regions 6 Regions 7 Regions Original Image 2 Regions

Find similar objects (feature vectors) among positive images At the same time, they should be as distinct from all objects in negative images as possible

Example Data Mining Three conceptual feature vectors Water, Rock, Trees.

7 Our Image Segmentation Approach Sample Segmentation Results Original Image 2 Regions 3 Regions 4 Regions 5 Regions 6 Regions 7 Regions Original Image 2 Regions 3 Regions 4 Regions Learning Semantically similar images may contain semantically similar objects. Find similar objects (feature vectors) among positive images At the same time, they should be as distinct from all objects in negative images as possible Conceptual feature vector: Multiple-instance Learning (MIL) using diverse density Learn which region represent the semantic meaning! Example Data Mining Three conceptual feature vectors Water, Rock, Trees. Rule description of a semantic concept If one of the regions is similar to water AND one of the regions is similar to rock, then it is a waterfall image, OR If one of the regions is similar to water AND one of the regions is similar to trees, then it is a waterfall image. Current Research -- Shape Representation and Matching What is What?

Shape Representation Shape representation methods: Region based Boundary based Shape descriptors: Fourier descriptors Moments Chain codes Etc.

similarity score using local signature Use a fuzzy method to combine the scores.

images with cluttered background and to return the image location and extent of each face if present Significance: The most important first step of face identification series.

8 Shape Representation Shape representation methods: Region based Boundary based Shape descriptors: Fourier descriptors Moments Chain codes Etc. Our Shape Representation and Matching Approach Shape indexing: global signature construction local signature construction Shape retrieval: calculate similarity score using global signature calculate similarity score using local signature Use a fuzzy method to combine the scores. Retrieval results are those with higher scores Our Shape Retrieval Results Current Research -- Face Detection Face detection: To determine whether or not there are any faces in the arbitrary still images with cluttered background and to return the image location and extent of each face if present Significance: The most important first step of face identification series. It is the preprocessing of face recognition, face tracking, etc. Our Approach 1. Apply color quantization and segmentation to preprocess the original image. 2. Apply a skin model to find possible skin regions. 3. Apply morphological processing to remove noise. 4. Merge skin regions if needed. 5. Apply some constraints to eliminate non-faces. 6. Apply wavelet packet to extract features. 7. Apply the neural networks to classify face and non-face. 8. Solve overlapping areas. Our Results

9 Current Research -- Digital Watermarking and steganography Watermarks: Secret messages used for protecting copyrights of digital multimedia data (images, audio, and video) Content and/or authentication For detecting unauthorized copies of images Our Watermarking Results -- Wavelet-based Approach Characteristics: Imperceptible, security, robustness, and blindness. Common Techniques Used: Spatial Domain Approach, Frequency Domain Approach, and Hybrid Approach. Watermark Our Watermarking Results -- Content-based Approach Stegnography Steganography: A way of hiding a classified message. Cover image + classified message = Stego Object Transmit over an insecure communication channel (Internet) The designated recipient will retrieve the classified message from the stego object, while others do not know the existence of the classified message in the innocent looking stego object.

Our Stegnography Approach -- Preliminary Test Study the OutGuess approach Current Research -- Vision-based Navigation (Road Detection) Study the JPEG images Study the characteristics of the DCT

10 Our Stegnography Approach -- Preliminary Test Study the OutGuess approach Current Research -- Vision-based Navigation (Road Detection) Study the JPEG images Study the characteristics of the DCT coefficients Study several attacks Histogram analysis 2 χ statistics OutGuess attacks Our Approach -- Preliminary Test Apply Principal Component Analysis (PCA) to find the remote scene. Apply Bayes statistics to learn the road features. Apply deformable templates to get rid of the shadows. Apply the curve functions to approximate the road. Research Interests Speech Recognition Intrusion Detection (One Student) Gene Sequence Analysis (One Student) Multi-media Data Mining Time/Spatial Data Mining Visualization Microarray Analysis Network simulation (One Student)

Overview of Computer Vision. CS308 Data Structures

Overview of Computer Vision. CS308 Data Structures Overview of Computer Vision CS308 Data Structures What is Computer Vision? Deals with the development of the theoretical and algorithmic basis by which useful information about the 3D world can be automatically