Contents I IMAGE FORMATION 1

Transcription

1 Contents I IMAGE FORMATION 1 1 Geometric Camera Models Image Formation Pinhole Perspective Weak Perspective Cameras with Lenses The Human Eye Intrinsic and Extrinsic Parameters Rigid Transformations and Homogeneous Coordinates Intrinsic Parameters Extrinsic Parameters Perspective Projection Matrices Weak-Perspective Projection Matrices Geometric Camera Calibration A Linear Approach to Camera Calibration A Nonlinear Approach to Camera Calibration Notes Light and Shading Modelling Pixel Brightness Reflection at Surfaces Sources and Their Effects The Lambertian+Specular Model Area Sources Inference from Shading Radiometric Calibration and High Dynamic Range Images The Shape of Specularities Inferring Lightness and Illumination Photometric Stereo: Shape from Multiple Shaded Images Modelling Interreflection The Illumination at a Patch Due to an Area Source Radiosity and Exitance An Interreflection Model Qualitative Properties of Interreflections Shape from One Shaded Image v

2 vi 2.5 Notes Color Human Color Perception Color Matching Color Receptors The Physics of Color The Color of Light Sources The Color of Surfaces Representing Color Linear Color Spaces Non-linear Color Spaces A Model of Image Color The Diffuse Term The Specular Term Inference from Color Finding Specularities Using Color Shadow Removal Using Color Color Constancy: Surface Color from Image Color Notes II EARLY VISION: JUST ONE IMAGE Linear Filters Linear Filters and Convolution Convolution Shift Invariant Linear Systems Discrete Convolution Continuous Convolution Edge Effects in Discrete Convolutions Spatial Frequency and Fourier Transforms Fourier Transforms Sampling and Aliasing Sampling Aliasing Smoothing and Resampling Filters as Templates Convolution as a Dot Product Changing Basis Technique: Normalized Correlation and Finding Patterns

3 vii Controlling the Television by Finding Hands by Normalized Correlation Technique: Scale and Image Pyramids The Gaussian Pyramid Applications of Scaled Representations Notes Local Image Features Computing the Image Gradient Derivative of Gaussian Filters Representing the Image Gradient Gradient-Based Edge Detectors Orientations Finding Corners and Building Neighborhoods Finding Corners Using Scale and Orientation to Build a Neighborhood Describing Neighborhoods with SIFT and HOG Features SIFT Features HOG Features Computing Local Features in Practice Notes Texture Local Texture Representations Using Filters Spots and Bars From Filter Outputs to Texture Representation Local Texture Representations in Practice Pooled Texture Representations by Discovering Textons Vector Quantization and Textons K-means Clustering for Vector Quantization Synthesizing Textures and Filling Holes in Images Synthesis by Sampling Local Models Filling in Holes in Images Image Denoising Non-local Means Block Matching 3D (BM3D) Learned Sparse Coding Results Shape from Texture Shape from Texture for Planes Shape from Texture for Curved Surfaces

4 viii 6.6 Notes III EARLY VISION: MULTIPLE IMAGES Stereopsis Binocular Camera Geometry and the Epipolar Constraint Epipolar Geometry The Essential Matrix The Fundamental Matrix Binocular Reconstruction Image Rectification Human Stereopsis Local Methods for Binocular Fusion Correlation Multi-Scale Edge Matching Global Methods for Binocular Fusion Ordering Constraints and Dynamic Programming Smoothness and Graphs Using More Cameras Application: Robot Navigation Notes Structure from Motion Internally Calibrated Perspective Cameras Natural Ambiguity of the Problem Euclidean Structure and Motion from Two Images Euclidean Structure and Motion from Multiple Images Uncalibrated Weak-Perspective Cameras Natural Ambiguity of the Problem Affine Structure and Motion from Two Images Affine Structure and Motion from Multiple Images From Affine to Euclidean Shape Uncalibrated Perspective Cameras Natural Ambiguity of the Problem Projective Structure and Motion from Two Images Projective Structure and Motion from Multiple Images From Projective to Euclidean Shape Notes

5 IV MID-LEVEL VISION Segmentation by Clustering Human Vision: Grouping and Gestalt Important Applications Background Subtraction Shot Boundary Detection Interactive Segmentation Forming Image Regions Image Segmentation by Clustering Pixels Basic Clustering Methods The Watershed Algorithm Segmentation Using K-means Mean Shift: Finding Local Modes in Data Clustering and Segmentation with Mean Shift Segmentation, Clustering, and Graphs Terminology and Facts for Graphs Agglomerative Clustering with a Graph Divisive Clustering with a Graph Normalized Cuts Image Segmentation in Practice Evaluating Segmenters Notes Grouping and Model Fitting The Hough Transform Fitting Lines with the Hough Transform Using the Hough Transform Fitting Lines and Planes Fitting a Single Line Fitting Planes Fitting Multiple Lines Fitting Curved Structures Robustness M-Estimators RANSAC: Searching for Good Points Fitting Using Probabilistic Models Missing Data Problems Mixture Models and Hidden Variables The EM Algorithm for Mixture Models Difficulties with the EM Algorithm ix

6 x 10.6 Motion Segmentation by Parameter Estimation Optical Flow and Motion Flow Models Motion Segmentation with Layers Model Selection: Which Model Is the Best Fit? Model Selection Using Cross-Validation Notes Tracking Simple Tracking Strategies Tracking by Detection Tracking Translations by Matching Using Affine Transformations to Confirm a Match Tracking Using Matching Matching Summary Representations Tracking Using Flow Tracking Linear Dynamical Models with Kalman Filters Linear Measurements and Linear Dynamics The Kalman Filter Forward-backward Smoothing Data Association Linking Kalman Filters with Detection Methods Key Methods of Data Association Particle Filtering Sampled Representations of Probability Distributions The Simplest Particle Filter The Tracking Algorithm A Workable Particle Filter Practical Issues in Particle Filters Notes V HIGH-LEVEL VISION Registration Registering Rigid Objects Iterated Closest Points Searching for Transformations via Correspondences Application: Building Image Mosaics Model-based Vision: Registering Rigid Objects with Projection

7 xi Verification: Comparing Transformed and Rendered Source to Target Registering Deformable Objects Deforming Texture with Active Appearance Models Active Appearance Models in Practice Application: Registration in Medical Imaging Systems Notes Smooth Surfaces and Their Outlines Elements of Differential Geometry Curves Surfaces Contour Geometry The Occluding Contour and the Image Contour The Cusps and Inflections of the Image Contour Koenderink s Theorem Visual Events: More Differential Geometry The Geometry of the Gauss Map Asymptotic Curves The Asymptotic Spherical Map Local Visual Events The Bitangent Ray Manifold Multilocal Visual Events The Aspect Graph Notes Range Data Active Range Sensors Range Data Segmentation Elements of Analytical Differential Geometry Finding Step and Roof Edges in Range Images Segmenting Range Images into Planar Regions Range Image Registration and Model Acquisition Quaternions Registering Range Images Fusing Multiple Range Images Object Recognition Matching Using Interpretation Trees Matching Free-Form Surfaces Using Spin Images Kinect Features

8 xii Technique: Decision Trees and Random Forests Labeling Pixels Computing Joint Positions Notes Learning to Classify Classification, Error, and Loss Using Loss to Determine Decisions Training Error, Test Error, and Overfitting Regularization Error Rate and Cross-Validation Receiver Operating Curves Major Classification Strategies Example: Mahalanobis Distance Example: Class-Conditional Histograms and Naive Bayes Example: Classification Using Nearest Neighbors Example: The Linear Support Vector Machine Example: Kernel Machines Example: Boosting and Adaboost Practical Methods for Building Classifiers Manipulating Training Data to Improve Performance Building Multi-Class Classifiers Out of Binary Classifiers Solving for SVMS and Kernel Machines Notes Classifying Images Building Good Image Features Example Applications Encoding Layout with GIST Features Summarizing Images with Visual Words The Spatial Pyramid Kernel Dimension Reduction with Principal Components Dimension Reduction with Canonical Variates Example Application: Identifying Explicit Images Example Application: Classifying Materials Example Application: Classifying Scenes Classifying Images of Single Objects Image Classification Strategies Evaluating Image Classification Systems Fixed Sets of Classes Large Numbers of Classes

9 xiii Flowers, Leaves, and Birds: Some Specialized Problems Image Classification in Practice Codes for Image Features Image Classification Datasets Dataset Bias Crowdsourcing Dataset Collection Notes Detecting Objects in Images The Sliding Window Method Face Detection Detecting Humans Detecting Boundaries Detecting Deformable Objects The State of the Art of Object Detection Datasets and Resources Notes Topics in Object Recognition What Should Object Recognition Do? What Should an Object Recognition System Do? Current Strategies for Object Recognition What Is Categorization? Selection: What Should Be Described? Feature Questions Improving Current Image Features Other Kinds of Image Feature Geometric Questions Semantic Questions Attributes and the Unfamiliar Parts, Poselets and Consistency Chunks of Meaning VI APPLICATIONS AND TOPICS Image-Based Modeling and Rendering Visual Hulls Main Elements of the Visual Hull Model Tracing Intersection Curves Clipping Intersection Curves

10 xiv Triangulating Cone Strips Results Going Further: Carved Visual Hulls Patch-Based Multi-View Stereopsis Main Elements of the PMVS Model Initial Feature Matching Expansion Filtering Results The Light Field Notes Looking at People HMM s, Dynamic Programming, and Tree-Structured Models Hidden Markov Models Inference for an HMM Fitting an HMM with EM Tree-Structured Energy Models Parsing People in Images Parsing with Pictorial Structure Models Estimating the Appearance of Clothing Tracking People Why Human Tracking Is Hard Kinematic Tracking by Appearance Kinematic Human Tracking Using Templates D from 2D: Lifting Reconstruction in an Orthographic View Exploiting Appearance for Unambiguous Reconstructions Exploiting Motion for Unambiguous Reconstructions Activity Recognition Background: Human Motion Data Body Configuration and Activity Recognition Recognizing Human Activities with Appearance Features Recognizing Human Activities with Compositional Models Resources Notes Image Search and Retrieval The Application Context Applications User Needs

11 xv Types of Image Query What Users Do with Image Collections Basic Technologies from Information Retrieval Word Counts Smoothing Word Counts Approximate Nearest Neighbors and Hashing Ranking Documents Images as Documents Matching Without Quantization Ranking Image Search Results Browsing and Layout Laying Out Images for Browsing Predicting Annotations for Pictures Annotations from Nearby Words Annotations from the Whole Image Predicting Correlated Words with Classifiers Names and Faces Generating Tags with Segments The State of the Art of Word Prediction Resources Comparing Methods Open Problems Notes VII BACKGROUND MATERIAL Optimization Techniques Linear Least-Squares Methods Normal Equations and the Pseudoinverse Homogeneous Systems and Eigenvalue Problems Generalized Eigenvalues Problems An Example: Fitting a Line to Points in a Plane Singular Value Decomposition Nonlinear Least-Squares Methods Newton s Method: Square Systems of Nonlinear Equations Newton s Method for Overconstrained Systems The Gauss Newton and Levenberg Marquardt Algorithms Sparse Coding and Dictionary Learning Sparse Coding Dictionary Learning

12 xvi Supervised Dictionary Learning Min-Cut/Max-Flow Problems and Combinatorial Optimization Min-Cut Problems Quadratic Pseudo-Boolean Functions Generalization to Integer Variables Notes Bibliography 684 Index 737 List of Algorithms 760