Object Tracking R. Venkatesh Babu
Primitive tracking Appearance based - Template Matching Assumptions: Object description derived from first frame No change in object appearance Movement only 2D translation Restrictive approach Not suitable for real-world videos
Basic Template Matching Assumptions: a snapshot of object from first frame can be used to describe appearance Object will look nearly identical in new image Movement is nearly pure 2D translation The last two are very restrictive.
Template Matching Is a search problem: Given an intensity patch element in the left (t) image, search for the corresponding patch in the right image (t+1). We will typically need geometric constraints to reduce the size of the search space
Matching Functions (SAD) (SSD) (Correlation)
Example (Correlation) SE 263 R. Venkatesh Babu
C = SE 263 R. Venkatesh Babu
Problem with Correlation of Raw Image Templates > Solution: Subtract off the mean value of the template. Now, the correlation score is higher only when darker parts of the template overlap darker parts of the image, and brighter parts of the template overlap brighter parts of the image.
SSD or block matching (Sum of Squared Differences) Best match (highest score) in image coincides with correct match in this case!
Practical Issues Object shape might not be well described by a scan line-oriented bounding rectangle End up including lots of background pixels Solution : Gaussian Weighting? Segmentation? Search Range Need some estimate of object motion. Appearance change Solution: Adaptive template update Problem : Drift
Gradient Descent Tracking Need a more efficient method than explicit search over some large window If we have a good estimate of object position already, we can efficiently refine it using gradient descent. Assumption: Our estimate of position must be very close to where the object actually is! (however, we can relax this using multi-scale techniques image pyramids)
Harris Detector : Mathematics Consider the SSD Function This tells how match score changes if you shift the template by [u,v]
Gradient Descent Method Taylor Series for 2D funcions
Lucas-Kanade Derivation SE 263 R. Venkatesh Babu
Lucas-Kanade Tracking Traditional Lucas-Kanade is typically run on small, corner-like features (e.g. 5x5) to compute optic flow. Limitations : Assumption of constant flow (pure translation) for all pixels in a larger window is unreasonable. can easily generalize Lucas-Kanade approach to other 2D parametric motion models (like affine or projective)
JY Bouguet, Pyramidal Implementation of the Lucas Kanade Feature Tracker
Robust Object Tracking With RBF Networks Machine Learning-based object modeling Fast learning RBFN Classifier Use two classifiers (object and background) for reliable modeling Use of posterior probability measure for tracking R. Venkatesh Babu, S. Suresh and A. Makur, Robust Visual Tracking with Online Adaptive RBF-Networks CVIU, March 2010.
Object Modeling Training Phase Object Initialization Object Classifier Object/ Background Separation Feature Extraction Object Model Non-Object Classifier
Object Localization - Testing Phase Object Classifier Current Frame Feature Extraction Object Localization Object Position Non-Object Classifier Object Model
Foreground/Background separation Histogram of pixels in object window and background window L x i log max max h h o b x x i i,, Threshold L(x i ) to classify each pixel as foreground / background Robustness to clutter
Classifier - Architecture SE 263 R. Venkatesh Babu
Fast Learning RBFN Classifier The output of RBFN with K neurons: In matrix form
Computing Output Weights The output weights are estimated as: Where Ф K is Hidden Layer Output Matrix, (pseudo-inverse)
Classifier Development Select K (No. of hidden neurons) Assign (μ,σ) for the Gaussian neurons arbitrarily Compute output weights ( ) Select the object/background classifiers from a set of classifiers.
Object Modeling Posterior probability obtained using object Classifier: Posterior probability obtained using non-object Classifier: The final object model is obtained as:
Object Model Object Classifier (p o ) Non-Object Classifier (1-p b ) Object Model (p t )
Object Localization Object center is estimated by iteratively seeking the mode of posterior probability weighted by target model.
Online Adaptation For each new sample (U n,t n ) Compute Adapt output weight ( ) using RLS Where,
Online Adaptation Only few samples are used for model update
Results Proposed System Solid Yellow Meanshift Dashed Blue
Results With adaptation Solid Yellow Without adaptation Dashed Red Meanshift Dashed Blue
Results Proposed System Solid Yellow Meanshift Dashed Blue