MODEL-FREE, STATISTICAL DETECTION AND TRACKING OF MOVING OBJECTS University of Koblenz Germany 13th Int. Conference on Image Processing, Atlanta, USA, 2006
Introduction Features of the new approach: Automatic detection of moving objects Objects are formed by connected, similar moving regions Model-free, statistical tracking Modified particle filter Uses color and edges Even works with moving camera
Motion Model Motion Model Views Object Model Motion Vector A motion vector m R 6 contains the six components of an affine transform for 2d-translation, rotation, and scaling. m = (d x, d y,α, s x, s y, s d ) R 6 d x, d y α s x, s y, s d translation in the image plane rotation in the image plane anisotrop scaling Motion Trajectory The sequence of motion vectors [m i ] t t 0 = m t0, m t0 +1,...,m t is called trajectory.
Template Model View Motion Model Views Object Model A view q = (C, E,ψ) represents a visible part of an object in the real world consists of a set C of color sample points (sampled randomly), a contour E (sampled equidistantly), and a weight ψ [0, 1] (trustiness). is the template used for the tracking. is generated from one ore more segments of the same frame. Original Image Region Image View
View of n segments Motion Model Views Object Model The color point set C of n segments S i, 1 i n is the unification of the color point sets C i of participating segments. The contour E of n segments S i, 1 i n is the unification of the contours E i of participating segments; such a contour could be called multi-contour. C = E = n C(S i ) i=1 n E(S i ) i=1 Original Image Region Image View of n segments
Sequence of Views Motion Model Views Object Model Problem: A view might be faulty, e. g. because of segmentation errors or occlusions. Solution: The system uses a sequence of views [q i ] t t 0 = q t0,..., q t as representation of the same object where q i is generated from segments of frame F i at time i. The view weights in a sequence of views sum to 1. Example: sequence of views of a trailer wich enters the scene (contour dotted blue)
Object Motion Model Views Object Model Motion and Views are modelled in stochastic terms to form object models. The motion of a view is not fact, but has a certain probability. The sampled probability distribution over the space of motions and views is represented as a particle set. A tracking object (short: object) is a particle set where each particle is a tuple ([q i ] t t 0,[m i ] t t 0,ω) consisting of a sequence of views [q i ] t t 0, a trajectory [m i ] t t 0, and a particle weight ω [0, 1] (probability). The sum of particle weights in a particle set must sum to 1.
Summary of Motion Model Views Object Model Object O (Particle Set) n Particle p View Seq. [q i ] t t 0 m View q Weight ω Trajectory [m i ] t t 0 m Motion Vec. m Color Points C Contour E View weight ψ Translation d x, d y Rotation α Scaling s x, s y, s d
Algorithm 1. Select a Particle: Sample a particle ([q i ] t t 1 0,[m i ] t 1 t 0,ω) O t 1 from the old particle set O t 1 with probability ω. 2. Select a View: Sample a certain view q τ = (C τ, E τ,ψ τ ) from the view sequence [q i ] t 1 t 0, t 0 τ < t with probability ψ τ. 3. Predict: Calculate a new trajectory [m i ] t t 0 based on [m i ] t 1 t 0. 4. Measure: Evaluate the view q τ with trajectory [m i ] t τ depending on the current Frame F t by a weight ω, i. e. measure, how good the view q τ fits the current image, after it was transformed by [m i ] t τ. 5. Normalize: Normalize the weights.
Step 4. Measure Color error e C = 1 C Contour error (x,c) C e E = 1 E ( min ě C, (c F t ([m] x)) 2) [0, ě C ] C [m] x F t ě C color point set transformation of location x current frame max. color error, system parameter ) min (ě E, D 2 t ([m] x) [0, ě E ] x E E contour, locations of contour points D t distance image of current edge image ě E max. contour error, system parameter Particle weight (before normalization) ( ω = exp e ) E 2σE 2 exp ( e ) C 2σC 2.
Distance Image (a) Original image (b) Region image (c) Distance image (d) Detail of (c) with a contour fitted in
Merge Objects Problem: Natural objects consist of more than a single segment. Solution: Basic objects 1 which are in spatial nearness and have similar motion trajectories are merged to complex objects where its trajectory is the component-wise, weighted average of trajectories of the basic objects, its color point sets of its views are obtained by unifying the respective color point sets of the basic objects, and the contours of its views are the unification of the respective contours of basic objects. 1 objects initialized by a single segment
Bobby-Car Foucault Pendulum Example 1. Two objects with identical color and partial occlusion
Bobby-Car Foucault Pendulum Example 2. Foucault Pendulum captured with a moving camera
Appendix References References M. Isard and A. Blake CONDENSATION - conditional density propagation for visual tracking. Int. Journal on Computer Vision, vol. 1, no. 29, pp. 5-28, 1998 M. Ross. Statistical Motion Segmentation and Object. (to apear) Vision, Modeling, and Visualization, Aachen, Germany, 2006.