9.913 Pattern Recognition for Vision. Class 8-2 An Application of Clustering. Bernd Heisele

Transcription

1 9.913 Class 8-2 An Application of Clustering Bernd Heisele Fall 2003

2 Overview Problem Background Clustering for Tracking Examples Literature Homework

3 Problem Detect objects on the road: Cars, trucks, motorbikes, pedestrians.

4 Image Motion Determine the image motion (vector field) ( uxy vxy) v( xy, ) = (, ), (, ). T

5 Object Segmentation using Image Motion Motion-based segmentation

6 Image Motion Equations for Rigid Motion 2 u= Ω xy+ Ω ( 1+ x ) Ω y+ ( V V x)/ Z X Y Z X Z 2 v= Ω ( 1+ y ) + Ω xy+ Ω x+ ( V V y)/ Z X Y Z Y Z

7 Image Motion Estimation Optical Flow Image intensity over time is f( x, y, t) The intensity of a point over time is given by: gt ( ) = f( xt ( ), yt ( ), t), where xt ( ), yt ( ) is the trajectory of the point in the image plane. Assume the intensity of the point does not change over time: dg() t f x f y f = = 0 dt x t y t t x y, = ( uv, ), ( uv, ) f + ft = 0 t t

8 Image Motion Estimation Optical Flow Gradient equation of optical flow: f f f u+ v+ = 0 x y t f f ( x) f( t) f( x) f( t) =, u = x u t x t u f ( x) x f () t x = u t x

9 Image Motion Estimation problems Aperture Problem t 1 t 2 Aperture y Aperture t 1 y t 2 x x No change over time, optical flow=?

10 Object Segmentation Problem Objects: To determine the image motion we have to know what the objects are, i.e. which points belong together. However, we can t extract objects without motion.

11 An Idea Are there methods that help us to determine the image motion (avoid the aperture problem)? Neighbored pixels of similar color belong to the same object color segmentation. Objects usually consist of several regions of different color color segmentation alone does not solve the problem, we still need motion.

12 Color Segmentation & Motion Color segmentation of single images Motion-based segmentation: Objects Find regions (connected component Analysis) Track regions

13 Color Segmentation & Motion, why it did not work Color regions are not stable over time, they merge & break apart which makes tracking extremely difficult. Need consistent color segmentation over time!

14 Color Cluster Flow Cluster Centers Segmented Objects Image n Each pixel is a data point in (R, G, B, x, y) Clustering by parallel K-means Trajectories of Cluster Centers Motion Segmentation

15 Color/Position Clustering Original Image Clustered Image Boundaries of Clusters Each pixel is represented by a its color/positoin features (R, G, B, wx, wy), where w is a constant. Clustering is applied to group pixels with similar color and position.

16 Color versus Position Cluserting in (R, G, B, wx, wy) For large w the clusters are compact (Voronoi tesselation). For small w the color dominates and the clusters lose their spatial connectivity.

17 Clustering of Consecutive Images with k-means y We know the cluster centers of the previous image and use them for clustering the current image consistent segmentation over time Assign each data point to closest center Recompute center as average of data points The trajectories (displacemets of cluster centers) are for free. x

18 Parallel k-means Clustering 1. Partitioning step in iteration k : { 2 } 2 s s r s r C ( k) = ( k 1) ( k 1) i q q n n q n i C : cluster q, s : data point n, r : prototype of cluster q q n q 2. Computing prototypes in iteration k : r 1 ( k) = s S : data points in cluster q ( ) i q q i q Sq k s C ( k) k-means leads to a local minimum of the quantization error N 1 mse = s r, r = argmin s r N i= i q() i q() i i n 1 n Q

19 Fast parallel k-means Clustering Q 1 circles around each r with diameters d = r r check if s are inside the circles. i m n m n 3 Speed-up for one iteration

20 Initial Clustering Clustering of the first image of a sequence K-means is not a good choice for the first image because we don t know a good initialization of the cluster centers. many iterations required (slow) could lead to a bad local minimum (large mse) We need a fast algorithm which doesn t require initialization. Divisive clustering (tree-based clustering)

21 Initial Clustering y No prior knowledge about color clusters. Select cluster with highest variance Split by hyperplane vertical to direction of highest variance. x

22 Initial Clustering Binary Tree Root cluster contains all points. -Use mse bound or max. number of clusters as stop criteria. -The cluster centers of the leafs initialize k-means of the next image.

23 Examples First Image of Sequence Last Image of Sequence Trajectories of Cluster Centroids

24 Not Perfect Clusters jump when objects appear or leave the scene. (Number of clusters if fixed) Over-segmentation and spurious motion in homogenous regions

25 Literature B. Heisele, U. Kressel, and W. Ritter. Tracking non-rigid, moving objects based on color cluster flow. Proc. Computer Vision and Pattern Recognition (CVPR), pp , San Juan, Clustering Classics: J. MacQueen. Some methods for classification and analysis of multivariate observations. Proc. 5 th Berkeley Symp. Mathematics, Statistics and Probablility, pp , Y. Linde, A. Buzo, and R. Gray. An algorithm for vector quantizer design. IEEE Transactions on Communications, COM-28/1, pp , 1980.

26 Homework No homework!