Online Adaptive Visual Tracking

Transcription

1 Presentazioni finali XXIII ciclo 17/01/2011 Ing. Tutors: Prof. Tullio Salmon Cinotti Prof. Luigi Di Stefano

2 Visual Tracking It is a key building block for many applications, such as Automatic Visual Surveillance and Video Analytics Automatic Traffic Monitoring Autonomous Vehicle or Robot Navigation Natural Interfaces and Human Computer Interaction

3 Problem formalization Long-term tracking requires on-line adaptation of Filter parameters Appearance model No standard solution has emerged in the recent research

4 Recursive Bayesian Estimation A general solution to the recursion is only conceptually possible. An analytic solution is available if the functions are linear the uncertainty on the measurements and the state are zero-mean Gaussians T x = F x 1 + υ, E[ υυ ] = Q k k k k k k k T z = H x + η, E[ ηη ] = R k k k k k k k In such a case the Kalman filter is the optimal observer of the state If one of these conditions is not met, an approximate solution is possible using a particle filter

5 Why Adaptive Parameters? A major limitation of RBE is the requirement to a priori specify the transition model. [Mehra, 72], [Oussalah et al, 00], [Weng et al, 06] can estimate some parameters in the case of linear & Gaussian systems, but require time-invariant parameters require complete observability of the system

6 Support Vector Kalman In case of a completely observable system, it is possible to learn the transition model on-line performing robust regression of the dynamic of the measurements. We use a modified Support Vector Machines in ε-regression mode (SVR) able to regress unbiased functions [Poggio et al. 01] to estimate the transition matrix, F k the associated noise covariance matrix, Q k Improvements over [Mehra, 72], [Oussalah et al, 00], [Weng et al, 06] Less assumptions (i.e. only complete observability) Transition Matrix Identification Time varying estimation of both F k and Q k (partially done by CMT [Mehra, 72]) Noise model assumed by the SVR is a superset of that assumed by the Kalman filter Drawback is the risk of filter coalescence Empirically limited by bounding the estimation of Q k

7 SVK Results Ground Truth SVK 2x2 matrix

8 Change Detection-based Tracking In case of a static camera, tracking has been usually tackled by using change detection In such scenario, standard approaches are not adaptive [Stauffer & Grimson, 99] [Haritaoglu et al, 00] [Harville et al., 04] fix hard thresholds to perform change detection [Haritaoglu et al, 00], require a fixed, off-line learned foreground model [Isard et al., 01] use heuristic analyses of the change map to obtain measures for the tracker [Stauffer & Grimson, 99] [Haritaoglu et al, 00] [Harville et al., 04]

9 Why Adaptive Parameters?

10 We have proposed Synergistic Bayesian Loop an almost parameters-free (in particular threshold-free) approach to provide measurements for the tracker from a change map a sound way to estimate the time-variant uncertainty of measurements a method to let the current state of the tracker positively guide the change detector and provide an adaptive target model (cognitive feedback) The measure and the cognitive feedback are obtained as the marginalization of the joint probability on the state and the change map (unknown) Prior ij ( ) = (,, ) p c p x c c dx ij k ij k 4 ij ij R c Θ Tracker Change Detection p ( x ) p( x, c) k Measure = c Θ k

11 Time variant uncertainty

12 Bayesian Loop results Overlap score

13 3D Category Detection Detection 2D : [Liebe & al., 04], [Dalal & Triggs, 05] Based on distinctive 2D local features

14 SHOT Features for Shape Matching 0,7 CSHOT (RGB) 0,6 0,5 CSHOT (CIELab) 0,4 SHOT 0,3 0,2 Matching of 3D local features 0,1 0-0,1 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9-0,1 Improved performance by seamless integration of shape and colour features Robust local RF + SHOT

15 Category Detection with 3D ISM Recognition based on Codebook of SHOT features Implicit Shape Model Avg Vehicle Animal House hold Building Furniture Plant Others 62,9 11,0 9,8 0,4 0,4 12,7 2,9 14,2 63,9 9,7 0,0 1,3 8,4 2,6 25,4 10,8 42,7 1,1 8,1 5,4 5,4 34,0 6,4 10,6 2,1 17,0 14,9 10,6 24,5 5,3 22,3 1,1 43,6 2,1 1,1 15,0 3,3 13,3 0,0 1,7 60,0 6,7 18,2 17,4 8,3 2,5 7,4 8,3 38,0

16 Conclusions & Future Work Improvements to the classical framework of visual RBE tracking Online and robust transition model adaptation Bayesian loop for static camera tracking A robust, descriptive and efficient 3D feature An approach for 3D category recognition and detection Future work: Online parameters adaptation for tracking in 3D data with initialization provided by 3D ISM Get rid of the static camera assumption and base the Bayesian loop on SHOT features of the target

17 Publications F. De Crescenzio, M. Fantini, F. Persiani, L. Di Stefano, P. Azzari, S. Salti, Augmented Reality for Aircraft Maintenance Training and Operations Support, Computer Graphics and Applications, IEEE, vol. 31, no. 1, pp , January-February S. Salti, F. Tombari, L. Di Stefano, On the use of Implicit Shape Models for recognition of object categories in 3D data, The 10th Asian Conference on Computer Vision (ACCV), Queenstown, New Zealand, 8-12 November, 2010 S. Salti, A. Lanza, L. Di Stefano, Bayesian Loop for Synergistic Change Detection and Tracking, The 10th International Workshop on Visual Surveillance (VS), Queenstown, New Zealand, 8 November, F. Tombari, S. Salti, L. Di Stefano, Unique Shape Context for 3D Data Description, ACM Int. Workshop on 3D Object ACM MM 2010, Firenze, Italy, October, F. Tombari, S. Salti, L. Di Stefano, Unique Signatures of Histograms for Local Surface Description, The 11th European Conference on Computer Vision(ECCV), Heraklion, Crete, Greece, 5-11 September, S. Salti, L. Di Stefano, On-line learning of the Transition Model for Recursive Bayesian Estimation, The 2nd International Workshop on Machine Learning for Vision-based Motion Analysis ICCV 2009, Kyoto, Japan, October S. Salti, L. Di Stefano, SVR-based jitter reduction for markerless Augmented Reality, International Conference on Image Analysis and Processing ICIAP 2009, Vietri sul Mare (SL), Italy, September S. Salti, O. Schreer, L. Di Stefano, Real-time 3D Arm Pose Estimation from Monocular Video for Enhanced HCI, 1st ACMInt. Workshop on Vision Networks for Behaviour Analysis (VNBA) in conjunction with ACM Multimedia 2008, Vancouver, Canada, October S. Salti, F. Tombari, L. Di Stefano, A Performance Evaluation of 3D Keypoint Detectors, (submitted to 3DIMPVT 2011)