ISMAR 2010 Tracking Competition & Alvar. ScandAR 2010 Alain Boyer Augmented Reality Team VTT Technical Research Centre of Finland

Size: px

Start display at page:

Download "ISMAR 2010 Tracking Competition & Alvar. ScandAR 2010 Alain Boyer Augmented Reality Team VTT Technical Research Centre of Finland"

Stephany Floyd
6 years ago
Views:

1 ISMAR 2010 Tracking Competition & Alvar ScandAR 2010 Alain Boyer Augmented Reality Team VTT Technical Research Centre of Finland

2 2 Agenda ISMAR 2010 Tracking Competition Alvar Overview Rules Approach Calibration Phase Detection Phase Results Overview Features Dependencies & Platforms Modules & Licensing Roadmap

3 3 Tracking Competition Overview What? Yearly academic and industrial tracking competition Who? VTT AR Team and Aalto University Where? ISMAR 2010 When? October 2010 Why? Advance the state-of-the-art in markerless tracking

4 16/12/2010 Tracking Competition Rules Environment A simulated storage room Many objects and features are present Fiducial markers with known 3D coordinates (calibration) Task Logistics scenario, identify and pick up certain items Items are identified by their 3D coordinates Visualize item using only crosshairs 4

5 5 Tracking Competition Approach Developed a client/server system Small mobile device Large computation power on server A server is required regardless in real world applications Meta data, augmentation data, instructions Server: a recent laptop Client: N900 mobile phone

Semi-automatically identify fiducial markers Add marker points to point cloud Align the point clouds to the global

6 6 Tracking Competition Calibration Phase Divide room into picking areas Perform 3D reconstruction for each area Take images, find GFT and upload to server Reconstruction via SURF matching Triangulate SURF and GFT features Semi-automatically identify fiducial markers Add marker points to point cloud Align the point clouds to the global reference frame Launch server for each picking area Database of 3D features (SURF, GFT, marker) Can do SURF matching and image registration

computes pose Client visualizes the current target (3D point) In view: crosshairs Out of view:

7 7 Tracking Competition Detection Phase Client takes image and uploads to server Server computes pose of image Returns list of GFT with 2D and 3D coordinates Client recursively tracks GFT and computes pose Client visualizes the current target (3D point) In view: crosshairs Out of view: arrow Client performs more requests periodically Random selection of server Shutter button activates new target

8 Tracking Competition Results A working tracking server was achieved Managed to identify all items Results of competition were unknown Problem item identification and scoring by

8 8 Tracking Competition Results A working tracking server was achieved Managed to identify all items Results of competition were unknown Problem item identification and scoring by organizers Major difficulty: obtaining GFT with 3D coordinates Not enough features after all the processing SURF matching failed with repetitive patterns Planned to use IMU, not enough time

9 9 Alvar Overview What? A Library for Virtual and Augmented Reality Who? Developed by VTT When? Initial release in October 2009 Why? To collect our technology and make it reusable Provide a framework for academic AR research Enable commercial use of AR

10 Alvar Features Detecting and tracking 2D markers Two types of markers, future marker types can be added Accurate pose estimation Heuristics for intelligent tracking Multiple marker

10 10 Alvar Features Detecting and tracking 2D markers Two types of markers, future marker types can be added Accurate pose estimation Heuristics for intelligent tracking Multiple marker setups are supported Marker relations set manually or detected automatically Recover from occlusions Computer vision tools Calibrating cameras Distorting and undistorting points Projecting points

11 Alvar Features Markerless tracking via SFM using image features Uses marker for initialization Hiding markers from the view Several methods for tracking using 2D optical flow

11 11 Alvar Features Markerless tracking via SFM using image features Uses marker for initialization Hiding markers from the view Several methods for tracking using 2D optical flow PSA, extended PSA for rotation Image feature tracking Statistical tracking Several basic filters Average, median, running average, double exponential smoothing Kalman filter, EKF, UKF

12 12 Alvar Dependencies & Platforms Dependencies OpenCV 1.0 Note: independent of any graphical libraries Platforms Windows XP 32-bit, VS 2003, 2005 and 2008 Linux 32-bit, GCC 4.4 Linux 64-bit, GCC 4.4 Mobile: Symbian, Maemo/Meego, iphone*, Android* Web: Flash*, Silverlight* (* platforms that will soon be available)

13 13 Alvar Modules & Licensing Modules Basic: core functionality and marker tracking Pro: advanced functionality and markerless tracking Licensing Free for personal and academic use 1 year time limit Commercial for paid applications unlimited

14 14 Alvar Roadmap Move to OpenCV 2.2 Provide binary packages Image-based markerless tracking Assumes a planar target Can use any feature detector Based on Ferns classifier for matching Improve mobile version More features, more platforms, deciding how to package Photorealistic rendering module Deciding how to make reusable and how to package Alvar 2.0

15 15 Thank You Questions? AR Team Alvar

Outline. Introduction System Overview Camera Calibration Marker Tracking Pose Estimation of Markers Conclusion. Media IC & System Lab Po-Chen Wu 2

Outline. Introduction System Overview Camera Calibration Marker Tracking Pose Estimation of Markers Conclusion. Media IC & System Lab Po-Chen Wu 2 Outline Introduction System Overview Camera Calibration Marker Tracking Pose Estimation of Markers Conclusion Media IC & System Lab Po-Chen Wu 2 Outline Introduction System Overview Camera Calibration