POME A mobile camera system for accurate indoor pose

Size: px

Start display at page:

Download "POME A mobile camera system for accurate indoor pose"

Anastasia Booker
5 years ago
Views:

ICT Intelligent Construction Tools A 50-50

2 ICT Intelligent Construction Tools A joint venture between Trimble and Hilti Vision: revolutionize the way construction is done Many inefficiencies persist on the construction site The central technical problem - Accurate and robust indoor positioning 11/7/2016 2

3 Overview 1. ICT and the construction market 2. A brief history of indoor positioning 3. POME concept, theory of operation, system tradeoffs 4. POME error budget 5. System components 6. The usual problems 7. POME accuracy 8. The future static kinematic 11/7/2016 3

5 Construction tools today A precision instrument (not a tool) 5 arc sec sensor ~2.5mm at 100 m (Horizontal, Vertical) $30-60K (expensive) Requires experience to set up and use careful installation on a stable tripod correct referencing Single user Subject to line of sight occlusion Issues in tracking at close range Most sites still use traditional tools for most tasks 11/7/2016 5

reliable installation Room size > 30 meters Challenges rapidly

6 Requirements Accuracy (< 6mm) Robustness to occlusion to drop dust/dirt Cost BOM <= $400 Ease of installation No cables Fast and reliable installation Room size > 30 meters Challenges rapidly changing environment variable lighting conditions many reflective surfaces 11/7/2016 6

8 Why POME? POME = Position & Orientation Measurement Engine Multi User Indoor (+ possibly outdoor) Low cost & fully solid state POME inside == GPS outside Similar weight & volume Similar cost Similar update rate Similar accuracy Like GPS, enables many applications, e.g.: Staked layout Projection systems Robotic systems Augmented Reality POME inside == GPS outside 11/7/ All rights reserved.

10 Principle of Operation Measure angles between known points Use redundant measurements Use least squares to solve a set of non- linear equations Q: How accurately can you measure angles with a (wide angle) camera? Θ Θ A Θ δθ Θ B B Θ A Simplified 2D example Θ A 10 11/7/2016

11 2D example: intersection of 2 circles Nonlinear problem D Intersection of 2 circles gives candidate solutions X & Y Positions of A,B,C,D must be known C Θ CD X Uncertainty in angle measurements results in a covariance ellipse Y Θ AB B A 11/7/ All rights reserved.

12 Error Budget require 0.2 pixel 1 sigma with multiple cameras 1 pixel = 2.2 um Error contributors: Achievable calibration accuracy Lenses mechanics Mechanical stability Centroid determination with: saturated signals weak signals Number and geometry of targets Accuracy of target survey 11/7/

13 System Design Considerations Number of cameras Arrangement of cameras for best F.O.V. Type of image sensors Number / size of pixels Rolling / global shutter Color / monochrome Dynamic range of pixels Type of lens projection function Image sensor matching Type of target LED s Visible / I.R., power, pattern Image processing considerations Image processing bandwidth Power Cost!! Early concept for 3 camera overlapping F.O.V. 11/7/

14 Active Targets Transmit at 850 nm (near IR) Approximately 350 mw Modulated intensity 11/7/

15 Projection from object space to image space Camera: a projection from object space to image space (x,y,z) -> (u,v) 1 to 1 mapping of rays (unit vectors) to image space points (u,v) Point of light becomes a blob (x,y,z) object space (x/z,y/z,1) optical axis θ Z a non-trivial mapping For pose calculation we need to convert from image space points to rays (angles) lens φ O Y X Mapping function is different for every camera => need to calibrate Image sensor (u,v) image space 11/7/

16 Fisheye (f-theta) lens projection For large F.O.V. pinhole camera needs a very large image sensor optical axis optical axis θ θ F-theta projection => equal angle increment maps to equal number of pixels optical center f θ Y optical center f Y Camera is an angle measuring sensor Image sensor R R = f tanθ X Image sensor R R = f θ X pinhole projection f-theta projection 11/7/ All rights reserved.

17 F-theta lens equal angle is mapped to equal distance on image sensor Camera measures angles Our cameras have ~14 pixels/degree => 1 pix = 250 arcsec => 0.2 pix = 50 arcsec ~ 5 20 meters 160 degree Image circle image sensor 11/7/

V Strongly affects centroid determination accuracy Non

18 Example blobs using off the shelf lenses Non symmetry of impulse response across the F.O.V Strongly affects centroid determination accuracy Non uniformity of energy distribution across the F.O.V. compounds near/far problem 11/7/

30 deg. elev. circle 0 deg. elev. circle -30 deg. elev. circle F.

Calibration residual After removing an f-theta model, we are left with residual Calibrate by fitting a function to the residual inverse residual function Blob

24 Calibration residual After removing an f-theta model, we are left with residual Calibrate by fitting a function to the residual inverse residual function Blob centroid (u,v) -> -> inverse f-theta -> angles -> pose solution Lens and camera mechanics must be stable over time and temperature 11/7/ All rights reserved.

25 Shock and Vibration Significant testing has been done to POME head to verify stability Below are test set up and results from shock and vibration testing with positive results 11/7/

27 The Usual Problems Calibration + mechanical / thermal stability range ratio (near / far problem) Registration (target determination) Interference rejection (strong signals) Multipath rejection (with and without direct ray) Initialization Data rate (~3000 Mb/s) / image processing / power Solution (target modulation, synchronization) 11/7/

natural lighting ~ 15 m x 10 m x 8m PLT for

30 Test results for unit 0x position stations 16 azimuth stations at each position station In table on following page Each row is one position station Each column is one azimuth station Numbers show the position error relative to the truth system (PLT) in units of mm Each number represents a static mode result Target locations are shown with black dots and numeric ID Robot trajectory is shown with blue x PLT location is shown with a red dot Position Station TP07 location shown with Warehouse dimensions are in units of meters 11/7/

32 Why is it difficult to state performance? Performance is characterized by error statistics To have significance, statistics need many measurements to validate We have 6 errors to characterize at each point in space 3 components of position error 3 components of orientation error Errors are worse in some directions than in other directions There are a variable number of targets and variable working volume geometry There are different modes of operation (here, we document static and survey modes) We plot the worst direction 1 sigma errors in a square working volume 11/7/

Example of scatter plot ellipsoids A scatter plot of results creates a clump of data points distributed around the truth value The statistics of the clump can be characterized by

33 Example of scatter plot ellipsoids A scatter plot of results creates a clump of data points distributed around the truth value The statistics of the clump can be characterized by an error ellipsoid Shown below are example 1 sigma ellipsoids for position and orientation Position Error Ellipsoid Orientation Error Ellipsoid 11/7/ All rights reserved.

Simulation Target Configurations Nominal square room of size 10x10 meters Consider 6 different target arrangements in room Targets installed at uniform height and positioned on walls around

34 Simulation Target Configurations Nominal square room of size 10x10 meters Consider 6 different target arrangements in room Targets installed at uniform height and positioned on walls around circumference of room Calculate position and orientation accuracy at grid points in the room Calculate solutions with: 1 azimuth station (static mode) 4 azimuth stations (survey mode) All simulations use 0.5 pixel 1 sigma, but we expect/hope to achieve 0.2 pixel 1 sigma in practice!! 11/7/

35 10x10 m room, 8 targets, 1 azim. station 0.5 pix 1 sigma 1 azimuth station 10x10 meter room Worst direction position Worst direction orientation 1 sigma results 11/7/

36 10x10 m room, 8 targets, 4 azim. stations 0.5 pix 1 sigma 4 azimuth stations 10x10 meter room Worst direction position Worst direction orientation 1 sigma results 11/7/

37 The future In the words of Yogi Berra, I never make predictions, especially about the future,' Image sensors and image processing continue to develop quickly reduced cost sophisticated image processing of real images lenses and stability will remain challenges to accuracy Mobile cameras and lightweight infrastructure (scalability, infrastructure) Step 1: Reduce the number of required active targets, use natural features Step 2: sensor fusion with inertial, ranging camera, stereo camera, 11/7/

Creating a distortion characterisation dataset for visual band cameras using fiducial markers.

Creating a distortion characterisation dataset for visual band cameras using fiducial markers. Robert Jermy Council for Scientific and Industrial Research Email: rjermy@csir.co.za Jason de Villiers Council