Perspective Sensing for Inertial Stabilization Dr. Bernard A. Schnaufer Jeremy Nadke Advanced Technology Center Rockwell Collins, Inc. Cedar Rapids, IA
Agenda Rockwell Collins & the Advanced Technology Center GPS Denied tradeoffs & approaches Perspective Sensing for Inertial Stabilization (PerSIS) PerSIS algorithm & processing Feature identification & tracking Breadboard description Breadboard performance Future work & conclusions Questions 2
Rockwell Collins: What We Do Aviation Electronics and Communication Systems for Commercial and Military Applications Worldwide Government Systems ~51% Commercial Systems ~49% Communications Navigation Automated Flight Control Displays / Surveillance In Flight Entertainment Integrated Aviation Solutions Information Management Aviation Services 2007 Sales: $4.4 Billion 3
Advanced Technology Center Overview Charter: Identify, Develop and Mature Technologies that Provide Growth for Rockwell Collins Technology Levels: TRL-2 to TRL-6 Support Business Units in Evaluation of Advanced Technology Partner with Companies and Universities for Advanced Research Key Technology Areas: Networked Communication Systems Radio Systems Navigation and Control Avionics & Cabin Systems ISR System Technologies Information Assurance Info & Computing Systems 10X Breakthroughs 4
GPS Denied Tradeoffs & Approaches Multi-Dimensional GPS Denied Factors Systematic GPS Avail. Clear RFI Jamming Extreme Space Weather Environmental GPS Avail. Open Terrain Urban Canyon Loss of Satellites Small Building Large Building Cave Time Scale Seconds Minutes Hours Days Forever Geographic Scale Building Block Neighborhood City Region Global Type of Navigation Relative Absolute Platform / Operation Dismounted Warfighter Ground Vehicle Small UAV Manned Aircraft / Large UAV Ship Other factors are important too: cost, deployability, Corner cases can make problem virtually intractable 5
High-Level GPS Denied Technology Assessment Building Neighborhood City Region Geographic Scale 10 5 m 10 4 m 10 3 m 100 m 10 m Image Dead-Reckoning MEMS IMUs Nav-Grade IMUs CSAC-Aided GPS Image Map-Matching RF Ranging Networks 1 s 10 s 100 s 10 3 s 10 4 s Time Scale Need assessment for different environments System solution required Image-Dead Reckoning provides utility on small scale / short timeframes Image-Map Matching provides utility on longer timeframes and larger geographic scales Hard indoor environments will likely require RF ranging augmentation No single technology will solve the problem 6
IMU vs. Image Navigation Performance Tradeoff MEMS-quality IMU Error grows as a function of time Free drift, no ZUPTs Image-based Nav Error grows as a function of distance traveled Analysis assumes 0.3% of distance traveled Contours indicate additional seconds that image-based navigation can meet Allowable Nav Error for a given constant User Velocity vs. IMU-only sensor Image-based Nav very effective for first-responder dynamic environment 7
Perspective Sensing for Inertial Stabilization (PerSIS) Use stereo image position updates to stabilize IMU drift Allows use of low-cost IMUs and low-cost stereo image sensors Inertial sensor provides orientation and coasting between image captures and image-based position updates PerSIS not limited to stereo optical sensors y z x 8
PerSIS Algorithm & Processing Operational environment Feature reuse may not be likely or easy to accomplish PerSIS design approach is less rigorous than pure SLAM solutions but optimized to be a practical solution PerSIS was motivated by a paper that did not dwell on the mapping aspect as much as treated features as transitory landmarks. Results in lower computational complexity The processing approach of the stereo image feature information for navigation correction updates is based on well-understood principles of photogrammetry PerSIS integrated navigation algorithms based on existing Rockwell Collins aided-imu designs Standalone extraction of image information for positioning, but integrated as slow-rate aiding data to the Aided Inertial system. Variant of the Extended Kalman Filter used with a distributed structure that is amenable for real-time embedded operation Utilizes sequential KF processing of the image-based feature data as opposed to batch least-squares computation 9
PerSIS Navigation Algorithm Flow Implemented as a Schmidt Kalman Filter for computationally efficient sequential processing of image features 10
IMU and Image Data Processing IMU and image data is collected with breadboard and postprocessed with custom feature tracking algorithm and PerSIS Simulink/Matlab simulation Acquire Stereo Images Feature ID & Tracking PerSIS Nav Strapdown Aided INS processing Integrated Position Output IMU 11
Feature Identification & Tracking Need to reliably find the same feature in consecutive images OpenCV Open Source Image Processing library Off-the-shelf processing components utilized to develop customized feature identification and tracking to minimize development time PerSIS image processing approach is currently based on a modified Kanade-Lucas-Tomasi optical flow algorithm Basic processing flow Acquire features in left image Identify corresponding features to right image Features sent to PerSiS to calculate position Time Acquire new features in left image Track features while moving and remove outliers Identify corresponding features to right image Features sent to PerSiS to calculate position 12
PerSIS Breadboard Description Videre Design Stereo Camera System Micron CMOS Imagers High sensitivity, low noise Low pixel cross-talk Rolling shutter Fully synchronized stereo High frame rates 30 Hz for 640x480 (We're using 15 Hz for our feature tracking processing.) Focal length: 4.0 mm Monochrome IEEE 1394 interface to standard PC hardware Systron Donner MMQ-50 IMU MEMS Quartz technology, 3 axes angular rate, 3 axes acceleration Less than 9 cu in Bias stability of less than 10 deg/hr Angle random walks less than 0.15 deg/root hour (typical) RS-232 temp compensated outputs, 450 Hz measurements 13
PerSIS Breadboard MMQ-50 IMU Stereo Camera Power Supplies Data Collection Laptop 14
Breadboard Performance (1) Actual endpoint: (0, -31.4, 0); PerSiS: (-0.1, -31.5, 0.36); Distance traveled: 31.4m Error of 1.3% of distance traveled 5 0 PerSIS Nav Sol. True Path Northing in meters -5-10 -15-20 -25-30 -35-5 0 5 10 Easting in meters 15
Breadboard Performance (2) Actual endpoint: (-9.4, -4.1, 0); PerSiS: (-9.3, -4.15, -0.05); Distance traveled: 13.5m Error of 1.0% of distance traveled Northing in meters 2 0-2 -4 PerSIS Nav Sol. True Path -6-10 -8-6 -4-2 0 2 Easting in meters 16
Future Work & Conclusions Future improvements to improve accuracy and integrity of the solution Collect IMU and image for body-worn stereo image and IMU sensors Cameras with higher dynamic range (current camera is 7 bit black & white) Evaluate other image sensors, e.g., LIDAR and/or thermal imagers, for operation in smoke and low-light indoor environments Image segmentation and intelligent feature selection to ignore nonstationary objects (people, etc ) Conclusions PerSIS image navigation is an effective dead-reckoning approach to limit IMU drift for body-worn walking/running applications Drift performance of better than 1% of distance is realizable 17
Questions? 18