Center for Advanced Imaging LADAR. A 3D Texel Camera

Transcription

1 Center for Advanced Imaging LADAR Utah State University A 3D Texel Camera Kylee Sealy Industry Day 12/28/2006

2 Problem/Opportunity Existing 3D Cameras: > Systems are relatively slow and awkward > Hard to get error free color 3D images > Do not work well at long distances > Scene/camera motion creates irrecoverable errors Mass Market Acceptance Requires: > Streamlined speedy operation > Error free operation > Ability to work at a variety of distances > Able to handle scene/camera motion > Lower cost

3 CAIL 3D Texel Camera LIDAR (LIght Detection And Ranging) > Laser Radar > x-y-z point cloud of scene > Techniques Time of flight Spatial resolution dependent on laser spot size on target Range resolution dependent on ability to accurately measure the time of flight Modulated continuous wave Electro-Optical (EO) or digital camera > Texture elements (Texels) of scene The method discussed is USU owned IP protected by U.S. Patent 6,664,529, 3D Multispectral Ladar.

4 CMOS Sensor and Processing

5 Temporal Decorrelation Illustrated time = T S Field-of-View time =T EO Flying Spot Lidar Scanner time = T E Independent Frame Imager TIMELINE ΔT = up to several seconds ΔT ΔT T S T EO = time image is taken time it takes for flying spot scan to be completed T E

6 Real-time Correlation Illustrated The Texel Camera uses a slaved frame (or line) imager that follows any LIDAR scan pattern in the field-of-view. T S Field-of-View T 1,1 T 10,1 Sequential flying spot from T S through T E. slaved Sequential pixel read-out from T 1,1 through T 10,10. T E T 1,10 T 10,10 Flying Spot Lidar Scanner Slaved Imager time for pixel readout sequence to be completed These Match so ΔT is small T 1,1 T 10,10 T S time it takes for flying spot scan to be completed T E

7 Example from Topographic Prototype Occluded areas common to the LIDAR and imager Open Pit Mine Wall Face: Upper colored LIDAR shots; Lower native texel dataset.

8 Same Example, New Point-of-View Same data as previous slide, viewed from a different perspective in 3D virtual space. Note how Texel data is carried with the LIDAR data.

9 Our Technology Our work is based on laser radar (LADAR). It can create error free 3D color models of real life scenes GREAT DEVICES EASY WORKFLOW INTEGRATION GROWING THIRST FOR 3D DATA PRODUCTS

10 Price Positioning Unit prices are compatible with current systems that have no or limited color capabilities. $2.5M $500k $150k PRICE $50k $10k CAPABILITY

11 Long Distance Tripod Product Mountainside in Logan Canyon Data & Display Software by CAIL Utah State University

12 Long Distance Tripod Product soil horizon bedrock OUR COLOR DETAIL scan tripod location Data Produced by CAIL Utah State University COMPETING DETAIL Display Software by Rappid Mapper Inc. 3D Rockslide near Ojiya, Japan

13 Medium Distance Tripod Product 340m Data Produced by CAIL Utah State University Ottinger Hall, Salt Lake City Display Software by Rappid Mapper Inc.

14 Short Distance Tripod Product 340m Data Produced by CAIL Utah State University Display Software by Rappid Mapper Inc. 3D Car in Kennecott Mine, Utah

15 Increasing Need for 3D Pictures in the Market Place 340m Movie and Gaming Industry > Inserting real-life scenes into 3D computer graphics Engineering Design and Land Development > Industrial plants, architecture, real estate, hazards analysis Law Enforcement / Crime Scene Analysis > On-the-spot documentation of evidence Utilities and Infrastructure Management > Includes documentation for insurance claims Manufacturing Quality Control and Assy Lines > Large to small object tracking and analysis Mapping, Targeting and Navigation > Military, shipping, aircraft and robot applications

16 CAIL IP Position Patent 3D Multispectral Lidar, Dec Other Patents in Process

17 Problems We Uniquely Solve With Our IP Faster acquisition of 3D color imagery > Other systems use fusion methods that require time-consuming error correction Real-time image viewing and analysis > We can view and QC data in the field we are the only ones Handles camera and or object motion > Errors are not introduced when moving as with other systems Can provide instantaneous earth coordinates for any object > Important in tactical military operations and land development Compact, lightweight, portable > We can be as small and portable as any other system

18 Current Commercial Products Tripod based Prototype 1 > Based on Riegl Lidar and Mikrotron CMOS camera > Lidar is mounted on two axis motion platform > Lidar and camera are looking at the same location all the time > For each Lidar shot, a 20 x 20 patch of pixels is captured centered around the Lidar shot at the same instant in time > Maximum range is on the order of 400 to 800 meters > About 800 Lidar shots per second relatively slow > Lidar data captured to portable PC over serial link > EO data captured to portable PC over CameraLink interface > Wireframe and associated texture information combined in post process to produce true 3D textured model > Used extensively by RappidMapper for preliminary business development

19 Current Commercial Products Tripod based Prototype II > Based on Riegl Z210i Lidar and ISG CMOS camera > Lidar rotates in azimuth while elevation scanning is performed with a mirror that rotates 20 times a second > Lidar and camera are looking at the same location at the same time > Lidar vertical scans occur 20 times per second. > A 2048 high by 16 wide slice of CMOS image is captured 60 times per second. Lidar scans and EO slices are time tagged as captured > Post processing aligns the time tagged data for temporal correlation > About 20,000 Lidar shots per second much faster > Lidar data captured to laptop PC over Ethernet > EO data captured to laptop PC over IEEE1394 (Firewire) interface > Wireframe and associated texture information combined in post process to produce true 3D textured model > Used exclusively by RappidMapper in business development and service model

20 Current Commercial Products Tripod based Prototype III > Based on Riegl Z420i Lidar and ISG CMOS camera > Very similar to Prototype II except that beam divergence is less. Gives better spatial resolution > A modified Lidar supplies a sync signal which is used to temporally correlate the capture of CMOS image slices. Lidar and texture data are now temporally correlated by default > Was delivered to Rappid Mapper in January 2006 > They expect to order 10 or 12 more over the next year > A company in SLC is partnering to design and build integrated Texel cameras for the commercial market based on the CAIL licensed technology

21 CEROS State of Hawaii program created in 1993 under a grant provided by DARPA Solicits proposals through annual BAAs Supports DoD technology requirements Encourages R&D in ocean sciences and technology in Hawaii Focus Areas > Ocean environmental preservation > Shallow water surveillance technologies > New ocean platform and ship concepts > Ocean measurement instrumentation > Unique properties of the deep ocean environment

22 Project Concept Compact Hydrographic Airborne Rapid Total Survey (CHARTS) Image courtesy of Naval Oceanographic Office

23 Existing CHARTS System SHOALS-3000T20E > 3,000 Hz bathymetric LIDAR > 20,000 Hz topographic LIDAR 1 Hz DuncanTech DT 4000 digital camera (RGB) CASI-1500-O hyperspectral imager

24 Current RGB Data Acquisition Between August and December 2000 the CHARTS predecessor, SHOALS, conducted coastal surveys of Hawaii. The figure below shows a series of example digital images (RGB) acquired along one of the Hawaiian coasts. The series of frames were post-processed into a mosaicked, co-registered, geo-referenced map (orthophoto map), a process we understand took several months for the project and was labor intensive. Technical improvements to CHARTS over SHOALS in terms of data precision and system speed are significant, however, the digital imager component of the system is operated fundamentally in the same way as before.

25 Temporal and Spatial Decorrelation The RGB frame imager takes snapshots during the LIDAR flying-spot scan. For a given spot in the scene > Reflected laser radiation is collected by the LIDAR receiver at one point in time. > Passive light is collected by the frame imager at another point in time. The time difference is unavoidable when a snapshot is taken sometime during an on-going flying-spot LIDAR scan. This temporal decorrelation, ΔT, requires post-mission processing to align the imagery accurately with the 3D point cloud. Negative 1: Image rectification will be expensive and time consuming. Negative 2: Spatial correlation can fail when sensor or object motion causes objects in the scene to come into and out of the field-of-view (FOV) during the decorrelation time ΔT. This is especially true around sharp boundaries. Independent 1 Hz Fame Imager 3000 Hz Lidar Scan Existing CHARTS System

26 Avoiding Temporal and Spatial Decorrelation Employ a Texel* Camera, which uses a slaved frame, line, or patch imager that follows the LIDAR scan pattern within the FOV. For a given spot in the scene > Reflected laser radiation is collected by the LIDAR receiver at one point in time. > Passive light is collected by the frame imager at the same time as the laser radiation, and from the same place in the FOV. > Temporal correlation is obtained. ΔT is zero or very small. No need for post-mission image rectification. The imagery is automatically fused with, and carried by, the LIDAR 3D point cloud. Positive 1: Spatially correlated image/lidar datasets are produced in real time. Saves post-mission processing time and expense. Positive 2: Spatial correlation is always successful, even when sensor or object motion causes objects to come into and out of the field-of-view at different times. Slaved 3000 Hz Patch Imager * texel = texture + pixel Texel Camera Installed in CHARTS System

27 Goal of This Project The CHARTS data will be utilized in the same way to create a texel image of the sea floor (the above is not real data).

28 Proposed Replace existing CHARTS DuncanTech digital imager with Texel Camera. Slave Texel Camera to CHARTS system Run data collection flights. Demonstrate real-time capability. Analyze alignment quality. Produce bathymetric maps. Fit and form test completed Actual flights and data gathering to be done in Hawaii in April or May

29 Future Projects and Products Short range high speed tripod scanner > Integrate a CMOS imager with a short range continuous wave Lidar > Very high speed scanning (50kHz) for short range (60 meter) applications > Smaller > Less expensive > Very fast

30 Future Projects and Products Airborne Commercial Texel Camera > Take what we have learned in the development of the commercial tripod products and apply the techniques to airborne applications > Further refine techniques and equipment for UAV applications

31 Future Projects and Products Handheld inexpensive Texel camera > Integrate a CMOS imager with a short range flash type Lidar > Very high speed scanning (30 frames/sec) for short range (<10 meter) applications > Very small and light. Truly handheld > Very inexpensive (eventually in the hundreds of dollars) > Very fast > The ability to capture freeform snapshots and video and stitch together the point clouds algorithmically > Large number of applications Automotive Facial and feature recognition Rapid architectural modeling 3D modeling of arbitrary objects. etc