A UAV MULTI-SENSOR RAPID MAPPING SYSTEM FOR DISASTER MANAGEMENT

Transcription

1 A UAV MULTI-SENSOR RAPID MAPPING SYSTEM FOR DISASTER MANAGEMENT Kyoungah Choi a, Jihun Lee a, Impyeong Lee a, * a Dept. of Geoinformatics, The University of Seoul, 90 Jeonnong-dong, Dongdaemun-gu, Seoul, Korea - (shale, fantest, iplee)@uos.ac.kr Commission IV, Working Group VIII KEY WORDS: Rapid Mapping, Disaster Response, Multi-Sensor System, UAV ABSTRACT: As natural and artificial disasters being increased, the demand for rapid responses for emergency situations also has been everincreasing. These disaster responses require geospatial information of the target areas in real-time or rapidly to manage the situation more effectively. We are in progress of developing a real-time aerial monitoring system based on a UAV to satisfy such requirements. Not only this system can access dangerous areas without interactions with human operators by employing a UAV as a platform but also can rapidly generate geospatial information of the sites through real-time transmission and processing of sensory data. This system consists of two main sectors: aerial sector and ground sector. The aerial sector includes a UAV platform equipped with different types of sensor (GPS/IMU/Camera/Laser Scanner) and sensor supporting modules for sensor integration, data transmission to the ground, data storage, time synchronization, and sensor stabilization. The ground sector includes a control/receiving/archiving subsystem, a data georeferencing subsystem and a spatial information generation subsystem. The development of this system is supported through Korean Land Spatialization Group by Korean government. As part of the four-year project, three years have been progressed. The development and testing of the individual modules and subsystems have been almost completed. Integration of the modules and subsystems is now in progress. In this paper, we will introduce the general concept of our system, including the main features, intermediate results, and expected outcome. 1. INTRODUCTION In recent years, disasters and accidents, such as the large scaled earthquakes happened in Japan and New Zealand, occur frequently and the extent of the damages is also huge. In those emergency situations, we need to build a rapid response system in order to minimize the damage (Choi et al., 2008). Moreover, the response is required to be customized to each individual site for more effective management of the emergency situations. These requirements can be satisfied with the decisions based on the spatial changes on the disaster area (Choi et al., 2009), which should be detected immediately or in real-time. Aerial monitoring without human operators is an appropriate mean because the disaster areas are usually inaccessible. Therefore, a UAV is a strong candidate as the platform of the aerial monitoring. The main advantage of rapid mapping systems based on a UAV is to acquire swiftly sensory data on the disaster area where ones can hardly access. For this reason, there have been many studies on the application of UAV systems to the disaster management. Eisenbeiss (2004) gave an overview about UAV systems and demonstrated their applications. MIRAMAP and E-Producties developed a UAV system with automatic triggering capability for rapid response operations (Haarbrink et al., 2005). For the responses to natural disasters, such as forest fire monitoring, A UAV-based data processing system was presented to rapidly generate the orthoimages of forest areas (Zhou et al., 2005; Zhou, 2006). Szendro fire department of Hungary has also conducted several real fire experiments using UAVs (Restas, 2006). Most of the systems developed from these previous studies do not include real-time transmission or processing of the data. Neither of them carries a laser scanner, which provides 3D point clouds for the rapid generation of the DEM of disaster areas. To overcome the limitations, we are developing a UAV rapid mapping system, which can transmit the sensory data promptly to a ground station and process the data automatically for the spatial information of the areas under emergency. This system is equipped with multi sensors (cameras, a laser scanner, GPS, IMU) and based on highly sophisticated software rather than a high-priced platform and sensors. In this paper, we will introduce the general concept of our system, including the main features, intermediate results, and expected outcome. 2. DISASTER MANAGEMENT SYSTEM We design a disaster management system in a national level, as shown in Figure 1. The disaster management system consists of six sectors, which are field, aerial, transmission, data processing, data analysis and data supply sector. If emergency situations such as earthquakes, floods, landslides and other possible disasters occur, the information of emergency area is transmitted from the field to the aerial sector. In the aerial sector, the sensory data of the emergency area is acquired by an aerial monitoring system. The sensory data will be transmitted to a ground station in real-time and be rapidly processed in order to obtain the spatial information of the area such as DEM/DSM, orthoimages, etc. The spatial information can be used to determine the appropriate emergency responses in the data analysis sector. Finally, the responses will be distributed from * Corresponding author.

2 the data supply sector to the field sector where there are individuals who need to evacuate from the area and rescue teams who have to save human life in the field sector. Figure 2. Overview of the real-time aerial mapping system 3.1 Aerial Sector Figure 1. Disaster management system based on real-time aerial mapping systems To promptly determine customized responses to the area and manage effectively the emergency situation it is important that the acquisition, transmission and processing of the sensory data should be performed in real-time. The three major sectors, which are aerial, transmission and data processing sector, can be supported by a UAV multi-sensor rapid mapping system. The UAV rapid mapping systems, which are equipped with GPS, IMU, a camera and a laser scanner, can acquire the airborne sensory data of the disaster area where people have a difficulty to access. In addition, the sensory data from the UAV systems usually have higher resolution than other system because the systems can operate at a lower altitude. If the transmission and processing of the data could be performed in real-time, the spatial changes of the damaged area can be detected with high spatial and temporal resolution by the UAV rapid mapping systems. As a result, we aim to develop a real-time aerial monitoring system based on a UAV, whose key features are the effective acquisition of the sensory data, real-time transmission and processing of the data. Our system will be introduced briefly in the next chapter. The purpose of the aerial sector is to acquire the sensory data and transmit the data to the ground sector in real-time. The aerial sector includes a UAV platform equipped with different types of sensor: GPS, IMU, camera, laser scanner and sensor supporting modules for sensor integration, data transmission to the ground, data storage, time synchronization and sensor stabilization. We select an appropriate UAV platform and sensors by considering our target applications for the disaster management Platform: We adopt Schiebel s Camcopter S-100 as the UAV platform, which can fly stably in unsettled weather, since the weather condition is usually windy when the disaster occurs. Moreover, its autonomy flight functions enable us to obtain the sensory data regularly. Its maximum payload is 50kg, so the platform can carry a laser scanner which relatively heavier than other sensors, such as a camera, GPS and IMU. This study aims at developing not a UAV system but a real-time mapping system. Therefore, this platform can be replaced with another model, which satisfies the requirements for system operations. The model is shown in Figure 3 and its main specifications are summarized in Table UAV MULTI-SENSOR RAPID MAPPING SYSTEM Our UAV multi-sensor rapid mapping system for the disaster management is composed of two main sectors, aerial sector and ground sector. The aerial sector performs data acquisition while the ground sector does real-time processing of the data. The sensory data and control commands are transmitted in real-time through a RF link between the aerial and ground sector. The overview of the entire system is illustrated in Figure 2. Figure 3. Camcopter S-100

3 Autonomy Endurance Power plant Typical D/L range Dash speed Cruise speed Typical payload Max. dimensions Fully autonomous takeoff, waypoint navigation and landing > 6 hours with 34 kg (75 lbs) payload plus optional external fuel tank extending endurance > 10 hours 50 HP rotary engine 80/180 km (43/97 nm) 120 kts 55 kts (for best endurance) 50 kg (110 lbs) 3110 mm (122 ) length 1120 mm (44 ) height 1240 mm (49 ) width Table 1. Main specifications of Camcopter S Sensors: This study focuses on the development of low cost, light and flexible system. Therefore, we employ inexpensive sensors and overcome their insufficient accuracy with our own sophisticated algorithm. Two kinds of CCD camera, a laser scanner, GPS and IMU are mounted on the UAV platform. One camera acquires image sequences. The other one supported by a gimbal is used to track a specified target. Both of them are medium format and non-metric. In order to produce DEM/DSM and orthoimages in real-time, our system should include a laser scanner. Riegl s LMS-Q240i is adopted and its measurement rate is 10,000 points/second. GPS and IMU provide initial EOP (Exterior Orientation Parameters) of the image sequences and laser points. We also select the GPS and IMU of medium grade. They are Novatel s OEMV3 and Honeywell s HG1700. OEMV3 has ±1.2 m and ±2.4 m as horizontal and vertical position accuracy, respectively. HG1700 s accelerometer bias is ±1 mg and gyroscope bias is ±1 deg/h. The main specifications of the sensors employed for our system summarized in Table 2. All the sensors are integrated with the sensor support modules and mounted on a rigid frame as shown in Figure 4. Figure 4. Configuration of the aerial sector Sensor supporting modules: We develop an OBC (On-Board Computer) and a gimbal. The OBC undertakes major five tasks: (1) sensor integration, which connects all the individual sensors; (2) transmission of data and control commands; (3) data storage; (4) image compression, which reduces the volume of image sequences to transmit in real-time; (5) time synchronization, which tags all the sensory data with GPS time. With the OBC, we can control all the sensors and obtain time tagged sensory data in real-time. The gimbal reduces the effects of vibration of the UAV platform on the camera and makes the camera look at a specific direction or track a particular target. The prototypes of OBC and gimbal are shown in Figure 5. In addition, there exists a power supply module, which supplies all sensors with appropriate amount of power. Component Model Important Specification Laser Scanner Digital Camera(A) Digital Camera(B) GPS IMU LMS-Q240i (Riegl) GD (Gevicam) XMV-16M (illunis) OEMV-3 (NOVATEL) HG1700 (Honeywell) weight : 7 kg, FOV : 80 (±40 ), scanning rate : 6~80 sps weight : kg, frame rate : 12 fps, effective pixels : , 3.4 μm weight : 0.47 kg, frame rate : 3 fps, effective pixels : , 7.4 μm position accuracy : 1.8 m, weight : kg data rate : 20 Hz velocity accuracy : 0.02 m/s, weight : 3.4 kg, data Rate : 100 Hz Table 2. Main specifications of the sensors Figure 5. Prototypes of the OBC and gimbal 3.2 Ground Sector The goal of the ground sector is to receive the sensory data from the aerial sector in real-time and generate spatial information rapidly. The ground sector is composed of a control/receiving /archiving subsystem, a data georeferencing subsystem and a spatial information generation subsystem. The sensory data from the aerial sector are transferred to the data georeferencing subsystem and stored archiving subsystem. Then, PAD (Position Attitude Determination), image georeferencing and LiDAR georeferencing are conducted in real-time at the georeferncing subsystem. With the georeferenced data, DEM and orthoimages are generated rapidly at the spatial information generation subsystem. The outline of the data processing in the ground sector is shown in Figure 6. Each subsystem will be explained briefly in following subsections.

4 Figure 6. Outline of the data processing in the ground sector Georeferencing subsystem: Georeferencing is to determine the EOP of sensory data, such as images and LiDAR data. To generate meaningful spatial information from the image sequences and laser point clouds acquired by our system, the georeferencing is one of the most important tasks. The georeferencing subsystem in our system is developed by integrating PAD, image georeferencing and LiDAR georeferencing. In the emergency situation, the processing time is more critical than the accuracy of the results. As a consequence, we should perform georeferencing in real-time. Moreover, we have to improve the low accuracy of the sensors through indirect georeferencing since we employ inexpensive sensors for a low-cost system, In our study, we develop a realtime indirect real-time georeferencing algorithm, where the data flows are illustrated in Figure Control/receiving/archiving subsystem: We construct a mobile ground station, which includes the ground vehicle loading several computers, data storage devices and communication equipment. The ground vehicle is a 2.5-ton truck and shown in Figure 7. We remodel the truck to accommodate some operators and devices. The communication equipment covers transmitting the control commands, receiving the sensory data and GUI (Graphic User Interface) for the operators. There are an antenna, tracking module and modem as the communication equipment. Their main specifications are listed in Table 3. Figure 8. Data flows in the georeferencing subsystem The algorithm consists of major three steps: integrating GPS/IMU for PAD, adjusting position/attitude information by sequential AT, and georeferencing laser scanning data. First, the GPS data and IMU data are integrated by EKF to provide initial EOPs of the image sequences. Second, the initial EOPs are adjusted by the sequential AT whenever a new image is acquired. Third, the laser scanning data is georeferenced by using the more precise EOPs. Finally, we can obtain more precise georeferenced image sequences and laser point clouds from the sensors with lower performance. In addition, the procedure is conducted in real-time without any GCP, which is hard to be surveyed or found in the disaster area. Component Antenna Tracking module Modem Figure 7. Mobile ground station Specification Type : Helix feeder ADE antenna / 2 axis link Size : 600 mm (diameter) Frequency band : 5.2 ~ 6.2 GHz Moving range : Elevation range : 0 ~ 90 deg Azimuth range : 450 deg Tracking rate : 30 deg/sec Actuator type : 2 axis step motor with timing belt Interface : Host PC: Ethernet, Console, RS-232 Antenna : N-type RF Connector Frequency : 5.72 ~ 5.82 GHz Receive sensitivity : Mbps Bandwidth : 20 MHz Effective speed : 10 Mbps Spatial information generation subsystem: Main products of the spatial information generation subsystem are DEM and orthoimages. Those products should be generated quickly for the disaster management. For rapid generation of them, we produce rough DEM using laser point clouds and orthorectify the georeferenced images using the DEM. In our process, rough DTM is generated through four steps, which are removing outlier, filtering terrain points, creating grid, and interpolation. Then, the orthoimages are produced by mapping the georeferenced images onto the rough DTM. Figure 9 describes the data processing in the subsystem. Table 3. Main specifications of the communication equipment

5 4.2 Data and Processing With our system, we could acquire about 200 images, 2,000,000 laser points, 192 GPS data and 1890 IMU data through two strips. According to the focal length of the selected lens, 50 mm, the image scale is 1:4000. The ground resolution of the images is about 3 cm, and the overlap ratio is about 88%. The density of laser points is 4.16 points/m 2. The main parameters of the data acquisition are shown in Table 4. All the data is synchronized with the GPS time. Therefore, the position/attitude data provided by the GPS/IMU are interpolated at the data acquisition rate and used for the EOP of the images and laser points. The acquired images and laser points georeferenced with the position/attitude data, are shown in Figure 12and 13, respectively. Figure 9. Outline of the whole data processing 4. INTERMEDIATE RESULTS We are still progressing in developing the UAV rapid mapping system. While most of the hardware in the system is constructed, the development of the data processing software is not completed. Hence, we introduce the intermediate results of this study in this chapter. Data Image LiDAR Parameters Single Image Coverage : m Ground resolution : 3 cm Overlap : 88 % Density : 4.16 points/ m 2 Scan width : 50 cm Scan rate : 30 lines/second Table 4. Main parameters of data acquisition 4.1 Experimental Test The test field is a small portion of Chungju in Korea and its area is approximately 300ⅹ1600 m 2. The test field includes residential zone, agricultural land and river. The flight height and velocity are about 200 m and 60 km/h, respectively. Figure 10 shows the range of the field and the flight trajectory of the test. In Figure 11, the flying system is shown. Figure 12. Images acquired by our system Figure 10. Test field and flight trajectory Figure 13. Georeferenced laser point cloud Figure 11. Our UAV rapid mapping system over the test field

6 5. CONCLUSIONS In this study, we introduce a UAV rapid mapping system for disaster management. The rapid mapping system consists of the aerial sector and the ground sector. The aerial sector covers a UAV platform, different types of sensor (GPS/IMU/camera/ laser scanner) and sensor supporting modules. The sensory data acquired by the aerial sector are transmitted to the ground sector. In the ground sector, which includes control/receiving /archiving subsystem, a data geo-referencing subsystem and a spatial information generation subsystem, the sensory data are processed in real-time and the spatial information is generated promptly. IEEE International, Geoscience and Remote Sensing Symposium, pp Restas, A., Wildfire Management Supported by UAV based Air Reconnaissance: Experiments and Results at the Szendro Fire Department, Hungary. In First International Workshop on Fire Management. We performed an experimental test for evaluating the performance of our system. Using our system, we could acquire aerial sensory data such as images, laser points data, GPS and IMU data in real-time. We also inspected the data and conducted georeferencing laser scanner data. Regarding the results of the experiments, we could obtain the sensory data with high resolution, which are useful to manage disasters. In the near future, we can obtain the DEM and orthoimages on the disaster area in real-time with completely developed data processing software. It is expected that this system will be applied to the management of various emergency situations, such as understanding the situations and rescuing lives. ACKNOWLEDGEMENTS This research was supported by a grant (07KLSGC03) from Cutting-edge Urban Development a Korean Land Spatialization Research Project funded by the Ministry of Land, Transport and Maritime Affairs. REFERENCE Choi, K., Lee, I., Shin, S.W. and Ahn, K., A project overview for the development of a light and flexible rapid mapping system for emergency response. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 37-B5, pp Choi, K., Lee, I., Shin, S.W. and Park, W., Developing a UAV-based rapid mapping system for emergency response. Proceedings of SPIE Defense, Security, Sensing, Orlando, FL, USA, Eisenbeiss, H., A mini unmanned aerial vehicle (UAV): system overview and image acquisition. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 36-5/W1, on CD-ROM. Haarbrink, R. B. and Koers, E., Helicopter UAV for photogrammetry and rapid response. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS Workshop of Inter-Commission WG I/V, Autonomous Navigation, Antwerp, Belgium. Zhou, G., UAV Real-time Data Processing through Flight Trajectory and Resample Rate Analysis. ASPRS 2006 Annual Conference, Reno, Nevada. Zhou, G., Li, C. and Cheng, P., Unmanned Aerial Vehicl (UAV) Real-time Video Registration for Forest Fire Monitoring.