Technical report from Automatic Control at Linköpings universitet The ARCUS Planning Framework for UAV Surveillance with EO/IR Sensors Per Skoglar Division of Automatic Control E-mail: skoglar@isy.liu.se 31st March 2009 Report no.: LiTH-ISY-R-2885 Address: Department of Electrical Engineering Linköpings universitet SE-581 83 Linköping, Sweden WWW: http://www.control.isy.liu.se AUTOMATIC CONTROL REGLERTEKNIK LINKÖPINGS UNIVERSITET Technical reports from the Automatic Control group in Linköping are available from http://www.control.isy.liu.se/publications.
Abstract This report gives an overview of the planner framework developed in the Arcus project. The framework consists of a number of planning modules and planning modes that are introduced. Keywords: ARCUS, UAV Surveillance, Planning
Contents 1 Introduction 3 1.1 The Arcus project.......................... 3 1.2 Background Discussion........................ 3 1.2.1 Autonomous Planning of a UAV with EO/IR Sensors.. 4 1.2.2 Planning Hierarchy...................... 5 1.3 Objective and Outline........................ 5 2 Software Framework 6 2.1 Matlab Simulation Framework................... 6 2.1.1 Scenario Engine....................... 7 2.1.2 World Simulator....................... 7 2.1.3 Target Tracking Filters................... 7 2.1.4 Planner............................ 7 2.1.5 Monitor............................ 7 2.1.6 User Interface......................... 7 2.1.7 Interactions.......................... 7 2.2 MSSLab Planning Module...................... 8 2.2.1 Introduction to MSSLab................... 8 2.2.2 The MSSLab Planner Federate............... 8 3 Planning Architecture 9 3.1 Receding Horizon Planning..................... 9 3.2 Sequential Planner Structure.................... 9 3.3 Sequential Planner Structure with Modes............. 9 3.4 Planning Modes............................ 10 3.4.1 Mode A; Uncertain Target in View Update........ 10 3.4.2 Mode B; Track Update Request............... 10 3.4.3 Mode C; Multi Target Search and Update......... 11 3.4.4 Mode D; Search and Exploration.............. 11 3.5 Planning Modules.......................... 11 3.5.1 Planner Module 1; Optimal Estimation Performance (OEP) Planner............................ 11 3.5.2 Planner Module 2; Information Exploration........ 12 3.5.3 Planner Module 3; Aim Planning of Gimballed EO/IR Sensor for Multi Target Tracking (MTT)......... 12 3.5.4 Planner Module 4; Geo-Tracking with Gimballed EO/IR Sensor............................. 12 1
4 Simulation Results 13 5 Conclusions 16 2
Chapter 1 Introduction The need for autonomous capabilities, such as on-board sensor data processing, sensor management, and path planning, will increase in both manned and unmanned platforms designed for future applications. This arises from the constantly growing quantity of sensors and associated raw data, as well as limitations in communication bandwidth and processing capacity of human sensor operators. Several basic functionalities of autonomous surveillance systems, e.g., target geo-location, robust navigation, collision avoidance, route and viewpoint planning all require advanced visual capabilities like target and landmark recognition, scene topography estimation, and image-motion computation. In order to raise the level of autonomy in these systems, it is necessary to take into account the uncertainty associated with the percepts of a cluttered and rapidly changing environment. These uncertainties arise from sensor noise, navigation errors, matching errors, prior knowledge model errors, and target prediction errors. 1.1 The Arcus project The Arcus project (Autonoma Reaktiva Certifierbara UAV-System) is a project in the TAIS-programme (Teknologier för Autonoma Intelligenta System) that is managed by FMV (Swedish Defence Material Administration) and VINNOVA. ARCUS is headed from FOI and partners are Linköping University (Automatic Control at the department of Electrical Engineering), Saab Aerosystems and Saab Bofors Dynamics. The goal is to demonstrate an enhanced level of surveillance capability including ground target detection, tracking and re-identification of moving objects without the direct control of a human operator. The demonstration will be within a HLA-based simulation environment developed at FOI. 1.2 Background Discussion All planning and control problems are in fact some kind of optimization problem where the goal is to minimize (or maximize) a criterion by choosing the right actions. Planning can be defined as the task of finding a sequence of actions that will achieve a goal [6]. Classical (AI) planning only considers environments 3
that are fully observable, deterministic, discrete and static. However, to be able to act in the real world with non-perfect sensors, the planner have to deal with incomplete, uncertain, and incorrect information in a robust manner. In classical control the goal is to control the behavior of a dynamic system. A model of the system is used in the design of a regulator that uses feedback to control the system in a desired way. Control design for linear systems and Gaussian noise is a well understood research area, but the problem is harder for nonlinear and non-gaussian systems. [3] provides a survey of different planning algorithms and [6] is a good introduction to planning as seen from the AI community. Planning and control algorithm suitable for our planning problem can be formulated as an optimal control problem [1, 2]. See the surveys [4] and [7] for more related references. 1.2.1 Autonomous Planning of a UAV with EO/IR Sensors Autonomous concurrent sensor and path planning of a UAV with EO/IR sensors, taking into account both platform and sensor constraints, as well as threats and environmental conditions, is a very challenging problem. Even more demanding, but still necessary, is the capability to dynamically adapt and replan the sensor utilization and the platform trajectory in response to changes in the environment as well as internal state, given feedback from new sensor data. Realistic models of the environment, sensors and platforms are very complex, due to the non-linear and stochastic properties of the world. Hence, algorithms and methods solving realistic planning problems are computationally very demanding. Furthermore, the optimal solution is in practice impossible to find, but this is not critical since there are, in principle, an infinite number of solutions that are sufficiently good. However, the problem of finding a sufficiently good local optimum is still very hard. Defining the planning problem is not straightforward. There are several objectives with different importance that also may vary from time to time, and from mission to mission. Two different planning algorithms can give completely different solutions to a surveillance and multi target tracking problem, since they focus on different aspects of the problem. In fact, even two identical algorithms can give different solutions depending on the planning horizon and due to the problem discussed above with the problem of finding the optimal solution. Different objectives of the planning problem that needs to be maximized are, among others, the number of simultaneously tracked targets, the number of found targets, the total surveyed area, the estimation performance of each target, the imagery quality for better association, classification and identification, energy-loss, i.e. minimizing fuel consumption, stealth behavior, 4
navigation performance. Most of these different utilities are conflicting and it is obvious that depending on the focus, the planning yields completely different results. The focuses in the Arcus project are on multi target tracking, area coverage, and estimation performance. Aspects like energy, stealth and navigation are ignored. 1.2.2 Planning Hierarchy Since the planning problem is very complex, we are forced to make simplifications. A monolithic planner is probably an unrealistic goal. Even if we could define a suitable objective function, this function has no obvious structure that can be utilized in the optimization and therefore it is also very hard to solve and analyze. Instead a hierarchical decomposition is required, but the question then is how to decompose the problem into sub-problems that guarantee that the overall objective is achieved. 1.3 Objective and Outline The objective of this report is to give an overview of the planner framework developed in the Arcus project. The conclusion of the discussion in Section 1.2 is that a hierarchical decomposition of the planner is needed. The planning architecture used in the project is presented in Chapter 3 and it contains a number of planning modules. The presentation of the modules is very brief, and the reader is directed to the references for details. Chapter 2 presents the software frameworks where the planner is developed and integrated. Finally, Chapter 4 shows the result from a simulation example. 5
Chapter 2 Software Framework 2.1 Matlab Simulation Framework A object-oriented simulation framework has been developed in Matlab, Figure 2.1 shows an overview. Important classes are, e.g., a world simulator, a Filter State Monitor Planners Measurements Plan GUI Simulation Nav State Environment Figure 2.1: The Matlab simulation framework. tracking filter module, and a planner. Furthermore, there are two classes for monitoring and controlling the simulations. The most important (super-)classes are presented briefly next. 6
2.1.1 Scenario Engine The ScenarioEngine is the heart of the simulation framework. It contains the main loop that drives the whole simulation. Every scenario description or simulation problem that will be run is defined in a subclass to the Scenario Engine. 2.1.2 World Simulator The world Simulator contains models of the sensor platform and the sensor gimbal simulating the platform and the aim trajectories given the plan. These trajectories together with the target trajectories, generated in the target simulator, and the environmental model are used to simulate measurement of the targets. The targets can be on-road or off-road targets and the environmental model contains information about the road network, the ground elevation and the terrain occlusion. 2.1.3 Target Tracking Filters The target tracking filter module contains a number of single target tracking filters, in this first version the association problem is simplified and the target identity is assumed to be given by the detection algorithm. The tracking filter can be of different filter types, Extended Kalman filter (EKF) or Unscented Kalman filter for off-road targets and Bootstrap Particle filter (BSPF) and Rao- Blackwellized Particle filter (RBPF) [12] for on-road targets. Furthermore, an Interactive Multiple Module Particle filter (IMM-PF) is developed for handling targets that can be either on-road or off-road [5]. All implemented filters are sub-classes to a general filter class Filter. 2.1.4 Planner The structure of the planner class Planner is presented in Chapter 3. 2.1.5 Monitor The Monitor class is a utility that stores simulation data and shows the result in plots. This class can also run a re-play of the simulation and create AVI-movies. 2.1.6 User Interface The Graphical User Interface (GUI) is a container for GUI-modules from all main classes. For instance, the GUI of the ScenarioEngine has buttons for running and stopping the simulation. The Monitor-GUI part contains a plot and an interface for selecting what will be shown in the plot. The Planner-GUI contains, e.g. buttons for manually selecting which planning module that should be run, and much more. 2.1.7 Interactions The data that is transmitted between the classes are called interactions. Several different interaction types are needed, e.g., the plan from the planner is a 7
PlanInteraction, the filter states from the tracking filters are FilterInteractions, and so on. 2.2 MSSLab Planning Module 2.2.1 Introduction to MSSLab Multi Sensor Simulation Lab (MSSLab) is a framework for simulation of advanced sensor systems developed at Information Systems, FOI (Swedish Defence Research Agency), Linköping. With MSSLab different sensor systems can be assessed in different environments, in various weather conditions and time of day. The framework makes it possible to simulate dynamic scenarios with moving platforms, sensor carriers, targets and so forth. High quality sensor simulations can be made in either IR, visual, laser and radar individually or combined as a multi sensor system. Advanced signal processing, tracking algorithms and data fusion methods are also integrated in the simulation tool. In MSSLab different simulation programs have been integrated in a HLA-environment (High Level Architecture). The simulation programs in MSSLab are not designed for real time simulations but are focused on realism and signal quality. 2.2.2 The MSSLab Planner Federate A federate is a simulation module in the MSSLab/HLA-framework. The Planner federate is written in Matlab and is essentially the Planner class in the Matlab framework together with an interface MSSLabPlannerInterface including transformation of coordinates and interactions. 8
Chapter 3 Planning Architecture A hierarchical decomposition of the planner is needed, due to the complexity of the planning problem. In this chapter we present a planner architecture with different planning modes. This architecture is not the "final solution" to the planning decomposition problem, but serves our needs in the ARCUS project well. The building blocks of the framework are a number of planning modules. The modules solve well-defined subtasks and they are briefly introduced in this report, for the details we refer the interested reader to the given references. 3.1 Receding Horizon Planning The planner is working in a Receding Horizon Control (RHC) manner, i.e., the planning horizon is T plan, but only the first part, with length T exe, are executed before a replanning is performed, and so on. Thus, T exe are less or equal to T plan. This is a rather conservative planning strategy, since in each planning step we assume that no more information will be obtained that can aid the planning in the future. See the discussion in [11] where open-loop-feedback-control (OLFC) [2] is introduced. 3.2 Sequential Planner Structure A simple hierarchy consists of a number of planner modules in a sequence. For instance, the first planner module creates a plan for the trajectory of the sensor platform and a second planner module adds trajectories of the sensor gimbal. However, it is difficult to handle more complex scenarios with this structure if we want the planner modules to be rather "pure". 3.3 Sequential Planner Structure with Modes An extension of the sequential planner can be obtained by having several parallel sequential planners. Each planner sequence is called a mode and the modes are ordered according to priority. Figure 3.1 shows a general sequential planner structure with a number of different modes. Only one mode is active in each planning epoch and which one is determined by a mode function. First the mode 9
Figure 3.1: A general sequential planner structure with modes. function of the highest priority mode is evaluated (top left in Figure 3.1). The output is either "yes" or "no", if "yes" then that planner sequence is executed, if "no" the mode function of the next mode is evaluated, and so on. A more concrete example of a sequential planner with modes is shown in Figure 3.2. In this example the first mode is "Track Update Request" and that mode is active if a track update request has been received. If that is the case two planner modules below is executed sequential, if not the planner checks if the next mode is active, and so on. An introduction to the planning modes and the planning modules will follow next. 3.4 Planning Modes In this section we describe the a number of planning modes, of which some are used in the simulation in Chapter 4 or shown in Figure 3.2. Of course, these modes may not be enough for a general surveillance mission, they only cover some basic functionalities that are needed in the presentation given in this report. 3.4.1 Mode A; Uncertain Target in View Update The Uncertain Target in View Update mode is active if any visible target has a state covariance that is larger than a certain threshold. If that is the case, this mode will facilitate the estimation process of that target. 3.4.2 Mode B; Track Update Request The Track Update Request mode makes it possible for an operator or other software modules to focus on one particular target/track/trajectory. For example, a (re-)identification module may need imagery data of a target during a 10
OEP Path Planner (Single Target) OEP Path Planner (Multiple Targets) Information Exploration Aim Planner for MTT Figure 3.2: Example of one specific sequential planner structure with modes used in Arcus few seconds. The identification module then send a track update request to the planner so the target will stay in the field of view for a number of seconds. A track update request is an interaction containing, among others, a track identity (i.e. the trajectory to track), a priority row vectors and its time vector, and an expiration time. 3.4.3 Mode C; Multi Target Search and Update The multi target update mode analyzes the current state and if one or more targets are in need of a measurement update, according some measure, then this mode will be active. A target is in need of a measurement update if, e.g., the uncertainty of the target state is above a certain threshold. Furthermore, the uncertainty may have to be below another threshold, otherwise the target is considered as lost. 3.4.4 Mode D; Search and Exploration The search and exploration mode determines if an area or road needs to be surveyed. If this mode is used as in Figure 3.2, this mode is always active if no one higher priority mode is active since it is the last non-trivial mode in that example. 3.5 Planning Modules 3.5.1 Planner Module 1; Optimal Estimation Performance (OEP) Planner The problem of optimal path for bearings-only estimation is described in [11] and [10]. 11
Planner Module 1A; OEP of Single Target The planner for the single target case is based on a Information Filter and the Certainty Equivalence Control-strategy and a detailed presentation is given in e.g. [11]. Planner Module 1B; OEP of Multiple Target The planner for the multiple target case is combination of independent single target planners, one for each target. 3.5.2 Planner Module 2; Information Exploration The information exploration is described in [8] and is related to the problem of optimal path for bearings-only estimation. The exploration approach can also be considered as an approach for searching for new targets. Planner Module 2A; Sensor Platform Path for Exploration The limited field-of-view of the vision sensor ignored and only the sensor platform trajectory is considered. Planner Module 2B; Concurrent Path and Sensor Planning for Exploration The vision sensor has limited field-of-view and the exploration planner optimizes both the sensor platform trajectory and the aiming direction (pan and tilt) of the sensor gimbal. 3.5.3 Planner Module 3; Aim Planning of Gimballed EO/IR Sensor for Multi Target Tracking (MTT) The aim planning method for tracking of multiple targets is described in [9]. Unlike the methods above that are using some kind of gradient search, this method must use a global optimization method, in this work a genetic algorithm. 3.5.4 Planner Module 4; Geo-Tracking with Gimballed EO/IR Sensor The geo-tracker is a function for computing the pan and tilt reference signals given the sensor platform trajectory and and a trajectory on ground that should be centered in the sensor view. 12
Chapter 4 Simulation Results The simulation example shown in this chapter is based on a multi-target tracking scenario with three moving road targets and three nearly stationary off-road targets. The planner architecture used here is shown in Figure 4.1. Thus, if OEP Path Planner OEP Path Planner Aim Planning for MTT Figure 4.1: The planner structure used in the simulation. a target is detected and its state covariance is above a certain threshold, the planner focuses on estimating the state of that target (Mode A using Planner 2A and 4.). If not the planner tries to get measurements from all targets, with focus on the most uncertain ones (Mode C using Planner 2A and 3). A Bootstrap Particle filter is used for each target. The result of the simulation is illustrated by Figures 4.2 and 4.3 where a number of snapshots, at irregular points, are shown. At time step 1 the Multi Target Search & Update mode (Mode C) is active. The top right subfigure in Figures 4.2 shows an example when the planner focus on a single target (Mode A). In the following snapshots the modes A and C can be seen alternately. Also note the replanning of the sensor platform path. 13
Figure 4.2: Simulation result of a multi-target tracking scenario. Snapshots from time steps k = 1, 2, 3, 4, 5, 6. Yellow/black line: sensor platform path. Cyan dotted line: current plan of the path. Green square: current sensor footprint on the ground. Red square: current sensor footprint plan. The roads are shown as thick lines in different colors. 14
Figure 4.3: Simulation result of a multi-target tracking scenario. Snapshots from time steps k = 100, 101, 102, 103, 104, 110. Yellow/black line: sensor platform path. Cyan dotted line: current plan of the path. Green square: current sensor footprint on the ground. Red square: current sensor footprint plan. The roads are shown as thick lines in different colors. 15
Chapter 5 Conclusions This report gives an introduction to the planner framework developed in the Arcus project. A planner architecture with different planning modes was presented. This architecture is not the "final solution", but it appeared to be a flexible planning decomposition that is easy to handle, but powerful enough for rather complex surveillance missions. The building blocks of the framework are a number of planning modules. The modules solve well-defined subtasks and they are briefly introduced in this report, for the details we refer the interested reader to the given references. 16
Bibliography [1] D. P. Bertsekas. Dynamic Programming and Optimal Control, volume 1. Athena Scientific, 2nd edition, 2000. [2] D. P. Bertsekas. Dynamic Programming and Optimal Control, volume 2. Athena Scientific, 2nd edition, 2001. [3] Steven M. LaValle. Planning Algorithms. Cambridge University Press, 2006. [4] J. Nygårds, P. Skoglar, J. Karlholm, M. Ulvklo, and R. Björström. Towards concurrent sensor and path planning - A survey of planning methods applicable to UAV surveillance. Scientific Report FOI-R 1711 SE, ISSN 1650-1942, Swedish Defence Research Agency (FOI), Division of Sensor Technology, SE-581 11 Linköping, Sweden, 2005. [5] Umut Orguner, Thomas B. Schön, and Fredrik Gustafsson. Improved target tracking with road network information. In Proceedings of IEEE Aerospace Conference 2009, Big Sky, Montana, USA, March 2009. [6] Stuart J. Russell and Peter Norvig. Artificial Intelligence - A Modern Approach. Prentice Hall; 2nd edition, 2002. [7] Per Skoglar. UAV path and sensor planning methods for multiple ground target search and tracking - A literature survey. Technical Report LiTH- ISY-R-2835, Department of Electrical Engineering, Linköping University, SE-581 83 Linköping, Sweden, December 2007. [8] Per Skoglar. Information based aerial exploration with a gimballed EO/IR sensor. Technical Report LiTH-ISY-R-2886, Department of Electrical Engineering, Linköping University, SE-581 83 Linköping, Sweden, March 2009. [9] Per Skoglar. A planning algorithm of a gimballed EO/IR sensor for multi target tracking. Technical Report LiTH-ISY-R-2887, Department of Electrical Engineering, Linköping University, SE-581 83 Linköping, Sweden, March 2009. [10] Per Skoglar and Umut Orguner. On information measures for bearingsonly estimation of a random walk target. Technical Report LiTH-ISY-R- 2888, Department of Electrical Engineering, Linköping University, SE-581 83 Linköping, Sweden, March 2009. 17
[11] Per Skoglar, Umut Orguner, and Fredrik Gustafsson. On information measures based on particle mixture for optimal bearings-only tracking. In Proceedings of IEEE Aerospace Conference 2009, Big Sky, Montana, USA, March 2009. [12] Per Skoglar, Umut Orguner, David Törnqvist, and Fredrik Gustafsson. Road target tracking with an approximative Rao-Blackwellized Particle filter. Submitted to the 12th International Conference on Information Fusion, 2009. 18
Avdelning, Institution Division, Department Datum Date Division of Automatic Control Department of Electrical Engineering 2009-03-31 Språk Language Rapporttyp Report category ISBN Svenska/Swedish Licentiatavhandling ISRN Engelska/English Examensarbete C-uppsats D-uppsats Övrig rapport Serietitel och serienummer Title of series, numbering ISSN 1400-3902 URL för elektronisk version http://www.control.isy.liu.se LiTH-ISY-R-2885 Titel Title The ARCUS Planning Framework for UAV Surveillance with EO/IR Sensors Författare Author Per Skoglar Sammanfattning Abstract This report gives an overview of the planner framework developed in the Arcus project. The framework consists of a number of planning modules and planning modes that are introduced. Nyckelord Keywords ARCUS, UAV Surveillance, Planning