P 1. e 1. Sectormap S. P 2 e 2

Transcription

1 In: Proc Int. Conf. on Advanced Robotics (ICAR'97), Monterey Active Controlled Exploration of 3D Environmental Models Based on a Binocular Stereo System Darius Burschka and Georg Farber Laboratory for Process Control and Real-Time Systems Technische Universitat Munchen D Munchen, Germany ~burschka Abstract This paper describes the exploration of indoor environments based on a binocular stereo system mounted on a mobile vehicle. It introduces the algorithms for the control of the vehicle and the camera mount depending on the explored and the a-priori known information about the environment. The controlled vehicle does not follow a precompiled path but nds its goals depending on the perceived information. The resulting environmental model is stored in a multilayer map storing the three-dimensional boundary lines of the objects at their geometrical positions in the world. The line description in the model is adapted to meet the requirements of a successive exploration with a video camera. Keywords: path planning, environmental models, active camera control, navigation 1 Motivation The knowledge about the current position and the surrounding environment is essential for the ecient use of an autonomous mobile robot (AMR). The localisation and navigation tasks require a reliable and exact representation of the environment. The stored information is continuously compared to the current sensor reading to compensate the errors of the deadreckoning. We have already presented an environmental model handling the a-priori known information about the environment [9]. This model was suitable for the navigation in a known environment supporting dierent kinds of sensor systems [6, 7]. The exact knowledge about the environment from architect's plans or measurements were necessary to generate this model. A typical indoor environment changes because of the human inuence or the operation of other mobile systems. The resulting discrepancies between the stored model and the modied environment may cause misinterpretations resulting in a false pose estimation of the AMR. An extension of our model, able to handle the uncertain information from the sensor during exploration, was presented in [2]. This new approach bases on the dynamic local map (DLM) ltering the explored information. This map stores the information on a low level of abstraction equivalent to the capabilities of the applied sensor system (section 3.1). The environmental representation in the DLM consists of the boundary line segments of the objects. The missing information about the planes in the map aggravates the use of this map for path planning. The problem is to decide based on this simple description whether the Figure 1: MARVIN planes enclosed by the line segments are passable for the vehicle or not (g. 2). The solution to this problem will be shown in this paper. It presents our exploration system and the way the sensor data is used to control the vehicle. 2 System structure The dynamic local map (DLM) storing the explored information of a local area is the essential part of the system presented in this paper. This map is constructed based on the data from three sources shown in gure 3: sensor system - the information from the current sensor reading 971

2 PSC 1 - known objects or structures are identi- ed and missing features are generated as hypothetical lines to be veried by the sensor system (section 3.4) global model - the a-priori information about the environment and the information from prior missions each other deteriorating the quality of the extracted information. This errors can be corrected as will be shown in section 3.3. Figure 3: System structure Figure 2: These two situations cannot be distinguished without any additional information ( a door in the wall $ a locker standing at a line on the oor). Our vehicle MARVIN based on the TRC-platform (Labmate) is equipped with a stereo camera mounted on a pan-tilt-vergence camera mount (g. 1). All processing is performed on two PC-computers with Intel Pentium{processor running Linux OS. All modules use the RPC 2 -communication to exchange data. This mechanism allows to distribute the processing among dierent machines to keep the average load at an uniform level without changes in the program code. The hardware of the vehicle and the camera mount are controlled by two RPC-servers allowing communication with dierent processes even on dierent computers. These servers (g. 3) are commanded by the path planning module, which determines their actions depending on the information stored in the DLM (section 4.2). The current position and orientation can be requested directly from the server. The sensor information from the stereo system is matched in the sensor feature extraction module to restore the threedimensional description of the extracted line segments. These features are stored in the dynamic local map (DLM) to improve their accuracy and to lter the misinterpretations. The stored information is used by the sensor system for the fast establishment of known correspondences between the features in the two camera images [3] and for the recalibration of the cameras. The recalibration is important for the exploration, because the knowledge about the exact position in the world is fundamental for the map construction. The vibrations of the cameras change their orientation to 1 Predictive Spatial Completion - a module reconstructing planes and known structures from the data stored in the DLM 2 Remote Procedure Call - interprocess communication allowing a communication between processes even on dierent machines. 3 Environmental model The \world" of the AMR can be subdivided into local operating areas, like: rooms, proximities of machines or docking stations, etc, relevant for specic tasks. These areas can be referenced relative to signicant structures, which can easily be recognized by the sensor system. These structures dene the origins of the according coordinate systems. These local areas are stored in the global map holding the information between the particular missions. But there are also areas, which are not relevant for a mission or change continuously and should not be stored permanently. In those areas the local map is only used for collision avoidance. The information stored in the DLM changes continuously because of the limited view angle and accuracy of the sensor. The inserted information is matched in a three-dimensional matching process to the already known content and stored eciently to reduce the storage space requirements of the map. On the other hand the access time to the stored information must be low to allow a closed-loop operation between the sensor and the map. The structure of the DLM is designed to cope with these contradictory requirements. 3.1 Internal structure The DLM has a multilayer structure to meet the requirements for short access time and simple update of the stored information (g. 4)[2]. We have tested dierent index structures storing the geometrical descriptions of the environment for their access times and eciencies [9, 11]. The access structures for extended spatial objects are surveyed in [10]. The spatial trees subdivide the world depending on the objects' distribution. They proved the shortest access times and lowest storage space requirements because of the adaptation of their structure to the environment. The 972

3 DLM uses an octree to store the environmental description. The octree subdivides a voxel 3 in four equal parts, which can further be subdivided until the stored elements are enclosed with the smallest possible cube. This increases the selectivity of the access functions. They supply only information stored in voxels intersected by the current sensor cone. A smaller voxel size results in a better approximation of the visible space. Only the visible elements are delivered. It is dicult to exchange parts of the tree if a new space segment is approached. The computation of all branches of the octree is necessary. Therefore, the upper layer of our map has a grid structure. Each grid element contains an octree storing a local part of the environment. If the AMR approaches a new space segment only the corresponding grid elements must be replaced. object recognition and path planning. The line segments are stored with additional information about their source and quality. They are described by their endpoints P 1 P 2. The accuracy e x is attached to each endpoint separately, because it depends on the distance from the sensor and the orientation of the projected line in the image in case of the stereo camera. The distance from the sensor is often dierent for the two endpoints. The resulting error is described by an error sphere around the estimated endpoint (g. 5). P 2 e 2 P 1 e 1 Sectormap S Figure 5: Line representation The sector map describes the directions from which the line segment was seen. The line segments are additionally described by their condence and age values. The condence c describes the reliability for the real existence of the line. If a new feature could be matched to one stored in the DLM, this value is increased according to the following equation: Figure 4: Internal structure of the DLM. The size of the DLM is adjusted to the typical size of the local operating area. The map has a virtual mapping of the world coordinates to the grid elements to reduce the data transfers within it (g. 4). The location in the \real world" is quantied by the size of a voxel. This so{called virtual coordinate is mapped 4 to the size of the grid array resulting in its real coordinate. The real coordinates identify the voxel's location, so every grid element may correspond to dierent locations in the real world, i.e. the world is being mapped on a torus. Every voxel contains the virtual coordinates of the represented real world location. This is similar to the tag{eld in computer caches. 3.2 Line representation The DLM is the source for the recent information about the environment. This information can be used for dierent purposes, like: sensor data processing, 3 a rectangular cube enclosing a space segment 4 the mapping is done using a modulus operation. c(x + x) = 1? e?k(x+x) ; 0 x 1 c(x + x) = 1? [1? c(x)] e?kx (1) The value x describes the condence in the current update (0!unsure, 1!sure). The factor k determines the speed at which the condence is changed and depends on the quality of the feature extraction algorithm. Line segments resulting from wrong correspondences are also stored in the map. These line segments cannot be veried from other positions. Each time the segment appears in the local view of the sensor and cannot be veried, the age of the segment is decreased. The feature is removed from the map if its age falls below a threshold. This method allows also to remove vanished obstacles from the map. An important information about the line segment is stored in a bitmap describing its sources. The possible sources for a feature are: the global map, the sensor system and the PSC. This information evaluates the features for the use in dierent tasks. For example hypothetical features generated by the PSC should not be used for path planning or localisation, because their real existence is not yet veried. The bitmap describes the history of a line. This information is very useful. 973

4 It helps to identify for example lines from the global model veried or rejected in the current sensor reading. 3.3 Interaction with the sensor system We use this information to recalibrate the sensor system and to correct the drifts of the dead-reckoning. The procedure is quite simple and shows the usefulness of the DLM. In each step the sensor system asks the map for the already known line segments in the local view. The returned lines are sorted according to their bitmap contents, condence values and accuracies. All lines originating from the global model or with high condence and accuracy are projected on the image planes of the sensors and matched with the current readings. The resulting matches are used for the calibration of the extrinsic camera parameters [13]. The resulting values are used to correct the orientation errors of the cameras or drifts of the vehicle's dead-reckoning. The remaining lines are matched in the same way to the extracted image line segments in both cameras. This step reduces the number of matchable lines to the still unknown lines, which are matched using geometric constraints [5]. If there are several matching candidates they are all stored in the DLM to be veri- ed from a dierent position exploiting the movement of the vehicle. The DLM is used to accelerate the sensor data processing and as an alternative to a third camera [1]. The DLM acts as a \reference camera" supplying three-dimensional information to the verication process. 3.4 Interaction with the PSC A new approach is the interaction between the local map and the predictive spatial completion module (PSC). The PSC scans the map and it tries to reconstruct planes and objects from the information explored by the sensor system [4]. An important result of this interaction are the hypothetical lines generated as a completion of assumed planes and known objects. These lines are stored back into the DLM to be veried by the sensor system in consecutive sensor readings. They are used to establish known correspondences as described in section 3.3. They help to nd correct correspondences in ambiguous situations. 4 Exploration The focus of this paper concentrates on the exploration of an unknown or partially known environment. There are two dierent ways to explore the environment depending on the current task and the knowledge level about the environment. The exploration can be done during a normal mission or be an exclusive task. We subdivide the path planning module (g. 3) into two parts with dierent knowledge levels about the abilities of the sensor system and the current mission. The higher level module -\strategist"- is responsible for the strategy, how to explore the whole \world" considering the time restrictions of the current mission. This module knows, how much time it has to explore a local part of the environment. It does not know how to control the sensors. It does not even know, what kind of sensors are used. It uses two commands to communicate with the lower module -\navigator"- responsible for the vehicle and sensor control. The rst command describes the coordinates of the next goal, the time to reach this goal and gives hints whether the vehicle may explore or not (exploration ag). It is allowed for the strategist to specify the starting point as the goal. This special case can be used in two situations: the vehicle is waiting for another vehicle and should stay at its current position or the vehicle has some time to explore the environment, but it should return to the starting point after this time. In unknown environments this is often the only known free point at the beginning. The exploration ag determines the behavior. The second command is used to rate the explored areas. Only the navigator can access the DLM and he is able to rate the quality of the stored information. The strategist does not know anything about the sensors. After an exploration of a local area the strategist may choose the next goal depending on the ratings of the possible goals proposed by the navigator and the global mission goal. The lower level module -\navigator"- operates in a local area and has no knowledge about global goals or missions. It knows the abilities of the used sensors and controls them. In our case the navigator controls the camera mount and the vehicle. It triggers also the sensor data processing. The sensor data processing is triggered while the vehicle does not move to reduce errors caused by vibrations. The binocular stereo system does not need any movement to reconstruct the three-dimensional information. The navigator interprets the explored information to estimate next goals for the exploration (section 4.2). 4.1 Estimation of free space The information stored in the DLM consists of single three-dimensional lines representing objects' boundaries (g. 6). There is no information about planes 974

5 or objects. Therefore, a new method was developed to estimate the free space based on this kind of data. The algorithm bases on the assumption that the current vehicle's position is a part of the free space. This space is restricted by the objects surrounding the vehicle. The boundaries of these objects are stored in the DLM. The space is subdivided into three regions. Figure 6: CAD-Model of an environment and the corresponding content of the DLM. The gray lines have insucient condence for path planning. The rst region lies above the vehicle and contains all lines, which cause no collisions with the AMR. The second region extends from the vehicle's height to some centimeters above the ground and contains obstacles causing a certain collision with the vehicle. The third region extends in a region passable for the vehicle (lines on the oor and small bumps). The interesting lines lie in the second and third region. The space between the vehicle and those lines is assumed to be free. There is a diculty to decide whether a line on the oor is a real obstacle or not. The information stored in the DLM does not support any hidden line removal. The vehicle decides to move across such a line, if there are further lines behind it and no line in the second region is found. The DLM supplies occasional the information about the area behind the walls, which is invisible in the current sensor reading, if it is already explored. Only the sensor can decide the visibility from a given position, but the wrong correspondences may lead to wrong decisions. The solution to this problem is an additional access function to the DLM ltering the current sensor reading without adding any further information not contained in the current view. Only the features from the current sensor view, that could be veried in the DLM, are used to solve the described problem (g. 7). 4.2 Generation of goals The section 3.2 describes the additional information stored with each line segment. This information is Figure 7: Free space estimation. used to determine the possible goals for the next step. All hypothetical features stored by the PSC or uncertain lines detected by the sensor system are clustered and each cluster is rated according to parameters describing the expected improvements, if the given cluster will be explored next. The \behavior" of the vehicle is determined by the results of this step. The covered path can be minimized or the inspection of the hypothetical lines can be preferred. The choice of the parameters changes the \behavior". The vehicle can decide to explore a fast uncertain model of the environment, if the costs for movements are low and uninspected areas in the map promise a high increase of the information content, or it can try to verify the detected features from an other side, if the costs for movement are high and the stored hypothesis have a high priority. The detected clusters are encircled in gure 8. Figure 8: Path planning in the environment shown in gure 6 The clusters determine the regions of interest in the local area. The goal point for the vehicle and the view angle for the cameras is computed to reach an optimal distance from the obstacle to achieve the best results with the applied sensor.??? 975

6 4.3 Path planning The unsteady diusion equation strategy is used to plan the path in the extracted free space [12]. The following equation describes the diusion from the starting point x 0 to the goal x G = a 2 r 2 u? g u; u(t; x G ) = 1; x G 2 R n The distribution of the concentration u(t; x) for a given room structure and the resulting path is shown in gure 9. log Path approximation with straight line seg- Figure 10: ments is an iterative approach and the quality of the generated paths increases with the number of computed iterations. The time restrictions require a compromise between the quality and speed of the path planning. The gure 11 shows the generated paths for dierent numbers of iterations. The path generation takes ca. 2.5 s for a eld containing 8000 grid elements of a size 10cm x 10cm on a Pentium processor at 90 MHz Figure 9: The distribution of the concentration u(t; x) and the planned path The resulting paths are approximated by straight line segments generated by algorithms used in the computer vision. The pixel chain is rst subdivided at the point where it deviated most from a straight line joining its endpoints. Subdivision repeats recursively until the edge is reduced to short segments representing the minimal way length [14]. The algorithm can be optimized for minimal deviation from the planned path or for optimal way lengths (g. 10). The free space is rastered with a 10cm x 10cm grid for which the diusion is computed. We use a non-oscillating equation 4 to compute the diusion. ~u k+1;r = 1 MX 1 + M ( m=1 8< u k+1;r = u k;m + u k;r )? g u k;r (3) : eqn: 3 for r 2 0 0; for r 2 0 1; for r = r G (4) The value M describes the number of the neighbors of a grid element, 0 is the free space, 0 are the grid elements containing obstacles and r G is the goal. 5 Results The applied diusion equation strategy requires a high computational eort, but it generates paths in a secure distance from the objects in the world. It Figure 11: Generated paths after 100 and 400 iterations The DLM helps to navigate in an unknown environment and gives valuable hints about the local area. The stored information is used to control the vehicle and the camera orientation. The introduced strategy allows a maximum increase of the information content and helps to explore the local area eciently. 6 Future work The introduced system operates in a global coordinate system. It is dicult to keep the position error of the modelled objects small for long distances. The operation far away from the origin of the coordinate system is aliated with errors in the dead-reckoning and the modelled world. We change our models to local maps of the environment referenced to signicant objects in the local area instead of a global reference now. This step requires a robust object recognition to identify these reference objects to switch to the appropriate coordinate system, when a new area is approached. The object recognition operating on three{dimensional features stored in the DLM is already implemented. A topological modeling is aspired, where the distance between the particular local maps does not need to be known exactly. 976

7 Acknowledgment The work presented in this paper was supported by the Deutsche Forschungsgemeinschaft as a part of an interdisciplinary research project on \Information Processing in Autonomous Mobile Robots"(SFB331). References [1] N. Ayache and F. Lustman. Fast and Reliable Passive Trinocular Stereovision. Proceedings 1. Int. Conf. on Computer Vision, pages 165 { 243, [2] D. Burschka and C. Eberst. Exploration of Unknown or Partially Known Environments. In 2. Asian Conference on Computer Vision, pages (II) 727{731, Dec [3] D. Burschka, C. Eberst, and C. Robl. Vision Based Model Generation for Indoor Environments. Proc. IEEE Int. Conf. on Robotics and Automation, pages 1940{1945, [4] C. Eberst and J. Sicheneder. Generation of hypothetical landmarks supporting fast object recognition with autonomous mobile robots. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robot Systems, Nov [10] B.Ch. Ooi. Ecient Query Processing in Geographic Information Systems. Lecture Notes in Computer Science No. 471, Springer Verlag, [11] Achim Ru. An Environmental Model with Real- Time Access Functions for Use in Autonomous Mobile Robots operating in Complex structured Manufacturing Environments. ROVPIA `94, Ipoh, May [12] G. Schmidt and K. Azarm. Mobile Robot Navigation in a Dynamic World Using an Unsteady Diffusion Equation Strategy. Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 642{647, [13] Roger Y. Tsai. A Versatile Camera Calibration Technique for High Accuracy 3D Machine Vision Metrology Using O-the-Shelf TV Cameras and Lenses. IEEE Transactions of Robotics and Automation, RA-3(4):323{344, August [14] Ch.M. Williams. An ecient algorithm for the piecewise linear approximation of planar curves. Computer Graphics and Image Processing, 8:286{ 293, [5] Oliver Faugeras. Three-Dimensional Computer Vision. Massachusetts Institute of Technology, The MIT Press, Cambridge, Massachusetts London, England, [6] A. Hauck and N. O. Stoer. A hierarchic world model supporting video-based localisation, exploration and object identication. In 2. Asian Conference on Computer Vision, Singapore, 5. { 8. Dec., pages (III) 176{180, [7] J. Horn and A. Ru. Localization System for a Mobile Robot based on a 3D-Laser-Range- Camera and an Environmental Model. Proc. IEEE Int. Conf. on Intelligent Vehicles, Paris, [8] Xavier Lebegue and J.K. Aggarwal. Automatic Creation of Architectural CAD Models. Proceedings: Second CAD-based Vision Workshop Seven Springs, PA, pages 82{89, February 8{ [9] Gunter Magin, Achim Ru, Darius Burschka, and Georg Farber. A dynamic 3D environmental model with real-time access functions for use in autonomous mobile robots. Robotics and Autonomous Systems, 14:119 { 131,