HAREMS: Hierarchical Architecture for Robotics Experiments with Multiple Sensors

Transcription

1 HAREMS: Hierarchical Architecture for Robotics Experiments with Multiple Sensors Giorgio Buttazzo ARTS Lab Scuola Superiore S. Anna Via Carducci, Pisa, Italy Abstract - This paper presents a hierarchical computer architecture specifically designed for ling a multisensory robot system in a flexible and modular fashion. The approach we used for the implementation of the system is bottom-up, in the sense that high processes are developed from elementary sensory-motor activities, carried out at lower s of a hierarchical system. Sensor acquisition, planning and are functionally distributed among different processors, each one representing a node of a tree-structured network. In order to obtain an extremenly open and flexible programming environment, most of the nodes, including the supervisor, have been realized by using IBM compatible personal computers, under MS-DOS operating system. The software is organized in a number of layers, which add new capabilities to the robot system, and allow the user to program at different s of abstraction. Particular attention has been devoted to the real-time of the robotenvironment interactions, and different sensor-based explorative strategies have been implemented as basic capabilities of the robot system. I. INTRODUCTION As robotics applications become more sophisticated, they exceed the capabilities of computers traditionally used in robotics, therefore it becomes necessary to investigate alternative types of computing systems capable of supporting many sensing and actuating devices. Advanced robots are going to operate autonomously in unstructured environments and to exhibit intelligent behavior and great flexibility in performing complex manipulative tasks. A sophisticated sensory system is undoubtedly a necessary condition for developing these capabilities, but it must be supported by a suitably designed computer architecture for improving robot performance in manipulative tasks and perceptual activities. Active perception is a process that involves dynamic sensing, where movements are utilized as means for increasing and driving sensory information. For active perception we intend the ability to not only see and touch objects, but also to manipulate them, explore the environment, recognize situations. Active perception is the necessary quality that a robot should have to operate autonomously in uncertain environments, perform flexible operations and exhibit intelligent behavior [1] [2] [3]. The motivation for using multiple sensors resides in the non-ideality of the world and of the sensors. Measurements are noisy, partial, imperfect, and hence one sensor cannot provide the system with reliable data. On the contrary, by using multiple sensors, several different properties can be extracted from an explored object and the probability of a correct recognition increases substantially. These properties may include geometric features (such as shape, contours, holes, edges, protruding regions), mechanical characteristics (such as hardness, flexibility, elasticity) or thermal properties (such as temperature, thermal conductivity). In order to support active multisensory perception the system architecture must have the following properties: a) High degree of parallelism The system has to process multiple input data streams which simultaneously arrive from sensors. Each of these data stream may require a different algorithm to be treated, therefore a parallel architecture is fundamental to meet the processing requirements of this large amount of data. b) Hierarchical loops In order to support a wide range of sensory-motor capabilities, ranging from low reactions to complex exploratory procedures, the system architecture must be able to handle hierarchical loops operating at different frequencies. Multi communications among processors is also required to close real-time loops at each of the hierarchy, including short range reflex arcs, for effective support of guarded movements. c) Modularity

2 It is convenient to partition the hardware and software architecture into several modules, each of which can be independently developed to handle a particular function of the robot system. Such a functional distribution among local subsystems facilitates system maintenance and unloads the central supervisor of most of low computations. Furthermore, since the number of sensors may increase and the organization of the system may change, a modular approach simplifies system reconfiguration and expansibility. The architecture presented in this paper was designed according to these structural and algorithmic objectives, trying to match the processing capability to the functional requirements of the robot system. In the area of robotics, several multiprocessor architectures have been proposed for ling multilink systems [4] [5] [6] [7], but only few authors dealt with the problem of multisensory active perception. An anthropomorphic approach to this issue has been discussed by Albus [8], who described the organization of a behavior-oriented architecture based on multi loops, whereas some computer architectures for multisensory perception have been proposed by [9] [10] [11]. Although some of these solutions are based on highly parallel hardware and can be organized in a modular structure, they are diffucult to program and to test, requiring special hardware and software tools, and often they are very expensive. The consequence is that most of these architecture are not of practical use and they are not flexible enough to experiment sensor-based robot applications in a changeable laboratory environment. For the above reasons, the major effort of this work has been to design an open architecture, easy to use and to expand, and capable of supporting multiple sensors and different sensor-based strategies. The hierarchical structure of the software and the modular approach provide great flexibility and allow the user to develop robot applications at different s of abstraction. II. SYSTEM DESCRIPTION The robot system led by the computer architecture described in this paper is shown in Figure 1. It consists of the following components: - a PUMA 560 robot arm, including a Mark III ler with VAL II operating system; - a 6-components force/torque sensor mounted on the robot wrist; - a 2-D vision system, consisting of a CCD camera and a frame grabber; Fig. 1: The robot workstation - a proximity sensor, based on ultrasonic (US) transducers; - an electric gripper, which incorporates two force/torque sensors and two tactile arrays (one for each finger) composed of 128 sensing elements each; In order to obtain an extremely open and flexible programming environment, most of the software for data acquisition and has been implemented on a personal computers (PC), with Intel microprocessor and MS- DOS operating system. The interface between PC and periferal devices, such as sensors, actuators or other computers, is provided by a number of acquisition and communication boards, which directly plug in the PC bus. In particular, the following boards have been used: - A Data Acquisition Card (DAC) which incorporates a 16-bit bidirectional TTL parallel port, two 12-bit digital to analog (D/A) convertes, and a 12-bit analog to digital (A/D) converter, with 4 differential input channels. - Two serial communication boards, each one having two RS232 serial ports, working at a maximum rate of baud.

3 SP 1 Graphics Display modularity, all nodes communicate through a message passing mechanism, and no shared memory is used. Graphics Display SP 2 SP 3 PC AO AI PC 1 supervisor alter MARK III CONTROLLER SP 4 PI PO SP = serial port PI = parallel input PO = parallel output AI = analog input AO = analog output Fig. 2: Acquisition and communication channels available on each PC node PC 2 FG100 WRIST PROCESSOR PC 3 PUMA ROBOT - A parallel communication input/output card, composed of 3 bidirectional 8-bit parallel ports. - A VGA color graphics card used for real-time representation of sensory data and other system characteristics. By these boards the PC becomes a very flexible and independent node of a distributed system, capable to handle input and output analog signals and to communicate to other nodes of the system through standard channels, such as serial and parallel ports. Figure 2 illustrates all channels available on a PC node. The solution we decided to adopt here is to design a multiprocessor architecture that fits the hierarchical structure required by the robot system, so that each task carried out by a functional module can be assigned to a specific microprocessor. As a result, the structure of the physical architecture is also hierarchical and it is organized to equalize the computational load on the various processors through functional partitioning of local low and high tasks. The general organization of the computer architecture is outlined in Figure 3. As shown in Fig. 3, the system topology is organized as a tree structure, which allows to equally distribute the computational load to peripheral processors, facilitating parallel data acquisition and sensory integration. Each node of the network is an independent computer supporting its own operating environment and dedicated to a specific task, which can be data acquisition, signal analysis, pattern recognition, exploration planning or motor, depending on the node position in the hierarchy. For greater camera wrist sensor US gripper Z-80 F F TS Fig. 3: The computer architecture of the robot system In particular, PC 3 is dedicated to the tactile subsystem, which consists of the sensorized gripper. Through the analog channels, PC 3 the grasp and manage data acquisition from the gripper force sensor (F). Because of the large number of sensing elements, tactile sensors (TS) are locally treated by a dedicated processor (Z-80), connected to PC 3 through a parallel port. PC 3 deals also with the proximity sensor (US), which does not increase the computational load of PC 3, since the US transducer is utilized during the approaching phase, when the gripper is idle. The PC 2 node is dedicated to the vision subsystem. It performs image acquisition, feature exctraction, scene analysis, and provides the supervisor (PC 1 ) with the world coordinates of the target location that the robot should reach. Finally, the node PC 1 is the system supervisor. Its main functions are to manages the communication with the robot ler, to integrate information produced by sensory subsystems and to plan robot action according to a specific goal. All sensory data produced by the peripheral subsystems arrive at the supervisor node. The visual information processed by PC 2 and the tactile data treated by PC 3 arrive to node PC 1 through two RS232 serial lines, at TS

4 9600 baud rate. Forces and torques produced by the Wrist Processing Unit are transmitted to PC 1 through a 16 bit parallel port at the maximum frequency of 100 Hz. The Mark III robot ler is connencted to PC 1 through three different communication channels: an RS232 serial line at 9600 baud rate is dedicated to the supervisor protocol for host remote ; another RS232 serial line at baud rate is dedicated to the ALTER communication protocol provided by VAL II for real-time path ; a 16-bit bidirectional parallel port is used to additional peripheral devices, usefull for action synchronization and guarded movement implementation. III. SOFTWARE ARCHITECTURE All software has been written in C language and has been incapsulated in a set of library functions, which extend the language with several robot-oriented instructions. This choice allows the following advantages: - the system does not require a special compiler (that should be available for each type of microprocessor used in the architecture), but all programs can be written in standard C and easily ported on different computers; - the library approach simplifies the modular implementation of strategies and allows the user to program at different s of abstraction; - users can easily modify low- as high- functions to adapt them to different types of peripheral devices, strategies, and applications; - if new sensors or new actuators are introduced in the system, robot capabilities can be incremented by simply adding new functions to a specific library; this is extremely useful in a changing experimental environment. As shown in Figure 4, the software architecture on the supervisor node is organized in a hierarchical structure of layers, each of them provides the robot system with new functions and more sophisticated capabilities. The importance of the library approach is not simply that one can divide the program into parts, rather it is crucial that each procedure accomplishes an identifiable task that can be used as a building block in defining other procedures. For example, suppose that we have implemented a library function hybrid(), which realizes a position/force loop to move the robot along a specific path, while exerting a force against a desired direction. If we want the robot to follow a contour of an unkown object, we can see the function hybrid() as a "black box" to develop a new robot action, say explore(), which exploits the capabilities of the hybrid() function to fulfil its task. The details of how hybrid() has been implemented can be ignored, and, as far as the explore() procedure is concerned, hybrid() is viewed as an abstraction of a behavior. Applic. peg-in-hole insertion compliant motion Action Behavior Communic. Device A. Device position suplib object exploration serlib contour following force altlib parlib surface cleaning obstacle avoidance hybrid ftslib daclib Fig. 4: Software architecture gralib adaptive grasp impedance vislib assembling This includes a set of libraries intented to manage all boards connected to the PC bus. Each library provides a number of functions, whose purpose is to facilitate device handling and to incapsulate hardware detail, so that higher software can be developed indepentently of the specific knowledge of the peripheral devices. Referring to Fig. 5, the library SERLIB provides a set of functions for using the four RS232 serial ports, setting the communication parameters, sending and receiving different types of data, such as characters, integers, floats, strings, arrays of integers, and arrays of floats; Similarly, the library PARLIB manages the parallel port. Besides providing the functions for managing the parallel communication with an other processing node, this library includes a set of functions operating on a single bit of the port. This is useful for external event synchronization, device, and guarded movement implementation. The library DACLIB handles the A/D and the D/A converters, for sensory data acquisition and analog signal output. Finally, the GRALIB library provides a number of functions specifically implemented to the graphics adapter (VGA) as faster as possible, for real-time event display. For a matter of efficiency, at this all functions have been implemented in assembly language, and no DOS system calls have been used.

5 B. Communication Because of some nodes of the architecture are commercial processing units, such as the PUMA Mark III ler, or the LORD wrist processing unit, specific communication protocols must be developed in order to connect these nodes to the supervisor node. Based on the primitives developed at the device, the communication includes a set of libraries for managing the communication between such processors and the supervisor node. In particular, two protocols have been developed to connect the PC 1 to the PUMA ler. The ALTLIB library handles the communication through the ALTER port of the Mark III ler, for modifying the robot trajectory in real-time, every 28 ms. A number of C functions are provided to start the ALTER protocol, close the communication, send a new robot location, or receive the actual robot coordinates. The SUPLIB library is intended to manage the communication through the SUPERVISOR port of the Mark III, for ling the robot by a remote host. Making use of this facility it is possible to impart VAL II commands directly from PC 1, such as moving the arm, changing the speed, loading files, or execute programs. Two more libraries are available at this : FTSLIB allows the communication with the LORD wrist processor, for reading force/torque data; whereas VISLIB is used for exchanging message with the vision subsystem. C. Behavior This is the in which several sensor-based strategies have been implemented, in order to give the robot different kinds of behavior. The functions available at this of the hierarchy allows the user to close real-time loops, by which the robot can follow on line planned trajectories based on sensory information, apply desired forces and torques on the environment, operate according to hybrid schemes, or behave as a mechanical impedance. Most of the functions resident at this utilize the ALTER port to communicate with the PUMA robot ler. D. Action Based on the methods and primitives developed at the lower s of the hierarchy, this provides the robot system with more sophisticated sensory-motor actions which represent the basic capabilities for carrying out complex tasks in unstructured environments. Some representative actions developed at this include: a) the compliant motion; b) the ability of the robot to follow an unknown object contour, maintaining the endeffector in contact with the explored surface; c) the reflex to avoid obstacles, making use of the proximity sensor incorporated in the gripper; and d) the ability to adapt the end-effector to the orientation of the object to be grasped, based on the reaction forces sensed on the wrist. Many different actions can be easily implemented at this by the user utilizing other strategies and different sensory subsystems. E. Application This is the at which the user defines the sequences of robot actions in order to accomplish applicative tasks, such as the assembly of mechanical parts, the exploration of unknown objects, or the manipulation of delicate materials. One of the most significant applications realized on the robot system has been the classical problem of the peg-inhole insertion, where the hole direction is known with some degree of uncertainty. This is a typical task which requires the robot to exhibit active compliance during the insertion, in order to continuously correct its motion to adapt to the hole constraints. The software stratification is organized in such a way that each functional modifies the system interface with respect to the user, but it does not hide the lower s, so that an applicative program can use all primitives implemented in all underlying s. Such a penetrability property gives the system the maximum flexibility, since the user that wants to experiment new strategies, can easlily create its own modules by accessing any of the funtions implemented at lower s, or even developing new funtions. IV. EXPERIMENTAL RESULTS The robot architecture presented in this paper has been tested on a number of different tasks, such the ones developed in the application. The purpose of these experiments has been to determine the architectural characteristics and parameters that influence the system performance in terms of bandwidth and stability. In order to carry out a quantitative analysis of these parameters we have developed a mathematical model of the system, experimentally derived by the step response of the robot arm, shown in Figure 5. The step response has been obtained by initially putting the robot end-effector (having a knwon elastic coefficient) in contact with a rigid surface, and then commanding the robot through the ALTER mode to move 1 mm against the surface, while monitoring the force sensor output. The frequency analysis of the step response revealed the presence of a dominant pole at 2.2 Hz and some higher frequency components around 13 Hz. We have found that the dominant pole is independent of the arm configuration, whereas the higher frequency components are not.

6 root works as system supervisor. The main functions carried out by the supervisor are to integrate sensory information extracted from other nodes, to recognize explored features, to plan robot actions according to the desired goal and to manage the user interface. The hierarchical structure of the software and the modular approach to sensory-motor are the most important features of this work, which provide great flexibility and allow the user to develop robot applications at different s of abstraction. REFERENCES Fig. 5: Step response of the system Moreover, a finite delay of about 60 ms has been observed between the time in which the command is issued and the time in which the motion begins. In the model we approximate = 2T, being T = 28 ms the sampling period imposed by the Mark III ler. Since in all practical applications the robot system works within a bandwith of few Hz, we have neglected the 13 Hz components, thus the transfer function of the system in open force--loop resulted to be the following: H(s) = s 0 s(s+s 0 ) e 2Ts where s 0 = 2 f p = 14 rad/s and T = 28 ms. Notice that the pole in the origin takes in account the cumulative alter mode, which make VAL II to interprete commands in velocity. During the execution of all applicative tasks, this model helped us to choose the right parameters to guarantee the stability of the system. The most significant fact that emerges from the model, and also confirmed by the experiments, is that the bandwith and the stability of the system strongly depends on the internal delay and on the sampling period T at which the robot trajectory is updated. Note that these two parameters do not derive from the structure of the distributed architecture or from the hierarchical software organization, but they come from the robot ler, which is the real bottleneck of the system. V. CONCLUSIONS In this paper we presented a hardware and software architecture designed to a multisensory robot system. In order to handle the huge amount of information coming from the peripheral sensors and to close real-time loops, the acquisition and processes have been distributed among different independent processors. All processors are hierachically organized in a tree structure, where the leaves are dedicated to sensory acquisition and the [1] Bajcsy, R. "Active Perception", Proceedings of the IEEE, Vol. 76, No. 8, August 1988, pp [2] Dario, P., and Buttazzo, G., "An Anthropomorphic Robot Finger for Investigating Artificial Tactile Perception", Int. J. Robotics Research, 6(3), Fall 1987, pp [3] Buttazzo, G., Bajcsy, R., and Dario, P., "Finger Based Explorations", Proc. of SPIE Conf. on Advances in Intelligent Robotic Systems, Cambridge, MA, [4] Gaglianello, R. D., Katseff, H. P., "A Distributed Computing Environment for Robotics", Proc. of IEEE Int. Conf. on Robotics and Automation, San Francisco, CA, 1986, pp [5] Kriegman, D. J., Siegel, D. M., Narasimhan, S., Hollerbach, J. M., and Gerpheide, G. E., "Computational Architecture for the UTAH/MIT Hand", Proc. of IEEE Int. Conf. on Robotics and Automation, St. Louis, 1985, pp [6] Narasimhan, S., Siegel, D., Hollerbach, J. M., Biggers, K., and Gerpheide G., "Implementation of Control Methodologies on the Computational Architecture for the Utah/MIT Hand", Proc. of IEEE Int. Conf. on Robotics and Automation, San Francisco, CA, 1986, pp [7] Zheng, Y. F., and Chen B. F., "A Multiprocessor for Dynamic Control of Multilink Systems", Proc. of IEEE Int. Conf. on Robotics and Automation, St. Louis, pp [8] Albus, J. S., Brains, Behavior, and Robotics, McGraw-Hill, [9] Ma, Y. W. E., and Krishnamurti, R., "REPLICA - A Reconfigurable Partitionable Highly Parallel Computer Architecture for Active Multi-Sensory Perception of 3- Dimensional Objects", Proc. of IEEE Int. Conf. on Robotics, Atlanta, 1984, pp

7 [10] Goldwasser, S. M., "Computer Architecture for Grasping", Proc. of IEEE Int. Conf. on Robotics, Atlanta, 1984, pp [11] Lee, I., and Goldwasser, S. M., "A Distributed Testbed for Active Sensory Processing", Proc. of IEEE Int. Conf. on Robotics and Automation, St. Louis, 1985, pp