Real-Time Interface Board for Closed-Loop Robotic Tasks on the SpiNNaker Neural Computing System

Transcription

1 Real-Time Interface Board for Closed-Loop Robotic Tasks on the SpiNNaker Neural Computing System Christian Denk 1, Francisco Llobet-Blandino 1, Francesco Galluppi 2, Luis A. Plana 2, Steve Furber 2, and Jörg Conradt 1 1 Fachgebiet Neurowissenschaftliche Systemtheorie, Fakultät für Elektro- und Informationstechnik, Technische Universität München, München, Germany {christian.denk,llobetblandino,conradt}@tum.de 2 Advanced Processor Technologies Group, School of Computer Science, University of Manchester, Manchester M13 9PL, UK {francesco.galluppi,plana,sfurber}@cs.man.ac.uk Abstract. Various custom hardware solutions for simulation of neural circuitry have recently been developed, each focusing on particular aspects such as low power operation, high computation speed, or biologically detailed simulations. The SpiNNaker computing system has been developed to simulate large spiking neural circuits in real-time in a network of parallel operating microcontrollers, interconnected by a high-speed asynchronous interface. A potential application area is autonomous mobile robotics, which would tremendously benefit from on-board simulations of networks of tens of thousands of spiking neurons in real-time. Currently, the SpiNNaker hardware circuit boards provide a single Ethernet interface for booting, debug, and input and output of data, which results in a severe bottleneck for sensory perception and motor control signals. This paper describes a small and flexible real-time I/O-hardware interface to connect external devices such as robotic sensors and actuators directly to the fast asynchronous internal communication infrastructure of the SpiNNaker neural computing system. We evaluate performance in terms of package throughput and present a simple application demonstration of a closed loop mobile robot interpreting visual data to approach the most salient stimulus. Keywords: massively-parallel simulation of spiking neurons, SpiNNaker, hardware interface board, mobile robotics. 1 Introduction and Related Work In computational neuroscience research various customized computing systems for the simulation of neuronal networks are under development, such as Facets/ BrainScales/ HBP [1], Neuro-grid [2], dedicated avlsi computing chips [3], or the SpiNNaker spiking network computing system [4]. All such hardware resembles brain style information processing, which in contrast to traditional general purpose computers offers various advantages, e.g. reduced power consumption or increased V. Mladenov et al. (Eds.): ICANN 13, LNCS 8131, pp , 13. Springer-Verlag Berlin Heidelberg 13

2 468 C. Denk et al. processing speed for elaborate or complex neural models. Most larger systems are designed for neuronal number crunching (i.e. detailed neuronal modeling) and exist in a closed computer rack with well controlled digital input and output channels. At the other side of the spectrum, there are small scale test-case implementations of neuromorphic functionality. Only very few such neuromorphic computing systems have yet operated in real time on noisily perceived sensory data and produced reasonable motor outputs to interact with the surrounding environment. Engineers and robotics systems designers however can strongly benefit from flexible instantiations of real-time neural information processing algorithms. Many robotic research groups agree that the neuronal style of information processing is advantageous for real time sensory processing and motor control, as this is what the brain and especially cortex is largely devoted to do. Current applications of neuromorphic hardware in closed loop systems typically keep a desktop computer in the loop to acquire sensory data from a robot, translate it into neural form, and provide it to the neuromorphic hardware and vice versa for motor output. Such a setup faces various inherent drawbacks: (a) large size and power consumption, which defies application on most mobile robots; (b) processing delays that might break real-time control loops; (c) no autonomy, which limits the operation range as a connection to a stationary computer is needed and (d) waste of resources, as the computer often only translates data. In this paper we present a standalone interface solution for a direct connection of various types of external hardware to the SpiNNaker [4] neural computing system, and present an application example that is autonomously executed in real-time onboard a mobile robot. The developed board is compact in size, supports high data transfer rates, requires little operating power (thereby allowing extended runtime on batteries), operates autonomously, offers various connection options for existing robots and sensors, and provides simple customizable extensions to additional sensors and actuators (or other robots). 2 An Autonomous Mobile Robot with a Neuronal Computing System 2.1 The SpiNNaker Neural Network Computing System The SpiNNaker computing system [4] is designed for massively-parallel computations of spiking neural networks. Each SpiNNaker chip consists of 16+2 generic ARM968 cores, a shared 128MByte SDRAM module, and an asynchronous high-speed communication interface with six bi-directional links. The chips execute arbitrary code in each of the 16 user accessible cores, but the overall system design is optimized for simulations of large spiking neural networks (e.g. LIF or Izhikevich neurons). Current SpiNNaker systems offer 4 chips (64 cores) or 48 chips (768 cores) with each core simulating up to 1000 neurons in real time [5], thereby allowing networks of ( ) spiking neurons. The fast asynchronous communication interface is designed to route neural action potentials from arbitrary neurons to a large number of other neurons. Various options for neural network implementations exist, from API library calls to interpreters of neural description languages such as PyNN [6] or NeNGO[7].

3 Real-Time Interface Board for Closed-Loop Robotic Tasks The Holonomic Mobile Robot Platform The mobile robot used in this project (Figure 1, left) is an omni-directional platform of 26cm diameter, with embedded low-level motor control and elementary sensors. An embedded microcontroller obtains motion commands in x and y direction and rotation through a UART interface, and continuously adapts three motor signals to maintain requested velocities. The robot s on-board sensors include wheel encoders, a 9 Degrees of Freedom (DOF) inertial measurement unit and a simple bump-sensor ring to trigger binary contact switches upon contact with objects. Fig. 1. Left: Autonomous mobile robot with on-board vision sensors, SpiNNaker hardware and interface board. Right: exemplary robot task: select the strongest out of multiple visual stimuli. 2.3 The Embedded Dynamic Vision Sensor The dynamic vision sensor (DVS) [9] used as spiking sensory input in this project is an address-event silicon retina that responds to temporal contrast. Each output spike represents a quantized change of log intensity at a particular pixel since the last event from that pixel. All 128x128 pixels operate asynchronously and signal illumination changes within a few microseconds after occurrence. We developed an embedded DVS system (edvs) [10] composed of a DVS chip connected to an ARM7 microcontroller that initializes the DVS and captures events. In this project the microcontroller streams all obtained events over a UART port into the SpiNNaker interface board (Section 3). 3 Design and Specifications of the Real-Time Interface Board Distributed computing cores in SpiNNaker exchange data on an energy and speed efficient asynchronous interface, which unfortunately is tedious to connect to external hardware. This section presents our developed interface board that attaches to this interface to send and receive native SpiNNaker packets, and to act as customizable interpreter between external hardware and the SpiNNaker system.

4 470 C. Denk et al. 3.1 The SpiNNaker Inter-Chip Communication Protocol and Interface Messages in the SpiNNaker system (typically neuronal spikes ) are transferred as 40-bit packets, composed of an 8-bit header and a 32-bit routing key. Packets can also carry an optional 32-bit payload. [5]. Communication between chips happens on two unidirectional asynchronous interfaces composed of 7 data lines (plus acknowledge), which are optimized for low energy consumption and fast data transfer rates: static levels on the data lines are meaningless, only transitions of bits encode a value. Any double bit flip on those 7 lines encodes the next nibble of the 40/72bits data word ( 2- of-7 code ), which needs to be acknowledged by toggling the signal on the respective acknowledge line. This protocol is fast (allows up to 6M packets per second) and energy efficient, but is difficult to implement for existing sensors and/or mobile robots. Hence we developed a generic interface board that on one side follows the communication protocol enforced in SpiNNaker, and on the other side offers various generic options to connect external hardware. Fig. 2. Left: Sketch of information flow SpiNNaker external hardware (e.g. edvs, robot). Right: SpiNN-3 system (4 chip board) with attached interface board. 3.2 The Developed SpiNNaker Interface Board The Interface Board receives and transmits SpiNNaker data packets in the 2-of-7 bit toggling format on one side (Figure 2, red connectors) and offers a variety of flexible interfaces for sensors and/or actuators on the other side (Figure 2, purple connectors). A fast on-board microcontroller (STM 32F407, 32bit ARM Cortex M4, 168MHz; Figure 2, green) allows flexible customization of translation protocols between SpiNNaker packets and sensor or actuators signals as described below. For efficiency we added a CPLD (XILINX Coolrunner-II, XC2C64A; Figure 2, blue) in the communication path, which translates between 2-of-7 bit-toggling codes and 9 bit data bus level signals for the microcontroller (8 data bits and 1 handshaking signal). All communication (SpiNNaker CPLD microcontroller) in both directions generates appropriate handshake signals to guarantee lossless transmission of data. The microcontroller consecutively retrieves all available data from SpiNNaker and connected peripherals and translates the data into the respective other format. After the translation, the data are forwarded to the respective devices as soon as possible (e.g. the SpiNNaker and/or UART transmit ports free).

5 Real-Time Interface Board for Closed-Loop Robotic Tasks 471 The presented interface board is easy to extend for upcoming system demands, even for users inexperienced with electronic hardware and/or microcontroller programming: the main-loop that continuously processes data is essentially a large lookup-table, which makes it easy to include different sensors and actuators without the need to be aware of SpiNNaker low-level programming (such as 2-of-7 bit toggling ) or routines to communicate over UART, SPI, or TWI. The developed interface board allows neural models running on SpiNNaker to receive sensory input signals and to control actuators. Performance evaluation in terms of number of sent / received packages is shown in Figure 3, left. 4 Application: Winner-Takes-All Network on Mobile Robot We demonstrate the SpiNNaker robot platform equipped with a forward pointing edvs performing a simple closed loop robotic task: various visual stimuli (lights with different flashing frequencies) are positioned at some distance of the robot. The system should identify and approach the most active stimulus (Figure 1, right). In our demonstration we implement a nonlinear robust Winner Takes All (WTA) [8] (Figure 3 right) neuronal network on the visual input to identify the most salient stimulus. All elements of this network are spiking neurons, which is well suited for the SpiNNaker platform. We demonstrate and evaluate the performance of our implementation in a closed-loop experiment. [MP/s] 0.5 Spinnaker to Interface Data Throughput Interface to SpiNNaker [MP/s] IF IF IF IF IF IF IF Fig. 3. Left: Transfer rates for simultaneous transmission and reception of SpiNNaker packets (32bit) in million packets per second. Right: Sketch of WTA network to process visual stimuli. 4.1 Winner Takes All Networks The WTA network is a well-established computing principle [8] to sharpen spatially and/or temporally related diffuse input signals. A WTA network can be described as a robust max operation: instead of identifying the maximum of several inputs at each time step, a WTA filter is a dynamical system that selects the maximum over a sliding window in time and possibly space, implementing hysteresis and thereby generating robust output [8]. Our implementation of the WTA network uses a layer of Integrate and Fire (IF) neurons to which all edvs events of one input column are propagated (Figure 3,

6 472 C. Denk et al. right). These IF neurons compete for activation, but only the most active neuron will reach its firing threshold and thereby is identified network winner. Such firing of a winning neuron has two consequences: (a) inhibition of its competitors, and (b) resetting/initializing itself to a non-zero membrane potential (self-excitation). This recurrence generates the desired hysteresis as discussed in [8]. The sketch of a sub-region of the full 128-node network in Figure 3, right, depicts event propagation and WTA implementation: the top grid shows a part of the edvs pixel map. Our selection problem is essentially a one-dimensional task, as all pixels in a column (y) for any given row (x) support the same driving direction. This grouping results in a one-dimensional visual input vector of size 128, represented by the activity levels in designated neurons. We apply a spatial low-pass distribution on the input signal (green Gaussian in Figure 3, right) that produces strong excitation at the center neuron and symmetrically decayed excitation at its neighbors. Stimulus Location S S1,S Temporal S2 (125Hz) Activity: S3 (85Hz) S1 (100Hz) Stimulus Location S2 S1,S3 Neuron Index Neuron Index WTA Stationary Input and Output Neuron Potentials Time [s] Visual input event WTA winning neuron Magnified View firing threshold reached 29,00 29,025 29,05 Fig. 4. Top Left: Stimulus activation (black bars) and corresponding evoked visual events over time (red dots). Bottom Left: activation of WTA neurons. Right: time magnified view. 4.2 WTA Implementation on SpiNNaker Hardware The developed interface board (Section 3) translates and propagates all edvs visual input events as native SpiNNaker packets, conveying the x-coordinate as source address. These packets are injected at the lower left chip on the SpiNNaker board, and distributed evenly among several other SpiNNaker chips and cores. For simplicity (as this is a proof-of-concept implementation) each core implements only a single IF neuron, who s potential is augmented by incoming events according to the Gaussian weighting function. Upon reaching firing threshold, an output spike causes all other distributed neurons to decrease their respective potentials (see Figure 4, right). Various dedicated motor neurons detect WTA output spikes and compute temporally low-pass filtered rate-encoded driving signals, which are sent through the SpiNNaker interface board to the robot.

7 Real-Time Interface Board for Closed-Loop Robotic Tasks Evaluation of the Demonstration System We demonstrate our implementation of the WTA network in two different scenarios: (a) stationary with multiple different alternating stimuli and (b) on an autonomous robot driving towards partially occluded stimuli (Figure 1, right). Robot X/Y Location [cm] Sensor Pixel X Stimulus 85Hz Stimulus 100Hz Time [s] Fig. 5. Closed loop SpiNNaker-robot experiment: WTA alternatively identifying winner (blue) out of all stimuli (red dots); robot continuously approaches (and centers) the respective winner For scenario (a), we provide three distinct LED stimuli (S1-3), each flashing with a particular frequency (see Figure 4, left). We position all stimuli so that the two low frequency stimuli are at roughly similar x coordinates (around x=38), whereas the most active stimulus is located elsewhere in the field-of-view (around x=110). The LED stimuli are turned on according to the timeline (black bar) in Figure 4. The upper graph shows input events (red dots) and WTA output spikes (blue dots). The lower graph displays WTA neuron integrator activation over time (darker parts indicate higher activation). Initially, with only S1 active, the WTA network identifies x=38 as center of activation, and provides a unique, spatially stable output firing despite background noise and a slightly broadened input signal distribution. After activation of S2 (with increased activity compared to S1) at t=10s, the WTA network transitions to this stimulus as winner; again identifying a spatially stable unique winning location. Additionally activating S3, which has the lowest frequency of all, but is spatially co-located with S1, yields a return to the first winning location, due to the Gaussian distribution of input signals to neurons, which allows the two less active stimuli to surpass the most active stimulus in sum. The magnified view (Figure 4, right) shows the neuronal activation over time; note the spatially distributed increase of activation around stimuli, and global inhibition and local self-excitation after a neuron fired. In demonstration scenario (b), an autonomous mobile robot is controlled by the WTA output (Figures 1 and 5). The WTA network focuses on the most active stimulus, which causes the robot to continuously turn towards and approach that stimulus. In the closed loop system, the winning stimulus approaches the center of the vision sensor (pixel 64) as the robot turns (Figure 5; inset shows the robot s trajectory in top-down view). We repeatedly occlude the stronger stimulus (see activation bars in Figure 5), which produces alternating robot motion towards the stronger stimulus, thereby demonstrating switching behavior of the WTA network.

8 474 C. Denk et al. 5 Results and Discussion The SpiNNaker computing system provides a powerful and easy to learn neuronal computing infrastructure for computational modelers, which allows simulation of large scale spiking neural system in real-time. However, scenarios with real-time input/output currently require a PC in the loop or are custom developed FPGA chips for a particular piece of hardware, because of the SpiNNaker internal communication bus which is incompatible with existing sensors and/or robots. In this project we presented a solution to flexibly interface various external hardware (such as sensors and/or robots) to the SpiNNaker computing system. The developed interface is small, allows high data transfer rates (sufficient even for visual data), and is easily customizable for future additional sensors and actuators without requiring in-depth knowledge about the SpiNNaker communication protocol. We demonstrated the performance of the developed system in two example settings: (a) stationary sensors with variable stimuli and (b) in an autonomous closed loop robotic experiment. The presented application shall be viewed as a proof-of-principle, not as an exhaustive evaluation of the board. In fact the demonstration only requires a small subset of the implemented features (e.g. significantly higher data rates are possible, refer to Figure 3, left). We are currently using the interface board in ongoing research such as a stereo optic flow processing and a neural model of grid cells for navigation. References 1. Pfeil, T., Grübl, A., Jeltsch, S., Müller, E., Müller, P., Petrovici, M.A., Schmuker, M., Brüderle, D., Schemmel, J., Meier, K.: Six Networks on a Universal Neuromorphic Computing Substrate. Frontiers in Neuroscience 7 (13) 2. Choudhary, S., Sloan, S., Fok, S., Neckar, A., Trautmann, E., Gao, P., Stewart, T., Eliasmith, C., Boahen, K.: Silicon Neurons That Compute. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 12, Part I. LNCS, vol. 7552, pp Springer, Heidelberg (12) 3. Badoni, D., Giulioni, M., Dante, V., Del Giudice, P.: An Avlsi Recurrent Network of Spiking Neurons with Reconfigurable and Plastic Synapses. In: IEEE (ISCAS), pp (06) 4. Khan, M., Lester, D., Plana, L.A., Rast, A., Jin, X., Painkras, E., Furber, S.B.: SpiNNaker: Mapping Neural Networks onto a Massively-Parallel Chip Multiprocessor. In: IEEE International Joint Conference on Neural Networks (IJCNN), pp IEEE (08) 5. Plana, L.A., Bainbridge, J., Furber, S., Salisbury, S., Shi, Y., Wu, J.: An on-chip and Inter- Chip Communications Network for the SpiNNaker Massively-Parallel Neural Net Simulator. In: 2nd ACM/IEEE NoCS, pp IEEE (08) 6. Galluppi, F., Davies, S., Rast, A., Sharp, T., Plana, L.A., Furber, S.: A Hierachical Configuration System for a Massively Parallel Neural Hardware Platform. In: Proceedings of the 9th conference on Computing Frontiers, pp ACM (12) 7. Galluppi, F., Davies, S., Furber, S., Stewart, T., Eliasmith, C.: Real Time on-chip Implementation of Dynamical Systems with Spiking Neurons. In: IJCNN, pp IEEE (12) 8. Oster, M., Douglas, R., Liu, S.-C.: Computation with Spikes in a Winner-Take-All Network. Neural Computation 21, (09) 9. Lichtsteiner, P., Posch, C., Delbruck, T.: A dB 15ms Latency Asynchronous Temporal Contrast Vision Sensor. IEEE Solid-State Circuits 43, (08) 10. Conradt, J., Berner, R., Cook, M., Delbruck, T.: An Embedded AER Dynamic Vision Sensor for Low-Latency Pole Balancing. In: IEEE ECV, pp IEEE (09)