A programming environment to control switching. networks based on STC104 packet routing chip 1

Size: px

Start display at page:

Download "A programming environment to control switching. networks based on STC104 packet routing chip 1"

Lenard Baker
5 years ago
Views:

1 A programming environment to control switching networks based on STC104 packet routing chip 1 I.C. Legrand 2, U. Schwendicke, H. Leich, M. Medinnis, A. Koehler, P. Wegner, K. Sulanke, R. Dippel, A. Gellrich DESY - Institut fur Hochenergiephysik, Zeuthen, Platanenallee 6, D Zeuthen, Germany The software environment used to control a large switching architecture based on SGS-Thomson STC104 (an asynchronous 32-way dynamic packet routing chip) is presented. We are evaluating this switching technology for large scale, real-time parallel systems. A Graphical User Interface (GUI) written as a multi-thread application in Java allows to set the switch conguration and to continuously monitor the state of each link. This GUI connects to a multi-thread server via TCP/IP sockets. The server is running on a PC-Linux system and implements the virtual channel protocol in communicating with the STC104 switching units using the Data Strobe link or the VME bus. Linux I/O drivers to control the Data Strobe link parallel adaptor (STC101) were developed. For each client the server creates a new thread and allocates a new socket for communication. The Java code of the GUI may be transferred to any client using the http protocol providing a user friendly interface to the system with real-time monitoring which is also platform independent. 1 Introduction This work is part of a test project for the third level trigger and on-line full event reconstruction for the HERA-B experiment[1]. The high event rate (10 MHz - corresponding to the bunch crossing rate) with multiple interactions per bunch crossing will produce more than 10 7 particles per second per square centimeter in the innermost detector region. The event rate is expected to be reduced by about ve orders of magnitude by a three-level trigger system. The 1 AIHENP'96 SE Corresponding author. Tel.: , fax: , legrand@ifh.de Preprint submitted to Elsevier Preprint 11 November 1996

2 rst and second level trigger will operate on a limited range of data, due to the hard time constraints for these systems. In the data acquisition scheme the event building is performed after the second level trigger decision. The events are then routed to the third level trigger, a farm of high performance processors acting in parallel on successive events in order to keep up with an expected input rate of up to 2000 events/s. For the events which have passed the third level trigger, complete event analysis will have to be performed \online", on the same architecture, because of long running periods and high event rates of the experiment. High-performance switching units are necessary for the event building task[2] and to distribute events for triggering and on-line reconstruction to a large farm of processors. The same network has to be used also to control the load of the system. A test farm based on Data Strobe (DS) link technology[3] is under construction to evaluate the real-time behavior of such a large scale parallel system. 2 The Data Strobe Link Technology The DS link protocol is based on the IEEE P1355 standard. It is pointto-point serial bi-directional connection running at 100 Mbits/s. A DS link is a four wire protocol with two wires data and strobe in each direction. This technology has been developed in the Open Microprocessor Systems Initiative/Heterogeneous InterConnect Project. This standard has been implemented by SGS-THOMSON Microelectronics in the STC101 (link driver chip)[4] and STC104 (32 way routing chip)[5]. 2.1 The STC101 parallel DS-link adapter The STC101 adapter drives a DS link at 100 Mbits/s full duplex and has 16/32 bit parallel interface. It may be used in two modes: { If internal packetizing is exploited the application has to provide the routing and packet size information to control registers and then supply the data. { Without internal packetizing the characters are transmitted transparently. Interfacing this chip with a PCI-bus by using fast FIFOs for matching the speed provides a quite high bandwidth I/O channel for standard processor architectures. 2

3 2.2 The STC104 asynchronous packet switch The STC104 is a low latency asynchronous packet routing chip for the DS protocol. It connects 32 DS bidirectional link ports via a non-blocking crossbar switch thus enabling packets to be routed from any one of its links to any other link. The 32 links operate asynchronously allowing packets of any length to be routed between a link pair without aecting the packet routing between any other link pair. The maximum bandwidth is 300 Mbytes/s with a latency of less than 1 s. To avoid store and forward buering problems, the STC104 uses wormhole routing in which the routing decision is taken from the packet header as it arrives. The destination address is compared against pre-loaded intervals to select the packet exit port. In this way, packets of arbitrary length can be handled with full ow control. 3 Network Components Events are kept in the Second Level Buers (SLB) until decisions are taken by the Second Level Trigger. One SLB unit[6] is a VME board having six Digital Signal Processors (DSP), AD SHARC[7], interconnected through the global bus. These are the data sources for the farm. VME boards containing two AD DSPs and eight STC101s will provide the interface between the DSP link protocol and the DS protocol. These units are also used to perform a partial event building and to allow temporary data buering for randomizing the trac through the switch. This is necessary because the event building task is a \all to one" routing problem. The interface between the processing nodes and the DS links is done by a PCI card equipped with a STC101 and fast FIFO memories. Two versions have been developed for PCs and PowerPC VME boards. PC cards with two STC101 interfaces (SGS-Thomson B108) are also used for testing and controlling the switch units. A standard VME module containing a STC104 switch and a VME to DS link interface[8] was designed as a basic unit for this large switch. The block diagram of this module is shown in Figure 1. The STC104 on this module can be congured and tested either through the VME bus or through the control DS link from the front panel. 3

4 4 The software environment To control and monitor such a network several software packages are necessary. A PC running Linux 2.0 is used as the host system and is equipped with a VME interface (and therefore may act as a VME controller), a B108 card and PCI-C101 interface card. This system may be used to control and monitor a large switching network. 4.1 The Network Description Language The Network Description Language (NDL) allows the user to describe the hardware that makes up the network. In our case this description is used to initialize and to congure the hardware. NDL is a declarative language. The network description has to provide: { declarations of the devices and their communication ports, { denitions of values of attributes of devices, { a description of the connection between devices. The NDL sources can be checked and compiled by the NDL compiler to produce a Network Information File (NIF) which can be downloaded to initialize the network. 4.2 The Virtual Channel Library This C package implements a simple Virtual Channel Protocol to provide packetization and virtual channel support for communication between a processor interface and a DS-link and STC104 network. It contains modules which can read NIF les and produce special messages for the hardware conguration. Synchronous and asynchronous I/O functions are provided for the user level. All these I/O functions perform the handshake protocol for the data ow control. This library was interfaced with Linux I/O drivers for the B108 card and the VME interface. Both interfaces may be used to congure and monitor STC104 modules. 4

5 4.3 The Switch Server The Switch Server is an interface between the part which is accessing the hardware through the Virtual Channel Protocol and the user. For such large experiments it is necessary to serve multiple clients concurrently. A multi-thread approach is a natural way to solve this problem by creating dynamically a new thread for each new client. TCP/IP sockets are used for the communication between the server and the clients. 4.4 The Graphical User Interface The Graphical User Interface (GUI) has been done in Java(tm) (Sun Microsystems). Java is an object oriented language which is platform independent multi-threaded. It is an easy to use environment for creating complex, distributed applications. This GUI may be downloaded (the code) by any client using the http protocol and can run with most of the web browsers. Once the code is downloaded at the client side, it creates a TCP/IP connection with the main server task by using a \meeting" socket. The server is listening to this port and when a new client tries to connect, the server creates a new socket for communication with this client and a new thread for serving. The GUI is itself a multi-thread application allowing an ecient use of the resources. The main panel of the GUI handles the conguration buttons and starts the switch panel thread. This panel creates a client object which will handle the communication with the server. Each link is represented by an object which encapsulates the link parameters and provide methods for the graphical representation. These objects are instantiated by the switch panel thread. Each user command is transmitted to the server as a code allowing to choose a predened conguration for initializing the switch and to reset the chip or only one link. It also provides debugging functions to examine all the registers of a certain link or to read any STC104 register. The monitoring part is done by the switch panel thread which tries to update every second the status of all the links by exchanging messages with the server. In this way, the information necessary for monitoring which is transferred through the network is minimum (two encoded bytes per link) allowing a prompt graphical interface. A schematic diagram of the main objects and how they interact is presented in gure 2. Currently a demo version of these programs and related documents are available at 5

6 5 Summary We tried to develop a quite general software environment to control and monitor a large switching network based on STC104. Structuring the packages necessary for such a task combined with a client-server protocol provides a clear and easy to use software. An adequate graphical user interface developed in Java, as a multi-thread application, which communicates with a server task to control the hardware, allows multiple clients to set the conguration and to monitor the system. It is platform independent and uses the resources at the client side. Such graphical interfaces are useful to control, test and debug large systems. This approach may be easily extended to other components in the DAQ and trigger scheme for large experiments. Acknowledgement We would like to thank H. Kolanoski from the Humboldt-University/Berlin for useful suggestions. References [1] HERA-B An Experiment to Study CP Violation in B System Using an Internal Target at the HERA Proton Ring, Design Report, DESY - PRC 95/1. [2] R.Heely et al., The application of the T9000 Transputer to CPLEAR experiment at CERN, NIM A, 368 (1996) 666. [3] IEEE Draft Std P1355 Standard for Heterogeneous InterConnect (HIC). IEEE Inc [4] SGS-Thomson Microelectronics, ST C101 Parallel Link Adaptor, June [5] SGS-Thomson Microelectronics, ST C104 Asynchronous Packet Switch, June [6] D. Ressing, R. Wurth, Implementation of the SLB using SHARCs, HERA-B report [7] Analog Devices, ADSP-2106x SHARC User's Manual. [8] U. Schwendicke, WOROS, A Wormhole Routing Switch for Data Strobe Links, DESY-IFH Internal Note. 6

Hera-B DAQ System and its self-healing abilities

Hera-B DAQ System and its self-healing abilities V.Rybnikov, DESY, Hamburg 1. HERA-B experiment 2. DAQ architecture Read-out Self-healing tools Switch SLT nodes isolation 3. Run control system 4. Self-healing