Characterizing the Performance of SpaceWire on a LEON3FT. Ken Koontz, Andrew Harris, David Edell

Transcription

1 Characterizing the Performance of SpaceWire on a LEON3FT Ken Koontz, Andrew Harris, David Edell

2 Introduction SpaceWire is emerging as standard high-performance data interface Recent NASA missions include LRO/Mini-RF, SDO, JWST Future NASA missions may include SPP, ILN/RLL, JEO SpaceWire interfaces are becoming standard on new flight HW BAE and Aeroflex have processors with SpaceWire SpaceWire advertises 2 to 400 Mbps, but Are these speeds really achievable, and under what conditions? As part of an IR&D project, we characterized the performance of SpaceWire on a LEON3FT. What we found was interesting 2

3 SpaceWire Background ESA standard ECSS-E-ST-50-12C (31 July 2008), plus several protocol extensions (-51C, -52C, -53C, ) SpaceWire packets are composed of a sequence of data characters, followed by an End-Of-Packet (EOP) character No explicit length field, which simplifies routing and router design Data characters are 10 bits: 8-bits data, 2-bits control Maximum uni-directional transfer rate is 80% signaling rate (e.g., 10 MHz signaling rate == 8 Mbps data rate). SpaceWire packets include a destination address and protocol ID Destination address used for routing (path or logical addressing) Protocol ID used at each end-node to identify protocol stack For our experiment, we used: Logical addressing (single byte), point-to-point (no router) CCSDS Packet Transfer Protocol (CPTP), one SpaceWire packet contained one CCSDS telemetry packet, single byte protocol ID 3

4 LEON3FT Background SPARC V8 system-on-chip, developed by Gaisler Research Aeroflex UT699 is a flight-qualified, rad-hard LEON3FT realization SPARC V8 with FPU and MMU 4 on-chip SpaceWire ports (end-nodes only, no router functions) 2 CAN, 1 PCI, 1 Ethernet, 1 memory controller w/edac 4

5 The Experiment Measure uni-directional performance in flight-like conditions : Point-to-point SpaceWire link (no router, no blocking, no topology issues) Vary link speed over a predetermined range (2, 5, 10, 25, and 50 MHz) Vary packet size over a predetermined range (2^N bytes, N=4..16, 16 to 64 Kbytes) Set transmit and receive speeds the same (adjusted at outermost loop) Send 50,000 packets at each configuration in innermost loop Measure start, end, elapsed time at receiver Measure CPU load at receiver using flight-like background monitor Send CCSDS packets over SpaceWire using CPTP, with standard processing on transmitter and receiver (buffer pools, checksums, sequence counters) Hardware 2 Aeroflex LEON3FT commercially available development boards SYS_CLK = 66 MHz, SPW_CLK = 100 MHz System software VxWorks 6.5 Aeroflex/GR BSP version GNU compiler, -O2 optimization level 5

6 Packets Per Second vs. SpaceWire Packet Size 6

7 Link Utilization (%) vs. SpaceWire Packet Size 7

8 CPU Load (%) vs. SpaceWire Packet Size 8

9 Effective Data Throughput (Mbps) vs. SpaceWire Packet Size 9

10 Observations With small packets, we observed an upper limit of ~4200 packets/ sec (or 238 microsecs/packet). As packet size increased, processing load remained high until the effect of the DMA units kicked in. Where this occurred varied by link speed. We were able to achieve very low CPU load (< 10%) and very high link utilization (> 90%) at 2, 5, and 10 MHz. We pegged the CPU load at 100% at 25 & 50 MHz link speeds, until the DMA units kicked in (2K MHz, 8 MHz) We were only able to achieve 60% link utilization at 50 MHz, even with very large 64 Kbyte packets. This also cost 30% CPU load. We did not test 100 MHz (50 MHz LEON3FT max rate). Performance expected to be worse than 50 MHz, even for large packets. We were surprised that 1 Kbyte packets at 10 MHz required 50% CPU load and only 80% link utilization. 10

11 Limitations and Optimizations If you need to use checksums, and implement them in software then expect to pay a price! Avoid copying packets as much as possible! In the stock driver, packets are DMA d into a driver buffer, and then copied to an application buffer. This can kill performance with large packets! The driver and application should use a common buffer pool Pass pointers to buffers between software elements Copy data only on TX and RX, and with the aide of DMA This is common practice with slower interfaces such as 1553 (1 Mbps) SpaceWire drivers and applications need to play catch-up Use interrupts only when they are necessary in the application The GRSPW interfaces can support DMA wo/interrupts The VxWorks driver only supports DMA w/interrupts If speed is the utmost concern, remove VxWorks and code in Bare C! 11

12 Questions? 12