Performance of ORBs on Switched Fabric Transports

Size: px

Start display at page:

Download "Performance of ORBs on Switched Fabric Transports"

Andrea Stokes
6 years ago
Views:

1 Performance of ORBs on Switched Fabric Transports Victor Giddings Objective Interface Systems 2001 Objective Interface Systems, Inc.

2 Switched Fabrics High-speed interconnects High-bandwidth, low latency switched circuits Adaptive routing through alternate paths DMA transfers between memories of processors Processor Memory Cross bar Switch Processor Memory 2001 Objective Interface Systems, Inc. 2 2

3 Motivation: ORB Performance on Ethernet Most ORB performance studies have used TCP over Ethernet Most common use of CORBA Well-known performance Prediction of an ORBs performance using TCP over Ethernet CPU speed is largest determinant of performance Startup latency is dominated by processing in the protocol stack Data throughput is dominated by marshalling (copy) time Extrapolating performance is a matter of scaling CPU speeds Problem: how to predict ORB performance on switched fabric transports 2001 Objective Interface Systems, Inc. 3 3

4 Context: ORB Switched Fabric Transports ORBexpress transports developed for two different switched fabric technologies Mercury Computing s RACEway Joint development with Mercury Computing Myrinet (CSPI & Myricom) Performance results shows Extremely low latency Low variability 2001 Objective Interface Systems, Inc. 4 4

5 Example Latency Results CSPI 2841 ORBexpress Latency - CSPI 2841 Switched Fabric Transport - LongSeq TCP - LongSeq usec ,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000 Bytes Transferred 2001 Objective Interface Systems, Inc. 5 5

6 Comparison - Latency Comparsion Model 2641 vs. Model 2841 Latency 1,000, ,000 10,000 usec 1, ,000 10, ,000 1,000,000 10,000,000 bytes transferred 2841 DoubleSeq 2641 DoubleSeq 2001 Objective Interface Systems, Inc. 6 6

7 Ratio of Latencies Ratio - Latency Model 2641 vs. Model , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 bytes transferred 2001 Objective Interface Systems, Inc. 7 7

8 Performance Model First Attempt Use Ethernet model Latency = ORB overhead + Transport Propagation delay ORBexpress provides a mirror transport Reflects requests back to collocated sender Directly measures ORB overhead including marshalling Simple model: Latency = ORB latency + Number of bytes * Propagation per byte 2001 Objective Interface Systems, Inc. 8 8

9 Performance Model Result of First Attempt 80,000 70,000 60,000 50,000 usec 40,000 30,000 20,000 10, , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 bytes transferred Actual Simple Prediction 2001 Objective Interface Systems, Inc. 9 9

10 Performance Model - Refinement Refined hardware block diagram Processor Memory Bus ASIC Cross bar Switch Memory Account for propagation delay on memory bus 2001 Objective Interface Systems, Inc

11 Performance Model Result of Refinement 80,000 70,000 60,000 50,000 usec 40,000 30,000 20,000 10, , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 bytes transferred Simple Prediction Actual Refined Prediction 2001 Objective Interface Systems, Inc

12 Performance Model Prediction of Ratios Ratio vs , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 bytes transferred Actual Refined Prediction 2001 Objective Interface Systems, Inc

13 Bandwidth ORB over Switched Fabric Transport ORBexpress Throughput MyriTransport - LongSeq MB/s , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Bytes Transferred 2001 Objective Interface Systems, Inc

14 Bandwidth Attained bandwidth is small part of available transport bandwidth Mirror transport bandwidth offers clue to cause Since inverse bandwidths add (additional time per byte adds) Transport bandwidth must be combined with ORB bandwidth ORB bandwidth factors Startup latency insignificant for significant byte counts Memory copies 2001 Objective Interface Systems, Inc

15 Memory Bandwidth Copy Inverse Rate Memory Copy Inverse Rate ( MHz PPC7400 with Altivec) nanosec/byte , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Bytes Copied 2001 Objective Interface Systems, Inc

16 ORB Bandwidth ORB Bandwidth Measures the ability of the ORB to transfer volumes of data ORB bandwidth is dominated by The number of copies And the memory bandwidth of the processor Increasing an ORBs bandwidth Requires elimination of copying Motivation for High Performance Enablers RFP 2001 Objective Interface Systems, Inc

17 Bandwidth - Predicted Predicted vs. Attained Bandwidth MBps , , , ,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 bytes transferred Predicted Bandwidth Attained Bandwidth 2001 Objective Interface Systems, Inc

18 Summary Examined two aspects of ORB performance over switched fabric transports Latency Prediction is more complicated than for TCP/IP over Ethernet or Loopback More complex model needed Bandwidth Prediction is more straightforward Introduced concept of ORB Bandwidth ORB bandwidth is dependent on the number of copies 2001 Objective Interface Systems, Inc

Using a Real-time, QoS-based ORB to Intelligently Manage Communications Bandwidth in a Multi-Protocol Environment

Using a Real-time, QoS-based ORB to Intelligently Manage Communications Bandwidth in a Multi-Protocol Environment Bill Beckwith Objective Interface Systems, Inc. +1 703 295 6500 bill.beckwith@ois.com http://www.ois.com