Vertically and horizontally High-performance, Real-time ORBs Motivation Many applications require æ guarantees QoS e.g., telecom, avionics, WWW Existi

Transcription

1 Principles and Patterns of High-performance, Real-time Object Request Brokers C. Schmidt Douglas University, St. Louis Washington Typeset by FoilTEX

2 Vertically and horizontally High-performance, Real-time ORBs Motivation Many applications require æ guarantees QoS e.g., telecom, avionics, WWW Existing middleware doesn't æ QoS eæectively support e.g., CORBA, DCOM, DCE æ Solutions must be integrated Washington University, St. Louis 1

3 High-performance, Real-time ORBs æ Goals of CORBA Candidate Solution: CORBA Simplify distribution Provide foundation for higher-level services æ Limitations of CORBA Poor performance Lack of QoS features Washington University, St. Louis 2

4 High-performance, Real-time ORBs The ACE ORB ètaoè æ TAO Overview A high-performance, ORB real-time Networking and æ focus avionics Leverages the ACE framework Ported to VxWorks, æ and Win32 POSIX, æ Related work QuO at BBN ARMADA at U. Mich. Washington University, St. Louis 3

5 æ Related work High-performance, Real-time ORBs The ADAPTIVE Communication Environment èaceè æ ACE Overview A concurrent OO framework networking Very widely used in industry Available in C++ and Java Ported to VxWorks, and Win32 POSIX, x-kernel Washington University, St. Louis 4

6 High-performance, Real-time ORBs Applying ORBs to Real-time Avionics æ Domain Challenges Periodic hard deadlines real-time COTS infrastructure Open systems æ Related work Deng, Liu, and J. '96 Sun Gopalakrishnan and '96 Parulkar Wolfe et al. '96 Washington University, St. Louis 5

7 High-performance, Real-time ORBs Applying ORBs to Real-time Network Management æ Domain Challenges Periodic statistical deadlines real-time COTS infrastructure Open systems æ Related work Deng, Liu, and J. '96 Sun Gopalakrishnan and '96 Parulkar Wolfe et al. '96 Washington University, St. Louis 6

8 High-performance, Real-time ORBs Research Objectives æ Identify features and architectural patterns needed for real-time ORBs Both hard real-time and statistical real-time æ Develop optimizations required to build high-performance ORBs e.g., Gigabit bandwidth and ç10 microsecond latency æ Determine changes needed to CORBA speciæcation e.g., APIs for deæning end-to-end QoS requirements Washington University, St. Louis 7

9 High-performance, Real-time ORBs Real-time Features and Optimizations in TAO Washington University, St. Louis 8

10 High-performance, Real-time ORBs Experimental Setup for CORBAèATM Testbed Services Requests Client Object Adapter ORB Core Server ATM Switch Ultra 2 Ultra Washington University, St. Louis 9

11 High-performance, Real-time ORBs Problem: Meeting End-to-End QoS Requirements æ Design Challenges Specifying QoS requirements Reducing latency demultiplexing Meeting scheduling deadlines Reducing presentation overhead layer Washington University, St. Louis 10

12 Provide Oè1è demuxing operation High-performance, Real-time ORBs æ Design Challenges Minimize demuxing layers Problem: Reducing Demultiplexing Latency Avoid priority inversions Remain CORBA-compliant Washington University, St. Louis 11

13 High-performance, Real-time ORBs æ Solution Approach Pre-negotiate keys demuxing Tunnel demuxing with Object key key Solution: De-layered Active Demultiplexing Use ACT pattern validation for æ Related Work Yau and Lam '97 Dittia and Parulkar '97 Engler and Kaashoek '96 Washington University, St. Louis 12

14 æ Results at High-performance, Real-time ORBs Demultiplexing Performance Experiments Linear search based on Orbix demuxing strategy æ Perfect hashing based on GNU gperf æ Washington University, St. Louis 13

15 High-performance, Real-time ORBs Demultiplexing Performance Results æ Synopsis gperf solution is 100è but static compatible, Active demuxing isn't compatible, but 100è is dynamic Washington University, St. Louis 14 Latency in microseconds Number of Objects Active Demux (1 Method) Active Demux (10 Methods) Active Demux (100 Methods) GPERF (1 Method) GPERF (10 Methods) Demultiplexing scheme GPERF (100 Methods) Linear (1 method) Linear (10 methods) Linear (100 methods)

16 Focus on Objects and Operations High-performance, Real-time ORBs Problem: Meeting CORBA Request Deadlines æ Design Challenges Specifyingèenforcing requirements QoS Not on threads or æ channels comm. æ Assumptions Static scheduling Non-distributed èinitiallyè Washington University, St. Louis 15

17 High-performance, Real-time ORBs æ Solution Approach Integrate RT dispatcher ORB into Solution 1: Real-time Object Adapter Support multiple request strategies scheduling e.g., RMS, RMS with æ Preemption, Deferred and EDF æ Related work Zinky, Bakken, and '95 Schantz, Lee, Rajkumar, and '96 Mercer Washington University, St. Louis 16

18 High-performance, Real-time ORBs Solution 2: Real-time Scheduling Service Washington University, St. Louis 17

19 High-performance, Real-time ORBs Scheduling Service Roles æ Components Oæine Assess schedule æ feasibility Assign thread and æ priorities queue Online Supply priorities to æ Adapter's Object dispatcher Washington University, St. Louis 18

20 Application interface Use RT Infos æ High-performance, Real-time ORBs Scheduling Service Interfaces æ Components Privileged interface Used by system æ tasks and services Washington University, St. Louis 19

21 High-performance, Real-time ORBs Scheduling Steps During Conæguration Run Washington University, St. Louis 20

22 RT Info references Vector of RT Tasks by each RT Task called Vector records æ High-performance, Real-time ORBs Scheduling Service Internal Repository æ Components dependencies Called-task chains are to compute traversed CPU time and total minimum period Washington University, St. Louis 21

23 æ Available at High-performance, Real-time ORBs Real-time Dispatching Experiments Washington University, St. Louis 22

24 High-performance, Real-time ORBs Key Patterns in TAO æ Deænition ëa recurring solution a design problem in to a particular context" æ Beneæts of Patterns Facilitate design reuse Preserve crucial design information Guide design choices Document common and pitfalls traps Washington University, St. Louis 23

25 High-performance, Real-time ORBs Real-time Event Channel Overview Consumer Consumer Consumer Real-time Event Channel æ Features push (event) Consumer Proxies Scheduling Correlation dependencies EVENT CHANNEL Dispatching Module Event Correlation Event Flow Filtering Subscription & Filtering Priority Timers Supplier Proxies Supplier push (event) Supplier Supplier Washington University, St. Louis 24

26 High-performance, Real-time ORBs CONNECT_PUSH CONSUMER Consumer Object Ref Scheduling QoS Correlation Specs Subscription Info Timeout Registration EVENT CHANNEL Dispatching Module Event Correlation Subscription & Filtering Priority Timers Consumer Proxies Supplier Proxies CONNECT_PUSH SUPPLIER Publish Types Supplier Object Ref Collaboration in the RT Event Channel Washington University, St. Louis 25

27 High-performance, Real-time ORBs RT Event Channel Use-cases Avionics Network management Washington University, St. Louis 26

28 High-performance, Real-time ORBs Timeline for Multi-threaded Object Adapter Washington University, St. Louis 27

29 High-performance, Real-time ORBs Timeline for FIFO Object Adapter Washington University, St. Louis 28

30 High-performance, Real-time ORBs Applying CORBA to Medical Imaging æ Domain Challenges Large volume of ëblob" data e.g., 10 to 40 Mbps æ Lossy compression isn't viable Prioritization of requests Washington University, St. Louis 29

31 æ Design Challenges High-performance, Real-time ORBs Problem: Reducing Protocol Engine Overhead Small memory footprint Predictable performance Minimize the typecode interpreter overhead Washington University, St. Louis 30

32 High-performance, Real-time ORBs Solution: TypeCode Interpreter Optimizations æ Solution Approach Optimized Typecode Interpreter Based on SunSoft engine IIOP æ Related work Hoschka '97 O'Malley, Proebsting, and Montz '94 Washington University, St. Louis 31

33 32 bytes èè BinStruct struct s; char c; long l; short o; double d; octet padë8ë; octet ; sequenceébinstructé typedef StructSeq; High-performance, Real-time ORBs TypeCode Layout for Sequence of BinStructs TypeCode Description æ CDR format in Washington University, St. Louis 32

34 High-performance, Real-time ORBs Throughput of the SunSoft IIOP Implementation Throughput in Mbps shorts longs chars/octets doubles structs TCP/IP Sender Buffer Size in Kbytes æ Experimental design Transfer 64 Mbytes of data ëoneway" Various types of data Washington University, St. Louis 33

35 High-performance, Real-time ORBs æ Problem Challenges of Optimizing Complex Softare Optimizing complex software is hard Small ëmistakes" are costly over high-speed networks æ Solution Approach èiterativeè Pinpoint sources of overhead via white-box metrics e.g., Quantify, TNF, etc. æ Apply optimization principles Validate via white-box and black-box metrics Washington University, St. Louis 34

36 High-performance, Real-time ORBs Optimization Principles Principle Number Optimize for the common case 1 Eliminate gratuitous waste 2 Replace ineæcient general-purpose 3 with eæcient special-purpose ones methods Precompute values, when possible 4 Store redundant state to speed up expensive operations 5 Pass information between layers 6 Washington University, St. Louis 35

37 $ è High-performance, Real-time ORBs Sender-side Analysis of SunSoft IIOP Implementation ' 1.55 write put_longlong CDR::encoder TypeCode::traverse $ ' write get_long calc_nested_size_and_al ignment CDR::encoder & & è TypeCode::traverse double struct Percent Execution Time for doubles and structs Washington University, St. Louis 36

38 $ è High-performance, Real-time ORBs Receiver-side Analysis of SunSoft IIOP Implementation CDR::get_long ' Typecode::traverse CDR::get_longlong deep_free CDR::decoder read TypeCode::kind ' $ calc_nested_size_and_ alignment struct_traverse CDR::decoder TypeCode::traverse deep_free CDR::skip_string CDR::get_byte & double & è struct Percent Execution Time for doubles and structs Washington University, St. Louis 37

39 High-performance, Real-time ORBs æ Problems Problems and Solutions Invocation overhead for small, frequently called methods æ Solution Inline method calls æ Principle Optimize for the common case Washington University, St. Louis 38

40 & & è $ è High-performance, Real-time ORBs Throughput After 1st Optimization ' ' $ Throughput in Mbps Original Inlining Throughput in Mbps Original Inlining Sender Buffer Size in Kbytes Sender Buffer Size in Kbytes double struct Throughput for doubles and structs Washington University, St. Louis 39

41 $ è High-performance, Real-time ORBs Receiver-side Analysis of IIOP Implementation è1st Optè Typecode::traverse 7.97 ' calc_nested_size_and_ alignment ptr_align_binary struct_traverse ' CDR::get_longlong $ CDR::decoder deep_free CDR::skip_string & CDR::decoder TypeCode::kind & è TypeCode::traverse deep_free double struct Throughput for doubles and structs Washington University, St. Louis 40

42 High-performance, Real-time ORBs æ Problems Problems and Solutions Lack of C++ compiler support for aggressive inlining æ Solution Replace inline methods with preprocessor macros æ Principle Optimize for the common case Washington University, St. Louis 41

43 & & è $ è High-performance, Real-time ORBs Throughput After 2nd Optimization ' ' $ Throughput in Mbps Original Inlining Forced Inlining Throughput in Mbps Original Inlining Forced Inlining Sender Buffer Size in Kbytes Sender Buffer Size in Kbytes double struct Throughput for doubles and structs Washington University, St. Louis 42

44 $ $ è High-performance, Real-time ORBs Receiver-side Analysis of IIOP Implementation è2nd Optè 8.25 calc_nested_size_and_alignment ' CDR::decoder Typecode::traverse ' struct_traverse CDR::decoder deep_free TypeCode::traverse TypeCode::kind deep_free TypeCode::param_count & double & è struct Percent Execution Time for doubles and structs Washington University, St. Louis 43

45 High-performance, Real-time ORBs æ Problems Problems and Solutions Too many method calls Computing the same quantity repeatedly æ Principles Precompute Add extra state Pass information through layers Convert generic methods to special-purpose ones Washington University, St. Louis 44

46 & & è $ è High-performance, Real-time ORBs Throughput After 3rd Optimization ' ' $ Throughput in Mbps Original Inlining Forced Inlining Precomputation Throughput in Mbps Original Inlining Forced Inlining Precomputation Sender Buffer Size in Kbytes Sender Buffer Size in Kbytes double struct Throughput for doubles and structs Washington University, St. Louis 45

47 $ è High-performance, Real-time ORBs Receiver-side Analysis of IIOP Implementation è3rd Optè ' Typecode::traverse $ CDR::decoder deep_free ' TypeCode::traverse CDR::decoder TypeCode::typecode_p aram deep_free TypeCode::_duplicate & double & è struct Percent Execution Time for doubles and structs Washington University, St. Louis 46

48 High-performance, Real-time ORBs æ Problems Problems and Solutions Expensive no-ops for memory deallocation æ Principles Eliminate gratuitous waste Specialize generic methods Washington University, St. Louis 47

49 & $ & è $ è High-performance, Real-time ORBs Throughput After Optimizations ' ' Throughput in Mbps Original Inlining Forced Inlining Precomputation Final Throughput in Mbps Original Inlining Forced Inlining Precomputation Final Sender Buffer Size in Kbytes Sender Buffer Size in Kbytes double struct Throughput for doubles and structs Washington University, St. Louis 48

50 $ è High-performance, Real-time ORBs Analysis of IIOP Implementation after Receiver-side Optimizations ' ' $ read Typecode::traverse TypeCode::traverse TypeCode::typecode_ param & double & è struct Percent Execution Time for doubles and structs Washington University, St. Louis 49

51 $ è High-performance, Real-time ORBs Throughput Comparisons ' ' $ Throughput in Mbps shorts longs chars/octets doubles structs TCP/IP Throughput in Mbps Sender Buffer Size in Kbytes shorts longs chars/octets doubles structs TCP/IP Sender Buffer Size in Kbytes & Original SunSoft & è Optimized TAO Throughput for SunSoft and TAO Versions of IIOP Washington University, St. Louis 50

52 High-performance, Real-time ORBs Results for Typecode Interpreter Optimizations Our measurement-driven, principle-based optimization process improved æ IIOP protocol engine performance as follows TAO's 1.8 times for doubles 3.3 times for longs 3.75 times for shorts 5 times for charsèoctets 4.2 times for structs æ Results available at Washington University, St. Louis 51

53 High-performance, Real-time ORBs Current Status of TAO IDL Compiler æ Based on Sun "IDL" front-end + our back-end RIOP Protocol Engine æ Optimized version of Sun's GIOPèIIOP protocol engine with real-time enhancements ACE ORB Core æ Multi-threaded ORB run-time system based on ACE Real-time Object Adapter æ Demultiplex, schedule, and dispatch client requests in real-time Object Services æ Real-time Event Channels and Multimedia Streaming Service Washington University, St. Louis 52

54 æ Components High-performance, Real-time ORBs Developing an ORB Core with ACE AcceptorèConnector Parameterized via æ strategies Reactor Demuxes client æ requests Active Objects Processes client æ requests Washington University, St. Louis 53

55 High-performance, Real-time ORBs Concluding Remarks æ Current Focus: High-performance, Real-time ORBs Reducing latency via de-layered active demuxing Applying optimization principles to TypeCode interpreter Enforcing periodic deadlines via Real-time Object Adapter i.e., support static request scheduling æ Applying optimization principles to presentation layer æ Future Work Pinpoint non-determinism and priority inversions in ORBs Dynamic scheduling of requests Distributed QoS and integration with RT IèO Subsystem TypeCode compiler optimizations Washington University, St. Louis 54

56