C. Schmidt Douglas schmidt@cs.wustl.edu University, St. Louis Washington www.cs.wustl.eduèçschmidtètao4.ps.gz Sponsors Boeing, CDI, DARPA, Kodak, Bellcore, Motorola, NSF, OTI, SAIC, Lucent, SCR, Siemens MED, Siemens ZT, and Sprint Siemens Hardware gets smaller, cheaper faster, Software gets larger, slower, expensive more Accidental and inherent complexity Middleware, frameworks, and patterns components, Design Patterns for Developing and Using Real-time CORBA Object Request Brokers Motivation: the Distributed Software Crisis DX IMAGE STORE Symptoms ATM LAN DIAGNOSTIC STATIONS ATM MAN ATM LAN CLUSTER IMAGE STORE Culprits MODALITIES (CT, MR, CR) CENTRAL IMAGE STORE Solutions Washington University, St. Louis 1
Latency Reliability Partitioning Ordering Low-level APIs Poor debugging tools Algorithmic decomposition Components Self-contained, ëpluggable" ADTs Frameworks Reusable, ësemi-complete" applications Patterns Problemèsolution pairs in a context Architecture Families of related patterns components and Sources of Complexity for Distributed Applications Inherent complexity Accidental Complexity Continuous re-invention Washington University, St. Louis 2 for Improving Software Techniques and Productivity Quality Proven solutions Washington University, St. Louis 3
Many applications require guarantees QoS e.g., telecom, avionics, WWW Existing middleware doesn't QoS eectively support Solutions must be integrated Simplify distribution automating by Object location activation and Parameter marshaling Demultiplexing Error handling Provide foundation higher-level for Motivation for Real-time Middleware e.g., CORBA, DCOM, DCE Vertically and horizontally Washington University, St. Louis 4 Candidate Solution: CORBA Goals of CORBA www.cs.wustl.eduèçschmidtècorba.html services Washington University, St. Louis 5
www.cs.wustl.eduèçschmidtèorbendsystem.ps.gz Lack of QoS specications Lack of QoS enforcement Lack of real-time programming features Lack of performance A high-performance, ORB real-time Telecom and focus avionics Leverages the ACE framework Runs on RTOSs, and Win32 POSIX, QuO at BBN ARMADA at U. Limitations Limitations of CORBA for Real-time Systems optimizations Washington University, St. Louis 6 The ACE ORB ètaoè TAO Overview Related work www.cs.wustl.eduèçschmidtètao.html Mich. Washington University, St. Louis 7
The ADAPTIVE Communication Environment èaceè www.cs.wustl.eduèçschmidtèace.html ACE Overview Concurrent OO networking framework Ported to C++ Java and Runs on RTOSs, and POSIX, Win32 Related work x-kernel SysV STREAMS Washington University, St. Louis 8 ACE Statistics ACE contain é 125,000 lines of C++ Over 10 person-years of eort Ported to UNIX, Win32, MVS, and platforms embedded e.g., VxWorks, Chorus, LynxOS, psos Large user community www.cs.wustl.eduèçschmidtèaceusers.html Currently used by dozens companies of Bellcore, Boeing, Kodak, Ericsson, Motorola, Lucent, Siemens, SAIC, StorTek, etc. Supported commercially www.riverace.com Washington University, St. Louis 9
Periodic deterministic deadlines real-time COTS infrastructure Applying ORBs to Real-time Avionics Domain Challenges Open systems Washington University, St. Louis 10 Visualizing Periods for Avionics Operations Washington University, St. Louis 11
Specifyingèenforcing QoS requirements Focus on Operations upon Objects Rather than on communication or threadsèsynchronization channels Initial focus Determinisitic deadlines Static scheduling Servants publish resource èe.g., CPUè and èperiodicè deadlines requirements Real-time Features and Optimizations in TAO Washington University, St. Louis 12 Design Challenges Providing QoS to CORBA Operations Solution approach Most clients are also servants Washington University, St. Louis 13
Oine Assess schedule feasibility Assign thread and priorities queue Online Supply priorities to endsystem ORB via Oè1è dispatcher lookup table Integrate RT dispatcher into endsystem ORB Support multiple request strategies scheduling e.g., RMS, RMS with Preemption, and Deferred EDF Requests ordered across priorities by OS thread dispatcher Requests ordered within based on data priorities Components TAO's Real-time Scheduling Service www.cs.wustl.eduèçschmidtètao.ps.gz Washington University, St. Louis 14 Real-time ORB Endsystem Architecture TAO's Solution Approach dependencies and importance Washington University, St. Louis 15
Server factory implements thread-per-rate Highest real-time for high priority priority client Lowest real-time for low priority priority clients Chorus' latency is lower for è of clients small ç1.2msec vs. ç1.4sec vs. ç2.0sec TAO's latency is much for large è of clients lower ç2.3msec vs. ç7.6msec ç14.1msec vs. TAO avoids priority inversion i.e., high priority client has lowest latency always C 0 C 1...... C n Servants Object Adapter ORB Core SCHEDULER RUNTIME One high priority client 1..n low priority clients Requests Client I/O SUBSYSTEM Server Priority Inversion Experiments ATM Switch 01 00 11 01 Ultra 2 Ultra 2 1100 01 www.cs.wustl.eduèçschmidtèrt-perf.ps.gz Washington University, St. Louis 16 16 Priority Inversion Experiment Results Synopsis of results Latency per Twoway Request in Milliseconds 14 12 10 8 6 4 MT-Orbix High-Priority MT-Orbix Low-Priority Sun COOL High-Priority Sun COOL Low-Priority TAO High-Priority TAO Low-Priority 2 0 1 5 10 15 20 25 30 35 40 45 50 Number of Low Priority Clients MT-Orbix, Chorus, and TAO Priority Inversion Washington University, St. Louis 17
Denition High overhead Context switching Synchronization Priority inversions Jitter is the from the variance TAO's jitter is and most lowest consistent MT-Orbix's jitter is and more highest variable FIFO request queueing Improper thread priorities Lack of application control concurrency model over Jitter Experiment Results 100 90 80 Jitter (millisec) 70 60 50 40 30 20 10 0 1 5 10 15 20 25 30 35 40 45 50 Orbix Low Priority Jitter Orbix High Priority Jitter Sun COOL Low Priority Jitter Sun COOL High Priority Jitter TAO Low Priority Jitter TAO High Priority Jitter average latency Synopsis of results Number of Low Priority Clients MT-Orbix, Chorus, and TAO Jitter Washington University, St. Louis 18 Problem: Improper ORB Concurrency Model Common Problems Washington University, St. Louis 19
Priority inversions Sharing multiple on a priorities connection single Complex connection multiplexing Synchronization Common Problems Problem: ORB Shared Connection Model overhead Washington University, St. Louis 20 TAO's Inter-ORB Connection Topology www.cs.wustl.eduèçschmidtèrt-middleware.ps.gz Washington University, St. Louis 21
Minimize demuxing layers Provide Oè1è operation demuxing Avoid priority inversions Remain Results at www.cs.wustl.eduèçschmidtèglobecom-97.ps.gz Design Challenges CORBA-compliant Problem: Reducing Demultiplexing Latency Washington University, St. Louis 22 Demultiplexing Performance Experiments Linear search based on Orbix demuxing strategy Perfect hashing based on GNU gperf www.cs.wustl.eduèçschmidtègperf.ps.gz Washington University, St. Louis 23
Synopsis Linear search is far too costly Dynamic hashing is too erratic gperf solution is 100è but static compatible, Active demuxing may be 100è not but is compatible, dynamic Vertical integration of QoS ORB, OS, and ATM through network Provides rate-based QoS end-to-end Leverages APIC features for cell Demultiplexing Performance Results 140 120 Latency in microseconds 100 80 60 40 20 0 500 400 300 200 100 Number of Objects 1 Active Demux (1 Method) Active Demux (10 Methods) Active Demux (100 Methods) GPERF (1 Method) GPERF (10 Methods) GPERF (100 Methods) Dynamic Hash (1 Method) Dynamic Hash (10 Methods) Dynamic Hash (100 Methods) Linear (1 method) Linear (10 methods) Linear (100 methods) Demultiplexing scheme Washington University, St. Louis 24 Integrating TAO with RT IèO Subsystem and ATM Key Features pacing Washington University, St. Louis 25
Extensible to retarget on new platforms Extensible via custom implementation strategies Extensible via dynamic conguration of custom strategies www.cs.wustl.eduèçschmidtèorbpatterns.ps.gz Denition ëa recurring solution a design problem in to Benets of Patterns Facilitate design reuse Preserve crucial design information Guide design choices Document common and pitfalls traps Dimensions of ORB Extensibility Washington University, St. Louis 26 Applying Patterns to Develop Extensible ORBs a particular context" Washington University, St. Louis 27
Problem Forces Low-level APIs are tedious to program Low-level APIs are error-prone Solution Apply the Wrapper Facade pattern to encapsulate low-level OS details programming Encapsulates low-level, system stand-alone within mechanisms modular, and type-safe, Forces Resolved Avoid tedious, error-prone, non-portable system and APIs Create cohesive Addressing ORB Portability and Typesafety Challenges Building an ORB using low-level system APIs is hard Low-level APIs are non-portable Washington University, St. Louis 28 Portability and Typesafety with the Wrapper Enhancing Pattern Facade Intent portable class interfaces abstractions Washington University, St. Louis 29
TAO's wrapper facades are based on ACE The Wrapper Facade pattern substantially increased portability and the amount of ad hoc code reduced Problem Forces Multi-threading is not always available Multi-threading is not always ecient Tightly coupling general event processing with ORB-specic logic is inexible Solution Use the Reactor pattern to decouple generic event processing from processing ORB-specic $ è Using the Wrapper Facade Pattern in TAO ' $ ' & SunSoft IIOP & è TAO Washington University, St. Louis 30 Addressing ORB Demuxing and Dispatching Challenges ORBs must process many dierent types of events simultaneously Washington University, St. Louis 31
& & è The ACE Reactor pattern is widely used by industry Decouples event synchronous demuxingèdispatching event handling from Demuxing events within one eciently thread Extending applications changing without infrastructure demux $ è Enhancing Demuxing with the Reactor Pattern Intent Forces Resolved Washington University, St. Louis 32 ' Using the Reactor Pattern in TAO ' $ SunSoft IIOP TAO Washington University, St. Louis 33
ORB Endpoint Initialization Challenges Addressing Problem The communication protocol used between ORBs is often orthogonal its connection establishment and service handler initialization to Forces Low-level connection APIs are error-prone and non-portable Separating initialization from subsequent processing increases software Solution Use the Acceptor-Connector pattern to decouple passiveèactive establishment and GIOP connection handler initialization connection & Intent & è Decouple connection establishment and service handler initialization subsequent service processing from $ è protocol reuse for many types of communication software from the subsequent ORB interoperability protocol èe.g., IIOPè Washington University, St. Louis 34 Endpoint Initialization with the Enhancing Pattern Acceptor-Connector ' ' $ Acceptor Structure Connector Structure Washington University, St. Louis 35
& Forces Resolved Problem $ & è Multi-threaded ORBs are needed since Reactive ORBs are often non-scalable, and non-robust inecient, Forces Multi-threading can be very hard to program No single multi-threading model is always optimal Solution Use the Active Object pattern to allow multiple concurrent server using an OO programming style operations $ è Using the Acceptor-Connector Pattern in TAO ' ' SunSoft IIOP TAO è1è Improve portability and reuse and è2è avoid common mistakes Washington University, St. Louis 36 Addressing ORB Concurrency Challenges Washington University, St. Louis 37
Decouples the thread request execution of the thread of from reception request Allow blocking operations Permit exible strategies concurrency TAO supports several variants of Active Objects èe.g., Thread-per- Thread-per-Request, Thread Pool, Thread-per-Rate, etc.è Connection, $ è ORB Concurrency with the Active Object Enhancing Pattern Intent Forces Resolved Washington University, St. Louis 38 ' Using the Active Object Pattern in TAO ' $ & SunSoft IIOP & è TAO Washington University, St. Louis 39
Problem It is important to minimize the amount of locking required to serialize to resources shared by an ORB access Forces Locks increase performance overhead Locks increase potential for priority inversion Solution Use the Thread-Specic Storage pattern to maximize threading-model and minimize lock contention and priority inversion exibility Intent Allows multiple to use threads logically one access global to retrieve point ORB thread-specic without data locking incurring for each overhead Lock Contention and Priority Inversions with Reducing Thread-Specic Storage Pattern the Dierent concurrency schemes yield dierent locking costs Washington University, St. Louis 40 ORB Locking with the Thread-Specic Minimizing Pattern Storage Forces Resolved Minimizes overhead and priority inversion access Washington University, St. Louis 41
Problem Real-world ORBs must be exible to satisfy the requirements of many types of end-users and applications dierent Forces Ad hoc schemes for ORB exibility are too static and non-extensible Flexibility often has many èrelatedè dimensions Solution Use the Strategy pattern to support multiple transparently ëpluggable" strategies ORB Using Thread-Specic Storage in TAO Washington University, St. Louis 42 Addressing ORB Flexibility Challenges Washington University, St. Louis 43
& & è Factor out similarity algorithmic among Orthogonally replace subsets behavioral transparently Associating state with $ è Enhancing ORB Flexibility with the Strategy Pattern Intent alternatives Forces Resolved an algorithm Washington University, St. Louis 44 Using the Strategy Pattern in TAO ' $ ' SunSoft IIOP TAO Washington University, St. Louis 45
Forces Resolved Problem Addressing ORB Congurability Challenges Agressive use of the Strategy pattern can create a conguration nightmare Forces It's hard to manage large numbers of individually congured strategies It's hard to ensure that groups of semantically compatible strategies are congured Solution Use the Abstract Factory pattern to consolidate multiple ORB into semantically compatible congurations strategies Washington University, St. Louis 46 ORB Congurability with the Abstract Centralizing Pattern Factory Intent Integrate all strategies to congure an used ORB Consolidates of many customization strategies Ensures semantic of compatibility strategies Washington University, St. Louis 47
Problem Prematurely committing ourselves to a particular ORB conguration inexible and inecient is Forces Certain ORB conguration decisions can't be made eciently until run-time Solution Use the Service Congurator pattern to assemble the desired ORB dynamically components Using the Abstract Factory Pattern in TAO Washington University, St. Louis 48 Addressing ORB Dynamic Congurability Challenges Forcing users to pay for components they don't use is undesirable Washington University, St. Louis 49
Decouples ORB from time strategies they are when congured Reduce resource utilization Support dynamic èreèconguration Dynamic ORB Extensibility with the Service Enhancing Pattern Congurator Intent Forces Resolved CORBA::ORB_init èint &argc, char *argvëëè èè Configure the ORB. Service_Config tao èargc, argvè; èè Perform initialization... Washington University, St. Louis 50 Using the Service Congurator Pattern in TAO Run-time Conguration Washington University, St. Louis 51
Complexity Metric Scores for TAO Macabe SunSoft IIOP and Patterns greatly code reduce complexity e.g., Most TAO have components é 10 vègè TAO components are smaller substantially SunSoft IIOP than e.g., connection management by a factor reduced $ è Quantifying the Benets of Patterns 70.0 Statistics 60.0 SunSoft IIOP TAO % Methods in Range 50.0 40.0 30.0 20.0 10.0 0.0 1 5 6 10 > 10 MVG Range of 5 Washington University, St. Louis 52 Current Status of TAO ' ' $ è& Washington University, St. Louis 53 &
TAO Project Summary Current Focus: Reducing latency via de-layered active demuxing Applying optimization principles to TypeCode interpreter Enforcing periodic deadlines via Real-time ORB endystem i.e., support static scheduling for CORBA requests Applying optimization principles to presentation layer Future Work Pinpoint non-determinism and priority inversions in ORBs Dynamic scheduling of requests Distributed QoS and integration with RT IèO Subsystem IDL compiler and optimized stub generation TypeCode compiler optimizations Washington University, St. Louis 54 Concluding Remarks Developers of distributed applications confront recurring challenges that largely application-independent are e.g., service initialization and distribution, error handling, ow event demultiplexing, concurrency control, persistence, fault control, tolerance Successful developers resolve these challenges by applying appropriate patterns to create communication frameworks and components design ORBs are an eective way to achieve reuse of distributed software components The next-generation of ORBs will provide much better QoS support Washington University, St. Louis 55