Enhancing Real-time CORBA via Real-time Java features

Transcription

1 Enhancing Real-time CORBA via Real-time Java features Arvind S. Krishna, Douglas C. Schmidt Electrical Engineering and Computer Science Vanderbilt University Nashville, TN 3723, USA arvindk, Raymond Klefstad Electrical Engineering and Computer Science University of California, Irvine Irvine, CA 92697, USA Abstract End-to-end middleware predictability is essential to support quality of service (QoS) capabilities needed by distributed real-time and embedded (DRE) applications. Real-time CORBA is a middleware standard that allows DRE applications to allocate, schedule, and control the QoS of CPU, memory, and networking resources. Existing Real-time CORBA solutions are implemented in C++, which is generally more complicated and error-prone to program than Java. The Real-Time Specification for Java (RTSJ) provides extensions that enable Java to be used for developing DRE systems. Real-time CORBA does not currently leverage key RTSJ features, such as scoped memory and real-time threads. Thus, integration of Real-Time CORBA and RTSJ is essential to ensure the predictability required for Java-based DRE applications. This paper provides the following contributions to the study of middleware for DRE applications. First we analyze the architecture of ZEN, our implementation of Real-time CORBA, identifying sources for the application of RTSJ features. Second, we describe how RTSJ features, such as scoped memory and real-time threads, can be associated with key ORB components to enhance the predictability of DRE applications using Realtime CORBA and the RTSJ. Third, we perform preliminary qualitative and quantitative analysis of predictability enhancements arising from our application of RTSJ features. Our results show that use of RTSJ features can considerably improve the predictability of DRE applications written using Real-time CORBA and Real-time Java. Keywords. Distributed Real-time and Embedded Systems, Real-time CORBA, Real-time Java. I. INTRODUCTION DRE systems are becoming increasingly widespread and important. Common DRE systems include telecommunication networks (e.g., call processing services), process automation (e.g., hot rolling mills), and defense applications (e.g., total ship computing environments). Real-time CORBA [1] is a rapidly maturing standard middleware technology that allows DRE applications to configure and control the following resources: Processor resources via thread pools, priority mechanisms, and intraprocess mutexes; Communication resources via protocol properties; and explicit bindings with non-multiplexed connections and Memory resources via buffering requests in queues and bounding the size of thread pools. Experience during the past five years [2] [5] demonstrates that Real-time CORBA has been successfully used to develop middleware for DRE applications, just as CORBA [6], COM+ [7], and Java RMI [8] have been applied in the business and desktop domains. Although the Real-time CORBA specification was integrated into the OMG standard in 1998, it has not been adopted universally by DRE application developers. A key barrier to adoption arises from the steep learning curve caused by the complexity of the CORBA-C++ mapping [9] [11]. To address this problem, the Java programming language has emerged as an attractive alternative. Since Java has less inherent and accidental complexity than C++, it is easier for application programmers to master it. Java also has other desirable language features, such as strong typing, dynamic class loading, reflection/introspection, and native support for concurrency and synchronization. Conventional Java runtime systems and middleware have historically been unsuitable for DRE applications, however, due to 1) The under-specified scheduling semantics of Java threads, which can lead to the most eligible thread not always being run. 2) The ability of the Java Garbage Collector (GC) to preempt any other Java thread, which can yield very long preemption latencies. 3) The lack of a standard distributed computing model, which leads to ad hoc mechanisms for communicating between different nodes in a DRE application. To address problems 1 and 2, the Real-time Java Experts Group has defined the RTSJ [12], which extends Java in several ways, including (1) new memory management models that allow access to physical memory and can be used in lieu of garbage collection and (2) stronger guarantees on thread semantics than in conventional Java. To address problem 3, we have implemented ZEN, which is an an open-source 1 RTSJ-based Real-time CORBA Object Request Broker (ORB) that combines the benefits of these two standard technologies. ZEN is ported to both the TimeSys RTSJ reference implementation [13] (which uses a virtual machine architecture) and 1 The source code for ZEN can be downloaded from

2 jrate [14] (which uses an ahead-of-time compiler architecture). ZEN is inspired by many of the patterns, techniques, and lessons learned in The ACE ORB (TAO) [15], which is our other open-source implementation of Real-time CORBA written in C++. Our prior published work on ZEN focused on (1) the extensible component architecture [16] of its ORB Core [17] and Object Adapter [18] layers and (2) the predictable demultiplexing strategies [19] that it uses to ensure lookup time irrespective of the depth of the Object Adapter hierarchy. This paper extends our earlier published work by focusing on a previously unexplored dimension in real-time middleware: the integration of RTSJ features to support Real-time CORBA. Our results show that effective use of RTSJ features to implement Real-time CORBA can considerably improve the predictability of DRE applications written using Real-time CORBA and Java. The remainder of this paper is organized as follows: Section II provides an overview of the RTSJ focusing on its thread and memory models; Section III describes the main problems that arose while designing ZEN using conventional Java implementations, analyzes the critical request/response code path within ZEN to identify sources for the application of RTSJ features, illustrates how RTSJ features can be associated with key ORB components to enhance predictability, and empirically analyzes how the application of RTSJ features improved predictability; Section IV summarizes how our work on ZEN relates to other research efforts; and Section IV presents concluding remarks. II. OVERVIEW OF REAL-TIME JAVA Java has traditionally not been suitable for developing realtime systems [2]. The RTSJ extends the Java API and refines the semantics of certain Java constructs to support the development of real-time applications. The guiding principles followed by the designers of the RTSJ specification include: Backward compatibility with the Java 2 platform No syntactic extension to the Java language, i.e. no new keywords Write once carefully, run anywhere conditionally Address current real-time practices and provision for future advances This section presents a brief overview of the memory and threading extensions provided by the RTSJ. A comprehensive discussion of RTSJ features, and programming tips appears in [21]. Section III, then describes how these features may be applied effectively within Java based middleware solutions to ensure predictability required by DRE systems. A. Memory Model Garbage collection (GC) is unsuitable for real-time systems. Although there have been several advances in GC algorithms [22], [23], the RTSJ has devised memory regions that bypass GC, giving application programmers control over the time at which memory is allocated and reclaimed. The RTSJ extends Java s memory model by providing other memory areas than just the heap. These memory areas are characterized by the lifetime of the objects in the areas and their allocation times. Figure 1 depicts the class hierarchy for the new memory MemoryArea ImmortalMemory ScopedMemory ImmortalPhysicalMemory HeapMemory VTMemory LTMemory LTPhysicalMemory VTPhysicalMemory Fig. 1. Hierarchy of Classes in the RTSJ Memory Model areas provided by the RTSJ including: Immortal memory. The RTSJ introduces the concept of immortal memory, where allocated objects have the same lifetime as the Java Virtual Machine (JVM). The disadvantage with immortal memory is that objects allocated in it are never garbage collected. Thus, memory leaks would eventually cause an application to run out of immortal memory and die. Physical memory. The RTSJ also allows direct access to raw memory. Every RTSJ implementation manages the allocation of memory to physical memory allocation objects, e.g LTPhysicalMemory and VTPhysical- Memory. The RawMemoryAccess class provides a safe method of accessing memory through pointers, making it easy for RTSJ applications to implement memorymapped I/O. Scoped Memory. Like immortal memory, a scoped memory area is not garbage collected. Unlike immortal memory, however, the lifetime of a scoped memory region is not persistent. Instead, it is reference counted, indicating the number of active threads in that region. After the count drops to zero, the memory region is considered inactive and objects allocated in it are finalized. The RTSJ specification provides scoped memories with linear and variable allocation times (LTMemory, and VTMemory, respectively). B. Thread Model The RTSJ extends Java s threading model and introduces RealtimeThread and NoHeapRealtimeThread (NHRT) Threads. The relationship between regular Java and real-time threads is shown in Figure 2. As shown in the Fig. 2. Thread RealtimeThread «Interface» Schedulable NoHeapRealtimeThread RTSJ Real-time Thread class Hierarchy

3 figure, a real-time thread is also a normal Java thread. Further, a real-time thread is also a Schedulable object. The RTSJ defines that every schedulable entity implements the Schedulable interface. The execution of Schedulable entities is managed by the scheduler that holds a reference to them. Every RTSJ compliant implementations provides a priority preemptive scheduler that can distinguish 28 different priorities. NHRT threads are special real-time threads that do not touch the heap i.e., they cannot load or store a reference to an object in the heap. NoHeapRealtimeThread can therefore have execution eligibility higher than the garbage collector. 2. The scheduler controls the execution eligibility 3. The new RTSJ memory areas, e.g., scoped memory and NoHeapRealtimeThread, hold great promise for Realtime CORBA developers since they bypass the garbage collector. By allocating Real-time CORBA internal objects in these areas, demand garbage collection (i.e., when operator new cannot find enough memory) can be reduced. Proper use of scoped memory in conjunction with real-time threads would reduce the interference with the GC, thereby enhancing the predictability of the system. Section III-D provides empirical results that quantify predictability improvements by the association of RTSJ features in ZEN. III. ENHANCING THE ZEN ORB USING THE RTSJ The OMG Real-time CORBA specification was adopted several years before the RTSJ was standardized. Real-time CORBA s Java mapping therefore does not use any RTSJ features, such as NoHeapRealtimeThread and Scoped- Memory. To have a predictable Java-based Real-time CORBA ORB like ZEN, however, it is necessary to take advantage of RTSJ features to reduce interference with the GC and ensure predictability. This section first identifies problems in the original design of ZEN, which was initially based on regular (i.e., non-rtsj) Java. We then analyze a typical end-to-end critical code path of a CORBA request within the original ZEN ORB, which was based on regular Java. Based on this analysis, we describe how we are enhancing ZEN to use RTSJ features, such as real-time threads and scoped memory, to improve its predictability. A. Problems in the Original Design of ZEN In the original architecture of ZEN [17], shown in Figure 3, key ORB components that are involved in request/response processing (such as acceptors, connectors, transports, and thread pools) were originally allocated in the heap. This architecture suffered from the following problem: the ORB allocates several temporary objects during the processing of a remote request/response. This allocation can lead to demand garbage collection, i.e., execution of the GC when the Java 2 The RTSJ v1. specification states that the NoHeapRealtimeThread always has execution eligibility higher than the GC, but this has been changed in the RTSJ v Execution eligibility is defined as the position of a schedulable entity in a total ordering established by a scheduler over the available entities [24]. The total order depends on the scheduling policy. new operator cannot find enough memory. Further, the JVM also allocates heap memory as part of its execution which additionally entails garbage collection. Execution of the GC can cause unbounded preemption latency to the thread processing the request. The situation is exacerbated if the request is critical (i.e., highest priority), which can be catastrophic for certain types of safety- and mission-critical DRE applications. To eliminate priority inversions related to invocations of the Buffer Manager GIOP Messaging CDR Stream Object Resolvers Transport Layer Fig. 3. HEAP MEMORY Object Adapter Other ORB Components Acceptor Transport IOR Parsers The Original ZEN ORB Architecture garbage collector during a request upcall, it is essential that: ZEN avoids heap allocation by exploiting the RTSJ scoped and immortal memory Key ORB components be allocated either within scoped or immortal memory that would not cause demand garbage collection. The aforementioned factors would minimize the interference with the GC enhancing the predictability of the ORB and DRE application. Our redesign of ZEN for Real-time CORBA began by identifying the participants associated with processing a request at both the client and server sides. For each participant identified, we associated the component with non-heap regions and resolved challenges arising from this association. Allocating key ORB objects within scoped or immortal memory would not cause garbage collection, thus minimizing the interference with the GC and enhancing the predictability of the ORB and DRE application. B. Applying RTSJ Features to ZEN a) RTSJ application goals: The goals for applying RTSJ features to ZEN include: Minimizing interference with GC. Garbage collection is generally considered to be unsuitable for DRE applications with stringent real-time requirements. Although there have been recent advances in GC algorithms [22], [23], the Java GC can preempt any thread in the system, leading to the thread incurring unacceptably long preemption latencies. A key goal of ZEN is therefore to avoid allocating critical ORB components in heap memory to reduce the number of GC sweeps.

4 Compliance with the CORBA specification. To preserve compliance with the Real-time CORBA specification, the RTSJ features must be incorporated within ZEN s ORB Core and Object Adapter layers without requiring any modifications to the Real-time CORBA specification. Options that require the end-user to be RTSJ-aware, such as associating scoped memory at the POA level, are provided as non-standard ZEN-specific options. Interoperability with non RTSJ Java. ZEN is designed to use intelligent strategies for component creation and extensibility [16] that allow configurability of real-time features (such as the number of static/dynamic threads, thread priorities, and buffer size) using properties and policies. These strategies use techniques, such as reflection [25] [27] and aspects [28], to create real-time/vanilla Java components, thereby minimizing time/space overhead for applications that do not require real-time features. b) Identification of steps: Our redesign of ZEN for Realtime CORBA began by identifying the participants associated with processing a request at both the client and server sides. For each participant identified, we associated the component with non-heap regions and resolved challenges arising from this association. The first step needed to identify where to apply RTSJ features required the analysis of the end-to-end critical code path in ZEN. Figure 4 depicts the participants involved in servicing a CORBA two-way invocation. The discussion Client ORB GIOP Message Parsers Connection Cache C 1 C 2 C 3 C1 Waiting Strategy C3 1, 16 Fig. 4. C2 Client Application 8, 17 2 Connector 1,3 9, 14 Buffer Manager Transport T 1 9, GIOP Message Parsers Buffer Manager Acceptor 4 Server ORB Object Adapter 11, Transport T 1 6, 7,14 5 Connection Cache C 1 C 2 C 3 Tracing an Invocation Through the ZEN CORBA ORB of the critical code path has been generalized using the Acceptor-Connector [29] pattern and thread-per-connection concurrency strategy. We next describe the sequence of steps a client ORB performs to actively create a connection when a CORBA request is invoked by the application, i.e., result = object.operation (arg). c) Connection management: We first describe how ZEN establishes a connection between a CORBA client and a server. 1. The client ORB s connection cache (ConnectorRegistry class in ZEN) is queried for an existing connection to the server, obtained from the object reference on which the operation is invoked. 2. If no previous connection exists, a separate connection handler is created (Transport class in ZEN) T and the Connector connects to the server 3. This connection is added to the ConnectorRegistry since C is bidirectional. The activities of the server ORB for accepting a connection are described next: 4. An acceptor accepts the new incoming connection. 5. This connection C is then added to the server s connection cache (AcceptorRegistry class in ZEN) as the server may send requests to the client. 6. A new connection handler T is created to service requests. 7. The Transport s event loop waits for data events from the client. d) Synchronous request/reply processing: The following are the steps involved when a client invokes a synchronous two-way request to the server. 8. The BufferManager class is queried to obtain a buffer to marshal the parameters in the operation invocation. 9. The appropriate GIOP Message Writer marshals the request and the Transport sends the request to the server. 1. The WaitingStrategy class associated with the transport waits for a reply from the server. The server ORB performs the following activities to process the request. 11. The request header on connection C is read to determine the size of the request. 12. A buffer of the corresponding size is obtained from the buffer manager to hold the request and the request data is read into the buffer. 13. The request is the demultiplexed to obtain the target POA, servant, and skeleton servicing the request. The upcall is dispatched to the servant after demarshaling the request. 14. The reply is marshaled using the corresponding GIOP message writer; Transport sends reply to the client. The client ORB performs the following activities to process the reply from the server: 15. The Reader reads the reply from the server on the connection. 16. Using the request ID, the Waiting Strategy identifies the target Transport. 17. The parameters are then demarshaled and control is returned to the client application, which processes the reply. e) Analyzing request processing steps: The request processing steps described above reveal the following characteristics: Repetitive. The steps involved with request/reply processing are repetitive, i.e., carried out for every request.

5 Steps at the server side for request processing and steps at the client side remain the same for each request from the client/server. Similarly, steps 1-3 are performed for every remote request sent to the server. Independent and memoryless. Steps required for processing request/response from two different client/server(s) are independent, i.e., they do not share any context. Moreover, two requests from the same client do not share any context. Ephemeral. The objects created during the execution of these steps remain valid only for the duration of one cycle of request/response processing. ORBs therefore usually cache these resources to minimize resource management. Thread bound. Each of the steps are executed by a request processing thread. For example, steps at the server side are executed by the transport and threadpool threads. The aforementioned characteristics of the steps lend themselves to the application of RTSJ features in the following manner: Associating Real-time threads. The thread-bound property of the steps enables components e.g., acceptorconnector and transports to be associated with realtime threads. In particular, each of these components is designed based on an logic part, implemented as a Java class that implements the Runnable interface. This part is then associated with a scoped memory region and bound with the thread at creation time. Associating Scoped memory. The ephemeral property of the steps enable internal objects created during request/response processing to be associated with scoped memory regions. The repetitive, independence, and memoryless properties of the steps further shape how an ORB implementor can associate scoped memory. The repetitive and memoryless properties enable the request/response processing steps to be carried out within a scoped memory region 4, process the request, and send the response to the client. The memory region is then exited 5 enabling all the objects created to be freed, thus minimizing the number of GC sweeps. This cycle is repeated for the next request. The independence property validates the above mechanism, allowing objects created during request processing to be freed before processing a subsequent request. Unlike heap/immortal memory, creation of scoped memory regions requires the size of the memory region to be specified at creation time. However, the footprint required to process request/response is dynamic, i.e., varies based on: Request size. The request size at the server depends on the size of the request sent by the client. Options associated. The footprint required during request processing depends on the options enabled. 4 Using the enter() method the memory region can be made the current allocation context. 5 Exiting a memory region is implicit, done after the completion of the run method. Sidebar 1: Understanding Scoped Memory Below we describe important concepts associated with scoped memory regions: Assignment Rules. To maintain referential integrity the JVM enforces certain rules for assignment in all memory regions, summarized in the table shown below. Based on the above rules, the following can be derived: A memory region m may hold a reference to another memory region m if the lifetime of m is shorter than that of m. Any scoped region can therefore always hold references to heap/immortal memory, however, the reverse is never valid. Reference to: Heap Immortal Scoped Heap Yes Yes No Immortal Yes Yes No Scoped Yes Yes Inner Scope Local Variable Yes Yes Inner Scope Inner scopes. The RTSJ provides mechanisms for real-time threads to allocate objects within multiple scoped memory regions. A real-time thread can make a scoped region m its current allocation context using the enter() method on that scope. In a similar fashion, the same thread can enter another scoped region m, making this region the current allocation context. The lifetime of m is shorter than m, making it an inner scope. Single parent rule. Nesting scoped memory, describe above can thwart RTSJ referential integrity by causing cycles between memory areas (a thread entering m can re-enter m ). The single parent rule enforces the rule that every scoped region must have only one parent (a thread cannot enter an already entered memory region), eliminating the aforementioned problem. Type of Request. The request size directly depends on the type of GIOP request e.g., a LOCATE REQUEST message would be of a different size when compared to a normal request. The most appropriate memory size would therefore have to be chosen during initialization time. One solution to this problem is to create the one huge chunk of memory. However, this solution is non scalable. Further, some JVMs may not be able to allocate a huge chunk of scoped memory region. To address this problem, in ZEN we use Nested Scopes (explained in Sidebar 1) for every request/response demultiplexing phase, which is explained in Section III-C. C. Applying Scoped Memory within ZEN To enhance predictability, we apply RTSJ features e.g., scoped memory to ORB components along the critical request/response processing path. Moreover, to minimize the effect of pre-allocating memory regions, we use nested scope memory regions for each demultiplexing phase. Below, we explain the three broad phases of request processing, at the server ORB and describe how we associate scoped memory with each of the three phases. Similar condition exist at the client ORB. 1. I/O layer Steps. This phase of demultiplexing corresponds to the steps 4-7 described in Section III-B.

6 Participants. The participants for this phase include acceptors, connectors, and transports. RTSJ application. Each of these components are threadbound components and are designed based on the inner class paradigm. This class derives from the Runnable interface and corresponds to the logic run by the thread. Instead of creating the entire component in scoped memory, we create the inner logic class in a scoped memory region, m. This logic class is associated with the thread at creation time. During ORB execution, multiple clients may connect to it, creating transports for every active client. Each of the transports will have a dedicated m region. We collectively refer to these regions as a space. 2. ORB Core layer Steps. This phase of demultiplexing corresponds to the Steps in Section III-B. Participants. GIOP Message parsers, Buffer Allocators and CDR Streams. RTSJ application. On receipt of new data events from the socket, the Transport reads the message header from the stream. Based on the size of the header, a RequestMessage 6 is created. After reading the request from the stream, the appropriate message parser is associated based on the type of the request. The message parser and the RequestMessage buffer are created in a nested memory region, m. The ORB space is a nested memory region. Based on RTSJ memory rules (explained in Sidebar 1), references from the ORB to the I/O space are valid, i.e., every m scope may hold references to the corresponding m region. 3. POA layer Steps. This phase of demultiplexing corresponds to the steps in Section III-B. Participants. Upcall objects 7, and thread-pools RTSJ application. The message parser parses the request to find the target POA and servant. An Upcall object is created to hold all information necessary to perform the upcall on the skeleton. A worker thread in the threadpool then performs the upcall. A CDROutputStream is created, to hold the response, which is then sent to the client. The Upcall objects and the output buffers are created in a nested scoped memory region m. The POA space is the innermost memory region. Again, references from POA to ORB or I/O space are valid. 4. Application layer The application layer typically corresponds to the region where servant IDL skeletons are created. In the current design of ZEN, this application layer is heap allocated. Thus in our architecture, a NoHeapRealtimeThread cannot be used for request processing. The use of NoHeapRealtimeThread requires the 6 This class encapsulates a buffer to hold the request. 7 Upcall objects are per request objects that hold context necessary to perform upcall and send response. application developer to be RTSJ aware and also entails modifications to the Java mapping of Real-time CORBA specification which in conflict with our goals. The use of a NoHeapRealtimeThread, however, is critical to enhancing the predictability of a Real-time CORBA ORB. In ZEN, we plan to provide policies at the POA level that would determine the type of real-time thread to used for request processing. Thus an RTSJ aware server application can allocate servants in a memory region other than the heap and set the type of upcall processing thread to NHRT to enhance predictability. DIRECTION OF SCOPE NESTING SERVANT IDL Skeleton POA SCOPE I/O SCOPE Acceptor IDL Skeleton Upcall Objects SERVANT ORB CORE SCOPE Fig. 5. NETWORK HEAP SCOPE THREAD POOL CDR Streams Transport Transport Transport The Layered Architecture of ZEN Figure 5 illustrates the layered architecture of ZEN s RTSJ layers. The architecture does not violate any of the RTSJ reference rules as (1) any of the ORB layers may hold references to the application layer and (2) a real-time thread can always allocate from Heap memory (enter it) without violating the single-parent rule. Moreover, our application is compliant with the CORBA specification. Figure 6 (A) illustrates the nesting of scopes within the ZEN ORB Core. The I/O space is the outermost memory region while the POA layer is the innermost. Memory regions are entered from outer inner, while references are maintained from inner outer. On completion of a request, the memory regions are exited from innermost to outermost. All the objects thus created for request processing are finalized, thereby minimizing interference with the GC. Figure 6(B) depicts the parenthood tree of the memory regions in ZEN. D. Empirically Evaluating the Application of RTSJ in ZEN To measure the predictability improvements accrued by using RTSJ features, we first associated scoped memory and real-time threads within the innermost scope i.e., POA layer, allocating both I/O and ORB scopes in heap memory. The motivation for this application was to obtain a baseline for improvement in predictability when applying RTSJ features in just one layer. In this regard, applying RTSJ features in

7 NESTED SCOPES enter() Fig. 6. I/O SPACE ORB SPACE POA SPACE A REFERENCES: INNER OUTER exit Scope Nesting in ZEN ORB I/O SPACE ORB SPACE POA SPACE either the ORB or the I/O scopes alone would violate RTSJ memory reference rules (explained in Sidebar 1) making our application the only possible option. Although ZEN supports a variety of options, we make the following assumptions for this analysis: 1) Portable interceptors are not considered in request processing. 2) The sequence of steps analyzed is for a remote client request, not a collocated request. 3) Servants are normal CORBA servants that inherit from org.omg.portableserver.servant, i.e., we do not consider DII and DSI. 4) ZEN s buffer manager was disabled to prevent caching of I/O buffers 5) No proprietary policies are used in the ORB and 6) The scoped memory used was of type LTMemory, which is a RTSJ memory region with linear allocation time with respect to object size. 8 These assumptions are representative of ways in which DRE applications commonly apply ORB middleware. All the experiments in this section were performed on an Intel Pentium III 864 Mhz processor with 256 MB of main memory. For these experiments, ZEN version.8 was compiled using the GNU gcj compiler version and executed using jrate.3a on Linux timesys kernel. For each experiment, a sample size of 5, data points was used for result analysis. The term predictability has different connotations in different disciplines. For example, in real-time scheduling theory, a predictable system means that each task always meets its deadline. For these experiments, we define predictability as the measure of standard deviation of the data points. 1) Demultiplexing Predictability: a) Test overview: The following are the demultiplexing stages present in the POA layer: 1) POA demultiplexing using the addressing information in the object key, locate the target POA in the POA hierarchy; 8 This bound does not include the time taken by an object s constructor or a class s static initializers. B 2) Servant demultiplexing from the object id part of the object key, locate the servant within the POA; and 3) Operation demultiplexing finally using the operation name, dispatch the upcall on the skeleton. The predictability of the ORB depends directly on the predictability of the aforementioned demultiplexing stages. As discussed in [18], ZEN uses active demultiplexing along with perfect hashing to ensure lookup time for all cases. In this experiment, we focus on the POA demultiplexing stage and analyze the predictability improvements accrued by associating scoped memory with ZEN s Object Adapter. b) Test settings: To measure the variation in POA demultiplexing time with the depth of the POA hierarchy, the experiment increased the nesting of transient POAs from 1 to 15 in increments of 25. For each case the time to reach the leaf POA was measured. c) Analysis of results: We now analyze the results of benchmarks that measure the average latency, the dispersion and worst case behavior for the various POA demultiplexing test cases. Latency(us) Latency(us) Average Depth of POA Hierarchy Standard Deviation Depth of POA Hierarchy Fig. 7. Latency(us) Latency(us) % Scoped Memory Heap Memory Depth of POA Hierarchy Max Depth of POA Hierarchy POA Demultiplexing Predictability Analysis Average measures. Figure 7 shows that average latency for both heap and scoped memory does not increase with an increase in the POA hierarchy. However, the average measures for the scoped ( 1 secs) are more than that of heap memory ( 7 secs) by 3 secs. Although, this represents a increase in the POA location overhead by a factor of 1.4, predictability improves considerably. Dispersion measures. The dispersion of scoped memory is smaller than that of heap memory for all cases. Moreover, the measures are better by a factor of 4, indicating that our association scoped memory has considerably improved predictability for this stage of demultiplexing. Worst case measures. The 99% bound shows that using scoped memory 99% of the values are less than 12 secs for all cases while those for heap are 3 secs,a

8 factor of 2 improvement. Maximum measures show that scoped memory considerably improves the predictability for all cases by bounding the worst case values. The worst case measures for scoped memory are tighter across the POA hierarchy. Moreover, the maximum latency for scoped memory is nearly constant across the POA hierarchy, while the latency of heap incurred measurable variability. From the analysis above, it is evident that the use of scoped memory enhances demultiplexing predictability by bounding dispersion and maximum measures. Moreover, the use of scoped memory does not significantly degrade the average demultiplexing latency. 2) Scoped Memory Overhead: a) Test overview: The application of scoped memory at ZEN s Object Adapter layer involves the following additional steps: 1) The request processing thread needs to make the scoped memory region its current allocation context by explicitly entering it (using enter() method defined on the region), incrementing the reference count of the region. 2) After processing the request, the thread exits the region, which changes the allocation context to the enclosing memory region or the primordial scope (heap), in turn decrementing reference count of the region. 3) If the reference count is zero, then all the objects allocated within the region are to be finalized. The steps described above must be executed for every client request. Thus, the overhead and predictability involved in the steps directly affect the predictability of the ORB. In ZEN, the size of the scoped memory region is pre-allocated at initialization time. However, as explained earlier, the foot-print at the server varies depending on the message size, options associated and the type of request. In this experiment, we analyzed the behavior of the enter and exit methods as the request sizes increase. b) Test settings: The scoped memory region size was fixed at 512KB. A client then issued requests starting at 1K in increments of 25KB up to a maximum request size of 1K. c) Analysis of results: We now analyze the results of benchmarks that measure the average latency, the dispersion, and the worst case behavior of the memory region s enter() method and its exit time. Average measures. Figure 8 shows that the average additional overhead incurred by the enter() method is 8 secs and is constant across all request message sizes. However, the latency of the memory region s exit time increase gradually with the request size. For example, the average exit time for a 1K message is 4 secs and increases to 12 secs for a 1K message. This behavior occurs because finalizes for every allocated object in the scoped memory region are called before exiting the region, in which case the size of messages increase the finalization time. Dispersion measures. The dispersion latencies for the enter() method show a trend similar to the average Average Size (KB) 4 2 Standard Deviation Size (KB) Fig % Exit Latency Enter Latency Size (KB) Enter and Exit Time Analysis Maximum Size (KB) constant ( 1-1.5) across all message sizes. However, the dispersion for the exit time increases with the request size. For example, dispersion for 1K message is.7 and increases to 1.9 for 1K. This behavior is induced by the finalization required. Worst case measures. The 99% bound for both the enter and exit latencies follows a trend similar to the respective average cases. Moreover, the values are closer to the average measures. However, the worst case measures for exit values are higher than those of the enter method and increase with message sizes. The analysis above reveals that the average overhead incurred due to the enter() method is 8 secs and is constant across all request sizes. However, the behavior of the exit method varies with the request size, i.e., the larger the request size, the greater the exit latency and worse the dispersion. 3) Round-trip Delay: a) Test overview: This experiment analyzes (1) the effect of scoped memory on round-trip delay and (2) the request execution time, i.e., total time required to complete the following steps in the POA layer: 1) Demultiplex the request through the POA hierarchy; find the target servant and dispatch the upcall. 2) Send the response to the client. Since we associate scoped memory at ZEN s Object Adapter layer, the only factor affecting round-trip delay is the request execution time. While the request execution time measures the effect of enter, exit, and the actual request execution time, the round-trip latency experiment measures the effect of using scoped memory in the critical end-to-end request path. b) Test settings: The IDL_Cubit test was run to measure the request execution time at the server and roundtrip delay at the client. The thread pool size was set to 2 and the buffer size to 5. Size of the scoped memory region was set to 512 KB. The number of simultaneous client connections was increased from 1 to 2 in increments of 5.

9 1 1 1 Average Standard Deviation Fig % Heap Scoped Memory Maximum Request Execution Predictability Analysis c) Analysis of results: We now analyze the results of benchmarks that measure the average latency, the dispersion and worst case behavior of round-trip and request execution latencies Latency (msec) Average Scoped Memory Heap Memory Standard Deviation Fig. 1. Latency (usec) Latency (msec) % Round-trip Latency Analysis Maximum Average measures. Figure 9 illustrates that the average request execution latency for heap and scoped regions are constant for all the cases. Although, latency for scoped memory ( 137 secs) is greater than that of heap memory ( 12 secs), scoped memory provides increased predictability, for this small overhead of 17 secs. This overhead stems from the time required for entering and exiting the memory region. Thus, the association of scoped memory does not lead to undue performance reduction. Figure 1 shows that the use of scoped memory increases the average round-trip delay. This increase is due to the increase request execution time at the server side. As the number of clients increase, however, latency for heap memory also increases. At 2 clients the round-trip time for heap memory exceeds scoped memory by 2 secs. This behavior occurs due to the increased GC activity for heap memory with an increase in the number of clients. Dispersion measures. The dispersion measures for scoped memory are considerably less than those for heap. On average, the use of scoped memory enhances request execution predictability by a factor of 2. For roundtrip delay, dispersion for both heap and scope memory increases with the number of clients. However, measures for scope memory vary little with an increase in the number of clients, indicating better predictability. On the average, predictability improves as much as 5%. Worst case measures. Both experiments reveal that use of scoped memory significantly bounds worst case measures. Though the average request execution latency is greater for scoped memory, its 99% and worst case latencies are smaller. For the case of 5 clients, the worst case request execution measure for heap is greater by a factor of 3. A similar trend is observed for round-trip delay measures. The empirical results described above indicates that the use of RTSJ scoped memory enhances predictability without unduly degrading performance. Moreover, the worst case measures are tighter and significantly better compared to that of heap memory. IV. RELATED WORK In recent years, a considerable amount of research has focused on enhancing the predictability of real-time middleware for DRE applications. In this section, we summarize key efforts related to our work on ZEN. A. Real-time CORBA middleware research. Real-time CORBA 1. implementations are available from variety of suppliers including e*orb from PrismTechnologies and ORBExpress from Object Interface Systems. Real-time CORBA has also been extensively studied in the research literature. The Time-triggered Message-triggered Objects (TMO) project [3] at the University of California, Irvine, supports the integrated design of distributed OO systems and real-time simulators of their operating environments. The TMO model provides structured timing semantics for distributed real-time object-oriented applications by extending conventional invocation semantics for object methods, i.e., CORBA operations, to include (1) invocation of time-triggered operations based on system times and (2) invocation and time bounded execution of conventional message-triggered operations. The ROFES project [31] is a Real-time CORBA implementation for embedded systems. ROFES uses a microkernellike architecture [31]. ROFES has been adapted to work with several different hard real-time networks, including SCI [32], CAN, ATM, and an ethernet-based time-triggered protocol [33].

10 The URI project [34] is a Real-time CORBA system developed at the US Navy Research and Development Laboratories (NRaD) and the University of Rhode Island (URI). The system supports expression and enforcement of dynamic endto-end timing constraints through timed distributed method invocations (TDMIs) [35] The The ACE ORB TAO [36] is a widely-used, opensource ORB compliant with most of the CORBA 3. specification [37]. TAO has been used in mission-critical DRE applications for over six years [38]. TAO supports the Realtime CORBA 1. specification and portions of Real-time CORBA 2.. B. RTSJ middleware research. RTSJ middleware is an emerging field of study. Researchers are focusing at RTSJ implementations, benchmarking efforts, and program compositional techniques. The TimeSys corporation has developed the official RTSJ Reference Implementation (RI) [13], which is a fully compliant implementation of Java that implements all the mandatory features in the RTSJ. TimeSys has also released the commercial version, JTime, which is an integrated real-time JVM for embedded systems. In addition to supporting a real-time JVM, JTime also provides an ahead-of-time compilation model that can enhance RTSJ performance considerably. The jrate [14], [39] project is an open-source RTSJbased real-time Java implementation developed at Washington University, St. Louis. jrate extends the open-source GNU Compiler for Java (GCJ) run-time system [4] to provide an ahead-of-time compiled platform for RTSJ. The Real-Time Java for Embedded Systems (RTJES) program [41] is working to mature and demonstrate real-time Java technology. A key objective of the RTJES program is to assess important real-time capabilities of real-time Java technology via a comprehensive benchmarking effort. This effort is examining the applicability of real-time Java within the context of real-time embedded system requirements derived from Boeing s Bold Stroke avionics mission computing architecture [2]. The researchers at the Washington University, St Louis are investigating automatic mechanisms [42] that enable existing Java programs to become storage-aware RTSJ programs. Their work centers on validating RTSJ storage rules using program traces and introducing storage mechanisms automatically and reversibly into Java code. V. CONCLUDING REMARKS Distributed Real-time and Embedded (DRE) systems are growing in number and importance as software is increasingly used to automate and integrate information systems with physical systems. Over 99% of all microprocessors are now used for DRE systems [43]. Ensuring end-to-end middleware predictability is essential to support the QoS capabilities needed by DRE applications. Therefore integration of RTSJ and Real-time CORBA is essential to ensure predictability required for DRE applications. This paper described our R&D activities associated with an previously unexplored dimension in real-time middleware: the integration of RTSJ features to support Real-time CORBA. We showed how scoped memory and real-time threads can be associated within a real-time ORB Core without violating RTSJ rules, yet still remaining compatible with the CORBA specification. The empirical results presented in Section III- D demonstrate that significant predictability improvements can be achieved by applying RTSJ features in ZEN. All of our optimizations and enhancements are compliant with the CORBA specification and are transparent to DRE application developers. Based on our experience building TAO, we have devised an R&D plan for ZEN. As shown in Figure 11, the heart of ZEN consists of optimization strategies, e.g., buffer management strategies, demultiplexing techniques and connection management optimizations that ensure predictability and scalability required by DRE applications. These optimizations are applied at the algorithmic and data structural level and are independent of the RTSJ implementation. This foundation has attained MICRO ORB ARCH SCOPED MEM THREAD POOLS RT-CORBA IMPLEMENTATION RTSJ OPTIMIZATIONS VIRT COMP PATTERN MUTEXS RT-Thread ACTIVE DEMUX OPTIMIZATION STRATEGIES Fig. 11. ZEN R&D Plan PRIORITY BANDS NHRT Thread PERFECT HASHING maturity in the current release of ZEN. We are applying RTSJ optimizations over this foundation to further enhance ORB predictability. Our ultimate goal is to implement a fully compliant Real-time CORBA implementation atop a mature RTSJ layer based on the commercial TimeSys Real-time Java platform. We believe that there is an important synergy between the layers, i.e., optimizing a single level will not ensure the layer-to-layer and end-to-end predictability required for DRE applications. VI. ACKNOWLEDGMENTS We would like to acknowledge the efforts of the other members of the Distributed Object Computing (DOC) research group at UC Irvine who are contributing to the design and implementation of ZEN. Special thanks to Angelo Corsaro, Carlos O Ryan, and Ossama Othman for contributing to ZEN s initial design. REFERENCES [1] Realtime Platform SIG, Realtime CORBA, White Paper, Object Management Group, Dec. 1996, Editor: Judy McGoogan, Lucent Technologies. [2] David C. Sharp, Reducing Avionics Software Cost Through Component Based Product Line Development, in Proceedings of the 1th Annual Software Technology Conference, Apr

11 [3] David C. Sharp, Avionics Product Line Software Architecture Flow Policies, in Proceedings of the 18th IEEE/AIAA Digital Avionics Systems Conference (DASC), Oct [4] J. Loyall, J. Gossett, C. Gill, R. Schantz, J. Zinky, P. Pal, R. Shapiro, C. Rodrigues, M. Atighetchi, and D. Karr, Comparing and Contrasting Adaptive Middleware Support in Wide-Area and Embedded Distributed Object Applications, in Proceedings of the 21st International Conference on Distributed Computing Systems (ICDCS-21). IEEE, Apr. 21, pp [5] Chris Gill, Douglas C. Schmidt, and Ron Cytron, Multi-Paradigm Scheduling for Distributed Real-Time Embedded Computing, IEEE Proceedings, Special Issue on Modeling and Design of Embedded Software, Jan. 23. [6] Object Management Group, The Common Object Request Broker: Architecture and Specification, Revision 2.6, Dec. 21. [7] J. P. Morgenthal, Microsoft COM+ Will Challenge Application Server Market, [8] Ann Wollrath, Roger Riggs, and Jim Waldo, A Distributed Object Model for the Java System, USENIX Computing Systems, vol. 9, no. 4, November/December [9] Douglas C. Schmidt and Steve Vinoski, The History of the OMG C++ Mapping, C/C++ Users Journal, Nov. 2. [1] Douglas C. Schmidt and Steve Vinoski, Standard C++ and the OMG C++ Mapping, C/C++ Users Journal, Jan. 21. [11] Inc ZeroC, The Internet Communications Engine, com/ice.html, 23. [12] Bollella, Gosling, Brosgol, Dibble, Furr, Hardin, and Turnbull, The Real-Time Specification for Java, Addison-Wesley, 2. [13] TimeSys, Real-Time Specification for Java Reference Implementation, [14] Angelo Corsaro and Douglas C. Schmidt, Evaluating Real-Time Java Features and Performance for Real-time Embedded Systems, in Proceedings of the!#"%$ IEEE Real-Time Technology and Applications Symposium, San Jose, Sept. 22, IEEE. [15] Douglas C. Schmidt, David L. Levine, and Sumedh Mungee, The Design and Performance of Real-Time Object Request Brokers, Computer Communications, vol. 21, no. 4, pp , Apr [16] Angelo Corsaro, Douglas C. Schmidt, Raymond Klefstad, and Carlos O Ryan, Virtual Component: a Design Pattern for Memory-Constrained Embedded Applications, in Proceedings of the &'"%$ Annual Conference on the Pattern Languages of Programs, Monticello, Illinois, Sept. 22. [17] Raymond Klefstad, Douglas C. Schmidt, and Carlos O Ryan, The Design of a Real-time CORBA ORB using Real-time Java, in Proceedings of the International Symposium on Object-Oriented Realtime Distributed Computing. IEEE, Apr. 22. [18] Raymond Klefstad, Arvind Krishna, and Douglas C. Schmidt, Design and Performance of a Modular Portable Object Adapter for Distributed, Real-Time, and Embedded CORBA Applications, in Proceedings of the 4th International Symposium on Distributed Objects and Applications, Irvine, CA, October/November 22, OMG. [19] Atif M. Memon and Adam Porter and Douglas C. Schmidt and Cemal Yilmaz and Bala Natarajan and Adithya Nagarajan, Distributed Continuous Quality Assurance: Leveraging User Resources to Improve Software Quality Around-the-World, Around-the-Clock, in Submitted to the Foundations of Software Engineering Conference, Hensinki, Finland, Sept. 23, ACM. [2] David D. Nilsen, Adding real-time capabilities to java, Communications of the ACM, vol. 41, no. 7, pp , July [21] Peter C. Dibble, Real-Time Java Platform Programming, Prentice Hall, 22. [22] Perry Cheng and Guy Belloch, A parallel, real-time garbage collector, in ACM SIGPLAN Conference on Programming Language Design and Implementation, 21, pp [23] David F. Bacon, Perry Cheng, and V. T. Rajan, A real-time garbage collector with low overhead and consistent utilization, in Proceedings of the 3th ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 23, pp , ACM Press. [24] Angelo Corsaro, Douglas C. Schmidt, Ron K. Cytron, and Chris Gill, Formalizing Meta-Programming Techniques to Reconcile Heterogeneous Scheduling Disciplines in Open Distributed Real-Time Systems, in Proceedings of the 3rd International Symposium on Distributed Objects and Applications., Rome, Italy, Sept. 21, OMG, pp [25] Gordon S. Blair and G. Coulson and P. Robin and M. Papathomas, An Architecture for Next Generation Middleware, in Proceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing, London, 1998, pp , Springer- Verlag. [26] Fábio M. Costa and Gordon S. Blair, A Reflective Architecture for Middleware: Design and Implementation, in ECOOP 99, Workshop for PhD Students in Object Oriented Systems, June [27] Fabio Kon, Fabio Costa, Gordon Blair, and Roy H. Campbell, The Case for Reflective Middleware, Communications of the ACM, vol. 45, no. 6, pp , June 22. [28] Gregor Kiczales, John Lamping, Anurag Mendhekar, Chris Maeda, Cristina Videira Lopes, Jean-Marc Loingtier, and John Irwin, Aspect- Oriented Programming, in Proceedings of the 11th European Conference on Object-Oriented Programming, June [29] Douglas C. Schmidt, Michael Stal, Hans Rohnert, and Frank Buschmann, Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects, Volume 2, Wiley & Sons, New York, 2. [3] K. H. (Kane) Kim, Object Structures for Real-Time Systems and Simulators, IEEE Computer, pp. 62 7, Aug [31] RWTH Aachen, ROFES, [32] M. Pfeiffer S. Lankes and T. Bemmerl, Design and Implementation of a SCI-based Real-Time CORBA, in 4th IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 21), Magdeburg, Germany, May 21, IEEE. [33] M. Reke S. Lankes and A. Jabs, A Time-Triggered Ethernet Protocol for Real-Time CORBA, in 5th IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 22), Washington, DC, Apr. 22, IEEE. [34] Victor Fay Wolfe, Lisa Cingiser DiPippo, Roman Ginis, Michael Squadrito, Steven Wohlever, Igor Zykh, and Russel Johnston, Real- Time CORBA, in Proceedings of the Third IEEE Real-Time Technology and Applications Symposium, Montréal, Canada, June [35] Victor Fay-Wolfe, John K. Black, Bhavanai Thuraisingham, and Peter Krupp, Real-time Method Invocations in Distributed Environments, Tech. Rep , University of Rhode Island, Department of Computer Science and Statistics, [36] Center for Distributed Object Computing, The ACE ORB (TAO), schmidt/tao.html, Washington University. [37] Object Management Group, The Common Object Request Broker: Architecture and Specification, 3. edition, June 22. [38] Timothy H. Harrison, David L. Levine, and Douglas C. Schmidt, The Design and Performance of a Real-time CORBA Event Service, in Proceedings of OOPSLA 97, Atlanta, GA, Oct. 1997, ACM, pp [39] Angelo Corsaro and Douglas C. Schmidt, The Design and Performance of the jrate Real-Time Java Implementation, in On the Move to Meaningful Internet Systems 22: CoopIS, DOA, and ODBASE, Robert Meersman and Zahir Tari, Eds., Berlin, 22, Lecture Notes in Computer Science 2519, Springer Verlag, pp [4] GNU is Not Unix, GCJ: The GNU Compiler for Java, gcc.gnu.org/java, 22. [41] Jason Lawson, Real-Time Java for Embedded Systems (RTJES), java/lawson.pdf, 21. [42] Morgan Deters, Nicholas Leidenfrost, and Ron K. Cytron, Translation of Java to Real-Time Java using aspects, in Proceedings of the International Workshop on Aspect-Oriented Programming and Separation of Concerns, Lancaster, United Kingdom, Aug. 21, pp. 25 3, Proceedings published as Tech. Rep. CSEG/3/1 by the Computing Department, Lancaster University. [43] Alan Burns and Andy Wellings, Real-Time Systems and Programming Languages, 3rd Edition, Addison Wesley Longmain, Mar. 21.