As Michi Henning and Steve Vinoski showed 1, calling a remote

Transcription

1 Reducing CORBA Ca Latency by Caching and Prefetching Bernd Brügge and Christoph Vismeier Technische Universität München Method ca atency is a major probem in approaches based on object-oriented middeware such as CORBA, DCOM, and RMI, especiay for fine-grained server interfaces. A mechanism that groups method cas and caches resuts can reduce method ca atency. The middeware runtime system does this transparenty without affecting cient code or server impementation. As Michi Henning and Steve Vinoski showed 1, caing a remote method over a computer network is orders of magnitude sower than caing a method directy within the same process space. Factors such as marshaing rate, network bandwidth, and network atency determine remote method invocation (RMI) 2 duration. Figure 1 shows a typica CORBA-IDL definition of a person interface. A sampe phone book cient dispaying, say, 100 person entries woud iterate over a ist of 100 person object references. Each entry woud invove a reference ca getname(), foowed by getemai(), foowed by getphone(),for a tota of 300 RMIs, or 300 network round-trips. IEEE Distributed Systems Onine Pubished by the IEEE Computer Society 2003 IEEE

2 Figure 1. Sampe CORBA-IDL definition of a person interface. To overcome the ca atency probem, which becomes more serious as server interfaces become more fine-grained, software deveopers try to reduce messaging overhead by defining coarse-grained methods. By doing this, the cient can execute a specific task by caing one method that returns mutipe data eements instead of caing many methods that return one data eement each. We can improve cient performance in the phone book scenario by introducing a coarse-grained method, as Figure 2 shows. Given the task of dispaying 100 person entries, we can rewrite the cient to ca getnamepusemaipusphone(), reducing the number of network round-trips from 300 to

3 Figure 2. A coarse-grained method. Repacing object orientation with data structures that can be marshaed together, as Figure 3 shows, can achieve further improvements. Figure 3. Data structures instead of interfaces. Athough introducing coarse-grained methods and data structures is an easy and widey used approach, it has some disadvantages: Deveopment time. Adding a method in a cient-server interface changes the software system's reease cyce. You 3

4 must impement the newy defined method on the server side, and rewrite it on the cient side so it is actuay caed. After impementation, you must test the modifications, update the documentation, and finay, reease and ro out the new system. Appication dependency. Coarse-grained methods are usuay appication dependent, which is their strength. They deiver the exact amount of data required for a specific cient task. Deivering too much data wastes bandwidth (which can hurt appication performance). Deivering too itte data means that mutipe methods must be caed, increasing network atency and sowing the appication. In the phone book exampe, if the end-user requests that the phone book appication shows an additiona person attribute (for exampe, an address), the appication deveoper must define a new method (such as getnamepusemaipusphonepusaddress()). Redundancy. Introducing a new method that redundanty incudes another method's job vioates a basic software engineering principe: the separation of concerns. With every modification of a fine-grained method impementation, the deveoper must ensure that the coarse-grained methods that rey on the modified method are working correcty. This adds extra overhead to the deveopment effort. Even more important, this kind of redundancy is error-prone and hard to maintain. Putting it a together, using coarse-grained methods (or "fat" operations 1) maximizes appication performance, but requires more deveopment time and maintenance effort (see the "Reated Work" sidebar). Approach When using coarse-grained methods (or data structures), a deveoper effectivey impements a certain prefetching mechanism. When a cient invocates a coarse-grained method, the server 4

5 transfers mutipe data eements (such as person attributes) to the cient, minimizing ca atency at the cost of greater deveopment effort. Our middeware runtime system creates coarse-grained methods on the fy. The generation of these methods, the cient code, and the server impementation, are a transparent to the deveoper. This ets us minimize ca atency without increasing deveopment and maintenance effort. We've based the current impementation on IDL annotations (shown in Figure 4) and cient-side ca statistics. Methods that have neither parameters nor side effects (marked as const in IDL) can be grouped together, and their resuts stored in a cient cache. These cache entries wi expire after a specific time-to-ive (TTL) vaue, which is aso defined in IDL on a per-interface basis. Method dependencies are judged based on the ca statistic. Figure 4. IDL annotation for person interface. The appication programmer annotates the IDL and feeds it into an IDL compier that generates specia proxy and stub casses. Every time a method is caed on one of the extended proxy objects, the 5

6 method identifier (name or numeric id) is stored in a statistic with a timestamp. Later, the middeware runtime system (more exacty the generated cient-proxy) uses this statistica information to find out which methods might beong together and shoud be grouped. The basic steps in a remote method invocation are If the resut of the caed method is aready in the cient cache, and the cache entry is not expired, the cient proxy returns the cached resut. If the method resut is not in the cache or has expired, the cient proxy must find out which methods are ikey to be caed next. The cient proxy sends a method identifiers to the server for invocation. The server stub invokes a methods on the server impementation. The server stub returns a resuts to the cient. The cient proxy stores a resuts in cache. Impementation We deveoped an agorithm for cacuating which methods are to be prefetched when a remote method is invoked: We defined a ca history per proxy object. The ca history is a ist of proxy ca entries. Each proxy ca entry consists of a method identifier and a timestamp. We then introduced a dependency matrixa matrix that stores data about method dependencies-for every interface. Tabe 1 presents the dependency matrix for our person interface. We assign a coumn and a row to each method that can be prefetched. Every method that is decared const in IDL and does not have parameters is a candidate for prefetching. We aso insert an empty coumn. 6

7 Each time a method is caed on a proxy object, the vaue of ce (empty, <caed method>) of the associated interface dependency matrix increases by one. Thus, the empty coumn stores the number of times a method has been invoked. Next, we take the ca history of the caed proxy object and remove a proxy ca entries that are oder than the TTL vaue decared in IDL. The remaining entries are the preceding methods of the previousy caed method. For each of these preceding methods, we increment the ce (<preceding method>, <caed method>) in the dependency matrix. Because we discard od entries from the proxy ca history, and each IDL-defined interface has ony one dependency matrix, we have constant memory consumption over time, where the upper imit depends on method invocation rate, which in turn is imited by program execution speed. We can state the foowing: The vaue of ce (empty, X) indicates how often method X has been caed. The vaue of ce (X, Y) indicates how often method X has been caed before method Y within the specified TTL vaue. Knowing this, we can cacuate the probabiity p, which indicates the probabiity of method Y if X has been caed (see Equation 1). 7

8 Equation 1. Probabiity for X => Y For each proxy method ca that can't be taken from cache and must be sent over the network, we search the dependency matrix for methods that are ikey to be caed next, with a certain probabiity P (75 percent in our case). For exampe, if a cient cas getname()on a proxy object, we iterate over the remaining methods (getphone() and getemai() in our case) and cacuate p for each. We group methods with probabiity p greater or equa to P and prefetch them. The server executes a methods one after another and sends the resuts to the server stub, which sends them a back at once to the cient proxy. After receiving the resuts, the cient proxy stores the resut vaues in the cient cache, and returns the resut of the origina method. Resuts In our phone book scenario, the phone book cient fetches 100 person object references from the server and iterativey cas the methods getname(), getemai(), getphone(), and so on. After a whie, when the statistic has enough information, the cient proxy object groups the methods: If the cient cas getname() on a person proxy object, the statistic knows that getemai() and getphone() wi ikey be caed next. the cient proxy then sends the method names getname(), getemai(), and getphone() to the server, where the server stub invokes server impementations of the methods. After invocation, the resuts are sent back to the person proxy object, which stores them in a cache and returns the resut of getname(), the initiay caed method. When the cient cas the next method on the proxy object (getemai() in this case), the cient proxy object takes the resut from the cache without reconnecting to the server if the cache content is not oder than the TTL specified for the method. The same hods for the getphone ()method. 8

9 Tabe 2 shows test resuts from the phone book scenario. A cient fetches 100 person references from a server and iterates over these references, invoking eight methods on each object reference. We used the Java JDK CORBA impementation. We impemented cient proxies and server stubs with the JDK-integrated IDL compier and manuay integrated the extensions. Tabe 3 shows the testbed configuration (CPU type, CPU speed, and network connection). Open issues Many open questions remain in our CORBA caching/prefetching system. The most reevant are: Cache consistency. We impemented a TTL expiration mode, which performs we. The drawback of the TTL approach is that 9

10 the deveoper must trade accuracy for performance. We pan to investigate aternative cache consistency mechanisms. Method invocation cost mode. In our test scenario, we use attributes (name, emai, and so on) that are short strings. If we prefetched a getimage() method that returned a 100-KByte image, and the cient didn't ca this method, we woud have wasted network bandwidth. We need a mode of method ca expenses that we can use to judge whether a method is worth caing. Deveoper transparency. In the current system, we use IDL annotations to indicate which methods can be cached and prefetched. So, in a way, we have given up tota transparency. Research can determine the appropriate trade off between deveoper transparency and deveoper contro. Accuracy of ca statistics. Having coected statistics data for a whie, we can guess what wi be caed next for a certain cient-proxy method ca. If the cient changes (for exampe, if there is a new reease), our statistic must adapt to the change. We must integrate an aging mechanism in our ca statistics. In addition to addressing these open issues in future deveopment, we pan to conduct fied tests of our middeware impementation against rea-word appications. References 1. M. Henning and S. Vinoski, Advanced CORBA Programming with C++, Addison-Wesey, Sun Microsystems, Java Remote Method Invocation (RMI), Bernd Brügge is university professor of computer science with a chair for appied software engineering at the Technische Universität München and adjunct professor at Carnegie Meon University. His research interests incude software architectures for dynamic systems, agie software deveopment processes, and software engineering education. He received a PhD in computer science from Carnegie Meon University. Contact him at bruegge@in.tum.de. 10

11 Christoph Vismeier is a PhD candidate at the Institut fur Informatik of the Technische Universität München, Germany. His research interests incude distributed object-oriented middeware. Contact him at vismeie@in.tum.de. Reated work One approach to overcoming the probem of remote method ca atency is object migration and mobie agent technoogy 1. Severa ongoing research projects sove the probem of network overhead by seriaizing objects and sending them over the network to the caer. Gregory V. Chocker and coeagues present a system that is based on object migration technoogy and uses hierarchica caches to store objects 2. The probems of cache consistency and cache expiration modes arise in various other caching appications, such as Web caching 3 and network fie systems 4. We strive to earn from these fieds and, possiby, appy their soutions to our prefetching system. References 1. Voyager documentation, Recursion Software, 2. G. Chocker et a., "Impementing a Caching Service for Distributed CORBA Objects," Proc. IFIP/ACM Int' Conf. Distributed Systems Patforms and Open Distributed Processing (Middeware 2000)., Springer, 2000, pp D. Duchamp, "Prefetching Hyperinks," Proc. 2nd Usenix Symp. Internet Technoogies and Systems, Usenix, 1999, pp M. Satyanarayanan et a., "Coda: A Highy Avaiabe Fie System for a Distributed Workstation Environment," IEEE Trans. Computers, vo. 39, 1990, pp