Are distributed objects fast enough?

Size: px

Start display at page:

Download "Are distributed objects fast enough?"

Sharlene Hutchinson
5 years ago
Views:

1 Matjaz B. Juric, Ales Zivkovic, Ivan Rozman Are distributed objects fast enough? Java is becoming important for building real world, mission critical applications. Although Java is still not a perfect language, it is becoming more mature every day. We all know the advantages of Java, especially the write once, run anywhere approach, but we are also aware of the disadvantages. Performance issue is the most common reason for Java rarely use today. In spite of that, there are many large companies claiming they are developing their crucial business applications in Java. Modern applications are not monolithic programs, they are built of objects. Therefor developers need a glue for bringing all the pieces together and combine them into a functional application. Object location independence is an advantage that gives developers the ability to structure the application into multiple tiers. For building distributed applications in Java it is natural to choose Remote Method Invocation (RMI), but there is another possibility - CORBA. CORBA is a valuable alternative to RMI. We could describe CORBA as a superset of RMI, although both technologies are not compatible yet. Both CORBA and RMI allow remote method invocation independently of location, however CORBA is more than just an object request broker. It offers programming language independence and a rich set of object services and common facilities all the way to the business objects. There are multiple factors that can affect the decision. One of them is certainly the performance. To get an overview about the performance of RMI and CORBA we have done several measurements and experiments to simulate real world scenarios. We hope that the founding of our work will help you to get a clearer picture about both distributed object models. Testing method The goal was to measure the overhead which was introduced into method invocation by CORBA and RMI. Therefore we have developed several testing patterns and methods with different basic data types of parameters and return values. As a basis for performed tests we used an ATM (Automatic Teller Machine) application, developed for demonstration purposes for one of our banks. The invocation time was measured for six methods with different return types. The main idea was to investigate the influence of data types on performance. The significant code with methods and corresponding classes is shown in Listing 1. As you can see all the methods return just a value. We chose this approach to minimize the influence of time needed to execute the methods.

2 public class Atm { private long number; private boolean working=true; final public boolean Working() { return working; final public long getatmno() { return number; public class Card { private int number; final public int getnumber() { return number; public class Account { private String type; private float balance=0f; private double limit; final public float getbalance() { return balance; final public String gettype() { return type; final public double getlimit() { return limit; Listing 1: Methods, used for performance measurement The methods were invoked from an applet which had a panel for displaying results and a button for start. Time was measured with System.currentTimeMillis() method. It has returned time in milliseconds. To achieve more accurate results methods were invoked several times. In Listing 2 the skeleton of testing applet is shown. boolean is_working=false; my_account.setbalance((float)50f); for(int j=0;j<10;j++) { long starttime=system.currenttimemillis(); for (n=0;n<no_iter;n++) is_working=my_atm.working(); long stoptime=system.currenttimemillis();

3 tmessage.append("elapsed time "+(stoptime-starttime)+"\n"); Listing 2: Client applet To eliminate inaccuracy we have repeated all the measurements ten times and calculated the average values. To get an idea how the string size affected the performance, the method Account.getType() returned two different string sizes. First, the method returned a string with one character only. Then we repeated the test with a string of characters. All the test code was compiled and executed with Java Development Kit For testing CORBA's performance we chose Visigenic s VisiBroker for Java version 3.0. For local measurements and all server side tests we used Pentium II 233 MHz computer with 64 MB RAM and Windows NT 4.0 Workstation as operating system. At the client side we used 200 MHz Pentium computers with 64 MB RAM and the same operating system. Because the trend today is to use the applications over the Internet where the bandwidth is low, the computers were connected into a 10Mbps Ethernet network. Native Java performance In the first step we measured native Java performance for method invocation where we invoked all methods in a loop one million times to achieve the necessary accuracy. The results are in Table 1. We found results for data types integer, float, boolean and String (1 character) had identical values. Also the results for types long and double were practically the same. Data type Time in ns int 427,8 long 300,5 float 428,3 double 301,5 boolean 428,5 String (1) 428,6 Average 385,9 Table 1: Native Java performance Java RMI performance To measure the overhead distributed processing introduces into method invocation we reprogrammed tests using RMI. We defined interfaces, separated the functionality into a server process and changed the client applet to connect to the server objects. First we

4 started both, the server and the client processes on the same Pentium II computer. As expected, the results showed very substantial performance degradation. In native Java an invocation lasts about 400 nanosecond. When using RMI the time was not measured in nanoseconds but in milliseconds. Because of that, the client repeated all invocations one thousand times thus accurate times could be measured. We have repeated tests ten times and reported the average values. The results are shown in Table 2. Data type Time in ms int 1,87 long 1,95 float 1,83 double 1,96 boolean 1,82 String (1) 1,85 Average 1,88 String (10.000) 108,00 Table 2: Java RMI performance on a single computer The times for methods which returned results of types integer, long, float, double, boolean and String (1 character) were very close. A method invocation using RMI lasted around 1.9 milliseconds on the hardware configuration used for testing. When we increased the string size to characters, the response time grew to 108 ms. The performance degradation factor was slightly over 58. Secondly, the measurement was repeated with server and client on separate computers. The server was the Pentium II computer and the client a Pentium 200. As already stated, the computers were connected using a 10 Mbps Ethernet network. The results are shown in Table 3. Data type Time in ms Relative to local RMI int 2,08 11,4% long 2,13 9,0% float 2,09 14,2% double 2,14 9,3% boolean 2,08 14,2% String (1) 2,10 13,5% Average 2,10 11,9% String (10.000) 77,39-28,3% Table 3: Java RMI performance over the network

5 When the methods were invoked over the network, the time on average was 12% slower than when they were invoked on the same computer. This was true for all return values except for the string with characters. When returning such a string the performance was approximately 28% better over the network. The blame probably went to Java Virtual Machine, which appeared to be quite CPU intensive. In our opinion it is possible to show that using slower computers and method invocation over the network the measurement times will improve compared to local invocation. To get more real performances of a client server system we used eight computers for the same tests. In the next test we tried to figure out the performance degradation under load imposed on the system. We configured eight Pentium computers to run the client applet. For the server side we used the Pentium II computer. The client applets were running on all the client computers simultaneously. Each client invoked each test method one thousand times. All the test were repeated ten times to get an average value. The clients invoked methods without any delays. This did not coincide with typical user interaction, therefore we assumed that this load represented more than eight users. In the Table 4 the results and relative performance degradation are presented. Time in ms Data type PC 1 PC 2 PC 3 PC 4 PC 5 PC 6 PC 7 PC 8 Avg Deg % int 9,16 8,88 8,68 8,00 8,54 7,94 8,81 8,59 8,58 312% long 8,76 7,62 7,44 6,91 9,32 8,38 7,72 7,76 7,99 275% float 7,16 8,06 6,64 8,10 7,45 7,90 7,80 7,59 7,59 263% double 7,22 8,25 7,19 7,35 7,67 8,58 7,56 7,74 7,70 260% boolean 8,36 7,85 8,91 8,58 8,47 8,14 8,45 8,11 8,36 302% String (1) 8,38 7,93 8,15 7,60 8,22 8,28 8,56 8,42 8,19 290% Average 8,17 8,10 7,84 7,76 8,28 8,20 8,15 8,04 8,07 284% String (10.000) 226,38 208,43 310,95 198,60 200,22 228,98 298,56 317,42 248,69 221% Table 4: Java RMI performance under eight client load It was determined that the average response time increased for 284% from slightly over 2 to about 8 milliseconds. A similar degradation occurred by string method with string size of characters. The degradation there was slightly smaller, from 77 to 250 milliseconds (about 220%). The results are graphically presented in Graph 1 and Graph 2. The first graph shows the response times for method invocations with different return types for a single computer scenario, a scenario with two and with eight client computers. This are performances a developer might expect when using methods with short return values.

6 50,0HWKRGLQYR DWLRQWLPH 5HVSRQVH WLPH LQ PLOOLVHFRQGV LQW ORQJ IORDW GRXEOH ERRO 6WULQJ FOLHQWV &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHUV &OLHQW DQG VHUYHU RQ WKH VDPH FRPSXWHU 0 HWKRG W\SH Graph 1: RMI method invocation time for simple return values In Graph 2 you can find the invocation times for methods with large return values. For comparison the graph also shows an average performance of methods with simple return values. Please note that the scales for both lines are different. In the graph you can see that the performance of large string method increased when we place the client and server on different computers. This is due to the CPU intensity of Java Virtual Machine. 50, 0HWKRG LQYRFDWLRQ WLPH 7LPHLQ PV &OLHQW DQG VHUYHU RQ RQH &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHU FRPSXWHUV FOLHQWV 0HWKRG ZLWK FRPSOH[ UHWXUQ YDOXH $YHUDJH PHWKRG ZLWK VLPSOH UHWXUQ YDOXH Graph 2: RMI method invocation time for large string method in comparison with average of simple methods.

7 CORBA performance CORBA can be considered a more mature technology than RMI. It has been around for years and there are numerous real world projects using it. CORBA is robust, scalable technology and its performances (especially in C++) are quite good. So it would be expected to get good results from CORBA-Java connection as well. Until now very little research has been done in this area. Therefor we decided to compare Java RMI performance with Java-CORBA. Numerous CORBA implementations exist today and most of them offer Java support. For the comparison we selected Visibroker for Java version 3.0 by Visigenic. It is one of the best CORBA implementations on the market and it has ORB bundled with Netscape Navigator 4.0. We adapted the tests described in Java RMI chapter because our goal was to make the Java RMI and CORBA results comparable. The first step in development of CORBA compliant applications was the definition of interfaces in IDL (Interface Definition Language). Because the return types of methods had been already defined in Java this types had to be mapped into IDL. The correct mapping is shown in Table 5. Java Type int long float double boolean java.lang.string IDL Type long long long float double boolean string Table 5: Type mappings The next step was the definition of interfaces and methods. In IDL we defined the same interfaces and methods as in Java RMI. The important parts of IDL definitions are shown in Listing 3. interface Atm { boolean Working(); long long getatmno(); ; interface Card { long getnumber(); ; interface Account { float getbalance();

8 ; string gettype(); double getlimit(); Listing 3: IDL definitions In both the client and the server code we made only CORBA-specific changes. The testing procedure was the same as when using Java RMI. Again all the measurements were repeated ten times and the applet at client side has reported average values. In the first column of Table 6 you can see the times where server objects and client applet were on the same Pentium II computer. In the second column the times are given where server and client are communicating over network. In the third column the difference in percentage is calculated. Data type Client and server on the same computer (times in ms) Client and server on different computers Difference int 2,20 2,31 5,0% long 2,21 2,32 5,0% float 2,20 2,27 3,2% double 2,25 2,29 1,8% boolean 2,02 2,23 10,4% String (1) 2,24 2,30 2,7% Average 2,19 2,29 4,6% String (10.000) 19,59 28,12 43,5% Table 6: CORBA performance with one client The average slowdown for using methods over network was smaller than 5%. Only the method which returned characters was slower for 43,5%. Our next test was to measure the performance using eight clients. The test was executed under same conditions as when testing Java RMI. Again, on eight client computers the applets were started simultaneously. The results are shown in Table 7. Data type PC 1 PC 2 PC 3 PC 4 PC 5 PC 6 PC 7 PC 8 Avg Deg % int 8,36 8,18 6,72 7,09 7,87 7,32 7,59 8,60 7,72 234% long 9,08 8,82 6,36 7,23 7,81 6,55 7,23 7,47 7,57 226% float 6,51 7,45 7,32 7,75 6,92 7,89 7,51 7,50 7,36 224% double 6,39 6,77 7,87 7,04 6,92 8,21 9,54 6,76 7,44 225% boolean 7,79 7,75 7,91 8,05 8,24 9,76 8,04 7,16 8,09 263% String (1) 7,41 8,10 7,77 7,89 7,41 8,03 7,69 8,12 7,80 239% Average 7,59 7,84 7,32 7,51 7,53 7,96 7,94 7,60 7,66 235% String (10.000) 105,64 91,85 88,45 101,00 98,92 178,25 123,60 113,99 112,71 301% Table 7: CORBA performance under eight client load

9 On average it took 3,3 times longer to execute a method under load conditions, the execution times were around 7,7 instead of 2,3 milliseconds. Only the method which returned a large string executed 4 times longer. Rather than 28 it took more than 112 milliseconds to complete. The Graph 3 shows the response times for methods returning different types for all the three scenarios used for testing. The Graph 4 shows the average response time of tested methods compared to response time of large string method for all three scenarios. The scales for both lines are different. &25%$0HWKRGLQYR DWLRQWLPH 7LPH LQ PLOOLVHFRQGV LQW ORQJ IORDW GRXEOH ERRO 6WULQJ FOLHQWV &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHUV &OLHQW DQG VHUYHU RQ WKH VDPH FRPSXWHU 0HWKRG W\SH Graph 3: CORBA method invocation time for simple return values &25%$ 0HWKRG LQYRFDWLRQ WLPH 7LPH LQ PV &OLHQW DQG VHUYHU RQ RQH FRPSXWHU &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHUV FOLHQWV 0HWKRG ZLWK FRPSOH[ UHWXUQ YDOXH $YHUDJH PHWKRG ZLWK VLPSOH UHWXUQ YDOXH Graph 4: CORBA method invocation time for large string method compared to average of simple methods.

10 CORBA vs. RMI The results of tested performance were even more interesting when we compared RMI and CORBA. We could do this because the code used in both cases was the same. It differed only in the way the connection between the client and the server was established. The Graph 5 shows an average response time of CORBA and RMI in three different testing scenarios. $YHUDJHUHVSRQVH RPSDULVRQ 7LPH LQ PV &OLHQW DQG VHUYHU RQ WKH VDPH FRPSXWHU &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHUV FOLHQWV 50, &25%$ Graph 5: Average response time comparison between RMI and CORBA The fact that RMI was slightly faster on one computer and on separate client and server was not surprising. CORBA is much more complex than RMI is. We should not forget that CORBA supports different programming languages and has more layers. On the other hand the computers which we used for testing were quite fast. The deficits of Java Virtual Machine therefore were not so obvious. When the results under the load of eight clients were compared we found out that CORBA was faster. Still the difference was not significant. It is important to notice that the performance trade off by using CORBA was 235%. By using RMI on the other hand it was 284%. The assumption that CORBA is faster was confirmed when comparing responses by invoking a method which returned a character string. The Graph 6 shows that CORBA was faster in all the situations. Under load of eight clients CORBA was more than twice as fast as RMI. The difference was even bigger on one computer, where CORBA was five times faster.

11 /DUJH VWULQJ UHVSRQVH 7LPH LQ PV &OLHQW DQG VHUYHU RQ WKH VDPH FRPSXWHU &OLHQW DQG VHUYHU RQ GLIIHUHQW FRPSXWHUV FOLHQWV 50, &25%$ Graph 6: Large string response time comparison Conclusion The performance tests gave us a picture about method invocation times using different data types of different sizes. We saw that both distributed object architectures introduced large overhead in method invocation. Therefor a developers should have this in mind when developing distributed applications. They should pay a great deal of attention to object design and to communications between them. When applications are poorly designed and the communication between objects is too heavy the users will soon discover unacceptable performances and improving them will be difficult and time consuming. We also saw that on fast computers Java RMI performance was acceptable and in some cases comparable to CORBA. This was true especially when small amounts of information were transferred between objects. By evaluation of numbers we have to stress once again that the tested CORBA implementation Visibroker is not the only CORBA compliant ORB. According to some unexamined sources, mainly on the Internet, Visibroker is one of the best CORBA products available. To get a clearer picture how fast different CORBA products are we are already working on a separate comparisons. And who is the winner? Unfortunately it was not as easy to pick a winner as we hoped it would be. When the most important criteria was examined and that was the performance degradation under heavy client load then CORBA was the winner. CORBA s performance degradation was lower than RMI s. On the other hand it was once again proved that CORBA was a very complex product and this complexity was reflected in performance. This was why in simple cases (small client load, simple

12 return values) on fast computers RMI was faster. This result is also congruent with the goals of both architectures. In our opinion RMI is very useful for simple applications, where it can offer acceptable performances. When applications become more complex other reasons for choosing CORBA besides performance will have to be considered. Based on the announcements of JavaSoft, in the near future RMI will accept the IIOP protocol and both architectures will become compatible. As soon as this will happen we will not hesitate to make another performance comparison. Authors would like to thank Mr. M. Hericko, B. Brumen, T. Domajnko, S. Beloglavec and Ms. A. Marinsek for their help and useful suggestions by performance measurements.

Performance Comparison of CORBA and RMI

Abstract Performance Comparison of CORBA and RMI Matjaz B. Juric, Ivan Rozman, Marjan Hericko Institute of Informatics, Faculty of Electrical Engineering, Computer and Information Science, University of