General Performance Assesement of the L4/Fiasco Micro-kernel

Transcription

1 General Performance Assesement of the L4/Fiasco Micro-kernel Cheng Guanghui, Zhou Qingguo*, Nicholas Mc Guire, Wu Wenzhong SISE, Lanzhou University Tianshui South Road 222, Lanzhou, Gansu, P.R.China Abstract This paper presents the experimental study of the performance of L4Linux on top of L4/Fiasco and Linux. Holding the hardware constant and using the different benchmark tools to measure a list of performances in order to give us a global view for performance comparison of L4Linux/Fiasco and Linux which could separately stand for the micro-kernel and monolithic kernel. Our experiments and analysis show the concrete measurement about their performances and in the end we present our conclusions. In Addition through these benchmarks we find some problems and maybe they are new task for the L4 community. 1 Introduction The debate with micro-kernel and monolithic kernel has been very hot all the time since the concept of micro-kernel appeared and maybe everyone could remember that famous one named Linus vs Tanenbaum. About one decade years ago the Herman Hartig, Jochen Liedtke and others published a paper The performance of of u-kernel-based systems in which the authors gave us a conclusion about the performance loss when comparing with monolithic kernel and micro-kernel. As a result the 5%-10% is as a conclusion when talking about the performance loss of micro-kernel vs monolithic kernel. In recent 10 years the Fiasco, L4Linux and Linux have many new versions. Especially the Linux kernel updates very very fast and from 2.0 to 2.4 and from 2.4 to 2.6 it varied very much. We repeat the similar experiments they did 10 years ago but with the tools and system of newest version. L4 is a second-generation micro-kernel specification first designed and implemented by Jochen Liedtke. Now this specification of the L4 API has implementation in many different flavours, one of which explicitly targets hard real-time systems - L4/Fiasco. L4/Fiasco, which is being developed at TU Dresden Contact us: Qingguo Zhou; Phone: ; zhouqg@lzu.edu.cn since 1996 is currently an implementation of the L4 V2 API specificatoin. The L4/Fiasco is the core of DROPS (The Dresden Real-Time Operating System Project) system which supports running real-time and time-sharing applications concurrently on one computer. L4Linux is a port of the Linux kernel to the L4 u-kernel API. Because of existence of L4Linux we could use the same tool to test two systems with different architecture. In order to get a global and fair overview for performance comparison of L4Linux and Linux we have four different benchmarks such as AIM9, Lmbench, bonnie++ and NetPIPE. In other words it is also as the view of comparison of micro-kernel and monolithic kernel. The rest of this paper is organized as following. In section 2: first we describe the test environment for evaluating them. Then we separate these results and reorganize them into 7 parts such as Numerical Operations, Process Operations, File Operations, Memory Operations, Process Communications, Disk I/O Performance, Network Performance. This classification mostly origins from the report of Lmbench benchmark. In section 3 we could state our conclusions about this test and give our opinions about the L4/Fiasco even the micro-kernel. 1

2 2 Performance Benchmark 2.3 Performance Metrics 2.1 Experimental Setup Numerical Operations The goal of this paper is to evaluate the L4Linux/Fiasco and Linux as the example of microkernel and monolithic kernel. In micro-kernel side we choose the newest version of Fiasco and L4Linux : Fiasco 1.2 and L4Linux (All the source code is downloaded from the cvs). In monolithic kernel side we choose the Linux (The kernel is downloaded from the offical kernel ftp site). we use the same file system from Ubuntu-6.06 and the hardware with AMD Duron 1.6G and 512M RAM. But in order for accuracy of comparison we limit the Linux and L4Linux could only use 256M memory. This is the basic test envirionment and if the benchmark need special hardware we could describle in the following section. The version information about benchmark tools is as following: Lmbench-3.0-a3, AIM Bechmark Suite IX(AIM9), NetPIPE 3.6.1, bonnie a. Lmbench integer float double L4Linux Linux AIM9 add mul div L4Linux Linux TABLE 1: Basic Numerical Operations 2.2 Experimental Method The benchmark once time is not reliable very much becuase of occasionality so every group of experiment we must have enough samples. So we have Lmbench benchmark 30 times and finally we have the average value. Because difference of AIM benchmark results with both L4Linux/Fiasco and Linux is trivial according to the results themselves. So we have the AIM benchmark 10 times. The situation of bonnie++ is like the Lmbench we do 30 times. The NetPIPE benchmark is very special and this test in Linux is very stable but L4Linux/Fiasco is very unstable. As a result the NetPIPE test in Linux is 10 times but the NetPIPE test in L4Linux/Fiasco needs to be 30 times. We will describle the NetPIPE benchmark in the following in detail. In the Lmbench benchmark about numberic operations in fact we test three types of number such as integer, float and double. In every type they have many kinds of operations such as add, mul, div and others. As different type we also give some particular operations For example mod of integer number. In the AIM9 the types of benchmark are more including double,float,long,int,short and they only give 3 operations with add, mul and div. Becuase they have too many kinds of numeric operations and all the results between these two kernels are almost the same. Therefore, in the TABLE 1 we use 1 to replace the real value because the raw data of L4Linux and Linux is the same. From the TABLE 1 we could find the performance of basic numberic operations with L4Linux/Fiasco and Linux almost has no difference and the micro-kernel doesn t have any visible performance loss. 2

3 2.3.2 Process Operations File Operations null- null- stat open/ Slct- Lmbench call I/O close TCP Linux L4Linux sig- sig- fork exec sh Lmbench inst hdnl Linux L4Linux kFile 0KFile 10kFile 10KFile Lmbench Create Delete Create Delete Linux L4Linux TABLE 4: File Operations In the TABLE 4 the smaller is better. From this table we could find that performance of Linux is also higher than L4Linux when they have some file operations Process Communication TABLE 2: Process Operations-lmbench Lmbench pipe af unix tcp tcp-conn From the TABLE 2 we could find the system call of micro-kernel is usually about 3 times than the monolithic kernel at least except select-tcp but the performance of select-tcp has different actions as for others. The L4Linux is equivalent with Linux in select-tcp. In this table the smaller is better. exec fork signal function function AIM9 -trap -call1 -call15 L4Linux Linux TABLE 3: Process Operations-AIM9 From the TABLE 3 we could find in a solid period that creating or execting a process in L4Linux is much slower than in Linux. But when having function call they have the same performance measurement. In this TABLE the bigger is better because it could deal with tasks faster if the number is more. In most situations the system call of micro-kernel is much slower than monolithic kernel from the Lmbench and AIM9. But this conlusion is not constant and it is only something experimental. Linux L4Linux TABLE 5: Proc Communication Latencies In the TABLE 5 we compare the communication latencies and we could find that the latency of L4Linux is double of Linux at least. And in this table the smaller is better. Lmbench pipe af tcp fileunix reread Linux L4Linux Lmbench bcopy bcopy mem- mem- (libc) (hand) read write Linux L4Linux : Proc Communication Band- TABLE width In the TABLE 6 the bigger is better and we compare the communication bandwidth of L4Linux and Linux. According to the result they have the similar performance in the communication bandwidth. 3

4 From the TABLE 5 and TABLE 6 we could know the communication latencies of L4Linux is much slower than Linux but the communication bandwidth of L4Linux and Linux are the similar Disk I/O operations bonnie++ real(m) user(m) sys(m) AIM9 disk rr disk rw disk rd disk wrt L4Linux Linux L4Linux Linux TABLE 9: Time Expense AIM9 disk cp sync rw sync wrt sync cp L4Linux Linux TABLE 7: Disk Operations(1) In the TABLE 7 the bigger is better. disk rr means Disk Reads (K) Per Second and others are similar. According to this table the disk read and disk write in L4Linux is also slower than in the Linux. but the actions relative with sync in L4Linux is much more than ones in Linux. According to the data in the TA- BLE 7 the sync speed in L4Linux is 10 times than in Linux at least. bonnie++ seq char cpu% seq blk cpu% read(k/sec) read(k/sec) From the TABLE 8 we could find that the L4Linux reads or writes the disk slower than Linux but it takes less CPU resources,too. We have the bonnie++ 30 times in total. From the TABLE 9 it only needs 78 minutes to finish all 30 tests in L4Linux but in Linux it needs 76 minutes to accomplish it. In conclusion the performance of L4Linux in disk I/O speed is very close to the performance of Linux Network Performance L4Linux Linux "L4.15" "L4.19" "Linux" bonnie++ seq rewr- cpu% seq char cpu% ite(k/sec) write(k/sec) L4Linux Linux Bandwidth in Mbps bonnie++ seq blk cpu% random cpu% write(k/sec) seek(/sec) L4Linux Linux e+06 1e+07 Message Size in Bytes TABLE 8: Disk Operations(2) Figure 1: NetPIPE Figure(1) 4

5 Bandwidth in Mbps "L4.19" "L4.2" "Linux" e+06 1e+07 "average" "Linux" "L4.17" "L4.2" 80 Bandwidth in Mbps Message Size in Bytes Figure 2: NetPIPE Figure(2) 0 0 1e+06 2e+06 3e+06 4e+06 5e+06 6e+06 7e+06 8e+06 9e+06 1e+07 Message Size in Bytes Figure 3: NetPIPE Figure(3) The basic setup is simple, we run NetPIPE on the native Linux installation and then rerun it under the L4Linux paravirtualization of the version-same kernel(2.6.17). Some problems seem to have been related to the PCI bus, the first tests were made with mainstream PCI card (RTL8139 and EEPRO100) - these test runs did not complete but simply hung for ever all the time. And the L4Linux output the kernel panic words. We tried different types of PCI cards but the same problems still appeared. Later we modify the relative file in L4Linux the kernel panic problem was resolved but the test didn t complete in most of time yet (2 times in 30 times are completed normally). Now the problems seem to be related to fairly large packet sizes and not directly related to the PCI bus. We think maybe it is a more fundamental problem in the current L4Linux version (CVS version ). In the Figure 1 you could see the 2 samples in which the netpipe could exit normally named L4.19 and L4.15 in comparison with the Linux sample. In the Figure 2 you could see the maximum sample of L4Linux named L4.19, the minimum sample of L4Linux named L4.19 and the Linux sample. In the Figure 3 you could see the average L4Linux sample named average, the maximum sample named L4.19, the minimum named L4.2 and the standard Linux sample. From these tests we think the L4/Fiasco or L4Linux is not stable in some situation. But L4Linux has a strong potential in the improvement of network performance because sometimes the maximum bandwidth could be up to 87M but the Linux is only 89M. And in 30 times experiment they have 5 times that the bandwidth is more than 80M. So we think it is very urgent and important to improve the stability of L4/Fiasco or L4Linux. If somebody could fix it it is possible that the average maximum bandwidth is up to 80M. Maybe how to resolve it is our nextstep of studying L4/Fiasco. NetPIPE average maximum minimum Linux L4Linux TABLE 10: Network Performance Testing of network performance showed some serious problems. We are not yet able to clarify fully if these problems are related only to the newest release of L4Linux or more fundamental. Nevertheless we present the results here although we don t overinterprete due to the problems. 3 Conclusion In this paper we have a series of benchmarks about performance comparison with the L4Linux on top of Fiasco and Linux. First,we present the first-hand performance benchamrk data about the L4Linux based on fiasco-1.2 and the Linux Second,from the benchmark of bonnie++ and NetPIPE we could find the performances of micro-kernel and monolithic-kernel are very very close. I think the performance loss of micro-kernel is not the reason of Linus vs Tanebaum any more. 5

6 4 Acknowledgements We d like to thank Adam Lackozynski of Dresden University of Technology very much. We asked him many problems when building the benchmark envionment and he was very patient and quick to give me the answer. Thanks to the Martin Pohlack of Dresden University of Technology very much. He helped me to fix a bug in my former paper. Thanks to the support of Cold and Arid Regions Environment and Engineering Research Institution, Chinese Academy of Sciences. Thanks to all other students in DSLab of Lanzhou University. They help us to review this paper and make the latex format. References [1] Andrew S. Tanenbaum, Jorrit N. Herder, Herbert Bos, 2006, Can we Make Operating Systems Reliable and Secure?, Computer, vol 39, no.5 pp [2] Herman Hartig, Michael Hohmuth, Jochen Liedtke, Jean Wolter, 1997, The Performance of u-kernel-based Systems, 16th ACM Symposiumon Operating Systems principles [3] Larry McVoy, Carl Staelin, lmbench: Portable tools for Performance analysis [4] Qingguo Zhou, Wang Baojun, Nicholas McGuire, 2004, Case Study of Performance of Real-Time Linux On the X86 Architecture, The Sixth Real-Time Linux Workshop [5] Herman Hartig, Michael Hohmuth and Jean Wolter, 1998, Taming Linux, the proceedings of PART 98 [6] Michel R. Dagenais, Disks, Partitions, Volumes and RAID Performance with the Linux Operating System [7] Nicholas Mc Guire and Qingguo Zhou,Benchmarking - Cache issues, the real-time Linux workshop 2005 [8] Hermann Hartig, Michael Hohmuth, Noman Feske, Christian Helmuth, Adam Lackozynski, Frank Mehnert and Michael Peter, 2005, The Nizza Secure-System Architecture, the proceedings of CollaborationCom 2005, San- Jose,CA,USA 6