Java Virtual Machine

Transcription

1 Evaluation of Java Thread Performance on Two Dierent Multithreaded Kernels Yan Gu B. S. Lee Wentong Cai School of Applied Science Nanyang Technological University Singapore Abstract Modern programming languages and operating systems encourage the use of threads to exploit concurrency and simplify program structure. An integral and important part of the Java language is its multithreading capability. Despite the portability of Java threads across almost all platforms, the performance of Java threads varies according to the multithreading support of the underlying operating system and the way Java Virtual Machine maps Java threads to the native system threads. In this paper, a well-known compute-intensive benchmark, the EP benchmark, was used to examine various performance issues involved in the execution of threads on two dierent multithreaded platforms: Windows NT and Solaris. Experiments were carried out to investigate thread creation and computation behavior under dierent system loads, and to explore execution features under certain extreme situations such as the concurrent execution of very large number of Java threads. Some of the experimental results obtained from threads were compared with a similar implementation using processes. These results show that the performance of Java threads diers depending on the various mechanisms used to map Java threads to native system threads, as well as on the scheduling policies for these native threads. Thus, this paper provides important insights into the behavior and performance of Java threads on these two platforms, and highlights the pitfalls that may be encountered when developing multithreaded Java programs. 1 Introduction A thread [1] is a single unit of execution within a process. In a multithreaded program there are multiple threads running concurrently within a single address space while each has its own sequence of instructions. Modern programming languages (e.g., Java), and operating systems (e.g., Windows NT and Solaris), encourage the use of threads to mask communication latency and overhead of some operations. Recently, the scientic computing community has started using threads to mask network communication latency in massively parallel architectures, allowing computation and communication to be overlapped. In general, it would be desirable if threaded programs could be written to expose the largest degree of parallelism possible, or 1

2 to simplify the program design. With multiprocessing systems available on several popular processor architectures, multithreading has become an important programming paradigm. Over the past few years, Java has emerged as an important programming language in the computing community. It has a number of features that are widely received: object orientation, portability and multithreading support. The portability of the language oers a means of reducing the cost of maintaining software support across dierent platforms. Portability does not mean that the application will behave in a similar manner when running across dierent platforms. This is especially true in the case of a multithreaded Java application. The main reason for the diversity of Java thread execution models is that Java threads are mapped to the native system threads in dierent manners and the native system supports threads at dierent levels. Thus, Java threads represent a platform specic abstraction on top of native threads and the variations of their performance on dierent multithreading kernels reect the internal thread support in those operating systems accordingly. This paper is organized as follows: Section 2 describes two main mapping models of Java threads on two multithreaded kernel operating systems. Experimental results comparing the performance of Java threads on those two platforms are presented in Section 3, and evaluated in Section 4. Section 5 concludes the paper with the discussion on the major ndings. 2 Characteristics of Java Threads on Multithreaded Kernels The Java thread package [2] makes threads an integral part of Java language and provides a rich collection of methods to start, run, stop and query the state of Java threads. Java thread is preemptive, which means a lower priority thread can be forced o a processor in favor of a higher priority thread. Java Virtual Machine (Java VM) makes the implementation details transparent to the Java users and achieves the portability of threads among almost all platforms. Despite the uniform thread interface in Java, variations in the implementations of Java thread on dierent platforms aect its behavior and performance. Some operating systems, including older releases of UNIX and Macintoshes, do not support threads. On those systems, the Java VM has to emulate threads. With the increased popularity of symmetric multiprocessing (SMP) machines, operating systems, such as Solaris and Windows NT, oer multithreaded kernels to provide highly concurrent, responsive operations. When running on those systems, the Java VM maps Java threads to native threads. Two main multithreaded kernels on which Java VM maps Java threads are discussed as follows: One-to-one mapping on the Windows NT [3, 4] system (i.e., Section 2.1) Many-to-one/Many-to-Many mapping on the Solaris [5, 6] system (i.e., Section 2.2) 2.1 Windows NT The multithreading kernel support on Windows NT is among the rst implementations of true multithreading. Creating a Win32 process automatically creates the process's main thread. Auxiliary threads can be created in the process afterwards. Each thread has its own stack, the size of which is specied at creation time. The Windows NT operating system does preemptive 2

3 Java Process Java Java Java thread thread thread Java Applications Java Virtual Machine Kernel Win32 Win32 Win32 thread thread thread scheduling entity: Win32 thread Figure 1: One-to-One Mapping Model on Windows NT multitasking on a per-thread (not per-process) basis and threads are the scheduling entities. Unlike the Solaris two-tier thread model (Section 2.2), Windows NT uses a straightforward One-to-One mapping between a user thread and a kernel thread. In this implementation, each user-level thread created by the application is known to the kernel and all threads can access the kernel. The main problem with this model is that it places a restriction on the developer to be careful and frugal with threads, as each additional thread adds more weight to the system. As a consequence, the thread package in Windows NT limits the number of threads supported in the system. The default thread scheduling scheme for the Win32 kernel threads is round-robin with dynamic priority adjustment. With the round-robin scheduling, threads of equal priority, if that priority is the highest among other threads, are time-sliced. The operating system assigns a running thread a CPU time-slice, on the order of 20 milliseconds. If a thread with this policy becomes ready, and it has a higher priority than the currently running thread, the kernel preempts the current thread and immediately begins running the higher-priority thread. The Java Virtual Machine on Windows NT maps Java threads directly to native Win32 operating system threads with a One-to-One mapping as shown in Figure 1. Since the scheduling entities are native threads, the CPU time is time sliced among all the threads that have been created. 2.2 SUN Solaris The multithreaded programming model in Solaris has two levels. The rst level is the user thread interface with which programmers write their multithreaded programs. The second level is the lightweight process (LWP). The Solaris LWPs are the schedulable entities. Each LWP is independently dispatched and scheduled by the kernel onto the available CPU. Each process contains one or more LWPs and each of which runs one or more user threads. The thread library schedules user threads onto the pool of LWPs without the involvement of the kernel. Although the two-level thread model adds some extra responsibility to the user, it allows for the cheap creation of user threads without using a lot of system resources. The kernel schedules all LWPs in round-robin manner. In contrast, the Solaris user threads within the same process use 3

4 Java Process Java Process Java Java Java thread thread thread Java Applications Java Java Java thread thread thread Java Applications Java Virtual Machine Java Virtual Machine User Space User Space green green green thread thread thread native native native thread thread thread User Thread Library User Thread Library Kernel Multiprocessor Kernel LWP scheduling entity: LWP LWP LWP scheduling entity: LWP (a) Many-to-One Mapping (b) Many-to-Many Mapping Figure 2: Thread Mapping Model on Solaris a default FIFO (First-In-First-Out) scheduling policy. The highest priority thread is scheduled by the user thread library to run on the LWP until it is blocked. If there is more than one user thread with the same priority and that priority is the highest among all the threads, the rst running thread will continue until it is blocked. Prior to Solaris 2.6, Java threads run as pure user-level threads called green threads. All of the user threads within the same process are bound to one kernel LWP in a Many-to-One manner, as shown in Figure 2(a). With the FIFO scheduling policy in Solaris, user threads with the same priority running in the same process are not time-sliced and require the applications to schedule them explicitly. When Java threads run on Solaris 2.6 with the support of the newer version of Java Virtual Machine such as JDK1.2 beta, they are mapped onto native unbound Solaris threads as shown in Figure 2(b). In the Many-to-Many model supported in Solaris 2.6, a user-level thread library provides sophisticated scheduling of user-level threads above kernel threads. The Many-to-Many mapping can avoid many of the limitations of the Manyto-One mapping model on Solaris green threads. It allows a program to have as many threads as appropriate without making the process too heavily loaded or burdensome. However, our initial testing of the native thread support using JDK 1.2 beta on a two-processor UltraSPARC Sun SMP and a one-processor Sun SPARC 20 shows that the Many-to-Many mapping is only supported on the multi-processor platform. On the one-processor system, only one LWP is allocated to support all the threads belonging to the same process. On the dual processor system used in our experiment, threads created within the same process are divided equally among the two processors, each having one LWP to support their execution. In the best case, there is a speedup of 2 on the two-processor machine. Therefore, on a single processor system, 4

5 similar performance will be observed for both types of Java thread mappings. 3 Experiments and Results An EP benchmark [7] was implemented using Java threads and processes to assess the performance of Java threads under dierent multithreading platforms. The EP program adopts the master-slave model. The master task distributes computation work to all the slave tasks before computation, and slave tasks send results back to the master task after computation. All tasks carry out the same amount of computation. Tasks were implemented using both threads and processes for comparison. Two parameters were measured: computation time and spawn time. In addition, experiments were also conducted to test the maximum number of threads that can be created for each operating system. The set-up of the experiment environment is as follows: a PC Pentium, 200MHz with 64 Mbytes of memory, running Windows NT 4.0 a SUN Sparc 20 single-processor workstation, 60 MHz with 32 Mbytes of memory, running Solaris 2.4. The Java Development Kit used is JDK1.1.5 on both PC and workstation without the just-intime compiler. 3.1 Spawn Time Spawn time refers to the duration from the time when the main thread calling the spawn function till the time when the spawned task starts executing. Figure 3 shows the average spawn time per task on both Windows NT and SUN Solaris. Logarithmic scale on the Y axis (i.e., the spawn time) is used to highlight the relatively small spawn time of Java threads compared with that of Java processes. As seen from the gure, the spawn time of threads is less than that of the processes by an order of tens to hundreds. The reason for this sharp variation is that all the threads share the process address space and each of them only needs a stack, a program counter and registers to start their execution. On the other hand, in the case of process, a new Java Virtual Machine needs to be loaded for the creation of a Java process. In addition a new address space and other system resources also need to be allocated. 3.2 Computation Time A number of experiments were conducted to compare the computational performance of threads and processes using EP benchmark. Comparison is made under two typical run-time environments: one with light background load and the other with heavy background load. On the single processor PC and workstation used in this experiment, the operating system time-slices scheduling units (either LWPs on Solaris or Win32 kernel threads on Windows NT) and schedules them to run on the CPU alternatively. Intuitively, the EP execution time will remain unchanged no matter how many tasks are used, since the EP problem size is xed in 5

6 Thread Process Spawn Time(ms) Number of Tasks (a) Average Spawn Time on Windows NT Thread Process Spawn Time(ms) Number of Tasks (b) Average Spawn Time on Solaris Figure 3: Average Spawn Time 6

7 our experiment. However, this is not the case observed in the experiments, as seen from the following test results. As a matter of fact, the thread management overhead together with the load exerted by other scheduling entities running on the same machine aects the performance of EP execution. With more tasks (either processes or threads) executing at the same time, the operating system overhead to manage them will increase accordingly. If there are other user applications executing on the same machine with the same priority, the EP tasks have to share CPU time with those user applications Performance under Light Load Thread Process Computation Time(sec.) Number of Tasks (a) EP Computation Time under Light Load on Windows NT Thread Process Computation Time(sec.) Number of Tasks (b) EP Computation Time under Light Load on Solaris Figure 4: EP Computation Time under Light Load Figure 4 presents the computation time for EP benchmark under light system load. In this experiment light load refers to the situation when there are no other active user applications 7

8 in the system. Figure 4(a) is the result obtained on Windows NT. The major point worth noting is that it does not make too much dierence on the performance whether Java threads or processes are used as EP computing units. This agrees with what can be deduced from the One-to-One thread-mapping model. However, at the threshold of 14 tasks, the computation time of threads starts to increase dramatically. This sudden increase of computation time is due to the applicability of threads that is limited by the space and time overhead. The time overhead is incurred in the form of saving and restoring a set of registers to and from memory at every context switch. Thread applications incur space overhead because the system allocates memory for thread stack segments. The stack segment is allocated by the Java Virtual Machine automatically and it must accommodate the maximum activation record requirements of each thread. For those applications using large number of threads or with tight memory constraints, thread creation will exhaust memory assigned to a process. As a result, the execution of the application will be slowed down due to TLB (Translation look-aside Buer) misses and excessive paging. In Windows NT, each process has a set of resource quota limits that restrict how much memory the threads of a process can use for opening handles to objects. The quota limits regulate how much memory, paging le space, and execution time the threads of the process collectively use. In this experiment, the sudden increase of execution time for threads is mainly caused by the space overhead. All the EP threads share the limited system memory of a process, whereas each EP process has its own memory allocated by the system. Figure 4(b) compares the EP computation time for threads and processes on Solaris. The value on the Y axis (i.e., the computation time) is in the range of 77? 82 seconds in order to highlight the slight dierence of the execution time between threads and processes. It can be observed from this gure that there is an increasing tendency in the computation time for both threads and processes as more tasks are involved in the EP computation. This is due to the increasing overhead required to manage more threads or processes. The other point that can be noticed is that processes always need a little more execution time than threads and the time dierence between them is enlarged gradually as the number of tasks (either threads or processes) increases. The explanation for this overhead is that each process is mapped to at least one LWP, while each Java thread is mapped to a user thread. The context switch of user threads, whose activity is restricted to user space, does not need the involvement of the kernel and is much faster than that of a process. In contrast, the context switch of processes needs to change the whole address space in the system kernel Performance under Heavy Load Figure 5 illustrates the results obtained from the execution of EP benchmark under heavy load. The load on the system is generated using ten Java processes, each of which does oatingpoint multiplication in an innite loop. Figure 5(a) shows that the performance of threads and processes on Windows NT is quite similar before reaching the threshold value. This is obvious because the scheduling entities in Windows NT are NT threads and the Java threads are mapped to the NT threads on a One-to-One basis. When more EP tasks (threads/processes) run in the system, a larger portion of the CPU time can be obtained by those EP tasks, thus resulting in the speed up as shown in the gure. The sudden increase at the threshold in Figure 5(a) can be explained by the same reason that was given for Figure 4(a). Figure 5(b) reveals the fact 8

9 Thread Process Computation Time(sec.) Number of Tasks (a) EP Computation Time under Heavy Load on Windows NT Thread Process Computation Time(sec.) Number of Tasks (b) EP Computation Time under Heavy Load on Solaris Figure 5: EP Computation Time under Heavy Load 9

10 that processes have a better performance than threads under heavy load in Solaris. There is no uctuation in the computation time for dierent numbers of threads since they are always mapped to one LWP and the scheduling entity for CPU is LWP. However, the computation time for processes decreases when more processes are used. The reason lies in the fact that each process is mapped to at least one LWP. When more processes are created for the EP benchmark, the system will allocate more LWPs for scheduling the processes. Hence, a larger portion of the CPU cycles will be allocated for EP computation. 3.3 Performance Using Very Large Number of Threads In order to understand the eects of thread mapping on Java threads, two more experiments were conducted to test the characteristics of Java threads under certain extreme situations. To nd out the maximum number of Java threads that can run in the system as well as to explore the Java thread performance under such kind of situations, a large number of threads are spawned in each system for computing the EP benchmark with the same problem size. Figure 6 presents the EP execution time using very large number of threads. After the number of threads used exceeds the system threshold value (i.e., 14 threads in our experiments), the execution time increases signicantly as more threads are used in the computation. The execution time using 200 threads is around 90 times of that using one thread. This result is due to the thread space and time overhead. That is, the process memory is not enough to satisfy the stack segment memory requirements for such a large number of threads. As a result, there are a lot of TLB misses and excessive paging during the execution. Further more, with a large number of threads, the context switch overhead also becomes signicant. All the NT threads are executed in timesliced manner with the round-robin scheduling policy. This results in frequent context switches among these threads. In contrast, Figure 6(b) shows that the execution time is unchanged when the number of threads used increases to 200 in Solaris system. In Solaris, the Many-to-One model maps Java threads to user level threads. All the user threads in the same process are not time-sliced, and the context switch only happens when a running thread nishes its execution and the user thread library schedules another thread to run. Hence, the context switch overhead among these Solaris user threads is almost negligible compared with the computation time. However, Figure 6(c) shows that the performance deteriorates when the number of threads used exceeds a certain threshold value (i.e., 800 threads in this experiment). The computation time using 4000 threads is around 8 times of that using one thread. This performance deterioration is mainly caused by the overhead to maintain the large number of threads in the system. With the large number of threads working on the same problem, each thread bears much less computation work. Although the context switching overhead is not the major problem in user level thread package, the overhead to initialize, start, and terminate each user thread begins to dominate the thread's execution time. The results obtained in these experiments suggest that the additional step of Java thread mapping in Solaris, that is, the mapping from Java thread to the user-level thread, allows more Java threads to be executed without aecting the performance too much. However, in spite of the fact that thread (especially user thread) is a lighter weight execution unit compared with process, the use of excessively large number of threads should be avoided whenever possible. 10

11 Thread Execution Time(sec.) Number of Threads (a) EP Execution Time on Windows NT Thread Execution Time(sec.) Number of Threads (b) EP Execution Time on Solaris Thread 350 Execution Time(sec.) Number of Threads (c) EP Execution Time on Solaris Figure 6: Execution Time with Large Number of Threads 11

12 4 Evaluation of Java Thread Mapping Model Java Virtual Machine makes the low-level system implementation details transparent to the users. As a consequence, Java threads are portable across dierent platforms. The mapping of Java threads to the underlying operating system is hidden from the users. The results from our experiment show that the portability of Java threads in terms of performance is not as good as expected. The performance of Java threads depends a great deal on its mapping to the underlying operating system execution unit (e.g., LWP or the kernel thread) as well as the support provided by the underlying operating system for thread execution. Windows NT has a One-to-One mapping of Java threads to the system kernel threads. The performance of threads is similar to that when processes are used. However, having too many threads in such a system has the disadvantage that the space and time overhead for executing threads may signicantly aect the performance. Thus, when using Java thread on Windows NT the users should bear in mind not to have too many threads running in the same process. In the case of Many-to-One mapping, as in Solaris, the performance of Java threads deteriorates quickly when the load in the system increases. This is because all the threads in a process are mapped to one LWP and LWP, instead of threads, are scheduling entities in the system. Although JDK1.1 or above on Solaris 2.6 claims to support the Many-to-Many mapping from Java threads to Solaris threads, our test shows that this only works for multiprocessor machines. On a single processor machine, although Java threads are mapped to Solaris threads, only one LWP is created to schedule all the threads in the same process. 5 Conclusion In this paper, we investigated performance issues involved in the use of threads on two multithreaded kernels, that is, Windows NT and Solaris, with the goal of gaining a better understanding of the impact of thread mapping mechanisms and scheduling policies on Java thread behaviors across dierent operating systems. A number of experiments were conducted to characterize the behavior of Java threads, and their results were compared with that of processes. Our measurements show that Java thread spawn time is much less than that of process by an order of hundreds. We also found that the computation behaviors of Java threads under light and heavy system loads are dierent on both platforms. Our analysis shows that the behavior is determined by the mapping of Java thread to the system native threads, be it at user or kernel level, as well as the scheduling of native threads by the kernels. Two other experiments were also conducted to evaluate the system support for very large number of threads. The lessons learned from these experiments suggest that the use of excessively large number of threads should be avoided whenever possible, in order not to waste valuable CPU cycles in thread management overheads. In summary, the portability of Java thread does not represent the portability of Java thread's behavior and performance across dierent platforms. Care must be taken when developing multithreaded Java applications. 12

13 References [1] Shashi. Prasad, Multithreading Programming Techniques, New York: McGraw-Hill, [2] Daniel J. Berg, Java Threads - A White Paper, Sun Microsystems Computer Corporation, March 1996, available at: [3] Jim Beveridge and Rovert Wiener, Multithreading Applications in Win32, Addison-Wesley Developers Press, 1997 [4] Helen Custer, Inside Windows NT, Microsoft Press, 1995 [5] Sun Microsystems, Inc, Java on Solaris 2.6, A White Paper, September 1997, available at: [6] Sun Microsystems, Inc, SunOS 5.0 Multithread Architecture, a white paper, SunSoft, 1991, available at: [7] Wentong Cai, Alfred Heng, and Peter Varman, Benchmarking IBM SP1 System for SPMD Programming, in the Proceedings of 1996 International Conference on Parallel and Distributed Systems (ICPADS'96), IEEE Computer society. 13