Performance Management of Java-based SIP Application Servers

Transcription

1 12th IFIP/IEEE 1M 2011: Mini Conference Performance Management of Java-based SIP Application Servers Mauro Femminella, Emanuele Maccherani, Gianluca Reali Department of Electronic and Information Engineering, University of Perugia, Perugia, Italy Abstract- Within the activities of the Java APIs for Integrated Networks (JAIN), the Java community offers a set of standard frameworks and open protocol APIs to create advanced telecommunications services. However, recent works have pointed out that Java-based implementations of the Session Initiation Protocol (SIP) stack, the fundamental signalling protocol in the convergent telephone-ip world, perform poorly if they are executed by large multi-core servers, typically used in data centres. The problem lies in the combination of the SIP protocol semantics with the Java language features, which does not allow fully exploiting the computing capabilities of multi-core architectures. To face this problem, we propose a solution to improve the throughput and signalling latency of applications implemented by using Java-based SIP application servers. It consists of the joint usage of virtualization and parallelization techniques. We have performed an extensive measurement campaign by using an open source application server compliant with the JSLEE (JAIN Service Logic Execution Environment) specifications. The rationale of this choice is that JSLEE application servers are currently regarded as very promising candidates for deploying telecom services. Results show that it is possible improving performance in terms of throughput and signalling latency by running more instances of the JSLEE server in parallel, each of them into separate virtual machines deployed on the same server. This improvement can increase throughput values of about 64% and, in the maximum throughput condition, the call set up latency can be nearly halved. Keywords- Java; SIP; JSLEE; virtualization; parallelization I. INTRODUCTION Advanced telecom services are intrinsically asynchronous and should fulfill low latency, high throughput and high availability requirements. They may rely on different network protocols and interoperate with different software and hardware platforms. In terms of modularity and reusability, the development process can benefit from the usage of eventoriented architectures, new levels of abstraction, and well defmed logical components. In addition, the introduction of standard open interfaces, rather than proprietary ones, brings the great advantage of enabling services to support multiple networks and devices. In addition, it allows involving a higher nwnber of developers to speed up the service development. The Java technology is a natural candidate for implementing telecom services. Thanks to its intrinsic features, such as platform and operating system (OS) independence, networking support, dynamic adaptability, and availability of standardized technologies and APIs, Java simplifies the service implementation. The Java Enterprise Edition (EE) is the industry standard for implementing enterprise-class services. In addition, two further Java application frameworks, namely Java APIs for Integrated Networks (JAIN) Server Logic Execution Environment (SLEE) and SIP Servlets, allow creating, deploying and managing advanced telecom services [1]. However, a recent paper [2] shows that Java implementations of Session Initiation Protocol (SIP [10]) stack, which is the de facto standard signalling protocol for voice over IP (VoIP) and multimedia services, have severe scaling problems in large multi-core architectures. Since they currently represent the most widespread solution for implementing servers in data centres, this performance issue needs to be solved urgently. The problems come from the interactions between SIP protocol semantics with Java language features, which causes unbalanced pipeline stages and lock contention. The same performance problem has already been illustrated in [3], where we have observed that the average CPU utilization never exceeds 50% when a server equipped with 8 CPU cores is used. In addition, in [2] authors show that no effective solutions to solve this issue through some best practice configurations exist. In fact, although some improvements can be obtained with extensive profiling and optimization, this procedure nullifies one of the most important features of the Java language, i.e. easy and rapid service development. On the other hand, obviously, applications must exploit as much as possible the hardware capabilities, and it is clear that if more than half of the computing capabilities are unused in the average, there are spare CPU resources to manage a larger workload. The contribution of this paper is the proposal and analysis of an original usage of virtualization and parallelization techniques in order to better exploit the computing capabilities of servers hosting Java implementations of SIP application servers (ASs). Clearly, our goal is not to achieve loo% of the CPU utilization, but to increase the overall server throughput in terms of successful supported SIP calls, while maintaining the setup latency below an acceptable threshold. Our results are someway counterintuitive. Through an extensive experimental campaign we have shown that a very convenient solution consists of introducing a hypervisor and use virtual machines (VMs) to host SIP ASs. In more detail, our proposal is to run multiple, identical VMs on the same physical server, each of them running a single instance of the Java-based AS. Despite the increased overhead due to virtualization, the peculiarity of the Java over SIP operation allows increasing the overall SIP $ IEEE 493

2 call throughput with such a virtualized setting. We use of an open source JSLEE AS, Mobicents [4], running a SIP-based V oip service performing several database queries during call lifetime. Our choice of using a quite complex service test is motivated by our convincement that the frequent approach of showing improvements with a very simple service, such as the answering service used in [2], is not enough to assess improvements in real service scenarios. Instead, using a JSLEE AS with a realistic VoIP service is reasonable, since JSLEE ASs are primary candidates for the deployment of application services in the new convergent telecom paradigms [6]. Thus, our tests represent a realistic benchmark to verify if the proposed solution can be effective in operation. The paper is organized as follows. In Section II, we present an overview of the JSLEE specifications, focusing on the critical performance aspects. Section III describes the proposed AS deployment configurations and the VoIP service implemented to test them. Section IV presents the numerical results of the experimental campaign. Section V illustrates related works and, finally, Section VI draws our conclusions. II. BACKGROUND A. JSLEE specifications and available platforms The JSLEE activity aims to specify a Java-based, eventoriented container for the execution of carrier-grade telecom services [7]. The service logic is implemented in software components called Service Building Blocks (SBBs). A JSLEE AS creates a pool of SBB objects and manages them according to a well defined lifecycle. SBBs operate asynchronously by receiving, processing, and triggering events. They can be attached to data streams called Activity Contexts, by which they receive events from other entities. Also, SBBs may be linked together by parent-child relationships to implement the service logic in a modular fashion. Events are internally managed by a functional element called Event Router, which delivers each event to the appropriate SBB. External network events, such as SIP messages, are translated into internal Java events by the socalled Resource Adaptors (RAs). More generally, the set of implemented RAs constitutes an abstract interface layer that allows a JSLEE server to access external resources. The Mobicents Communication Platform is an open source project, currently owned by Red Hat [4]. It includes a JSLEE, a Media Server, a Presence Server, and a SIP Servlet Server. Other commercial JSLEE implementations are OpenCloud Rhino [l3], and Amdocs jnetx Convergent Service Platform [14]. In our experiments we have used the v GA of Mobicents JSLEE (MSLEE). It comes with a SIP RA which is already compliant with the new JSLEE V.U specifications [7]. The MSLEE includes several J2EE components, such as Container Managed Persistence (CMP) fields, which enable data persistence for SBB objects, Java Database Connector (JDBC) drivers, Java Management Extensions for the environment management and monitoring, and Java Naming and Directory Interface (JNDI), which offers lookup functions for service registration. The MSLEE is installed within the JBoss AS [5], which is a hosting environment offering special capabilities, such as service and JSLEE configuration management, deployment, and thread pooling. B. Critical issues in system configuration The Mobicents structure is characterized by an evident complexity, as it integrates the Java Virtual Machine (JVM), the JBoss AS, and the MSLEE. In order to achieve an efficient platform setup, some critical aspects must be considered. Below we briefly illustrate those we have faced in this work. 1) Java memory management Being based on the Java technology, the MSLEE relies on the automatic Java Garbage Collector (GC) mechanism for memory cleaning [9]. The drawback is that the developer has not the full control over the GC behavior, which may even pause and delay the application being executed [1]. Such pauses may be critical for real-time telecom applications. In fact, the AS may freeze due to post-pause avalanche restarts (since during a pause it can accumulate many unprocessed messages [8]). The JVM v.6 includes different GCs, in order to meet the requirements of different applications [1]. We have selected the Parallel GC (see also [3]), which is the default and most efficient GC (i.e., it uses the lowest CPU time), even if it produces longer pauses in program execution. In particular, the test results described in [3] shows that the Parallel GC exhibits better performance when used with the UDP transport protocol. Another issue to be considered when Java is used for real time services is the amount of memory to be allocated to the Java heap. As suggested in [8], this is another tricky point, since having a large memory allocated to Java may reduce the frequency of GC phases, but each collection may last a much longer time due to the larger amount of heap to be cleaned. 2) Operating System type: 32 vs. 64 bit Typically, when the hardware resources are suited to host a 64 bit operating system (OS), it is preferable over a 32 bit one. In fact, 64 bit OSs allow overcoming the well known limitation of 32 bits OSs that each application may use up to 3GB of RAM. Furthermore, it may also increase the CPU efficiency. Nevertheless, 64 bit systems may be still not mature enough for all carrier-grade applications, whilst 32 bit systems are generally more stable. We have evaluated many Linux distributions, both 32 and 64 bit versions. The 32 vs. 64 bit choice may impact on performance, but it does not affect service implementation, since using either a 32 or a 64 bit OS basically means using a different kernel and a different JVM, whilst the Java code of JSLEE and application services is unchanged. 3) Database Transactions In critical data transmission environments, transactions are typically used to preserve a computer framework (e.g. a database) in a known, consistent state, after system failures. As regards transactions, the MSLEE relies on the JBoss AS which natively supports Java Transaction API (JTA), thus allowing the usage of any transaction manager implementation. JBoss is by default configured to use the so-called "JTA compatible in VM" transaction manager. In some services, such as banking transactions, a provider may need a transactional behavior to ensure that all operations made on a remote database remain consistent during all system operation, as well as after a system 494

3 failure. Such a guarantee comes with a large performance overhead, in the form of disk writing operations, which are directly related to the number of service transactions processed. Hence, it is of great importance for a provider to carefully evaluate whether to use transactions or not, in order to ensure maximum performance. In fact, for some services such as VoIP ones, transactions may not be needed, whereas performance requirements are typically very stringent. This is the case of call forwarding, redirect and control services. For these reasons, in our experiments JTA has been disabled. III. PROPOSED SOLUTION AND TEST SERVICE DESCRIPTION A. Application server deployment configurations In this section, we present the three deployment solutions that we have evaluated by our experiments. Native configuration (Fig. La): this is the classic AS deployment configuration, in which a single instance of the server is installed within a single JVM, and the OS runs directly on top of the server hardware. Parallel configuration (Fig. I.b): this configuration is often used in Java-based AS deployments to improve scalability with respect to memory. In fact, it is well known that, even if 64-bit JVMs allows exploiting the availability of several GBs of RAM, an excessive Java heap allocation may cause performance degradation in real time systems due to large GC times, and usually ASs badly scale with the Java heap [20]. This solution consists of running multiple JVMs over a common OS installed directly onto the server hardware. Each JVM hosts an instance of JBoss, which includes an instance of MSLEE. Clearly, in order to correctly forward SIP calls to the appropriate MSLEE, it is necessary to introduce a SIP proxy acting as a call dispatcher. We have considered two versions of parallel configuration. The fust one, labeled "JVM", is the classical parallel deployment of n JVMs, each free to access all CPU cores without restrictions. In the second configuration, labeled "1VM-taskset", we have bound each JVM to a specific subset of CPU cores, thus allowing each core to be used by a single 1VM by using the system command "taskset". The rationale of this choice is that the Java-based SIP ASs do not scale well not only with the Java heap, but also with the number of CPU cores. Thus, this solution "emulates" separated Java computing environments within the same OS. Virtualized configuration (Fig. I.c): it consists of a hypervisor that virtualizes hardware resources and hosts some VMs. Within each VM, we replicate the native configuration (OS, JVM, JBoss, MSLEE AS). In this case, we really have completely insulated computing environments, and the MSLEE runs separately in each of them. In this case, we do not bind the virtual CPU s of each VM to real CPU cores, leaving the task of scheduling the VM access to computing resources to the hypervisor. This guarantees more degrees of freedom with respect to the previous solution. This configuration is labeled "VM". Looking at Fig. I, some considerations can be made. Without any particular performance indication, the most efficient configuration should be the first one, since it has the minimum overhead. In fact, the parallel configuration suffers from the presence of multiple JVM, JBoss, and MSLEE instances, which clearly imply additional CPU and memory consumption. This aspect is exacerbated in the third configuration, the virtualized one, which also includes the overhead of the hypervisor and multiple OS instances. However, since in the case of Java-based SIP stacks the native configuration does not exploit all the available computing resources, we expect that the best performance is achieved by the second one, and in particular the "JVM-taskset" one. It has less overhead than the virtualized one. In addition, by binding each JVM to a specific set of CPU cores, it emulates a physical server with few cores, thus mitigating the mentioned scalability problem. We expect that the improvement of the "JVM" configuration is inferior, since allowing all JVMs to access all CPU cores may cause resource contention and locking events. However, by considering both pros and cons, we are convinced that the preferred configuration is the third one, since operating a completely virtualized environment, including even the OS, can provide a degree of flexibility in server usage and services deployment [19] that by far compensates the small performance degradation expected [27]. In fact, a virtualized environment can allow server consolidation procedures for a more efficient hardware and energy usage, service migration through VM mobility, and so on. To the best of the authors' knowledge, there are no other proposals of using virtualization technologies (in particular bare metal hypervisors) in the literature to improve performance of a single AS through the deployment of multiple AS instances in different VMs on the same physical server. Instead, a common practice is a server consolidation through virtualization, which allows avoiding to run different and underutilized ASs in separated physical servers. Clearly, the benefit of using the parallel or the virtualized solutions has to be enough to balance at least the cost of an additional entity, the SIP proxy. Nevertheless, it can be easily implemented even by unpretentious hardware using a SER-based implementation [17]. It is worth noting that a SIP proxy acting as a load balancer is needed also with native configurations in large architectures. We show the performance of all considered architectures in section IV, highlighting the improvements of "1VM", "1VM-taskset", and "VM" over native configuration. B. Test service implementation The Mobicents community has published a set of performance achievements [8], obtained through an automatic answering service in which the MSLEE acts like a simple User Agent Server (UAS). It is one of the examples included in the MSLEE package. It utilizes only one SBB which responds to the incoming call, completes the SIP three-way handshake, and, after a short timer expiration, sends a BYE request to the caller User Agent Client (UAC) that has initiated the call. 495

4 [ Mobieents I JBossA5 Mobicents JBoss AS 5unJVM SunJVM # Obi"" JBoss AS L a - N_ t_ iv_e_0_5 1 LI N a t_ iv _e_0_5 Mobicents JBoss AS 5unJVM SunJVM #n VM 05#1 Hypervisor (a) (b) (e) I rl Mob;"",, JBoss AS 5unJVM VM 05#n Figure 1. Application servers deployments configurations. We have implemented a more complex SIP-based VoIP service to suitably characterize the MSLEE performance in a realistic telecom scenario. It could model an online charging system for pre-paid VoIP calls, which periodically updates the user credit in the subscriber profile by accessing a remote database (see also [16]). However, given the widespread usage of the SIP protocol beyond IP telephony, it could be also the skeleton of non VoIP services. The MSLEE manages the entire signalling by implementing a call control service through a SIP Back-to-Back User Agent (B2BUA) architecture [10], which easily allows the introduction of a third party call control mechanism [IS]. The MSLEE acts both as UAS and UAC, by splitting each call in two SIP dialogs over two distinct call-legs (caller-mslee and MSLEE-callee, see Fig. 2). This service implements two timers. One of them (Call Duration Timer - CDT) is used to control the maximum call duration, while the other one (Periodic Database Query Timer - PDQT) triggers periodic updates to the subscriber profile database. As regards the internal MSLEE service operation, when the initial INVITE is received by the MSLEE, its event routing subsystem is invoked and creates a Selector root SBB. This component queries the database to retrieve the subscriber profile, which includes the values of CDT and PQDT timers. Then, upon receiving the answer, it activates a child SBB called Call Control, and leaves the signalling control to it. In turn, this child SBB creates the second call leg towards the callee and establishes the media session between the two end points, starting both the CDT and the PQDT timers and querying the database upon each PQDT timeout. The call ends when the CDT expires and the CallControl SBB sends a BYE message on both call legs. The message exchange is illustrated in Fig. 2. Alternatively, each of the two end points can send a BYE to the other to close the call before CDT expiration. The service performs an additional, [mal database query on call termination, in order to update the user profile in the database. In the implementation used for this experimental campaign, the call is always closed by the MSLEE upon CDT expiration. IV. NUMERICAL RESULTS In what follows we first present the test bed used in the measurement campaign, then the results achieved by executing the MSLEE in all configurations illustrated in the previous lila section. Finally, we compare and discuss the results obtained with the three deployment schemes. CalierUAC I 100 INVITE -Trvinll OpenSips INVITE 100 -Trvine I. lbo-ringing 180'R;n,;" 200-0K ACK BYE 200 OK Figure 2. A. Test bed description 200-0K ACK BYE 200-0K SlEE UAS UAC -,,,, INVITE 100 Tcy;ng 180 -Rin ing 200-0K ACK BYE "'''. '1 i atabaseque request atat:.a5eque jspoose 200- OK "I l Signalling flow for the test service. Database The Mobicents JSLEE v GA, deployed on the JBoss v GA, has been installed on a Fujitsu-Siemens server PRIMERGY TX300 S4 with dual Intel Xeon 2.33 GHz (8 CPU cores) and 16 GB RAM. The 64 bit OS is the Novell Suse Linux Enterprise server x64 v.lo.l. For executing 32 bit experiments, we have used OpenS use Linux In the virtualized setting, we used the ESXi 4.1 hypervisor [12]. The JVM is the v.1.6.x. In the MSLEE configuration, we have used the JAIN SIP RA v.1.2, tuning the number of threads to allow a greater number of events to be processed simultaneously. All logs have been disabled during tests execution to improve performance. In addition, we have used the following tools: SIPp traffic generator [11], installed on two PCs with Ubuntu Linux v.9.l0, acts as UAC and UAS endpoints. MySQL database v.s.o.sla and MySQL Connector/J v.s.1.6 JDBC for database access, deployed on a PC with Arch Linux x64 (kernel ). Each database interrogation relies on an object-relational mapping by using the Hibernate technology, provided by JBoss AS. 496

5 The call dispatcher is an OpenSER [17] proxy running on a PC with Arch Linux x64 (kernel ). It forks SIP traffic among different MSLEEs according to a weighted round robin policy in the virtualized/parallel settings. The weight is proportional to the number of CPU cores allocated to each JVMiVM. As for the traffic statistics, the SIPp UAC has been set to generate new SIP calls with a constant rate, referred to as A, lasting 60 minutes for each A value. The value for the CDT timer (i.e. maximum call duration) has been set equal to 3 minutes. The value for the PQDT timer has been set equal to 10 seconds. This means that 18 queries are issued for each call, plus two additional queries at the call setup (INVITE received) and upon CDT timer expiration (call tear down). We have considered the following performance metrics: Maximum call throughput, defmed as the rate of successfully established SIP calls in calls per seconds (cps), with at least 95% of successfully handled calls. Session Request Delay (SRD [18], see Fig. 2). SRD is measured at UAC and defmed as the time interval from the initial INVITE to the first non-loo provisional response. It represents the latency experienced by the UAC for setting up a call. Since our SIPp UAS is configured with no pause between the 180 RINGING and the 200 OK messages, we have triggered the measuring timer at the reception of the 200 OK. B. Performance of Native corifiguration The initial set of experiments has been done to find the best configuration for the MSLEE when it runs in an OS natively installed in the physical server. We recall that this configuration uses UDP as transport protocol and the Parallel GC, and JTA is disabled. Fig. 3 shows the maximum call throughput as a function of the amount of RAM allocated to the JVM for a 64 bit and 32 bit OSs. In the 64 bit case, the Java heap ranges from 2 GB to 15 GB, whereas for the 32 bit OS the maximum value is 2.5 GB. First let us consider the maximum throughput achieved by using the 64 bit OS. When the amount of memory allocated to the JVM increases from 2 to 7 GB, the relevant throughput increases. The major increase is observed when the memory allocated ranges from 4 to 6 GB, and it is almost negligible from 6 to 7 GB. Increasing the memory allocation beyond 7 GB does not produce any throughput increase, but rather a slight decrease, especially for values larger than 8 GB. This is in line with what the Mobicents team has stated in [8], that is any increase of the memory allocation causes the GC to be executed less frequently, but increases the garbage time. In fact, polishing a larger memory increases the relevant service pauses, which may cause an avalanche restart and a consequent server overload. A different consideration is needed for the 32 bit OS. It is interesting to note that, even if the amount of memory allocated to the JVM is only 2 or 2.5 GB (the maximum allowed with a 32 bit JVM), its performance is comparable with a 64 bit system with a double amount of memory allocated to the JVM. This result is not surprising, since 32 bit systems are much more stable and can benefit from a long history of optimization. However, increasing the Java heap the improvement with a 64 bit OS is not negligible (about 10%), although it requires about a triple amount of memory than a 32 bit system using the same cpu. The conclusion is that, for a 64 bit OS, the optimal amount of memory allocated to the Java heap is a number of GBs equal or slightly lower than the number of CPU cores, which in our case are 8. In addition, since 32 bit OS are much less greedy of memory, they are better candidates for parallel or virtualized deployments !: OJ ::J E 'x 20 Figure Java heap size (G6) Maximum throughput vs. amount of memory allocated to JVM. Fig. 4 shows the average call setup signaling latency (i.e. SRD) as a function of the offered load. It can be seen that SRD increases almost linearly with the offered load up to the maximum throughput. Beyond this value, the server enters the overload status. In this condition (not shown to improve figure neatness), the latency needed to establish few calls exhibits a steep increase, since the Java heap needs to be continuously polished and the MSLEE has to process the queued messages, thus causing timeouts and retransmissions, which further increase the server load and slow down the overall call processing. 0.6 rr==== ' '11 2 GB -32 bit -B- 2.5 GB -32 bit GB -64 bit ---A--- 5 GB -64 bit --B GB -64 bit 0.4. f:> 7 GB -64 bit. + 8GB-64bit "' GB -64 bit Ql 0.3 '-----.,-----;;..--;Y o II: (/) OL- -L L- -L L- -L Figure Offered load (cps) SRD vs. offered load for different sizes of JVM. The second comment is that, up to 50 cps, one of the best performing configurations is the 32 bit OS with 2.5 GB allocated to the Java heap. For a workload of 55 cps, which is maximum call throughput for this configuration, the SRD has a sharp improvement, which doubles the value achieved at 50 cps. Finally, a further significant comment is that, in 64 bit configurations, the average SRD improves with the amount of memory allocated to the Java heap. This is quite evident for 4,

6 5 and 6 GB configurations. Perfonnance of 64 bit OS with 7 and 8 GBs is almost equivalent, and 7 GB configuration slightly outperfonns the 8 GB one only at 60 cps (at this value the 8 GB setup reaches its maximum throughput). Any further increase of the Java heap does not cause significant performance changes, altough this performance is slightly better than in all the other 64 bit configurations. Also this behavior can be explained by resorting to the GC operation: the larger the amount of memory allocated to the JVM, the fewer the pauses due to the GC collections, the longer their duration. Thus, the backlogged SIP messages may produce an avalanche restart causing a server collapse. This is the reason why the server does not achieve the best performance in tenns of call throughput with 15 GB memory allocated to the JVM. Finally, we have verified that, even for a complex AS like Mobicents, it is still valid the proposition of paper [2]. To this end, we have set an increasing number of CPU cores to be used by the JVM in native configuration through the system command "taskset". The results achieved are shown in Fig. 5, where the maximum call throughput is plotted versus the number of used CPU cores, for both 32 and 64 bit OSs. We first analyze the 32 bit OS behavior. As expected, the maximum call throughput scales with CPU cores only up to 2 cores, then it slightly increases for 3 and 4 cores, and remains stable up to 8 cores. As for the 64 bit version, it exhibits a trend which is nearly linear up to 3 cores, then the slope decreases. However, passing from 1 to 2 cores the relevant throughput does not double, and the same happens with 3 cores. In conclusion, 32 bit OS results the best candidate for parallelization/virtualization purposes, since, even with much less memory allocated to Java heap, with the same number of CPU cores (up to 4), it outperforms 64 bit versions. By using the MSLEE on a 64 bit OS, performance still improves with a number of CPU cores beyond 4, but very slowly. Thus it exhibits poor scalability properties along with a high memory req uirement. Fig. 6 shows the maximum call throughput values achieved by the three schemes as a function of the number of MSLEE instances. The first comment is that the intuition at the basis of this paper is correct: since Java-based SIP ASs badly scale with CPU cores, the best option to improve hardware resources exploitation is to use several ASs in parallel. Basically, all schemes outperfonn the native configuration, in both 32 bit (55 cps) and 64 bit (61 cps) versions. Up to 3 MSLEE instances, the configuration that shows better results is the "VM" ones, even if the difference with "JVM-taskset" is really small. We ascribe this result to the better insulation capabilities of the "VM" approach. From 2 up to 8 MSLEE instances, the "JVM" approach has very little improvement, since it passes from 80 cps (2 MSLEE instances) to 90 cps (4 and 8 MSLEE instances). Beyond 3 MSLEE instances, the best configuration is definitely the "JVM-taskset". This result is reasonable, since it causes less overhead than the "VM" one, and better CPU resource insulation with respect to "JVM". In particular, any increase of the number of VMs beyond 3 seems to cause an excessive overhead thus, differently from "JVM" and "JVMtaskset", performance starts decreasing. The best result in term of maximum throughput is reached by deploying 8 MSLEE instances, each using a single CPU core with 1.8 GB of memory allocated to the Java heap (all other configurations has 2.5 GB allocated to each JVM). In this condition, the maximum throughput reaches 125 cps. # of MSLEE instances 5 7 Figure 6. Maximum call throughput vs. number of MSLEE instances. C. Figure Number of CPU cores 7 8 Call throughput vs. number of CPU core used by MSLEE. Performance of ParallelellVirtualized configurations In this sub-section, we analyze the results relevant to "JVM", "JVM-taskset", and "VM" configurations. On the other hand, considering only the maximum call throughput as performance metric is not suitable, since also the SRD is important. In fact, the delay in establishing a session cannot be arbitrarily high, especially if during an ongoing call it is necessary to redirect the call towards another participant (e.g. a media server or a person). Fig. 7 shows the average and 95th percentile of the SRD as a function of the number of MSLEE instances, evaluated in the maximum throughput condition. We recall that the 64 bit version (native configuration) in the maximum throughput condition (61 cps) achieves an average SRD equal to about 500 ms and a 95th percentile of the SRD equal to about 2 s. It is evident that most configurations exhibit setup delays that cannot be acceptable in real settings. To this end, the reader should bear in mind that the measured delay is relevant only to the component of SRD 498

7 accumulated inside the data center where ASs run, since the UAC and UAS used in our tests represent inbound and outbound SIP proxies or PSTN gateways in the data center. Thus, in order to obtain the actual SRD values perceived by users, we have to add the delay contributions "caller-as" and "AS-callee", which could be two segments of wide area networks and thus with significant delays. To this end, Fig. 8 shows the maximum throughput for the three considered approaches as a function of the number of MSLEE instances, with the constraint of limiting the 95th percentile of SRD to 200 and 500 ms. Results are very interesting. First of all, we observe that striving for maximum throughput may cause unacceptable SRD values. In fact, the configuration showing the best performance in Fig. 6 (8 MSLEE instances in "JVM-taskset" setting) is now the worst. This happens since the use of a single core for executing GC causes excessive delays, and it is not suitable for real time systems. Instead, the best configuration consists of using 4 MSLEE instances in the "JVM-taskset" configuration (110 cps with 500 ms constraint and 105 cps with 200 ms constraint). This is convincing, since a 32 bit system with 2 CPU cores scales very well (see Fig. 7). A further interesting result is that the "VM" approach can support up to 95 cps for both constraints. This means that adding the reasonable constraint on SRD, the performance in terms of maximum call throughput of "JVM-taskset" and "VM" approaches gets closer. In fact, performance loss of the "VM" approach with respect to the "JVM-taskset" one is 13.5% if the 95th percentile of SRD is bounded to 500 ms, and 9.5% with a constraint of 200 ms, whereas without such a constraint the performance loss is 20%. In addition, the reader should bear in mind that running an AS inside a VM can provide a much better performing solution than a monolithic solution such as the "JVM-taskset" one. To this regard, we have verified that it is possible to perform live migration of VMs [21] without server crash up to the maximum throughput of 95 cps, by using the vmotion function in the vsphere package [12]. Further details are omitted for space limitations. Thus, considering this enormous potential in management and flexibility, our conclusion is that the preferable deployment is "VM" with 3 MSLEE instances. V. RELATED WORK A. JVM performance Papers [24][25] analyze the JVM scalability on multi-core systems. They analyze the performance of benchmarks using various numbers of processor cores and application threads. They correlate low-level hardware performance to the number of JVM threads and system components, in order to observe potential bottlenecks. Lock contentions and memory stalls cycles, produced by insufficient L2 cache and cache-to-cache transfers, are the main observed bottlenecks. JVM includes a parameter called thread-local allocation buffer (TLAB) that limits these issues [24]. A further performance optimization [24] is achievable by using the Parallel GC configuration and a suitable ratio of new and old generation memory heap to reduce the overhead of minor memory collection. It can be set by using the NewRatio flag. The TLAB and Parallel GC options have been already used in our experiments, whereas the NewRatio will be included in future works. However, from first tests, it seems that in our case its use improves mainly SRD and does not have a large impact on maximum throughput. B. OpenSER SIP Server Performance OpenSER [17] is a modular SIP proxy server, call SIP router, and SIP registrar server written in C language. It is widely adopted in VoIP environments and focuses on proxying and routing performance. For this reason, it has fewer capabilities than other capability oriented packages, such as a JSLEE AS. Therefore, due to its compact and lightweight software architecture, OpenSER is faster than a JSLEE AS, but it cannot be used as an execution environment for hosting complex SIP services. Other papers [22][23] have investigated the OpenSER scalability problems on multi-core systems, proposing solution to improve user registration, authentication, and localization. Even if also OpenSER can benefit from our proposed solution, the reverse is not possible, since the JSLEE internal architecture is not proxy-centric. C. Web Server Performance Performance scalability problem in multi-core environment is relevant also for other type of ASs, e.g. web servers. "'.s 10 3 <D <D e <D a..<:.., co '" g 10 2 :.. en # of MSLEE instances 10 ' ' ' # of MSLEE instances Figure 7. Average and 95th percentile values for SRD vs. number of MSLEE instances. 499

8 110 r----,-----r----r_--_,----_r----, configuration (64 bit, 7 GB of memory allocated to Java heap), with the flexibility of a virtualized environment. "' Co :5 Co.<:: g> e ; 5 E x L L----L L # of MSLEE instances Figure 8. Maximum call throughput vs. number of MSLEE instances limiting the 95'h percentile value for SRD to 200 and 500 ms. Veal at al. [26] have analyzed this issue by using Linux, Apache, SPECWeb2005, and a server with two quad-core processors. They have identified memory stalls as the primary cause of poor scaling. They propose a TCP flow level parallelism to minimize cache and Translation Lookaside Buffer (TLB) shortage. The nearly equivalent solution in Java consists of using the TLAB option, which has been used in our experiments. VI. CONCLUSION In this paper, we have presented a detailed experimental performance evaluation of a Java-based SIP AS, the open source JSLEE Mobicents platform. We have considered as test service a typical VoIP service, with a back-to-back user agent executing a call control function with multiple database queries. The considered module can be regarded either as a simple VoIP standalone service or as a module for more complex services needing a third party call control function, e.g. it can be used to switch the call among multiple parties. The main result achieved is that the standard deployments of Java-based ASs ("native" or also "JVM") do not provide the best performance, which is obtained when the server runs in multiple systems in parallel, each one with few CPU cores. In particular, considering not only the performance in terms of maximum call throughput and set up latency, but also ease of management and flexibility, the best evaluated solution consists of running a single AS instance inside VMs, and deploying VMs with 32 bit OS and 2/3 CPU cores. Clearly, these numbers are valid for our testbed, which consists of a server with 8 CPU cores and 16 GB of RAM, but provides useful information also for other settings, recalling that using more that 3 CPU cores for VM is not convenient (see also Fig. 5). Using this shrewdness, we have been able to evaluate a performance improvement of about 64% in terms of signalling call throughput, with an average setup latency halved with respect to native OS deployment with optimal REFERENCES [I] Bruno Van Den Bossche et ai., "J2EE-based middleware for low latency service enabling platforms," IEEE Globecom 2006, San Francisco, USA, Nov.-Dec [2] Y. Zhao, K. Zheng, H. Wang, Z. Gao, 'Tale in the multi-core era: is Java still competitive to host SIP applications?," IEEE ICC 2009, Dresden, Germany, June [3] M. Femminella, R. Francescangeli, F. Giacinti, E. Maccherani, A Parisi, G. Reali, "Scalability and performance evaluation of a JAIN SLEEbased platform for VoIP services," ITC 21, Paris, France, Sept [4] Mobicents Web Site, [5] JBoss Web Site, [6] H. Khlifi, J.-C.Gregoire, "IMS application servers: roles, requirements, and implementation technologies," IEEE Internet Computing, May-June 2008, Volume: 12, Issue: 3, pp [7] JSR 240, JAIN SLEE vl.1 Web Site, [8] Mobicents Performance Guide, Web Site: [9] Java SE HotSpot Virtual Machine GC Tuning Web Site, [10] 1. Rosenberg et ai., "SIP: Session Initiation Protocol," JETF RFC 3261, June [II] SIPp Web Site: [12] VMware ESXi Web Site, [l3] Open Cloud Web Site, [14] jnetx Web Site, [15] J. Rosenberg, 1. Peterson, H. Schulzrinne, G. Camarillo, "Best current practices for third party call control (3pcc) in the Session Initiation Protocol (SIP)", JETF RFC 3725, April [16] 3GPP TS , "IP Multimedia Subsystem (IMS) charging (Release 8)," V8.5.0, [17] E. Nahum, 1. Tracey, C. Wright, "Evaluating SIP proxy server performance," NOSSDAV 2007, Urbana-Champaign, USA, June [18] D. Malas, "SIP end-to-end performance metrics," IETF Internet Draft, draft-ietf-pmol-sip-perf-metrics-ol.txt, June [19] ASingh, "An introduction to virtualization," Janaury 2004, [20] C. Renouf, "More WAS internals," IBM System Magazine, Oct. 2007, age=%2faix% 2Fenewsletterexclusive%2 F I 7888printp I.aspx&string_referer=/aixlene wsletterexc1usive/17888p4.aspx. [21] F. Checconi, T. Cucinotta, M. Stein, "Real-time issues in live migration of virtual machines," VHPC'09, Delft, The Netherlands, Aug [22] J. Zou et ai., "Scalability evaluation and optimization of multi-core SIP proxy server", TCPP [23] c.p. Wright et ai., "SIP server performance on multicore systems," IBM Journal of Research &Development, 54(1), 20 I O. [24] K.-Y. Chen, J. M. Chang, T.-W. Hou, "Multi-Threading in Java: Performance and Scalability on Multi-Core Systems," IEEE Trans. on Computers, 02 Dec. 2010, DOl: /TC [25] K. Ban et ai., "Java Application Server Optimization for Multi-core Systems", Intel Tech. Paper, [26] B.Veal, A Foong, "Performance scalability of a multi-core web server", ACMlIEEE ANCS [27] M. H. Jamal, A Qadeer, W. Mahmood, A Waheed, J.J. Ding, "Virtual Machine Scalability on Multi-Core Processors Based Servers for Cloud Computing Workloads", IEEE International Conference on Networking, Architecture, and Storage (NAS), 2009, Zhang Jia Jie, China. 500