WebLogic Server- Tips & Tricks for Troubleshooting Performance Issues By: Abhay Kumar AST Corporation March 1st, 2016
Contents INTRODUCTION... 3 UNDERSTAND YOUR PERFORMANCE OBJECTIVES AND SET REALISTIC GOALS... 3 MEASURE YOUR PERFORMANCE METRICS... 3 TOP TUNING RECOMMENDATIONS... 4 TUNE POOL SIZE... 4 THREAD MANAGEMENT... 4 TUNING NETWORK I/O... 5 TUNING DATASOURCES... 5 TUNING TRANSACTIONS... 5 WEBLOGIC CLUSTERS... 6 TUNE PERSISTENT STORES... 6 USE BEST DESIGN PRACTICES... 6 CONCLUSION... 6 tips & ticks Page ii
Introduction There is no magical wand to tune the Weblogic server. It is a complex and iterative process Oracle WebLogic Server is the #1 application server across conventional and cloud environments. With Oracle WebLogic we can deliver next-generation applications on a mission-critical cloud platform, simplify operations with native cloud management, and accelerate time to market with a modern development platform and integrated tools. WebLogic Server is the industry's best application server for building and deploying enterprise Java EE applications with support for new features for lowering cost of operations, improving performance, enhancing scalability and supporting the Oracle Applications portfolio. Performance tuning WebLogic Server and your WebLogic Server application is a complex and iterative process. This paper will provide tips to get you started, and provide a short list of recommendations to help you optimize your application s performance. U N D E R S T A N D Y O U R P E R F O R M A N C E O B J E C T I V E S AN D S E T R E A L I S T I C G O A L S Understand your deployed application- Bad Design can never be a good candidate for performance tuning Find out the number of users using the system. Find out the number of requests on the system. The amount of data flowing in/out of the system. Determine the target CPU Utilization- Should not be 100%. M E A S U R E Y O U R PERFORMANCE M E T R I C S Take measurements of metrics which will be used to quantify the performance goals. Monitor Application Server (CPU and Disk) Monitor Database Server(CPU and Disk) Monitor data transfer across networks Locate bottlenecks and break them. Page 3
T O P T U N I N G RECO M M E N D A T I O N S Following is the list of recommendations for performance tuning. Tune Application- Overly-complex or poorlydesigned applications will perform poorly regardless of the system-level tuning and best practices employed to improve performance. Tune DB - Database can be a major enterpriselevel bottleneck. Database optimization can be complex and vender dependent. Tune WebLogic Server Performance Parameters- The WebLogic Server uses a number of OOTB (out-ofthe-box) performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system can greatly improve both single-node performance and the scalability characteristics of an application. Tune Your JVM - The Java virtual machine (JVM) is a virtual "execution engine" instance that executes the bytecodes in Java class files on a microprocessor. Tune the Operating System - Each operating system sets default tuning parameters differently. For Windows platforms, the default settings are usually sufficient. However, the UNIX and Linux operating systems usually need to be tuned appropriately. T U N E POOL SIZE Provide pool sizes (such as pools for JDBC connections, Stateless Session EJBs, and MDBs) that maximize concurrency for the expected thread utilization. In general, the number of connections should equal the number of threads that are expected to be required to process the requests handled by the pool. The most effective way to ensure the right pool size is to monitor it and make sure it does not shrink and grow. T H R E A D M A N A G E M E N T WebLogic Server provides the following mechanisms to manage threads to perform work. Tuning a Work Manager: Tune the thread utilization of a server instance by defining rules and constraints for your application by defining a Work Manger. Tuning the Stuck Thread Detection Behavior: WebLogic Server automatically detects when a thread in an execute queue Page 4
becomes "stuck." Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. WebLogic Server diagnoses a thread as stuck if it is continually working (not idle) for a set period of time. You can tune a server's thread detection behavior by changing the length of time before a thread is diagnosed as stuck, and by changing the frequency with which the server checks for stuck threads. T U N I N G NETWORK I/O Network channels are also known as network access points. These access points let us specify variant quality of service parameters for network communication. The major role of a channel is to control the network traffic for a server. But we can leverage the capacity to build multiple custom channels to acknowledge a multi-threaded client to interact with server instances over multiple connections. So this helps in reducing the potential for a bottleneck. T U N I N G DATASOURCES An easy way to boost performance of a data source in WebLogic Server applications is to set the value of Initial Capacity equal to the value for Maximum Capacity when configuring connection pools in your data source. Cache JNDI connections. Use Test Connections on Reserve with Care. T U N I N G TRANSACTIONS Typical two-phase transactions in JMS applications usually involve both a JMS server and a database server. The LLR option can as much as double performance compared to XA.The safety of the JDBC LLR option contrasts with wellknown but less-safe XA optimizations such as "last-agent", "last-participant", and "emulate-two-phase-commit" that are available from other vendors as well as WebLogic.JDBC LLR works by storing two-phase transaction records in a database table rather than in the transaction manager log (the TLOG). Page 5
W E B L O G I C CLUSTERS A weblogic Server cluster is a group of WebLogic Servers instances that together provide fail-over and replicated services to support scalable high-availability operations for clients within a domain. A cluster appears to its clients as a single server but is in fact a group of servers acting as one to provide increased scalability and reliability. T U N E PERSISTENT S T O R E S For subsystems that share the same server instance, share one store between multiple subsystems rather than using a store per subsystem. Sharing a store is more efficient for the following reasons: A single store batches concurrent requests into single I/Os which reduces overall disk usage. Transactions in which only one resource participates are lightweight one-phase transactions. Conversely, transactions in which multiple stores participate become heavier weight two-phase transactions. Add a new store only when the old store(s) no longer scale. Under heavy JDBC store I/O loads, you can improve performance by configuring a JDBC store to use multiple JDBC connections to concurrently process I/O operations. U S E BEST DESIGN PRACTICES Most performance gains or losses in a database application is not determined by the application language, but by how the application is designed. The number and location of clients, size and structure of DBMS tables and indexes, and the number and types of queries all affect application performance. Conclusion This white paper provides some input on the major performance tuning considerations for WebLogic Server. However keep following things in mind: Performance tuning is not a silver bullet. Simply put, good system performance depends on: good design, good implementation, defined performance objectives, and performance tuning. Performance tuning is ongoing process. Implement Page 6
mechanisms that provide performance metrics which you can compare against your performance objectives, allowing you to schedule a tuning phase before your system fails. The object is to meet your performance objectives, not eliminate all bottlenecks. Resources within a system are finite. By definition, at least one resource (CPU, memory, or I/O) will be a bottleneck in the system. Tuning allows you minimize the impact of bottlenecks on your performance objectives. Design your applications with performance in mind. o Keep things simple - avoid inappropriate use of published patterns. o Apply Java EE performance patterns. o Optimize your Java code. Page 7