Performance and Scalability Benchmark: Siebel CRM Release 8.1.1.4 Industry Applications and Oracle 11.2.0.3 Database on Oracle's SPARC T5 Servers and Oracle Solaris An Oracle White Paper Released April 2013
The current Siebel Platform Sizing and Performance Program (PSPP) benchmark is done using Siebel Release 8.1.1.4 and covers a number of areas of the Siebel product suite like Call Center and Siebel Order Management. It is significantly different from all previous Siebel PSPP benchmarks done against prior Siebel releases. A number of complex scenarios in high demand areas like Siebel Order Management have been added to the PSPP kit. Therefore, any comparison of this benchmark results with the previous ones is not meaningful and hence is not permitted. The Description of the benchmark scenarios and percentage of the load for each is documented in methodology section of this document. 2
Performance and Scalability Benchmark: Siebel CRM Release 8.1.1.4 Industry Applications INTRODUCTION This white paper describes the performance and scalability capabilities of Oracle s Siebel Customer Relationship Management (CRM) Applications Release 8.1.1.4. The benchmark comprised 40,000 concurrent users running Siebel CRM Release 8.1.1.4 industry applications and Oracle 11.2.0.3 Database on Oracle's SPARC T5 Servers. All servers ran Oracle Solaris. Oracle s Siebel Platform Sizing and Performance Program is a test suite certified by Siebel and executed independently by Oracle. Oracle completed the benchmark on 03/22/2013; Siebel certified it on 03/25/2013. Note that this benchmark data is intended for general information purposes and not as a substitute for implementation-specific sizing or benchmarks. Results Summary: 40,000-Active-Concurrent-User Benchmark 1 2 Workload Number of Users Average Operation Response Time (sec) Business Transactions Throughput/hour Projected Daily Transactions Financial Services Call Center 28,000 0.110 273,786 2,190,288 Order Management 12,000 0.608 59,553 476,424 Totals N/A Test Component Software Version Hardware OS Database Server Oracle 11gR2 11.2.0.3 SPARC T4-2 Oracle Solaris 11 App/Gtwy Server Siebel 8.1.1.4 SPARC T5-2 Oracle Solaris 10 Web Server iplanet Web Server 7 update 9 SPARC T4-1 Oracle Solaris 10 1 Actual results may vary, based on a broad range of implementation-specific factors, such as transaction mix, hardware platform, network parameters, and database size. Oracle does not warrant or guarantee that customers will obtain the same or similar results, even if they use the same or similar equipment and/or software applications. Oracle does not warrant, endorse, or guarantee any performance of any products, any results desired or achieved, or any statements made within this document. 2 Siebel CRM Release 8.1.1.4 Industry Application Platform Sizing and Performance benchmarks are based on Siebel CRM Release 8.1.1.4 customized industry applications and reflect a heavier scenario mix and more-aggressive think times than earlier versions. Results of this benchmark are not comparable with those of prior Siebel CRM Release or 8.0 benchmarks. 3
OVERVIEW Oracle s Siebel Platform Sizing and Performance Program is designed to stress the Siebel CRM Release 8.1.1.4 architecture and to demonstrate that large customers can successfully deploy many thousands of concurrent users. Among the Siebel CRM Release 8.1.1.4 architecture features exercised are the following: Smart Web Architecture Takes advantage of the newest Web browser technology to deliver a highly interactive experience. The interaction model, which is similar to Windows-based applications, also improves productivity. Utilization rates on the Web server are low, allowing customers to retain existing Web server infrastructure. Smart Network Architecture Allows Siebel CRM Release 8.1.1.4 customers to leverage their existing network infrastructure by compressing and caching user interface components, so that browser/web server interaction occurs only when the application requests data. This allows customers to avoid expensive network upgrades that can be necessary with competing products. Server Connection Broker The Siebel Connection Broker (SCBroker) is a server component that provides intraserver loadbalancing. SCBroker distributes server requests across multiple instances of Application Object Managers (AOMs) running on a Siebel Server. Smart Database Connection Pooling and Multiplexing Allows customers to scale their databases without introducing expensive and complex transaction-processing monitors. Server Request Broker Server Request Broker (SRBroker) processes synchronous server requests requests that must be run immediately, and for which the calling process waits for completion. This test simulated a large corporation with 40,000 concurrent active users in multiple departments and addressed key business requirements. Siebel Financial Services Call Center Provides the most complete solution for sales and service, allowing customer service and telesales representatives to provide superior customer support, improve customer loyalty, and increase revenues through cross-selling and up-selling. Siebel Order Management Oracle's Siebel Order Management allows employees such as salespeople and call center agents to create and manage quotes and orders through their entire life cycle. Siebel Order Management can be tightly integrated with back-office applications, allowing users to perform tasks such as checking credit, confirming availability, and monitoring the fulfillment process. Siebel Workflow Automates user interaction, business processes, and integration through use of a businessprocess-management engine. It allows simple administration and customization through a graphical drag-anddrop user interface. Administrators can add custom or predefined business services and specifies logical branching, updates, inserts, and sub processes to create a workflow process tailored to their unique business requirements. 4
METHODOLOGY This benchmark was executed independently by Oracle under the Siebel CRM Release 8.1.1.4 Industry Applications Platform Sizing and Performance Program (PSPP) guidelines. Test cases are based on Siebel customer requirements and exercise some of the most critical and frequently used components of the Siebel CRM application. The test cases must run in steady state for at least one hour, and certification is dependent on the achievement of certain key performance indicators. The test simulated real-world requirements of a large organization, consisting of 40,000 concurrent, active users in a call center organization. Test conditions simulated service representatives running Siebel Financial Services Call Center and order capture agents organizations running Siebel Order Management. The number of Call Center representatives is 70% and the number of Call Center and order capture agents represents 30% of the total number of users. End users were simulated by use of Oracle Application Testing Suite 9.21. The think time range between user operations was 10 to 67 seconds. Database Setup Prior to benchmark execution, the database was approximately 115GB. It was constructed based on Siebel customer experience and requirements and on the Siebel CRM Industry Application repository and data model representing the most common data distribution and volumes in high-transaction-rate implementations. The table below shows a sampling of record volumes for key business entities in the Siebel CRM Industry Application volume database. Business Entity Number of Records Accounts 2,203,840 Activities 6,099,058 Addresses 4,148,701 Contacts 3,711,496 Employees 62,401 Opportunities 3,395,085 Orders 624,546 Products 288,619 Quote Items 1,984,584 Quotes 301,568 Service Requests 5,875,416 Siebel Financial Services Call Center Business Transactions Three complex business transactions were executed simultaneously for 28,000 concurrent users. The ratios of these 3 scenarios were 30%, 40%, 30% respectively, which together were totaling 70% of all transactions simulated in this benchmark. Between each user operation and the next one, the think time averaged approximately 15 seconds. This section provides a high-level description of the use cases tested. Incoming Call Creates Opportunity, Quote and Order Create a new contact Create a new Opportunity for that contact Add two products to Opportunity 5
Navigate to Opportunities - Quotes View Click "AutoQuote" button to generate quote Enter Quote Name, and Price List Drilldown on the quote name to go to Quote - Line Items View and specify discount Click "Reprice All" button Update opportunity Navigate to Quotes - Orders View Click on "AutoOrder" button to automatically generate order Navigate back to Opportunity Incoming Call Creates Service Request Create a new Service Request Associate Contact and Account for that Service Request Click Verify button to bring up pick applet Select Entitlement Query and Select Policy Select Product and add Product to Service Request Save Service Request Go to Service Request Activity Plan Select Activity Plan and Save Service Request Incoming Call Updates Service Request Click on Service Screen Tab Go to My Service Request Drill down on Service Request and go to SR Activity Navigate to SR Related SR Select, Add Solution and Save Update Service Request, set it to Pending and Save Navigate back to Service Request Activity Siebel Communication Order Management Business Transactions Two complex Order Management transactions were executed simultaneously for 12,000 concurrent users concurrently with aforementioned three Call Center scenarios above. The ratio of these 2 scenarios was 50% each, which together were totaling 30% of all transactions simulated in this benchmark. Between each user operation and the next one, the think time averaged approximately 20 and 67 seconds respectively. This section provides a highlevel description of the use cases tested. 6
Order & Order Items Creation Create new accounts, address Navigate to Account Summary Click the New button in Installed Assets Create New Order and New Order Item Associate a Price List Customize Order Accept Promotion Create 8 more Order Items and Select Products Click Submit on the Order header Complete order Click AutoAsset Order Updates Navigate to Account Summary Click the New button in Installed Assets Create New Order and New Order Item Associate a Price List Select Product Click Submit on the Order header Complete order Click AutoAsset 7
TOPOLOGY This section describes the hardware topology of the systems used for the test as well as the hardware and software combinations used. PSPP Components Siebel CRM Release 8.1.1.4 Industry Applications Oracle 11gR2 Database Server Oracle iplanet Web Server Oracle Solaris 10 Oracle s SPARC T5 Servers SPARC T5 servers configured with 3.6GHz, 16 cores per processor running Oracle Solaris provides the most scalable, secure, and highly integrated platform for the optimized deployment of enterprise and mission-critical applications. Oracle Solaris The world's leading enterprise operating system - Solaris, can span the entire enterprise from the Web tier, to the datawarehouse, to the most demanding applications, all while delivering the performance, stability and security that could satisfy even the most demanding customers. Database Server 1 x SPARC T4-2 server o 1 x SPARC T4 processor, 2.85 GHz o 128 GB RAM o Oracle Solaris 11 11/11 o Oracle Database 11g Release 2 (11.2.0.3) Gateway/Application Server 2 x SPARC T5-2 servers, each with o 2 x SPARC T5 processor, 3.6 GHz o 512 GB RAM o Oracle Solaris 10 8/11 o Oracle 11gR2 Database Client v11.2.0.1 (32-bit) o Siebel CRM 8.1.1.4 SIA [21225] 8
Web Server 1 x SPARC T4-1 server o 1 x SPARC T4 processor, 2.85 GHz o 128 GB RAM o Oracle Solaris 10 8/11 o Oracle iplanet Web Server 7 OATS (Oracle Application Test Suite) Controller 1 x Sun Fire X4200 o 2 x 2.6 GHz Dual Core AMD Opteron 285 SE o 16 GB RAM o Microsoft Windows Server 2003 R2 EE SP2 o OATS v9.21 OATS (Oracle Application Test Suite) Remote Agent Servers: 8 x Sun Fire X4170 o 2 x 2.93 GHz 6-Core Intel Xeon X5670 o 48 GB RAM o Microsoft Windows Server 2003 R2 EE SP2 o OATS v9.21 9
TOPOLOGY DIAGRAM OATS Remote Agent Servers OATS v9.21 OATS Controller OATS v9.21 4 x X4170 4 x X4170 X4200 Private GigE SPARC T4-1 Web Server SPARC T5-2 Gateway Server / App Server 1 DB Server Private GigE App Server 2 SPARC T4-2 SPARC T5-2 Flash Storage RESULTS Response Times and Transaction Throughput 3 4 5 Workload Number of Users Average Operation Response Time (sec) Business Transactions Throughput/hour Projected Daily Transactions Financial Services Call Center 28,000 0.110 273,786 2,190,288 Order Management 12,000 0.608 59,553 476,424 Totals N/A Server Resource Utilization Node Users Functional Use % CPU Memory Utilization (GB) 1 40,000 Web Server 29.5 14 GB 2 20,000 Gateway/Application Server 67 206 GB 3 20,000 Application Server 66 206 GB 4 40,000 Database Server 33.5 109 GB 10
Network Utilization For 40,000 concurrent users, the network total average utilization measured was 816.37 Mbps for the Web to Application traffic, an average of 28.83 Kbps per user. CONCLUSION The test system demonstrated that Oracle s Siebel CRM Release 8.1.1.4 architecture and SPARC T5 Servers form a very powerful and cost effective business solution. Siebel on Oracle Solaris This benchmark is first published with Siebel 8.1.1.4 running on the Oracle Solaris Operating System. The Web and Application tiers of the Siebel CRM Release 8.1.1.4 architecture ran Oracle Solaris 10, and Oracle 11gR2 Database Server on Oracle Solaris 11. Vertical scalability The Siebel CRM Release 8.1.1.4 server showed excellent scalability on SPARC T5 server. Many users can be supported with minimal hardware. Low network utilization The Siebel CRM Release 8.1.1.4 Smart Web Architecture and Smart Network Architecture efficiently managed the network, consuming only 28.83 kilobits per second per user. Efficient use of the database server Siebel CRM Release 8.1.1.4 Smart Database Connection Pooling and Multiplexing allowed the database to service 40,000 concurrent users and the supporting Siebel CRM Release 8.1.1.4 server application services with approximately 2000 database connections using Siebel database pooling ratio 15:1. 3 Response times are measured at the Web server instead of at the end user. The response times at the end user would depend on the network latency, the bandwidth between Web server and browser, and the time for browser rendering of content. 4 A business transaction is a defined set of steps, activities, and application interactions used to complete a business process, such as Create and Assign Service Requests. Search for a contact is an example of a step in a business transaction. For a detailed description of business transactions, see the Business Transactions section. 5 Actual results may vary, based on a broad range of implementation-specific factors, such as transaction mix, hardware platform, network parameters, and database size. Oracle does not warrant or guarantee that customers will obtain the same or similar results, even if they use the same or similar equipment and/or software applications. Oracle does not warrant, endorse, or guarantee any performance of any products, any results desired or achieved, or any statements made within this document. Released April 24, 2013 11
Performance and Scalability Benchmark: Siebel CRM Release 8.1.1.4 Industry Applications and Oracle 11.2.0.3 Database on Oracle's SPARC T5 Servers and Oracle Solaris Released 24-April-2013 Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com Copyright 2013 Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. 12