IBM B2B INTEGRATOR BENCHMARKING IN THE SOFTLAYER ENVIRONMENT

IBM B2B INTEGRATOR BENCHMARKING IN THE SOFTLAYER ENVIRONMENT 215-4-14 Authors: Deep Chatterji (dchatter@us.ibm.com) Steve McDuff (mcduffs@ca.ibm.com)

CONTENTS Disclaimer...3 Pushing the limits of B2B Integrator...4 Benchmark setup...4 Hardware...4 Software...5 Overview of softlayer...5 network topology...5 CPU Sockets, cores & Affinity on B2Bi...6 Oracle 12c features...7 Advanced Table Compression...7 Advanced Index Compression...7 Compression test results...8 Tests and results...9 SFTP Transfer...9 Test results summary...9 SFTP 2 kb files using database storage... 1 SFTP 1 MB files using database storage... 11 SFTP 1 MB files using file system storage... 12 SFTP 1 MB files using file system storage and encryption... 13 SFTP 1 MB files using database storage and encryption... 14 X12... 15 X12 on two B2Bi nodes... 16 X12 on a single B2Bi node... 17 SFG Routing... 18 SFG Routing test result... 18 X12 index and purge... 19 X12 Index... 2 X12 Purge... 21 Conclusions... 22

DISCLAIMER IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

PUSHING THE LIMITS OF B2B INTEGRATOR As the speed of computer hardware improves and new versions of software become available, we inevitably want to know how much our applications would benefits from those improvements. The way we run our software is also changing with the increased popularity of cloud computing providers such as Softlayer and Amazon. The goal of this white paper is to measure the throughput that IBM B2B Integrator can achieve using the 5.2.5 release of B2Bi and Oracle 12c on recent hardware, hosted in a Softlayer environment. BENCHMARK SETUP For this benchmark, we used the following hardware and software topology: Server 1 Server 2 B2Bi Application Node 1 Performance Testing Tool 1 Gbps link B2Bi Application Node 2 Oracle 12c Database HARDWARE Server #1: Application Processor: 2 X 2.4GHz Intel Xeon-IvyBridge (E5-465-V2-DecaCore) RAM: 512 GB DDR3 Registered 1333 Disk: RAID 1 using an array of 16 disks: 147GB SA-SCSI 15K RPM. Note: the disks were connected directly to the server. They were not shared with other machines. Network speed: 1 Gbps Server #2: Database Processor: 4 X 2.4GHz Intel Xeon-IvyBridge (E5-465-V2-DecaCore)

RAM: 512 GB DDR3 Registered 1333 Disk: RAID 1 using an array of 16 disks: 147GB SA-SCSI 15K RPM Note: the disks were connected directly to the server. They were not shared with other machines. Network speed: 1 Gbps SOFTWARE Server #1: Application Operating System: Red Hat Enterprise Linux 6.x (64 bit) Two instances of B2Bi 5.2.5 configured as a vertical cluster running the internal perimeter server. Minimum heap size: 4 GB Maximum heap size: 8 GB B2Bi performance testing software. Server #2: Database Operating System: Red Hat Enterprise Linux 6.x (64 bit) Oracle 12c Database. OVERVIEW OF SOFTLAYER Softlayer enables it s users to rent both Bare Metal and Virtual hardware. Bare Metal hardware guarantees that the hardware capacity will not be shared with other Softlayer tenants. Bare Metal hardware can also reach higher total capacity in terms of CPU core count and memory. Bare Metal hardware was chosen for this benchmark as the goal was to measure the highest possible throughput. NETWORK TOPOLOGY Each server on Softlayer is connected to both a public and a private network. For this benchmark, we used the 1 Gbps private network to communicate between B2Bi and the database. There was no firewall between the test servers. While we didn t have control over the network load produced by other servers on the private network, we didn t experience network latency problems during out tests.

CPU SOCKETS, CORES & AFFINITY ON B2BI In order to determine the optimal configuration for the B2Bi vertical cluster, we considered the following aspects: CPU Caches are more efficient when a process is consistently using the same CPU core or physical socket. A process can be bound to a CPU using CPU affinity. To get the most throughput on B2Bi using the X12 protocol, we settled on the following configuration: 2 B2Bi nodes 2 physical CPU with 1 cores each. CPU Affinity turned off. Our experiments showed that Going from 1 B2Bi node to 2 lead to a 5% throughput improvement on the same hardware. With 2 B2Bi nodes, going from 1 physical CPU to 2 lead to a 12% throughput improvement. With 2 B2Bi nodes, going from 2 physical CPU to 4 lead to a 22% throughput degradation. This is due to the increased frequency at which threads switch between physical CPU. It is therefore recommended to use horizontal clustering to fully leverage more than 2 physical CPUs. Enabling CPU affinity lead to no statistically significant changes in throughput for our specific workload.

ORACLE 12C FEATURES ADVANCED TABLE COMPRESSION Advanced Row Compression uses a compression algorithm designed to work with OLTP applications. The algorithm works by eliminating duplicate values within a database block, even across multiple columns. Compressed blocks contain a structure called a symbol table that maintains compression metadata. The benefits of Advanced Row Compression go beyond just on-disk storage savings. A significant advantage is Oracle s ability to read compressed blocks directly without uncompressing the blocks. This helps improve performance due to the reduction in I/O, and the reduction in system calls related to the I/O operations. In general, organizations can expect to reduce their storage space consumption by a factor of 2x to 4x by using Advanced Row Compression. It s much better than Basic compression available in Oracle before as it uses more highly developed compression algorithms and data access policies. We used this feature on the set of top 5 tables in the B2Bi schema which experience very rapid growth and high I/O overhead. The tables are: CORRELATION_SET DATA_TABLE DOCUMENT TRANS_DATA WORKFLOW_CONTEXT An example of how this was achieved: As the tables would be pre-exiting in the B2Bi schema, we ll move an existing table to use compressed format. Command: ALTER TABLE DATA_TABLE MOVE ROW STORE COMPRESS ADVANCED; After table has been compressed its indexes need to be rebuilt. Command: ALTER INDEX SCI_PK_14 REBUILD; ALTER INDEX SCI_IDX_56 REBUILD; ADVANCED INDEX COMPRESSION Advanced Index Compression has the potential to reduce the overall size of non-unique indexes and multi-column unique indexes. A smaller index would fit/stay in the buffer cache (RAM). It is a large performance improvement if it stays permanently smaller without subsequent expensive maintenance operations. Not only

in X12 Cycles per hour will it potentially save storage space, but if the resultant index contains fewer leaf blocks, it means fewer IO s with a better execution plan. A limitation being, it cannot be specified on single column unique index. This may not be applicable to some of the indexes on B2Bi tables as some would be primary keys. All indexes on the above 5 tables were compressed except where the limitation applied. An example: For the indexes on DATA_TABLE, Command: ALTER INDEX SCI_PK_14 REBUILD COMPRESS ADVANCED LOW; ALTER INDEX SCI_IDX_56 REBUILD COMPRESS ADVANCED LOW; COMPRESSION TEST RESULTS 1 Oracle 12c compression improvements 75 5 +58% +71% 25 No compression (baseline) Table compression Table & Index compression Baseline Improvement - EDI tests were run to gauge the impact of using compression. - When compared to non-compressed tables, 1.7X throughput improvement was achieved when using table & index compression. - Further, as more data could fit in the database buffer, it decreased the number of disk I/O operations. All the tests described in this benchmark used both table and index compression.

TESTS AND RESULTS This benchmark focuses on the SFTP protocol, X12 payload processing and SFG Routing with a minimum amount of work performed within the business processes. To determine how fast B2Bi will handle the workload for your environment, we recommend to pick the test results which best fits your type of workload and use it as a baseline. SFTP TRANSFER For this test, an external Java SFTP client built into the test harness was used to push the payload files into B2Bi. In B2Bi, the SFTP Server Adapter was being used. The test performed the following actions during a cycle: login, cd and put. We picked the most commonly used file sizes of 2 KB and 1 MB. The file is sent 1, times. The throughput is measured as function of time taken for the test to complete. There were some permutations of this test where storage was initially set to DB (database) then changed to FS (file-system); the payload was first being stored as plain text then changed to encrypted, the encryption/decryption was being done in B2Bi. To keep tests consistent, each test starts from an empty database. This explains why the CPU and IO per second take a few minutes to stabilize as the database size grows. The encryption was being controlled using ENC_DECR_DOC in security.properties, the possible values are: ENC_ALL: Encrypt documents both in the database and in the file system. ENC_DB: Encrypt documents in the database. ENC_FS: Encrypt documents in the file system. ENC_DB and ENC_FS were used in the tests. TEST RESULTS SUMMARY B2Bi cluster File size node count (kb) (cycles / hour) 2 2 DB clear text 372,839 1.4 97.9 2 1, DB clear text 459,567 31.8 78.8 2 1, FS clear text 315, 22.2 9.2 2 1, FS encrypted 123,748 11.1 95.8 2 1, DB encrypted 317,617 42 71.7 type impact: Database storage is more performant than file system storage for small and compressible documents. This matches the type of documents we used in our test. While larger documents weren t covered by this benchmark, past performance tests showed that file system storage is preferable when dealing with documents of 5mb or more. impact: For database storage, the encryption results in a higher CPU cost per transfer for the B2Bi application.

16:35 16:41 16:47 16:53 16:59 17:5 17:11 17:17 17:23 17:29 17:35 17:41 17:47 17:53 17:59 18:5 18:11 18:17 18:23 18:29 18:35 18:41 18:47 18:53 18:59 19:5 19:11 16:35 16:41 16:47 16:53 16:59 17:5 17:11 17:17 17:23 17:29 17:35 17:41 17:47 17:53 17:59 18:5 18:11 18:17 18:23 18:29 18:35 18:41 18:47 18:53 18:59 19:5 19:11 SFTP 2 KB FILES USING DATABASE STORAGE Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 1 SFTP 2 2 DB plain 372,839 1.4 97.9 Resource snapshots: SFTP 2KB - DB - B2Bi Application Server 1. 8. 6. 4. 2.. 9 8 7 6 5 4 3 2 1 SFTP 2KB - DB - DB Server 1 9 8 7 6 5 4 3 2 1 2 18 16 14 12 1 8 6 4 2

1:55 1:56 1:57 1:58 1:59 11: 11:1 11:2 11:3 11:4 11:5 11:6 11:7 11:8 11:9 11:1 11:11 11:12 11:13 11:14 11:15 11:16 11:17 11:18 11:19 11:2 11:21 11:22 11:23 1:55 1:56 1:57 1:58 1:59 11: 11:1 11:2 11:3 11:4 11:5 11:6 11:7 11:8 11:9 11:1 11:11 11:12 11:13 11:14 11:15 11:16 11:17 11:18 11:19 11:2 11:21 11:22 11:23 SFTP 1 MB FILES USING DATABASE STORAGE Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 2 SFTP 2 1, DB plain 459,567 31.8 78.8 Resource snapshots: SFTP 1MB - DB - B2Bi Application Server 1 8 6 4 2 7 6 5 4 3 2 1 SFTP 1MB - DB - DB Server 1 8 6 4 2 8 7 6 5 4 3 2 1

11:37 11:39 11:41 11:43 11:45 11:47 11:49 11:51 11:53 11:55 11:57 11:59 12:1 12:3 12:5 12:7 12:9 12:11 12:13 11:37 11:39 11:41 11:43 11:45 11:47 11:49 11:51 11:53 11:55 11:57 11:59 12:1 12:3 12:5 12:7 12:9 12:11 12:13 12:15 SFTP 1 MB FILES USING FILE SYSTEM STORAGE Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 3 SFTP 2 1, FS plain 315, 22.2 9.2 Resource snapshots: SFTP 1MB - FS - B2Bi Application Server 1 8 6 4 2 4 35 3 25 2 15 1 5 SFTP 1MB - FS - DB Server 1 8 6 4 2 7 6 5 4 3 2 1

17:59 18:3 18:7 18:11 18:15 18:19 18:23 18:27 18:32 18:36 18:4 18:44 18:48 18:52 18:56 19: 19:4 19:8 19:12 19:16 19:2 19:24 19:28 19:32 19:36 17:59 18:3 18:7 18:11 18:15 18:19 18:23 18:27 18:31 18:35 18:39 18:43 18:47 18:51 18:55 18:59 19:3 19:7 19:11 19:15 19:19 19:23 19:27 19:31 19:35 SFTP 1 MB FILES USING FILE SYSTEM STORAGE AND ENCRYPTION Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 4 SFTP 2 1, FS encrypted 123,748 11.1 95.8 Resource snapshots: SFTP 1MB - FS - Encrypted B2Bi Application Server 1 8 6 4 2 3 25 2 15 1 5 SFTP 1MB - FS - Encrypted - DB Server 1 8 6 4 2 6 5 4 3 2 1

17:39 17:4 17:41 17:42 17:43 17:44 17:45 17:46 17:47 17:48 17:49 17:5 17:51 17:52 17:53 17:54 17:39 17:4 17:41 17:42 17:43 17:44 17:45 17:46 17:47 17:48 17:49 17:5 17:51 17:52 17:53 17:54 SFTP 1 MB FILES USING DATABASE STORAGE AND ENCRYPTION Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 5 SFTP 2 1, DB encrypted 317,617 42 71.7 Resource snapshots: SFTP 1MB - DB - Encrypted B2Bi Application Server 1 8 6 4 2 6 5 4 3 2 1 SFTP 1MB - DB - Encrypted - DB Server 1 8 1 8 6 6 4 4 2 2

X12 A sample X12 payload is picked up via FileSystemAdapter and handed over to a BP which does de-enveloping, the payload file-size is ~ 1 KB (5 * 2 KB) and the file is sent 1 K times. The throughput is measured as function of time taken for the test to complete. To keep tests consistent, each test starts from an empty database. This explains why the CPU and IO per second take a few minutes to stabilize as the database size grows. Note: The time taken to move files by the file adapter isn t measured in this test. Result summary: Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 6 X12 2 2 DB plain 1,251,66 85 5. 7 X12 1 2 DB plain 1,18,538 67 5. Going from 1 to 2 B2Bi nodes to handle X12 results in a relatively small improvement in throughput. Clustering should therefore be used if it s necessary to handle more CPU intensive tasks during the workflow.

13:28 13:29 13:3 13:31 13:32 13:33 13:34 13:35 13:36 13:37 13:38 13:39 13:4 13:41 13:28 13:29 13:3 13:31 13:32 13:33 13:34 13:35 13:36 13:37 13:38 13:39 13:4 13:41 X12 ON TWO B2BI NODES Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 6 X12 2 2 DB plain 1,251,66 85 5. Resource snapshots: X12 on 2 B2Bi Nodes - B2Bi Server 1 8 6 4 2 2 15 1 5 X12 on 2 B2Bi Nodes - DBServer 1. 8. 1 8 6. 6 4. 4 2. 2.

21:5 21:51 21:52 21:53 21:54 21:55 21:56 21:57 21:58 21:59 22: 22:1 22:2 22:3 22:4 22:5 22:6 22:7 22:8 21:5 21:51 21:52 21:53 21:54 21:55 21:56 21:57 21:58 21:59 22: 22:1 22:2 22:3 X12 ON A SINGLE B2BI NODE Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 7 X12 1 2 DB plain 1,18,538 67 5. Resource snapshots: X12 on 1 B2Bi Nodes - B2Bi Server 1 8 25 2 6 15 4 1 2 5 X12 on 1 B2Bi Nodes - DB Server 1 12 8 6 4 2 1 8 6 4 2

16:2 16:3 16:4 16:5 16:6 16:7 16:8 16:9 16:1 16:11 16:12 16:13 16:14 16:15 16:16 16:17 16:18 16:19 16:2 16:21 16:22 16:2 16:3 16:4 16:5 16:6 16:7 16:8 16:9 16:1 16:11 16:12 16:13 16:14 16:15 16:16 16:17 16:18 16:19 16:2 16:21 16:22 SFG ROUTING In this test a sample X12 payload is sent into B2Bi/FileGateway using FTP from there FileGateway processing takes place. The payload is sorted and stored in mailbox/s per the FileGateway routing rules. File size = ~ 2 KB and the file is sent 1 K times. The throughput is measured as function of time taken for the test to complete SFG ROUTING TEST RESULT B2Bi cluster Test# Protocol node count File size (in kb) (cycles / hour) 8 SFG routing 2 2 DB plain 487,815 52 12.1 Resource snapshots: SFG Routing - B2Bi Server 1 8 6 4 2 7 6 5 4 3 2 1 SFG Routing - DB Server 1 8 6 4 2 16 14 12 1 8 6 4 2

X12 INDEX AND PURGE This test is used to measure how fast the system can take documents out of the system, it s used to determine how many documents per day the system can handle. The speed of index and purge is typically linked to the number of business process steps. Out of the box B2Bi Indexing functionality was used but with the batch size changed to 5 K in a batch. B2Bi command line multi-threaded Purge was invoked with these commands: control_extpurge.sh start control_extpurge.sh stop Note: As for the Index and Purge tests, these were in ideal condition with no other background load running other than the test data. Multithreaded command line Purge was used.

14:13 14:14 14:15 14:16 14:17 14:18 14:19 14:2 14:21 14:22 14:23 14:12 14:13 14:14 14:15 14:16 14:17 14:18 14:19 14:2 14:21 14:22 14:23 X12 INDEX Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 9 Index X12 2 2 DB plain 3,,.1.3 Resource snapshots: X12 Index - B2Bi Server 1. 8. 6. 4. 2.. 4 3.5 3 2.5 2 1.5 1.5 X12 Index - DB Server 1. 8. 6. 4. 2. 2 15 1 5.

14:3 14:31 14:32 14:33 14:34 14:35 14:36 14:37 14:38 14:39 14:4 14:41 14:3 14:31 14:32 14:33 14:34 14:35 14:36 14:37 14:38 14:39 14:4 X12 PURGE Test# Protocol B2Bi cluster node count File size (in kb) (cycles / hour) 9 Purge X12 2 2 DB plain 4,285,.3 1.2 Resource snapshots: X12 Purge - B2Bi Server 1. 12 8. 6. 4. 2. 1 8 6 4 2. X12 Purge - DB Server 1 8 6 4 2 2 15 1 5 The spike in on the database was caused by the database persisting a lot of changes at once. Note that the scale remains relatively small at 2 transfer per second.

CONCLUSIONS The performance of B2Bi is very good when it runs on Softlayer Bare Metal servers. When using Oracle 12c, it s highly recommended to use Advanced Table and Index compression. It reduces disk I/O s and results in better throughput. It is recommended to use horizontal clustering on B2Bi to leverage the capacity of more than 2 physical CPUs to avoid thread switching overhead.