Intelligent Storage Results from real life testing Ingo Brenckmann Jochen Kirsten Storage Technology Strategists SAS EMEA
SAS Intelligent Storage components! OLAP Server! Scalable Performance Data Server! Scalable Performance Data Engine! Base SAS Engine
DELL PowerEdge 6600 relevant hardware components! 4 Intel L3 XEON CPUs 1,5GHz, 1MB L3 Cache, Hyperthreading Support! System Bus 64-bit/100MHz PCI! Main Memory 4GB DDR RAM The CPUs and the memory
4 hyperthreaded XEONs make Windows 2000 Advanced Server "see" 8 CPUs The CPUs and the memory in Windows Task Manager
Relevant hardware components! 2 SCSI System Hard Disks Hitachi 32DJ-18MC, 18GB, Ultra/160, 10000RPM! 1 SCSI Controller Adaptec 2940UW, Ultra3/160 LVD The SCSI controllers
Relevant hardware components! 8 SCSI Test Hard Disks Fujitsu MAM 3367MC, 36GB, Ultra 160, 15000RPM! 2 RAID Controllers PERC 3/DC, 128MB, 2 Channel, Ultra3/160 LVD The eight test hard disks and four still empty drive bays
Ready to go!
Simplified system chart XEON XEON RAM XEON XEON PCI System Bus Channel 0 Channel 1 Channel 0 Channel 1 Controller 0 Controller 1
2 controllers/1 channel vs. 1 controller/2 channels I/O-set-up - take 1 Channel 0 Channel 1 Controller 0 Data1 Data2 Index / Work / General Purpose Data3 Data4
Take 1 - results 10GB Full Table Scan Elapsed Time [s] 300 250 200 150 100 50 0 100.0 80.0 60.0 40.0 20.0 0.0 Throughput [MB/s] READ 1 thread READ 2 thread READ 4 thread READ 8 thread Number of Threads
2 controllers/1 channel vs. 1 controller/2 channels I/O-set-up - take 2 Controller 0 Controller 1 Channel 0 Channel 0 Data1 Data2 Index Data3 Data4 Work
Take 2 - results 10GB Full Table Scan 300 300 Elapsed Time [s] 250 200 150 100 50 0 READ 1 thread READ 2 thread READ 4 thread READ 8 thread 250 200 150 100 50 0 Throughput [MB/s] Number of Threads
I/O-set-up Channel 0 Channel 1 Controller 0 Data Index
I/O-Setup various RAID-levels! 1 Controller/2 Channels 2*RAID0 (4 disks) 1*RAID10 (8 disks) 2*RAID0 (2 disks) 4*RAID0 (2 disks) 8*RAID0 (1 disk) 1*RAID0 (8 disks)! 2 Controllers/1 Channel 2*RAID0 (4 disks) 2*RAID5 (4 disks) 2*RAID0 (2 disks) 4*RAID0 (2 disks) 8*RAID0 (1 disk) 2*RAID0 (3 disks)
Bandwidth of different configurations 140.0 120.0 100.0 80.0 60.0 40.0 1 Contr. / 2 Channels 2 Contr. / 1 Channel each 20.0 0.0 1 2 3 4 5 6
Append Performance With Constant Append Size (10% of base table) And Growing Result Table (4 indexes) 1400 1200 1000 Runtime [sec] 800 600 400 200 0 SPD Engine (index online) Base Engine (index offline) Base Engine (index online) 11 12 13 Result Table [million rows] 14 15
Append Performance With Same Base Table Size (10 million rows) And Increasing Append Size (4 indexes) 800 700 600 500 Runtime [sec] 400 300 200 100 0 SPD Engine (index online) Base Engine (index offline) Base Engine (index online) 1000 10000 100000 1000000 Append Size [rows] 10000000
Sort Performance (by 1 variable) 16000 14000 12000 Runtime [sec] 10000 8000 6000 4000 2000 SPDE BY-Sort SPDE MP-Sort BASE MP-Sort 0 1000000 10000000 BASE Normal Sort 100000000 1000000000 SPDE Normal Sort
SPDE index creation performance! Table 100GB (1000789200 rows)! Create two indexes, 16 byte char, 8 byte numeric Synchronously: 3 hours Asynchronously: 2,5 hours
SAS 8 runtime statistics! CPU time is always lower than real time NOTE: PROCEDURE APPEND used (Total process time): real time 13:24.56 cpu time 12:15.42 NOTE: PROCEDURE DATASETS used (Total process time): real time 0.32 seconds cpu time 0.23 seconds
SAS 8 at work
SAS 9 runtime statistics! CPU time might be higher than real time NOTE: PROCEDURE APPEND used (Total process time): real time 14.96 seconds cpu time 45.06 seconds NOTE: DATA statement used (Total process time): real time 3:24.18 cpu time 9:53.67 NOTE: PROCEDURE DATASETS used (Total process time): real time 2:38.17 cpu time 4:47.28
SAS 9 at work
SAS OLAP Server Scalability Inside Parallel Parallel Storage Storage One One Thread Thread for for every every Query Query One One Aggregation Aggregation Selection Selection for for every every Region Region affected affected Configurable Configurable Cube Cube Cache Cache
SAS OLAP Server - Parallel Storage data data data data One One file file system system for for all all data data Data Data is is spread spread across across multiple multiple file file systems systems 1 Thread Multi-threaded
C02_05T3 Build Performance CPU TIME 1000 900 800 700 600 500 Overall CPU Time increases slightly due to thread handling. 400 300 200 100 0 CB_02_01 CB_02_02 CB_02_03 CB_02_04 CB_02_05 C02_05D2 C02_05C2 C02_05S2 C02_05D3 C02_05C3 C02_05S3 C02_05D4 C02_05C4 C02_05S4 C02_05T1 C02_05T2 1 Processor 4 Processors
C02_05T3 Build Performance REAL TIME 3000 2500 2000 Overall Real Time decreased by 78%!!! 1500 1000 500 0 CB_02_01 CB_02_02 CB_02_03 CB_02_04 CB_02_05 C02_05D2 C02_05C2 C02_05S2 C02_05D3 C02_05C3 C02_05S3 C02_05D4 C02_05C4 C02_05S4 C02_05T1 C02_05T2 1 Processor 4 Processors
Building the cube with multiple threads The The magic magic behind behind the the lower lower Real Real Time: Time: MULTI-THREADING MULTI-THREADING
Query Performance Cubes CB_02_05 934MB C02_05D2 469MB C02_05D3 314MB C02_05D4 236MB C02_05C2 463MB C02_05C3 314MB C02_05C4 228MB C02_05S2 447MB C02_05S3 293MB S02_05S4 220MB S02_05T1 468MB C02_05T2 368MB C02_05T3 299MB
Multi User Environments Up to 1000 Users and still an average response time of less than a second.
Scaling for REAL Sun Fire 15K "72 UltraSPARC CPUs running at 1050 MHz and 8 MB E$ "288 GB RAM "Qty 16 T3 Sun StorEdge Arrays (100 MB/s Fiber Connection Per Array) "256 GB of usable disk space each array "Solaris 9 12/02 "SAS Version 8.2 32bit for Sun "SPDS Version 3.0TSM4
SPD Server on SUN 15K benchmark result highlights! 577GB table (738199305 rows, 84 columns)! Creation of table: 2:18 hours! 4000 rows subset full table scan: 15min! Sustained I/O-bandwidth ~720MB/s! Parallel creation of four indexes: 2:11 hours! 4000 rows subset indexed search: >1s
34