IBM s Data Warehouse Appliance Offerings RChaitanya IBM India Software Labs
Agenda 1 IBM Smart Analytics System (D5600) System Overview Technical Architecture Software / Hardware stack details 2 Netezza Performance Server (TwinFin) NPS Overview Technical Architecture Software / Hardware stack
IBM Smart Analytics (D5600)
IBM Smart Analytics System 5600 / 5600S Overcoming The Obstacles To Business Transformation An integrated, high-performance analytics solution for accelerating delivery of insights for faster, smarter action Able to adjust and grow based on your company s ever changing business needs broad analytic capabilities powerful warehouse capabilities scalable and fully-integrated IBM hardware set-up services and single point of premium support Delivering results in days instead of months
IBM Smart Analytics System 5600 / 5600S What's in the system? Analytics Software Options Cognos 8 Business Intelligence InfoSphere Warehouse Cubing Services InfoSphere Warehouse Text Analytics & Data Mining Data Warehouse Software InfoSphere Warehouse InfoSphere Warehouse Advanced Workload Management Tivoli System Automation Hardware/OS IBM System x3650 M2 IBM System Storage DS3400 SLES10 SP2
IBM Smart Analytics System 5600 / 5600S Highly Flexible and Scaleable Choose the way that your Smart Analytics System is deployed Add capacity and analytic capabilities as your business requirements evolve Start with a System that fits your needs Add capacity for growing data & users Add analytic capability to meet new business needs IBM SMART ANALYTICS SYSTEM
IBM Smart Analytics System 5600 / 5600S Transparent Modular Architecture Foundation Structure Add-On s Foundation + + Data User Failover Application (s) 1 1 to x Nodes 0 to y Nodes 0 or (x+y)/5 Nodes 0 to z s Add-On s Analytics BI RSM 3 rd Party s
Definition for Smart Analytics System 5600 / 5600S Core Warehouse s Foundation Data User Failover Management Server: 1 x3650 M2 Admin (1 st user) server: 1 x3650 M2 Storage: 2 DS3400 (24 disks) Server: 1 x3650 M2 Storage: 2 DS3400 (24 disks) Server: 1 x3650 M2 Storage: 2 DS3400 (24 disks) Server: 1 x3650 M2 Storage: None
Definition for Smart Analytics System 5600 / 5600S Application (Add-on) s Analytics Server: 1 x3650 M2 (option of 1 additional server for HA) Storage: 1 DS3400 (6 disks) BI Server: 2 x3650 M2 (option of up to 4 additional servers for expansion) Storage: 1 DS3400 (6 disks) RSM Server: 1 x3650 M2 Storage: None
Hardware Stack 5600: IBM System x3650 M2 Server 1 quad-core Intel Xeon Processor X5570 293GHz with 8MB L3 Cache 32 GB (8 x 4 GB) PC3-10600-999 1333 MHz ECC DDR3 registered SDRAM DIMMs 2 8Gbps Fibre Channel dual-port PCI-e HBAs for external storage 4 integrated Gigabit Ethernet ports: 2 146 GB 25-inch 15 000 RPM (RAID-1 mirrored) managed by ServeRAID-BR10i SAS/SATA controller IBM System Storage DS3400 300 GB 35-inch 15 000 RPM SAS drives (RAID-6 4+P+Q array)* SMC 8848M Stackable Ethernet Switches IBM System Storage SAN24B-4 Switches * default drive size, may choose from 300GB, 450GB, or 600GB drives 5600S (for core warehouse): IBM System x3650 M2 Server 2 quad-core Intel Xeon Processor X5570 293GHz with 8MB L3 Cache 64 GB (16 x 4 GB) PC3-10600-999 1333 MHz ECC DDR3 registered SDRAM DIMMs 2 8Gbps Fibre Channel dual-port PCI-e HBAs for external storage 4 integrated Gigabit Ethernet ports: 2 146 GB 25-inch 15 000 RPM (RAID-1 mirrored) managed by ServeRAID-BR10i SAS/SATA controller 2 FusionIO 320GB PCI-e SSD drives IBM System Storage DS3400 450 GB 35-inch 15 000 RPM SAS drives (RAID-6 4+P+Q array)* SMC 8848M Stackable Ethernet Switches IBM System Storage SAN24B-4 Switches Changes in blue
Software Stack SUSE Linux Enterprise Server 10 Service Pack 2 InfoSphere Warehouse 971 DB2 Performance Expert 322 IBM Cognos 8 Business Intelligence 841 IBM WebSphere Application Server 7005 IBM Tivoli System Automation for Platforms 3104 IBM Systems Director 612 DS Storage Manager 1060x511 IBM Remote Support Manager (RSM) for Linux 232 IBM Advanced Settings Utility (ASU) 350 Changes in blue
5600S Option for Core Warehouse Foundation Structure Add-On s Foundation + + Data User Failover Application (s) 1 1 to x Nodes 0 to y Nodes 0 or (x+y)/5 Nodes 0 to z s Add-On s Analytics BI RSM 3 rd Party s
Core Warehouse Difference Between 5600 and 5600S IBM Smart Analytic System 5600 X3650 M2 IBM Smart Analytic System 5600S X3650 M2 Quad-core CPU Quad-core CPU Quad-core CPU Core Core Core Core Core Core Core Core Core Core Core Core Memory (32GB) Memory (64GB) OS 146GB 15K 146GB 15K OS 146GB 15K 146GB 15K DB2 Database Partition 1 Fibre HBA DB2 Database Partition 2 DB2 Database Partition 3 Fibre HBA DB2 Database Partition 4 DB2 Database Partition 1 SSD DB2 Database Partition 2 320GB DB2 Database Partition 3 320GB DB2 Database Partition 4 Fibre HBA Fibre HBA SAN Switch 2 SAN Switch 1 SAN Switch 2 SAN Switch 1 DS3400 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K DS3400 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K 300GB 15K DS3400 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K DS3400 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K 450GB 15K
Configuration Overview of 5600
Configuration Overview of 5600S
HA Group 5+1 HA group SAN switch 1 SAN switch 2 Administration node Database partition 0 Data node 1 Database partitions 1, 2, 3, 4 Data node 2 Database partitions 5, 6, 7, 8 Data node 3 Database partitions 9, 10, 11, 12 Data node 4 Database partitions 13, 14, 15, 16 Failover node Each HA group consists of 5 Admin/User/Data s + 1 Failover, and 1 SAN switch pair Each HA group fills up one rack
Netezza Performance Server (TwinFin)
Netezza Eliminates the I/O Bottleneck Move the SQL to the hardware to where the data lives Just send the Answer, not Raw Data
FPGA in the NPS Architecture FPGA in each SPU Auxiliary processor to the CPU Customized instruction set optimized for SQL Flexible for the addition of new instructions Millions of dynamically programmable logic gates We are only using half the logic gates today
The Netezza AMPP Architecture FPGA Memory CPU Advanced Analytics FPGA Memory CPU Host Hosts ETL BI FPGA Memory CPU Loader Disk Enclosures Network S-Blades Fabric Netezza Appliance Applications
FPGA An example usecase select DISTRICT, PRODUCTGRP, sum(nrx) from MTHLY_RX_TERR_DATA where MONTH = '20091201' and MARKET = 509123 and SPECIALTY = 'GASTRO' FPGA Core CPU Core Uncompress Project Restrict, Visibility Complex Joins, Aggs, etc Slice of table MTHLY_RX_TERR_DATA (compressed) select DISTRICT, PRODUCTGRP, sum(nrx) where MONTH = '20091201' and MARKET = 509123 and SPECIALTY = 'GASTRO' sum(nrx)
S-Blade Components SAS Expander DRAM SAS Expander Dual-Core FPGA Intel Quad-Core IBM BladeCenter Server Netezza DB Accelerator
The IBM Netezza TwinFin Architecture Optimized Hardware + Software Purpose-built for high performance analytics Requires no tuning Streaming Data Hardware-based query acceleration Blistering fast results True MPP All processors fully utilized Maximum speed and efficiency Deep Analytics Complex analytics executed in-database Deeper insights
Disk Mirroring and Failover Primary Mirror Temp All user data and temp space mirrored Disk failures transparent to queries and transactions
S-Blade Failover S-Blades Drives automatically reassigned to active S-Blades within a chassis Read-only queries that have not returned data yet automatically restarted Transactions and loads interrupted Loads automatically restarted from last successful checkpoint
TwinFin 24 Specification 16 (8*2) Disk Enclosures 192 (96*2) 1TB SAS Drives (8 hot spares) RAID 1 Mirroring 2 Hosts (Active-Passive): 24 Cores (Quad-Core Intel 26 GHz) 96 GB Memory 4x146 GB SAS Drives Red Hat Linux 5 64-bit 10G Internal Network 24 Netezza S-Blades: 192 Core s ( Intel Quad-Core 25 GHz) 192 FPGA s ( 125 MHz ) 384 GB DDR2 RAM (1+TB compressed) Linux 64-bit Kernel User Data Capacity: Data Scan Speed: Load Speed (per system): 250 TB 290 TB/hr 20 TB/hr Power/Rack: Cooling/Rack: 7,400 Watts 25,500 BTU/Hour