Inteligent data center/next generation data center

Transcription

1 Inteligent data center/next generation data center Internet Users' Conference - CUC 2005 Dubrovnik, November , Wednesday, :00-13:00 Josip Zimet Cisco Systems Cisco Public 1

2 List of architectures 2000/2001 AVVID - sla 2001/2002 C a m SAFE pu s security 2002/2003 V3PN 2004/2005 SDN/SWAN -sla -security -Integrated security - collaborative security systems -adaptiive threat defense -standardization - virtualization - b a c k b o n e 2005/2006 IIN: AON/DNA/SONA 2

3 Architectures V L 2000/2001 A AVVID N s - sla C a m p u s b a c k b o n e 2001/2002 SAFE 2002/2003 V3PN - security -sla -security 2004/2005 SDN -Integrated security - collaborative security systems -adaptiive threat defense 2005/2006 SONA -standardization - virtualization - Next generation switches : XL 2950/ /2960/3560 Next Generation routers : XM 2800 ISR Next generation PIX : PIX 5xx PIX 5xxE ASA 55xx Next generation IDS., Next generation Wireless 3

4 List of acquisitions 4

5 Acquisitions related to the DNA Networking ACE,AVS Optical Cat6500 ACNS Cat6500 CSM,.. Cat 4948 Cat6K Blades Application Delivery Security Storage Cat G,SAT IOS Server Provisioning High Performance Computing 5

6 DNA Today Data Network Video Network Voice Network DNA Storage Network Compute Network 6

7 Data Center DNA Products Application Optimization Application Velocity System NETWORKED INFRASTRUCTURE LAYER Data Center Services DC Security DC L4-7 Switching Server Fabric Compute ServicesServices Compute Server Fabric Catalyst Switching Storage Fabric Services Storage Storage Switching Optical Transport Fabric Manager Network Management Application/Content Services VFrame Virtualization and Provisioning INTERCATIVE SERVICES LAYER Wide-Area File Services 7

8 Cisco DNA Infrastructure Evolution/Roadmap Fabric Enabled Integrated Svcs Cat6000 Family Classic Family 32Gb/System FE Server Aggregation L2/L3 Services Storage and Compute Family 1999 Continued 6K Evolution Sup720 CEF256 Family CEF720 Family CEF1440 Family 720Gb/System Accelerated IP Svcs 10/100/1000 and 10GbE high density App Compute Engines 1.4TB/System 8th Gen Fwd Engine Dense 10GbE 256Gb/System Distributed Forwarding L4- L7 Services Integration Security Services MDS9500 Family 1TB+ System Datacenter Storage Virtual SANs Highly Available Modular Scalable OS SFS Family Compute System Low Latency Switching High Density IB Gateway to FC,GbE Virtualization 2004 Next Generation Future 5Tb+/System Datacenter Convergence Virtualization I/O Consolidation

9 The Big Picture - The Cisco Data Center Enterprise Mainframe Connectivity Tape Storage Enterprise Disk Storage Enterprise SAN Switching The Cisco Data Center Virtual Fabrics (VSANs) MDS 9000 Family Embedded Intelligent Storage Services Storage Virtualization Data Replication Svcs Embedded Intelligent Network Services Fabric Routing Svcs Embedded Intelligent Virtualization Services Server Balancing Multiprotocol Gateway Services VPN Termination V Virtual I/O SSL Termination Topspin Family Catalyst 6500 Family Firewall Services Intrusion Detection Grid/Utility Computing Low Latency RDMA Services Clustering Server Farm Switching NAS Enterprise NAS Storage Server Virtualization VFrame WIN Server Fabric Switching Enterprise Grid UNIX UNIX/Windows Servers Blade Servers Virtual Private Server Fabric #1 Virtual Private Virtual Private Server Server Fabric #2 Fabric #3 (Blade-based) 9

10 Business Ready Data Center Architecture to Topology SERVER FARM NETWORK DDOS Guard Catalyst INTELLIGENT INFORMATION Firewall Services 6500 Intrusion Prevention Server Load Balancing SSL Off-load Application Message Services Network NETWORK Attached Appliances Caches NAS EMPLOYEE / PARTNER / CUSTOMER ACCESS NETWORK Application Message Services Gig E, 10 Gig E Internet MPLS VPN IPSEC/SSL VPN INTEGRATED NETWORK SERVICES SERVER FABRIC NETWORK Storage Grid/Utility ServerFabric V Applications Computing Virtualization Low Latency RDMA Adaptive Threat Infiniband Defense Virtual I/O INTEGRATED VIRTUALIZATION SERVICES SFS 3000 Virtual Server Clusters Enterprise Grids STORAGE AREA NETWORK Virtual Fabrics (VSANs) Fabric Assisted Applications Storage Storage Server Virtualization Network Farm Data Replication Server Blade Servers UNIX/NT Servers Mainframes DC DC Interconnect Access FC, FICON, iscsi, FCIP Fabric Services INTEGRATED STORAGE SERVICES Application Optimization MDS 9500 DATA CENTER INTERCONNECT NETWORK SONET/SDH xwdm Metro Ethernet FCIP ONS Storage & Tape Arrays 10

11 Enterprise Data Center Network Topology Server Farm Network DDOS Guard Intrusion Prevention Employee/Partner/Customer Access Network Catalyst 6500 AVS Firewall Services Internet MPLS VPN IPSEC/SSL VPN Application Network Services Server Load Balancing Gig E, 10 Gig E Application Message Services SSL Off-load WAEE Embedded Network Services Server Fabric Network Grid/Utility Computing Server Virtualization Low Latency RDMA Virtual I/O V Infiniband SFS 3000 Embedded Virtualization Services Virtual Server Clusters Enterprise Grids Storage Area Network Virtual Fabrics Storage Virtualization Fabric Assisted Applications Data Replication Services Embedded Storage Services Blade Servers UNIX/NT Servers Mainframes Data Center Interconnect Network SONET/SDH xwdm Metro Ethernet FCIP FC, FICON, iscsi, FCIP ONS MDS 9500 Storage and Tape Arrays 11

12 The Evolution of the Data Center Enterprise Applications AUTOMATION Self-Healing and Self Optimizing Networks Data Network Storage Network VIRTUALIZATION Networked Intelligence And Open Standards Compute Network CONSOLIDATION Networked Resource Pools Business Policies On-Demand Service Oriented Compute Network Storage Storage 12

13 Server Switching Architectural Evolution Server/Compute Consolidation Server and I/O Virtualization Automated Application Services LAN LAN Virtual Servers Grid Fabric Cluster SAN Servers SAN Server Selects Relevant Transport and Infrastructure Diskless Server Hardware Shared, Virtual I/O Hardware Server Images Network Selects Relevant Transport and Infrastructure I/O Network Services Storage Network Automates Resource Allocation Today: Complex servers with Next: Unified fabric enables shared Future: Virtual Resource parallel networks for LAN, SAN, pools of stateless commodity Networks dynamically map apps and Clustering servers to resources 13

14 The Evolution of the Data Center Dis-aggregation SMP Pro: Standard Con: Lots of managed Pro: Reduced # of managed servers, components, lowcomponents, virtual I/O, fast inexpensive performing interconnect standards backplane Network and Storage Virtualization Fibre Channel Ethernet Network Pro: Single Con: Expensive, managed entity, Proprietary server + fast backplane backplane Virtualization Server Virtualization - The Server Switch 14

15 The Evolution of the Data Center Serviceware Layer Cluster Mgmt Cluster Mgmt Virtualized Mgmt Middleware Layer Data Middleware Compute Middleware Web Services Blade Servers Compute-Network Farms Resource Layer Legacy Servers Network Layer 10/100/1000 Ethernet 1990 Session Number CONFIDENTIAL CISCO 2000 GigE/10GigE/40GigE/100GigE 2G/10G FC

16 Cisco DNA Architecture 3rd Party Management and Provisioning Tools Extensibility Layer Protocols and APIs (for Third-Party Management Tools) Control Plane Triggers Policy Enforcement Actions Topology Transparency Switching Fabric Ethernet I/O (LAN) Network InfiniBand RDMA (InfiniBand, 10GigE) High Performance Server-Server Connectivity Servers Fibre Channel Storage (SAN) Storage 16

17 A New Category of Data Center InfrastructureThe Server Fabric Switch Network Switch Storage Switch Clients Storage (SAN) Network Resources Server (Internet, Printer, Server) Server Switch Servers Network Storage 17

18 DNA Virtualization Vision VFrame Web Services Computing Fabric Storage Pool V-LAN Networking V-SAN Storage Management Virtual Server 1 Virtual Server 2 Virtual Server 3 18

19 What Makes The Server Fabric Switch Different? High Performance Server-to-Server Interconnect Virtualization (I/O, Storage, and CPU) RDMA High Bandwidth Low Latency InfiniBand today; Policy-Based Ethernet with Dynamic RDMA and TCP Offload Resource Shared Resources Mapping Across Entire Cluster Routing, Aggregation, Load Balancing App/OS to CPU provisioning Performance and Control 19

20 Server Fabric Switch Applications Why Performance and Control? Server Clustering High Performance Computing (HPC) Enterprise-class HPC Database scalability I/O Virtualization I/O consolidation I/O aggregation Server consolidation Utility or Grid Computing Applications Application provisioning Server re-purposing Server migration 20

21 CPU System Memory Mem Cntlr App Buffer OS Buffer CPU interconnect NIC Server (Host) System Memory Mem Cntlr App Buffer OS Buffer HCA Data traverses bus 3 times Kernel Bypass Model interconnect CPU Server (Host) Host Interconnect CPU Host Interconnect RDMA & OS Bypass, Kernel Bypass Application User Kernel Sockets Layer RDMA ULP TCP/IP Transport Driver Hardware 21

22 Cluster Application Interconnect Application Category Application Characteristics Modeling, Rendering, etc Data Mining Transaction Processing Computationally Complex Batch Cluster Application I/O Intensive Cluster Application Batch I/O Intensive Oracle Web Non-Parallel > 30 usec Standard GbE/10GbE usec Optimized GbE/10GbE < 4-8 usec InfiniBand Message Latency Requirement Interconnect Technology 22

23 InfiniBand Performance Measured Results Application BSD Sockets udapl SRP Async I/O MPI extension Direct Access TCP IP SDP IPoIB 1GE Throughput Latency 10G IB 1 Gb/s 4.1Gb/s 4.5 Gb/s 7.9Gb/s 8Gb/s 8Gb/s usec 30 usec 18 usec 18 usec 8 usec 3.5 usec 23

24 I/O Gateways for Network and Storage Eliminating Technology Islands Server Cluster Single InfiniBand link for: - Storage - Network Cisco MDS 9000 Series Cisco SFS 3012 Fibre Channel to InfiniBand gateway for storage access Two 2-Gbps Fibre Channel ports per gateway Create 10-Gbps virtual storage pipe to each server InfiniBand switches for cluster interconnect Twelve 10Gbps InfiniBand ports per switch card Up to 72 ports total ports with optional modules Single fat pipe to each server for all network traffic Cisco Catalyst 6500 Series Ethernet to InfiniBand gateway for LAN access Six Gigabit Ethernet ports per gateway Create virtual GigE pipe to each server 24

25 Programmability VFrame 1) Server Switch receives policy from VFrame Director or 3rd party software. Virtual Server Policy 2) Based on policy, Server Switch assembles the virtual server Selects server(s) that meet minimum criteria (e.g. CPU, memory) Boot server(s) over the network with appropriate app/os image Creates virtual IPs in servers and maps to VLANs for client access. vip vip vhba vhba vhba Cisco SFS 3012 CPUs LAN SAN Creates virtual HBAs in servers and maps to Zones, LUNs, and WWNNs for storage access 25

26 Grid or Utility Computing Virtual Server CPUs Policy App/OS vhba vhba vhba $3,500 SFS-3000 $3,000 VFrame SFS 3000 GbE App/OS CPUs $2,500 IB HCA $2,000(NIC) FC IB Baselined at 175 Data servers incenter a traditionalvirtualization environment 10% servers in 1:1 HA mode VFrame Software to create virtual servers by provisioning $4,500 applications to CPUs and then mapping to $4,000 VLANs (clients) and LUNs (storage) Cost of Ownership ($Thousands) vip vip Products Used T otal Cost of Ownership in a VFrame Environment $1,500 Catalyst 6500 $1,000 LAN CPUs SAN LAN MDS-9000 $500 Connectivity costs (from MFIO model) Server outage cost ~40% Mulitfabric Server Switch with Fibre Pow er and Channel and Ethernet gatewayscooling to VFrame softw are cost connect unified server fabric w/ LAN (Catalyst) and SAN (MDS)CX4 Cabling cost VFrame and IB Training cost InfiniBand host adapter. Support for Linux IB technology administration and Windows OS and Application cost Provide Connectivity from server Server cost fabric to Clients and VLANs Systems Administration Provide Access from server fabric to App/OS Images stored in SAN $Reference Customers Traditional HA VFrame HA EDS, General Dynamics, Cisco (soon) Model 26

27 VFrame Data Center Administrator Server GRID Policy SFS 3012 Image VFrame picks server with right criteria to run application and boots server GigE Performance FC Security Availability Accounting MDS 9500 Catalyst 6500 SAN CSM Load Balancer FWSM Firewall Servers Application Service Provisioned! VFrame translates policies to actions and passes to infrastructure VFrame identifies right App / OS Image From storage IB VFrame Application: SAP Define application services and pass policy to VFrame Campus/ WAN/VPN VFrame gives new server right VLAN and LUN info so it can find/be found by right clients and storage VFrame provisions security policies to FWSM VFrame provisions CSM to add new server to load balancing pool 27

28 The Next Frontier: Datacenter Management Device and Service Provisioning + Virtualization Policies APIs Management APIs e.g. Tivoli App/OS VFrame CSM, FM,DCM App/OS Virtualization Orchestration Provisioning App/OS Management Compute ApplicationCentric Management Security ServiceOriented Management Storage End-to-End Management Network (L4-7) Management Network (L2-3) 28

29 Horizontal versus Vertical Provisioning Policies VFrame APIs APIs Management App/OS App/OS Virtualization Orchestration Provisioning App/OS Management Compute Management Security Management Storage Networking Management Network Application Services Management Network Connectivity Services 29

30 Datacenter Ecosystem Applications Data Center Mgt Cisco Works Tivoli Virtual Machines P, SNM P S MT SO AP Sun SD Monitoring / Reporting P XML/ SNMP HP Intel Sun AMD Content Switch Cisco Data Center Network and Security Infrastructure Nativ e Ci (CDP sco APIs, etc. ) Catalyst PIX Routers EMC MI Dell CL I, S IBM BMC Tivoli CA -S VFrame SFS Platform FI 3, E Server Partner Infrastructure EMC 1 Int INFRASTRUCTURE Grid/Job Management SAP P, TC ML X MANAGEMENT VMware Microsoft Xen Virtual Iron Altiris OpsWare Veritas Microsoft HP A, M L R D XM Software Updates/ Patch Management Oracle MDS AON MANAGEMENT INFRASTRUCTURE Hitachi IBM o cr Ma PIs A HP SAN Storage Partner Infrastructure NTAP LAN Security 3rd Party Network and Security Infrastructure 30

31 What is InfiniBand? InfiniBand is a high speed low latency technology used to interconnect servers, storage and networks within the datacenter Standards Based InfiniBand Trade Association Scalable Interconnect: 1X = 2.5Gb/s (2Gb/s data) 4X = 10Gb/s (8Gb/s data) 12X = 30Gb/s (24Gb/s data) 31

32 High Performance Server Interconnect InfiniBand Industry Standard Manageability RDMA for Ultra-Low Latency 10Gbps Bandwidth (moving to 30Gbps) Connection Oriented Control GigE, 10GigE, FC, iscsi, etc Myrinet Topspin 600 GigE CPU Utilization 40-50% 5 Q 3 0 Q 2 0 Q Q MBps Node Count Q usec Q 1 0 Q 4 0 Q MBps 4 Topspin 4 GigE Q 4 0 Price Per Port ($) Interconnect Agnostic Storage and I/O 1200 Latency Connection Oriented Partitionable 0 Throughput Boot Over IB Built-in Control Economics Performance Standard 5 usec 1-3% 32

33 Cluster Application Interconnect Application Category Modeling, Rendering, etc Data Mining Transaction Processing Application Characteristics Computationally Complex Batch Cluster Application I/O Intensive Cluster Application Batch I/O Intensive Oracle Web Non-Parallel > 30 usec Standard GbE/10GbE usec Optimized GbE/10GbE < 10 usec InfiniBand Message Latency Requirement Interconnect Technology 33

34 Price / Performance Comparative InfiniBand Offers the Best Price / Performance for HPC InfiniBand PCI-Express Myrinet D Myrinet E 10GbE GbE GbE/RNIC Data Bandwidth (Large Messages) 950MB/s 245MB/s 495MB/s 900MB/s 100MB/s 100MB/s MPI Latency (Small Messages) 5us 6.5us 5.7us 50us 50us 18us HCA Cost (Street Price) $550 $535 $880 $2K-$5K Free $500 Switch Port $250 $400 $400 $2K-$6K $100-$300 $100-$300 Cable Cost (3m Street Price) $100 $175 $175 $50 $25 $25 Note: MPI User Space to User Space latency switch latency is less * Myrinet pricing data from Myricom Web Site (Dec 2004) utilizing Myrinet s latest switches ** InfiniBand pricing data based on Topspin avg. sales price (Dec 2004) *** Myrinet, GigE, and IB performance data from independent June 2004 OSU study **** 10GigE and GigE Cost and Performance data from Cisco Internal document 34

35 InfiniBand Performance Measured Results Application Transparent BSD Sockets Custom API Required udapl SRP Async I/O MPI extension Direct Access TCP IP SDP IPoIB 1GE Throughput Latency 10G IB 1 Gb/s 4.1Gb/s 4.5 Gb/s 7.9Gb/s 8Gb/s 8Gb/s usec 30 usec 18 usec 18 usec 8 usec 4.5 usec 35

36 InfiniBand Protocol Summary Protocol / Application Summary Application Example IPoIB (IP over InfiniBand) Enables IP-based applications to run over InfiniBand transport. SDP (Sockets Direct Protocol) Accelerates sockets-based applications using RDMA. Standard IP-based applications. When used in conjunction with Ethernet Gateway, allows connectivity between IB network and LAN. Communication between database nodes and application nodes, as well as between database instances. SRP (SCSI RDMA Protocol) Allows InfiniBand-attached servers to utilize block storage devices. When used in conjunction with the Fibre Channel gateway, allows connectivity between IB network and SAN. udapl (Direct Access Programming Library) Enables maximum advantage of RDMA flexible programming API. Used for IPC communication between cluster nodes for Oracle 10G RAC. MPI (Message Passing Interface) Low latency protocol used widely in HPC environments. HPC applications. 36

37 The InfiniBand Driver Architecture APPLICATION INFINIBAND NETWORK BSD Sockets BSD Sockets SAN NFS-RDMA UDAPL TCP FILE SYSTEM SDP TS IP SDP TS IPoIB Drivers SCSI TS API ETHER INFINIBAND HCA ETHER SWITCH INFINIBAND SWITCH ETH GW E Kernel DAT SRP FCP VAPI LAN/WAN User FS API FC FC GW UNIFIED FABRIC FC SWITCH SAN 37

38 IP over InfiniBand Transmission of IP over Infiniband Use IB as a link layer for IP Define data link and link layer address Encapsulation for ARP, IPv4 and IPv6 Address resolution Transport IP multicast over IB Provides highest level of application compatibility. Applications do not need to be re-written or re-compiled Standard IP utilities and applications work as usual: Ifconfig, ping, telnet, File sharing (NFS, CIFS); Login access (ssh, telent, etc); Cluster heartbeat DHCP over IB IP over InfiniBand MIB 38

39 How IP over InfiniBand works 10Gbps 10Gbps Ethernet GigE Link Link IB Link IB Link Server Switch InfiniBand-only Switch Application TCP Application TCP Session IP IP over IB TCP IP IP Connection IP IP IP over IB Verbs API Verbs API IB Transport IB Transport IB Network IB Network IB Link IB Link IB Link IB Link Ethernet Link Eth Link Eth Link Ethernet Link IB Physical IB Phy IB Phy IB Physical Ethernet Physical Eth Phy Eth Phy Ethernet Physical Server TCP / IP Layers in Software TS-360 TS-120 Infiniband layers in Software Infiniband Layers in Hardware Ethernet Switch Desktop Ethernet Layers in Hardware * Notes: Uses standard Berkeley TCP/IP libraries 39

40 Sockets Direct Protocol Sockets Direct Protocol Runs socket based TCP/IP traffic with TCP and copy offload Highly configurable: By process By port By destination By environment variable No application recompile or rework necessary Zero copy capability using Asynchronous I/O (AIO) 40

41 Tangible Benefits of Server Fabric Switching Purchase 20X more compute power for same dollars (pay as you grow, moore s law, expense vs capitalization) 50% cost savings from resource consolidation delivers instantaneous ROI (Single Server Fabric- eliminates adapter, cables, ports) Dramatically Reduce TCO - Manage enterprise-wide Server GRID centrally (wire once, control servers over the network) Provision New Servers in seconds, not days (or weeks) Help Eliminate Server Downtime (Failover provision, add/remove I/O or storage bandwidth on the fly) Control Ballooning Investments in Real Estate, Power & Cooling (capitalize on dense server packaging and Blade architectures) Political Power and Self-Rule for the Server Team (Eliminate dependence on other teams to get apps provisioned quickly) 41

42 Cisco DNA Impact : Improved Server Utilization VFrame Provisioning 60+% Server Utilization ~30% Server Reduction 42

43 Dramatic Cisco DNA Impact: Application Acceleration Improvement on Response Times Application Software Before After Transaction Time Reduction Call center PeopleSoft 63 sec 23 sec 63% ( 270%) JIT manufacturing SAP 76 sec 22 sec 71% ( 350%) Store management IBM WebSphere 46 sec 16 sec 66% ( 290%) Claims management IBM WebSphere 42 sec 19 sec 55% ( 220%) Collaboration Lotus inotes 90 sec 28 sec 68% ( 320%) Employee portal Plumtree 204 sec 59 sec 71% ( 350%) 43 sec 6 sec 85% ( 670%) Portal consolidation SunOne, Vignette Employee portal PeopleSoft 103 sec 32 sec 69% ( 320%) CRM Siebel 389 sec 133 sec 66% ( 290%) Notes: Bandwidth reduction averages 80-90% All timings are customer-verified using either LoadRunner, FineGround AppScope, or in-house customer tools 43

44 Topspin Building Blocks Gateway Modules - InfiniBand to Ethernet - InfiniBand to Fibre Channel Switches Host Channel Adapter (HCA) With upper layer protocols Integrated System and Fabric management SRP SDP udapl MPI IPoIB Linux and Windows driver support 44

45 VFrame Server Fabric Virtualization Software R3.0 InfiniBand Multifabric Server Fabric Switch Software The Cisco SFS Product Line SFS 3001 (TS90) SFS 3012 (TS360) (12) 4XIB + 1 Gw (24) 4XIB + 12 Gws SFS 7000 (TS120) (24) 4XIB (2) 2G FC GW (6) GE GW SFS 7008 (TS270) (96) 4XIB SFS 7012 (TS540) (144) 4XIB SFS 7024 (TS740) (288) 4XIB HCA Blade Server IBM BladeCenter HCA (2) 1XIB PCI-X Embedded switch (14) 1XIB (Internal) + (1) 4XIB and (1) 12XIB (External) (2) 4XIB PCI-X (2) 4XIB PCI-ex *plus InfiniBand Cables Dell 1855 HCA (2) 4XIB PCI-ex Passthru Module (10) 4XIB Remote Boot Linux Host Driver Windows Host Driver 45

46 Cisco InfiniBand Blade Switch Modules For IBM eserver BladeCenter IB switch module For Dell PowerEdge 1855 Blades IB pass through module 1855 chassis BladeCenter chassis IB daughter card Plug one card into each server blade Plug one or two switch modules into chassis Each server blade gets one or two 1x IB (2.5Gbps) connections Target markets: HPC, Multifabric I/O (MFIO), On-Demand data centers See IBM Redbook for more details: IB daughter card Plug one daughter card into each server blade Plug one or two pass through modules into chassis Each server blade gets one or two 4x (10Gbps) IB connections Target markets: HPC, Multifabric I/O (MFIO), Scalable Enterprise centers 49.html?Open 46

47 Case Study: Leading Research Facility High Performance Computing Cluster Application: Core Fabric: 6x SFS 7008 (TS270) High Performance Computing Cluster Compute time outsourced to Commercial Enterprises (major oil & gas) Environment: 174 Uplink cables Edge Fabric: 29x SFS 7000 (TS120) 520 Dell Servers 3:1 Blocking ratio 6x SFS 7008 (TS270) 29x SFS 7000 (TS120) NCSA Tungsten Node Supercomputer Benefits Compelling Price & Performance Measured MPI latency 5.2µs 47

48 Case Study: Large Wall Street Bank Enterprise Grid Computing Application: HDS Storage Replace proprietary platforms with standards-based components Build scalable on-demand compute grid for financial applications LAN Existing N/W Catalyst Switch Environment: 500+ Intel Servers per slice Topspin Server Switch with Ethernet and Fibre Channel Gateways Grid I/O SFS 3012 (TS360) 96 ports Core Fabric SFS 7008 (TS270) Hitachi RAID Storage SAN Switches Ethernet Switches Benefits: 20X Price/Performance Improvement over four years 30-50% Application Performance Improvement Standards-based solution for on-demand computing Environment that scales using 500-node building blocks Edge 44x 24-port SFS 7000 (TS120) 512 Nodes 12 hosts 12 hosts 12 hosts 12 hosts 48

49 Case Study: Major System Vendor Utility Computing Service Application: Build scalable on-demand compute service for enterprise customers (license $/CPU) Key initiatives around Financial Services and Energy verticals 768 Nodes currently deployed 1024x Sun V20z Nodes 22x SFS 7008 (TS270) Environment 1024x Sun V20z Nodes 34 TS270 Server Fabric Switches Non-Blocking CLOS Network 12x SFS 7008 (TS270) 8 TS360 s with Gateways Sun Storage Enterprise-Class Reliability Benefits: Ability to outsource computing services to many customers with common infrastructure 8x SFS 3012 (TS360) with IB to FC Gateways FC SAN Sun Storage Arrays 49

50 Case Study- Large Government Lab Worlds 2nd Largest Super Computer Application: High Performance SuperComputing Cluster Environment: 8x SFS 7008 (TS270) 288 ports each Core Fabric 4096 Dell Servers 2048 uplinks (7m/10m/15m/20m) 50% Blocking Ratio 8 TS 740s Edge 256 TS120s 256x SFS 7000 (TS120) 24-ports each Benefits: Compelling Price/Performance Largest Cluster Ever Built (by approx. 2X) Expected to be 2nd Largest Supercomputer in the world 18 Compute Nodes) Dell 1850 Servers Dual 3.6Ghz EMT64 6GB Cisco PCIe HCA 18 Compute Nodes) 8192 Processor 60TFlop SuperCluster 50

51 Leading UK Telecom Provider Oracle 10g Deployment (BT) Broadband billing application Server 12 identical regional deployments Running Oracle 10g No single-point of failure Each server has a single HCA, with ports dual connected to two TS90s 10Gbps IB SFS 3001 SFS Gbps FC Storage Server Storage 51

52 Sandia National Labs 4096 Nodes Cluster Application: High Performance SuperComputing Cluster Environment: 8x SFS ports each Core Fabric 4096 Dell Servers 50% Blocking Ratio 8 SFS SFS 7000 s Edge 256x SFS ports each Benefits: Compelling Price/Performance Largest IB Cluster ever built Expected to be 3rd Largest Supercomputer in the world 18 Compute Nodes) Dell 1850 Servers Dual 3.6Ghz EMT64 6GB Cisco PCIe HCA 18 Compute Nodes) 8192 Processor 60TFlop SuperCluster 52

53 Oracle 10g: Broad Scope of IB Benefits Intra RAC: IPC over IPoIB FC gateway: host/lun mapping Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analy si s Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analy si s Ethernet Gateway Sniff er Ser ver m onit ori ng/ analy si s Sniff er Ser ver m onit ori ng/ analy si s OracleNet: SDP over IB Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s SAN Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver Network m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Shared Storage Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Sniff er Ser ver m onit ori ng/ analysi s Consolidate and share Storage amongst servers Application Servers Oracle 10g RAC 20% improvement in throughput 2x improvement in throughput and 45% less CPU 30% improvement in DB performance 53

54 Bio-Informatics Cluster: 1,066 Node Supercomputer 1,066 Fully Non-Blocking Fault Tolerant IB Cluster Fault Tolerant Core Fabric port TS-270 1,068 5m/7m/10m/15m uplink cables port TS-120 Edge Fabric 1,066 1m cables 12 Compute Nodes 12 Compute Nodes Key decision factors: Topspin benchmarked and tuned customer MPI application Best operational experience with large clusters best references Rapid Service architecture proved 2-min vs. 2-day MTTR. 54

55 Cisco InfiniBand Landscape Vendors working with Cisco Server Switches IB Fabric SFS FC GbE 55

56 Topspin and Top Tier Server Vendors IBM and Topspin Communications Forge Key Agreement Sun and Topspin Partner to Deliver Grid Computing Solutions Dell Adds Topspin Switches to High-Performance Computing Clusters Topspin Selected As NEC's Strategic InfiniBand Technology Provider HP To Leverage Topspin Technology 56

57 Who Owns the Datacenter? Applications I own Network THE NETWORK I own Server I own Storage I own Mainframe 57

58 Thank You! Cisco Public 58