Toward a unified architecture for LAN/WAN/WLAN/SAN switches and routers Silvano Gai 1
The sellable HPSR Seamless LAN/WLAN/SAN/WAN Network as a platform System-wide network intelligence as platform for applications/services and resource virtualization Chassis based crossbar architecture Routing, Switching, Software, Services Dedicated linecards Ethernet, Sonet, DWDM, Fibre Channel, Wireless, Security, Module Carrier, Telephone Gateway, Virtualization, Caching, Computing Global manageability/provisioning High Availability Software Modularity/API 2
Important Characteristics Reliable, 24x7, We must be always on Very high availability/non-stop operation Modern SW architecture Distributed/Modular OS Non Disruptive Software Upgrade Hitless firmware update Virtual Routers support 3
True High Availability Platforms Hardware Availability Availability inside the system.. Resiliency throughout implementation Dual OOB management channels Non-disruptive software upgrades Dual System Clocks Modular software for resiliency Majority voting for critical signals Stateful process re-startability Power and Cooling management Stateful failover of supervisors Environmental monitoring No re-learning on fail-over System health monitor Dual Supervisors and Cross-bars 9 Fans POSIX Processes IPC Modular Line Cards Dual Power Supplies State Synchronization PSS PSS HA Platform Distributed frame forwarding HA Platform Software Availability POSIX Processes IPC Kernel Kernel Primary Supervisor Stateful Standby Supervisor 4
Agenda Introduction Fabric Forwarding Engine Line Cards Service Cards Fibre Channel Requirements 5
A Few Basic Architectures Bus Central Memory Mesh Two-Tier Cross-bar 6
Advantages of Cross-bar Scalable Cross-Bar Multi-speed capable Non-blocking Efficient Line Card Line Card Line Card Line Card Line Card 7
Cross-Bar IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 Cross-Bar OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 8
10 Times 1 Does Not Equal 1 Times 10 Switch A Switch B Switch A Switch B 10 Links @ 1Gbps Each Bandwidth = 10Gbps Flow Bandwidth = 1Gbps Serialization Delay = 20uS 1 Link @ 10Gbps Each Bandwidth = 10Gbps Flow Bandwidth = 10Gbps Serialization Delay = 2uS 9
Cross-Bar IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 IN 1 IN 2 IN 3 IN 4 IN 5 IN 6 IN 7 IN 8 Cross-Bar OUT 1 OUT 2 OUT 3 OUT 4 OUT 5 OUT 6 OUT 7 OUT 8 10
Example of Today Cross-bar 20 Gbps Each 60 Gbps Each 60 Gbps Each 20 Gbps Each FIFO 1 FIFO 1 FIFO 2 FIFO 2 FIFO 3 Classic Cross-Bar 2.16Tbps FIFO 3 FIFO 18 FIFO 18 Buffered Cross-Bar with 3X Over-Speed 11
Possible improvements Crossbar are designed with Serial Links Today 3-1/8 GHz, i.e 2.5 Gb/s To obtain 20 Gb/s full duplex it requires 16 differential pairs 40 + 40 Gb/s per slot Tomorrow Integrated SerDes Moving from 3-1/8 to 6-1/4 40Gb/s on the same number of pair 80 + 80 Gb/s per slot enough for a while due to front pannel port limitation (8 10Gb/s Ethernet or FC or 8 OC192) Eye openers to extend the backplane 12
Add Missing Components MAC FWD MAC MAC FWD MAC MAC FWD MAC Port 13
Re-Draw FWD FWD FWD MAC MAC MAC 14
Agenda Introduction Fabric Forwarding Engine Line Cards Service Cards Fibre Channel Requirements 15
Requirement for a Forwarding Engine IPv4 and IPv6 forwarding With Multicast Support MPLS with ATOM GRE, L2TP, IP in IP Ethernet forwarding Extremely expensive Sometime it can be shared across linecards 16
Next-generation ASIC requirements More logic gates More integration of memories on-chip Higher clock frequencies Wider on-chip busses Higher external I/O bandwidth Better power density Larger chip packages 17
ASIC Feature Size Evolution Feature size (drawn) (mm) Qual. Year Usable Gates (M) DRAM density (Mbit/mm 2 ) Gate delay (ps) Power (nw/mhz/gate) Core Voltage Metal layers 0.25 1999 10-50 50 2.5/1.8V 5/Al Today 0.18 (0.15) 2000 24 0.81 33 20 1.8V 6/Cu 0.13 (0.10) 2002 40 1.5 27 9 1.2V/1.5V 7/Cu Future 0.09 (0.07) 0.065 2004 2005? 72 144 2.9? 21 17 6? 0.7V-1.3V 0.8V-1.0V 7/Cu? Source: IBM SA-12E, SA-27E, Cu-11, Cu-08 18
Increasing the clock speed Standard ASIC development flows are very limiting and generally conservative (but hide a lot of complexity from the user) Customer Owned Tooling (COT) allows the customer more control over the decisions COT allows very fine optimisation of gate placement, gate selection, drive strengths, etc Leads to potential speed increases of 4-5x The extreme case of this is full-custom (e.g. Intel) 19
Custom circuit design (COT) A =!(B & C) 2-input NAND gate 20
Agenda Introduction Fabric Forwarding Engine Line Cards Service Cards Fibre Channel Requirements 21
Native Line Cards Ethernet 10/100/1000 Ethernet 10 Gb/s OC-3, 12, 48, 192 DWDM Fibre Channel 1, 2, 10 Gb/s 22
Requirement for buffering 100 Mb/s Cheap LAN switch Few KBs Good LAN switch.1 MB Carrier Class Router 2-8 MB 1 Gb/s 8 64 KB.5 MB 16-64 MB 10 Gb/s N/A Few MBs 256-512 MB 23
Agenda Introduction Fabric Forwarding Engine Line Cards Service Cards Fibre Channel Requirements 24
Security Modules Dedicated line cards for Firewall VPN termination Intrusion Detection Content Switching SSL Services Network Analysis & Monitoring 25
IP telephony modules Ethernet Inline Power (48-volt DC power) Digital T1 or E1 PSTN and PBX gateways Transcoding (G.711, G.729a, G.723) Conference bridging Analog Interface Module - foreign exchange station (FXS) 26
WLAN Wireless Service Card Simple Radio conversion Encryption Ethernet Protocol Termination 27
Agenda Introduction Fabric Forwarding Engine Line Cards Service Cards Fibre Channel Requirements 28
Additional Requirements for FC Director Credit Management No packet drop Throughput Port count Modular No single point of failure In-order delivery 29
Becomes This ARB 30
Re-Draw ARB FWD FWD FWD MAC MAC MAC 31
32 2003, 2001, Cisco Systems, Inc. All rights reserved.