Sungsoo Kang Network Technology Division
Background & review From 10Mb/s to 1 Gb/s Ethernet 10GbE Background & activity Technical detail 10GbE prospective 2
ALOHA Based on contention of broadcast (air) channel Pure ALOHA Slotted ALOHA Packet Packet Packet Packet 3
Extremely inefficient, since the worst-case period of vulnerability is the time to transmit two frames. Collides with the start of the shaded frame Collides with the end of the shaded frame time Vulnerable 4
Invention of Ethernet Robert Metcalf s Idea borrow ALOHA s idea to LAN With adding collision detection by carrier sensing Now, let see MAC protocol Invented by Metcalf at Xerox in 1973 and patented in 1976 Xerox, Digital and Intel jointly made DIX.xx IEEE standard in 1989 of the first Ethernet design 5
Take Slotted ALOHA to the next level, use the slots as contention periods. If no collision occurs before the end of the period, then complete transmission of the frame. CSMA/CD can be in one of three states: contention, transmission, or idle Variable length of packet 6
10,000 Mbps 1,000 Mbps Ethernet Standards Approval Timeline Gigabit Ethernet 802.3z(98) : Fiber, STP 802.3ab(99) : UTP5 10Gigabit Ethernet 802.3ae(02) : Fiber Only 100 Mbps 10 Mbps Xerox Ethernet 2.94 Mbps Ethernet 10BASE5(80) : Thick Coaxial 802.3a&b(85) : Thin Coaxial 802.3i(90) : UTP3(or better) 802.3j(93) : Fiber Fast Ethernet 802.3u(95) : UTP3&5, Fiber 802.3y(97) : UTP3 FDX/Flow control, Auto-negotiation Carrier, Packet bursting 1973 1980 1990 1995 1998 2000 2002 FDX only 7
Change of Market Drivers 6000 Global Voice / Data Crossover 200 Voice and Data Revenue Growth Gbps 4000 3000 5000 2000 1000 0 97 98 99 00 01 voice 02 03 04 05 06 07 data 08 Billions of Dollars 150 100 50 0 1998 1999 2000 2001 2002 voice data Source: A.D. Little Source: The Yankee Group Source: NGN Ventures 2000 8
Circuit traffic : increase 8~10 % per year Packet traffic : increase 200 % per year How do we efficiently collect traffic from Access and Metro Area Networks?? 10 Gig Ethernet might be best candidate 9
Gigabit BW vs. ASP Ethernet SONET 85% 10Gigabit Ethernet : $950(2001), $485(2004) OC-192 SONET : $7,653(2001), $3,143(2004) ASP(Average Selling Price), Normalized to 1Gbps Service. Ethernet SONET Source : Dell Oro Group 10
Most simple way to expand enterprise & service provider s network Can use already installed more than 300 million Ethernet switch port Support all data service Support not only LAN but also MAN, WAN Simple, cheap and fast comparing to competitive technologies Match to MAN/WAN backbone speed(oc-192) Support all link without protocol conversion!! 11
1999 2000 2001 2002 2003 SSG rmed 802.3ae formed Draft 1 Draft 2 Draft 3 Draft 4 Draft 5 PAR draft PAR approved First draft WG ballot LMSG ballot standard 10 GbE Standard draft near complete. Vendors build pre-standard products based on draft Commodity 10 GbE Transceivers Hit the market Commodity 10 GbE Solutions in market 12
Preserve 802.3 Ethernet frame format Preserve 802.3 min/max frame size Full duplex operation only Fiber cabling only 10.0 Gbps at MAC-PHY interface LAN PHY data rate of 10 Gbps WAN PHY data rate of ~9.29 Gbps 13
10Gig Ethernet Reference Model 10GBASE-X 10GBASE-LX4 10GBASE-R 10GBASE-SR 10GBASE-LR 10GbASE-ER 10GBAE-W 10GBASE-SW 10GBASE-LW 10GBASE-EW 10GBASE-W is subset of 10GBASE-R 14
10 GbE 1550 nm 1310 nm 1310 nm 850 nm 1000BaseLX ~ 1300 nm 9 µm SMF, serial 9 µm SMF, serial & WWDM 62.5 µm MMF, WWDM 50 µm MMF, serial 9 µm SMF 50 µm / 62.5 µm MMF GbE 1000BaseSX ~ 850 nm 1000BaseT 1999 (802.3ab) 1000BaseCX Copper 50 µm MMF 62.5 µm MMF 4 pair UTP 5 65 m 300 m 25 m 275 m 550 m 5 km Wiring closet Building backbones Campus backbones > 10 km > 40 km MAN / WAN 15
Ref: IEEE Draft P802.3ae/D 2.3, March 2001 16
PMA Physical Media Attachment Serialization/Deserialization(SerDes) Octet delineation [SONET framing] [Scrambling function for SONET frame (x 7 + x 6 + 1)] PCS Physical Coding Sublayer MAC packet delineation Header control 64B/66B Coding Scrambling function for data (x 43 + 1) PMD Physical Media Dependent interfaces Optics Clock Recovery PCS PMA PMD MAC PHY 17
MAC Ethernet Packet + min. IPG 10 Gbps XGMII 8b 8b 8b 8b 10 Gbps 8b/10b PCS 8b/10b encoder 8b/10b encoder 8b/10b encoder 8b/10b encoder 10b 10b 10b 10b 12.5 Gbps, 4 @ 3.125 Gbps PMA SERDES SERDES SERDES SERDES 12.5 Gbps, 4 @ 3.125 Gbps 18
10GBASE GBASE-R MAC 64b/66b PCS Ethernet Packet + min. IPG 64b 64b 64b 64b 64b 64b 64 bit scrambler 10 Gbps XGMII or XAUI 10 Gbps PMA Sync. Bits (2) 64b SERDES 10.3 Gbps XSBI 10.3 Gbps 19
10GBASE GBASE-W(1) MAC 64b/66b PCS Ethernet Packet + min. IPG 64b 64b 64b 64b 64b 64b 64 bit scrambler Extra IPG 10 Gbps XGMII or XAUI 9.29 Gbps Extra IPG dumped Sync. Bits (2) 64b 9.58 Gbps WIS PMA simplified SONET framer SERDES 9.95 Gbps XSBI 9.95 Gbps 20
10GBASE GBASE-W(2) WIS is to allow to 10GBASE-W equipment to generate Ethernet data streams that may be mapped directly to STS-192c or VC-4-64c streams at the PHY levels, without requiring MAC or higher-layer processing. WIS Frame = STS-192c Frame 576 octets 17280 octets 9 rows RSOH MSOH (STS-192c) Envelope Capacity (STS-192c) SPE IDLE packet IDLE Path Overhead column Fixed stuff Payload Capacity 9 rows IDLE packet PCS data stream 1 63 16640 16704 octets STS-192c = Synchronous Transport Signal level 192, c=concatenated SPE=Synchronous Payload Envelope IDLE 21
Structure of SONET Overhead RSOH pointer MSOH POH J1 B3 1 2 3 193 385 576 A1 A1 A1 A1 A2 A2 A2 A2 J0 Z0 Z0 Z0 B1 E1 F1 X X D1 D2 D3 H1 H1 H1 H1 H2 H2 H2 H2 H3 H3 H3 H3 B2 B2 B2 B2 K1 K2 D4 D4 D5 D5 D6 D5 C2 G1 F2 H4 Z3 Z4 Z5 D7 D7 D8 D8 D9 D8 D10 D10 D11 D11 D12 D11 S1 M1 E2 22
Only minimum SOH process for OC-192c interface Pointer process (H1-H2, H3) Automatic protection (K1,K2) Framing & regenerator (A1, A2, J0, Z0) Error check, Mux error monitor & remote error indicator (B1, B2, M1, S1) Others : un-supported (DCC, order wire, sync. etc.) Only 1349 octets are defined in WIS total:5184 octets Overhead(SOH, LOH) are defined in SONET Also minimum POH process for OC-192c interface STS path signal label (C2) STS path trace (J1) Others : un-supported (H4, F2, Z3, etc.) 23
SONET friendly does NOT mean SONET compliant SONET frame (bits) are SONET compliant No layer 2 bridging required Overhead will be interoperable with existing equipment Does NOT meet SONET jitter requirements match the ITU Grid Does NEED a PHYSICAL layer conversion 24
XGMII is interface between MAC and PHY Short reach interface: XGMII is ~ 7 cm Need to extend reach for many 10GbE port Reconciliation sublayer 36 1 XGMII signals TXD (32 data bits, 4 control bits) TXC (transmit clock) PHY 36 1 RXD (32 data bits, 4 control bits) RXC (receive clock) Station management 1 1 MDC MDIO (serial 2 wire interface) 25
Higher Layer LLC-Logical Link Control MAC Control (Optional) Media Access Control (MAC) XGMII XAUI XGMII Reconciliation XGXS XGXS Physical Coding Sublayer (PCS) Physical Medium Attachment (PMA) Physical Medium Dependent (PMD) MEDIUM Optional XGMII extender PHY 26
XGMII (10G Media Independent I/F) 4 byte-wide lanes with 1 control bit per lane XAUI (10G Attachment Unit I/F) Extends XGMII reach (3 vs. 20 ) 4 differential lanes at 3.125 Gbps XSBI (10G Sixteen-Bit Interface) Based on the OIF SFI-4 interface 16 differential signals at 622-645 Mbps 27
XAUI : 10 Gigabit extended Attachment Unit Interface XGXS : XGMII extender Sublayer Increased reach XGMII is ~ 7 cm XAUI is ~ 50 cm Lower connection count XGMII is 74 wires (2 sets of 32 data, 4 control & 1 clock) XAUI is 16 wires (2 sets of 4 differential pairs) 3.125 Gbaud, 8B/10B encoded over 50cm FR-4 PCB traces CDR-based, 4-lane serial, self-timed interface May be implemented in CMOS, BiCMOS, SiGe Direct mapping of RS/XGMII data to/from PCS http:// grouper.ieee.org/groups/802/3/ae/public/may00/taborek_2_0500.pdf 28
8B/10B transmission code Column stripping across 4 independent serial lanes Identified as lane 0, lane 1, lane 2, lane 3 Perform XAUI lane and interface(lane) synchronization Perform lane-to-lane deskew Provide packet delimiters Perform error control to prevent error propagation 29
10 GbE PMD and distances 300 meter, 850 nm VCSEL 2 Km, 1310 nm Fabry-Perot laser, singlemode 10 Km, 1310 nm DFB laser or VCSEL, singlemode 40 Km, 1310 nm DFB laser, cooled, singlemode 40 Km, 1550 nm DFB laser, singlemode Source: Business Communication Review, April,2000 30
fine wire contact emiconductor sample (GaAs) Conventional LD p-n junction box base of transistor can current Laser beam output Mounting post SIO AlGaAs GaAs MQW AlGaAs Top layer (99.9% reflective) Laser cavity (length) Bottom layer (99.9% reflective) VCSEL Oxid layer Gain regio 31
Comparison with Edge-emit LD Vertical : longitudinal structure Circular : elliptical coupling efficiency 80% : 10% Low threshold currents (typically less than 1 ma) Very stable over temperature performance Yield improvement On wafer : after wafer & individually Trends 850 nm commercially available now 1310 nm just announced ~ 10 Km with range 1550 nm by end of 2001, be announced 32
Muti Source Agreement(MSA) SFP(Small Form-Factor Pluggable) 14 Participating Transceiver Companies Target IEEE 802.3ae, other 10Giga standard Support 4 lane electrical interface www.xgpmsa.org 33
802.3 Layer Model Logical Link Control MAC Control (Opt) Media Access Control Reconciliation Sublayer (XG)MII Physical Coding Sublayer Physical Medium Attachment Physical Medium Dependent MDI Media 34
Pin in Hole 1x9 GBIC SFP Pluggable SFF 35
XENPACK FTRX XGXS 36
MAC It s just Ethernet Maintains 802.3 Frame Format and Size Full Duplex operation only PHY LAN PHY uses simple encoding mechanisms to transmit data on dark fiber & dark wavelengths WAN PHY adds a SONET framing sublayer to enable transmission of Ethernet on SONET transport infrastructure PMD Support distances from 65m on installed MMF to 40km on SMF No copper solution proposed 37
Campus A Campus B 10GbE 10GbE Internet Extranet 10GbE 10GbE Data Center Server Farm 10 GbE in service provider data centers & enterprise LANs Switch to switch Switch to server Data centers Between buildings 38
Metro Location A 10GbE 10GbE Metro Metropolitan Networks 10GbE Enterprises: Enables dark fiber to become a 10 Gbps fat pipe for metro networks Service Providers: enables Gigabit services at costs less than T3 or OC-3 Location B 10GbE 10GbE Location C Remote Servers 10GbE 39
Location A 10GbE DWDM mux MAN DWDM Optical Network Enterprises: 10 GbE enables serverless buildings, remote backup, disaster recovery Service Providers: 10 GbE enables dark wavelength Gigabit services at costs less than T3 or OC-3 Location C Remote Servers 10GbE Location B 10GbE Distance: 100 km 40
Attachment to the optical cloud with WAN physical layer Compatibility with the installed base of SONET OC-192 Interface & link between SP to IXC network can be co-located No need for protocol conversion, traffic remains IP/Ethernet Service Provider Point of Presence (PoP) Carrier Central Office (CO) Carrier Central Office (CO) Service Provider Point of Presence (PoP) 10GbE Optical Transport Optical Transport 10GbE Core DWDM Optical Network National Backbone Carrier DWDM device collocated with SP 10 GbE Switch 41
WAN interconnection Market issue Industry issue What s our next step? 42
Path(s) Line Path Terminating Path Equipment Terminating (PTE) Path Equipment Terminating Path Equipment (PTE) Terminating Equipment (PTE) (PTE) Line Terminating Equipment (LTE) Section Regenerator (STE) Section Line Terminating Equipment (LTE) Path Terminating Path Terminating Equipment Path Terminating Equipment (PTE) Path Terminating Equipment (PTE) Equipment (PTE) (PTE) Local Clock Stratum Clock Local Clock Note: A Line can be longer than two Sections 43
Higher Layers Ethernet Line Terminating Equipment (ELTE) 802.3 MAC Reconciliation Sublayer 64B/66B Physical Coding Sublayer WAN Interface Sublayer 802.3 WAN Physical Medium Attachment 802.3 WAN Physical Medium Dependent Partial WAN Interface Sublayer 802.3 WAN Physical Medium Attachment 802.3 WAN Physical Medium Dependent Layer 1 Path Relay SONET Line Termination SONET Section Termination SONET Physical Media Adaptation 802.3 Link Segment SONET OC-192c λ A The client-side link (private network) uses Ethernet optics and implements part of the WAN Interface Sublayer (terminates section and line overhead) The system-side link (to public network) contains an OC-192c or OC-768 SONET LTE λ B A Layer-1 Path Relay transports the SPE from the client side to the system side and vice versa Source: PMC-Sierra, T1X1.5,2001, May 44
GbE : $1,400 (1999) $550 (2004) NIC : $ 570 (1999) $ 80 (2004) 10GbE : $2,800 (2000) $2,000 (2004) NIC : $3,000 (2001) $1,200 (2004) : Linley Group, June 2001 45
Chip set trend? (10GbE Transceiver Agere 2003 WWDM XAUI $1000 (uncooled DFB LD) 10GEA MEF, RPR 10GEA: 2001 1, 111 / : 2001 10, 60 IEEE802.17, 2003.3 RPR 10GbE : 1550nm, 50Km (CISCO12000 ), $80,000/card Avaya: 850nm/1310nm/1550nm($20,000 ~ $60,000) 2002.01 Ethernet system itself is still not yet 10G NPU, Switch EZchip: 2000.10 $28M funding Xelerated: $12M funding (2001.10) Silicon Access: $85 M funding IBM, 10G switch fabric Ezchip interoperability IEEE802.3ae : (wait for enough prototype) Source: Network World High-speed LANs news letter, Aug., 2001, by Jeff Caruso 46
100,000 Megabits per second 10,000 1998 1,000 1995 100 1983 10 10 Gigabit Ethernet Moore s Law 1990 10BASE-T 2006 2002 Gigabit Ethernet Gigabit Ethernet Fast Ethernet 1980 1985 1990 1995 2000 2005 2010 47
50% of 802.3ae thinks next speed should be 40 GbE People are already talking about combining 10 GbE Serial with 4 Color WDM to provide 40 GbE Ethernet Utilizing 802.3ad (link aggregation) Or, does this mean OC-768-like? But, 100 GbE is not unlikely (year 2006)?? 48
Textbook J. Kadambi, et. al, Gigabit ethernet, prentice hall, 2000 Tanenbaun, Computer Network, 5 th edition, 2000,, 1996, Ohm 3,, 2000, Other useful articles IEEE 802.3ae, http://rouper.ieee.org/groups/802/3/ae/index.html 10GEA White Papers, http://www.10gea.org C.Simoneaux & J. Jewell, MSA Group overview and status report, IEEE 802.3ae interim meeting, Sept., 2000 H. Kaplan & B. Noseworthy, The Ethernet: Evolution from 10 to 10,000 Mbps How it all works!, N+I 2000 Atlanta workshop Optical Ethernet, http://www.optical-ethernet.com/optical/index.htm IEEE 802.3, http://www.ieee802.org/3/efm/index.html 49