150000 100000 50000 0 Trident+ Trident II NG 300 200 100 IPv4 FIB LPM IPv6 FIB LPM 0 Trident+ Trident II or + NG LAB Data Cente r Internet WAN Bandwidth in 10G Increment 40GE Ports 10GE Ports 100GE Ports Core Routing Module Building Aggregation 20 30 Buildings Building Aggregation 20 30 Buildings Building Building Building Building Today: MPLS, OSPF, BGP Future: OSPF, BGP Today: L2VPN, L3VPN Labs Security Devices Developer Internet Business Partners Others Future: VXLAN
Use Cases Simple Connectivity Over an IP Only Network Instant Connectivity From Service Source & Destination Tenant in a Data Center with an IP Only L3 Clos Fabric. Cost Efficient Service Extender Over a 3 rd Party IP Network Use Cost Efficient Ports When you Can and Expensive Ports Where you Must Cheap Pseudo Wires without MPLS API Driven Provisioning: Simplified and Easily Consumed by Developers Removal of additional Protocols to Manage Reduction in Cost, Driven by Merchant Silicon & Competition Remote Tap Termination Transport Wire Tap (SPAN) Aggregation Over an IP Network Lab Patch Panel Anywhere to Anywhere Patch Panel for Labs
Why VXLAN VXLAN for this presentation is not utilized for its intended use-case. NVGRE NOTE: - Utilized in Other Parts of Microsoft - Used in Overlay Network Creation UDP Header: Hashing Works Just Like Any Other UDP Traffic Supported in Wide Variety of Platforms Broadcom Trident 2: Arista Networks, Cisco Systems, Dell, Juniper Networks White Label: Bring Your Own Switch (BYOS) & Purchase Operating System (Cumulus & Others to Follow) Widely Deployed in Many Networks Dense, Power/Space Efficient & Cheap $60 100 per 10GE, Sometimes Better Supports Point to Point & Multi-Point Point to Point Doesn t Require Control Plane Software Multi-Point Does Require Control Plane Software or Self Provided Control Plane Software Wide Vendor Support & Interest
What Worked & Didn t Worked Tagged Packets (VLAN) VRRP OPSFv2, OSPFv3 BGP Address Family: IPv4, IPv6, VPNv4 RSVP NOTICE: Didn t Work or In Progress LACP Doesn t Pass Through (Ethernet Channel) LLDP Doesn t Pass Through, but works locally LDP (Possibly 224.0.0.2/1 Multicast Issue & Interception) IS-IS (Investigating) Continuously Investigating What Else Doesn t Work and What Can Work or be Fixed
Use Case: Lab Patch Panel Today: MPLS L2VPN Future: T2 Based Patch Panel Puget Sound Campus: 100+ Building Distributed Labs Building F Why VXLAN: IP Only is Cheap & MPLS is Expensive Wide Vendor Choice UDP: Hashing Less Protocols & Dependencies API Driven + Apps (Self Service with Windows Phone & Tablet) Building A Building E Building B Building D Building C
Use Case: VRF Extension Any T2 VTEP / SPINE Any T2 VTEP / SPINE
Use Case: TAP Remote Transport Reasons & Benefits: Limit Expensive Tools Deployment Redirect Captured Traffic for Personal Analysis No Tools in Local Location SPAN to VXLAN Transport Yet another RSPAN, but IP based Tap Output Loopback Cable VXLAN Input Port Terminating to Remote VXLAN Switch
Lab Environment: Physical Connectivity N3132Q Underlay Network IP Fabric N3132Q Overlay Network.10 switch
interface TenGigabitEthernet 0/0 ip address 192.168.254.0/31 interface TenGigabitEthernet 0/1 ip address 192.168.254.2/31 interface TenGigabitEthernet 0/2 ip address 192.168.254.4/31 interface TenGigabitEthernet 0/3 ip address 192.168.254.6/31 interface Loopback 0 ip address 192.168.254.100/32 router ospf 1 router-id 192.168.254.100 network 192.168.254.0/24 area 0 interface Ethernet1/49/1 no switchport ip address 192.168.254.1/31 interface Ethernet1/49/2 no switchport ip address 192.168.254.3/31 interface Ethernet1/49/3 no switchport ip address 192.168.254.5/31 interface Ethernet1/49/4 no switchport ip address 192.168.254.7/31 interface Ethernet1/50/1 no switchport ip address 192.168.254.9/31 Underlay Network Configuration interface Ethernet1/50/2 no switchport ip address 192.168.254.11/31 interface Ethernet1/50/3 no switchport ip address 192.168.254.13/31 interface Ethernet1/50/4 no switchport ip address 192.168.254.15/31 feature ospf router ospf 1 router-id 192.168.254.150 network 192.168.254.0/24 area 0.0.0.0 interface TenGigabitEthernet 0/0 ip address 192.168.254.8/31 interface TenGigabitEthernet 0/1 ip address 192.168.254.10/31 interface TenGigabitEthernet 0/2 ip address 192.168.254.12/31 interface TenGigabitEthernet 0/3 ip address 192.168.254.14/31 interface Loopback 0 ip address 192.168.254.200/32 router ospf 1 router-id 192.168.254.200 network 192.168.254.0/24 area 0
Overlay Network Configuration Part 1 Client Facing Base Client Facing Base feature vxlan gateway-ip 192.168.254.100 controller 1 10.37.33.35 port 6632 ptcp interface ManagementEthernet 0/0 ip address 10.37.33.35/24 management route 10.0.0.0/8 10.37.33.1 management route 172.16.0.0/12 10.37.33.1 interface TenGigabitEthernet 0/116 no ip address interface TenGigabitEthernet 0/117 no ip address interface TenGigabitEthernet 0/118 no ip address interface TenGigabitEthernet 0/119 no ip address feature vxlan gateway-ip 192.168.254.200 controller 1 10.37.33.35 port 6632 ptcp interface ManagementEthernet 0/0 ip address 10.37.33.36/24 management route 10.0.0.0/8 10.37.33.1 management route 172.16.0.0/12 10.37.33.1 interface TenGigabitEthernet 0/8 no ip address interface TenGigabitEthernet 0/116 no ip address interface TenGigabitEthernet 0/117 no ip address interface TenGigabitEthernet 0/118 no ip address interface TenGigabitEthernet 0/119 no ip address
Overlay Network Configuration Part 2 Controller: Flow Provisioning S6000-1 Controller: Flow Provisioning S6000-2 Creation of Logical Network: vtep-ctl --db=tcp:10.37.33.35:6632 add-ls "LN1:ELINE:1001" vtep-ctl --db=tcp:10.37.33.35:6632 add-ls "LN2:ELINE:1002" vtep-ctl --db=tcp:10.37.33.35:6632 add-ls "LN3:ELINE:1003" vtep-ctl --db=tcp:10.37.33.35:6632 add-ls "LN4:ELINE:1004" vtep-ctl --db=tcp:10.37.33.35:6632 add-ls "LN7:ELINE:1007 Creation of Logical Network: vtep-ctl --db=tcp:10.37.33.36:6632 add-ls "LN1:ELINE:1001" vtep-ctl --db=tcp:10.37.33.36:6632 add-ls "LN2:ELINE:1002" vtep-ctl --db=tcp:10.37.33.36:6632 add-ls "LN3:ELINE:1003" vtep-ctl --db=tcp:10.37.33.36:6632 add-ls "LN4:ELINE:1004" vtep-ctl --db=tcp:10.37.33.36:6632 add-ls "LN7:ELINE:1007 Bind Port to the Logical Network: vtep-ctl --db=tcp:10.37.33.35:6632 bind-ls "10.37.33.35" "Te 0/116" 0 LN1:ELINE:1001 vtep-ctl --db=tcp:10.37.33.35:6632 bind-ls "10.37.33.35" "Te 0/117" 0 LN2:ELINE:1002 vtep-ctl --db=tcp:10.37.33.35:6632 bind-ls "10.37.33.35" "Te 0/118" 100 LN3:ELINE:1003 vtep-ctl --db=tcp:10.37.33.35:6632 bind-ls "10.37.33.35" "Te 0/118" 200 LN4:ELINE:1004 vtep-ctl --db=tcp:10.37.33.35:6632 bind-ls "10.37.33.35" "Te 0/119" 0 LN7:ELINE:1007 Tunnel Termination to the Remote Node: vtep-ctl --db=tcp:10.37.33.35:6632 add-ucast-remote "LN1:ELINE:1001" "00:11:11:11:11:11" "192.168.254.200" vtep-ctl --db=tcp:10.37.33.35:6632 add-ucast-remote "LN2:ELINE:1002" "00:33:33:33:33:33" "192.168.254.200" vtep-ctl --db=tcp:10.37.33.35:6632 add-ucast-remote "LN3:ELINE:1003" "00:55:55:55:55:55" "192.168.254.200" vtep-ctl --db=tcp:10.37.33.35:6632 add-ucast-remote "LN4:ELINE:1004" "00:77:77:77:77:77" "192.168.254.200" vtep-ctl --db=tcp:10.37.33.35:6632 add-ucast-remote "LN7:ELINE:1007" "00:14:14:14:14:14" "192.168.254.200" Bind Port to the Logical Network: vtep-ctl --db=tcp:10.37.33.36:6632 bind-ls "10.37.33.36" "Te 0/116" 0 LN1:ELINE:1001 vtep-ctl --db=tcp:10.37.33.36:6632 bind-ls "10.37.33.36" "Te 0/117" 0 LN2:ELINE:1002 vtep-ctl --db=tcp:10.37.33.36:6632 bind-ls "10.37.33.36" "Te 0/118" 100 LN3:ELINE:1003 vtep-ctl --db=tcp:10.37.33.36:6632 bind-ls "10.37.33.36" "Te 0/119" 200 LN4:ELINE:1004 vtep-ctl --db=tcp:10.37.33.36:6632 bind-ls "10.37.33.36" "Te 0/8" 0 LN7:ELINE:1007 Tunnel Termination to the Remote Node: vtep-ctl --db=tcp:10.37.33.36:6632 add-ucast-remote "LN1:ELINE:1001" "00:22:22:22:22:22" "192.168.254.100" vtep-ctl --db=tcp:10.37.33.36:6632 add-ucast-remote "LN2:ELINE:1002" "00:44:44:44:44:44" "192.168.254.100" vtep-ctl --db=tcp:10.37.33.36:6632 add-ucast-remote "LN3:ELINE:1003" "00:66:66:66:66:66" "192.168.254.100" vtep-ctl --db=tcp:10.37.33.36:6632 add-ucast-remote "LN4:ELINE:1004" "00:88:88:88:88:88" "192.168.254.100" vtep-ctl --db=tcp:10.37.33.36:6632 add-ucast-remote "LN7:ELINE:1007" "00:15:15:15:15:15" "192.168.254.100"
Overlay Network Configuration Part 3 root@mx80# run show vrrp logical-system RTR1 Interface State Group VR state VR Mode Timer Type Address xe-0/0/0.0 up 1 master Active A 0.588 lcl 192.168.1.2 vip 192.168.1.1 root@mx80# run show vrrp logical-system RTR2 Interface State Group VR state VR Mode Timer Type Address xe-0/0/1.0 up 1 backup Active D 3.130 lcl 192.168.1.3 vip 192.168.1.1 mas 192.168.1.2 N3132Q-2# ping 192.168.1.1 vrf VRF1 packet-size 1422 count 3 df-bit PING 192.168.1.1 (192.168.1.1): 1422 data bytes 1430 bytes from 192.168.1.1: icmp_seq=0 ttl=63 time=2.233 ms 1430 bytes from 192.168.1.1: icmp_seq=1 ttl=63 time=1.922 ms 1430 bytes from 192.168.1.1: icmp_seq=2 ttl=63 time=2.079 ms --- 192.168.1.1 ping statistics --- 3 packets transmitted, 3 packets received, 0.00% packet loss round-trip min/avg/max = 1.922/2.077/2.233 ms RTR2 { interfaces { xe-0/0/1 { unit 0 { family inet { address 192.168.1.3/24 { vrrp-group 1 { virtual-address 192.168.1.1; priority 90; accept-data; RTR1 { interfaces { xe-0/0/0 { unit 0 { family inet { address 192.168.1.2/24 { vrrp-group 1 { virtual-address 192.168.1.1; accept-data; feature interface-vlan interface Ethernet1/2/1 switchport access vlan 300 spanning-tree port type edge speed 10000 interface Ethernet1/2/2 switchport access vlan 300 spanning-tree port type edge interface Vlan300 vrf member VRF1 ip address 192.168.1.10/24 switch
Verification Part 1 S6000-1#sho vxlan Instance : 1 Admin State : enabled Management IP : 10.37.33.35 Gateway IP : 192.168.254.100 MAX Backoff : 30000 Controller 1 : 10.37.33.35:6632 ptcp (connected) Fail Mode : non secure Port List : Te 0/8 Te 0/9 Te 0/10 Te 0/11 Te 0/116 Te 0/117 Te 0/118 Te 0/119 Te 0/120 Te 0/121 Te 0/124 S6000-1#sho vxlan logical-network Instance : 1 Total LN count : 7 Name VNID LN1:ELINE:1001 4656 LN2:ELINE:1002 4657 LN3:ELINE:1003 4658 LN4:ELINE:1004 4659 LN7:ELINE:1007 4660 LN5:ELINE:1005 4661 LN6:ELINE:1006 4662 S6000-1#sho vxlan logical-network name LN1:ELINE: 1001 Name : LN1:ELINE:1001 Description : Tunnel Key : 4656 VFI : 28673 Port Vlan Bindings: Te 0/116: VLAN: 0 (0x80000001), S6000-1#show vxlan physicallocator Instance : 1 Tunnel : count 1 192.168.254.200 : vxlan_over_ipv4 (up) S6000-1#sho vxlan unicast-mac-remote Total Local Mac Count: 7 VNI MAC TUNNEL 4656 00:11:11:11:11:11 192.168.254.200 4657 00:33:33:33:33:33 192.168.254.200 4658 00:55:55:55:55:55 192.168.254.200 4659 00:77:77:77:77:77 192.168.254.200 4660 00:14:14:14:14:14 192.168.254.200 4661 00:99:99:99:99:99 192.168.254.200 4662 00:12:12:12:12:12 192.168.254.200 S6000-1#sho vxlan statistics interface te 0/116 0 Port : Te 0/116 Vlan : 0 Rx Packets : 326100 Rx Bytes : 23360185 Tx Packets : 91579 Tx Bytes : 8487292
Verification Part 2
Summary Operationally Simple Ethernet Line Services Eliminate Protocol Dependency & Complexity Expand Vendor Choice Practical & Economical Service Deployments Deploy Services Over Legacy Networks or IP Only Networks Deploy Services Over Other Providers Networks Future Use Cases in the Works: ELINE Health Status Details Encapsulation / De-encapsulation Gateway Bridging of Overlay Networks Hub and Spoke ELINE Services More Mobile Management Applications to deliver Self Service Pass Through Encryption Module
- IPv4 & IPv6 VRRP - IPv4: TAG + OSPF + BGP - IPv6: TAG + OSPFv3 + BGP - Ethernet Bundle: TAG + IPv4 - MPLS: RSVP+ Remote PE - OPEN: N3132Q-2: VRF1 N3132Q-2 MX80: LSYS1 MX80 MX80: LSYS2 S6000-1 N3064 S6000-2 QFX5100-48s-6q PE MX80: LSYS3 N3132Q-1 N3132Q-2: VRF2 MX80: LSYS4 N3132Q-2: VRF3