All-in-One CCIE Data Center V Written Exam Cert Guide

Transcription

1

2 2 All-in-One CCIE Data Center V Written Exam Cert Guide 1st Edition

3 3 Contents at a Glance Part 1 Cisco Data Center L2/L3 Technologies Chapter 1: Design Implement and Troubleshoot Complex Layer 2 Technologies Chapter 2: Design Implement and Troubleshoot Complex Layer 3 Technologies Chapter 3: Design, Implement and Troubleshoot External Fabric Connectivity Part 2 Cisco Data Center Network Services Chapter 4: Design, Implement and Troubleshoot Service Insertion and Redirection Chapter 5: Design, Implement and Troubleshoot network services Part 3 Data Center Storage Networking and Compute Chapter 6: Configure and Troubleshoot Data Center Storage Protocols Chapter 7: Design, Implement and Troubleshoot Data Center Storage Networking Features Chapter 8: Implement and Troubleshoot Compute resources Chapter 9: Design, Implement and Troubleshoot Data Center Compute Connectivity Chapter 10: Design, Implement and Troubleshoot Virtual and Physical End Point Connectivity Part 4 Data Center Automation and Orchestration Chapter 11: Implement and Troubleshoot Data Center Automation Chapter 12: Implement and Troubleshoot Data Center Orchestration Tools Chapter 13: Integrate Cisco Cloud offerings into existing Data Center infrastructure Part 5 Data Center Fabric Infrastructure Chapter 14: Design, Implement and Troubleshoot policy and non-policy driven Internal Fabric Connectivity Chapter 15: Design, Implement and Troubleshoot Infrastructure policies and elements Part 6 Evolving Technologies Chapter 16: Cloud Chapter 17: Network Programmability (SDN) Chapter 18: Internet of Things (IoT)

4 4 Table of Contents Preface What this study guide covers How to use this study guide What's available on the CCIEin8Weeks website Part 1 Cisco Data Center L2/L3 Technologies Chapter 1: Cisco Data Center Architecture, Products, Unified I/O and Platform Selection Design, Implement and Troubleshoot Highly Available and redundant topologies for example vpc NX-OS Version Requirement for vpc NX-OS License Requirement for vpc Components of vpc vpc Peer Link vpc Peer-Keepalive vpc Ports, and Orphaned Ports Traffic Flows Dual-Control Plane with Single Layer 2 Node Behavior Link Aggregation Group Identifier Dual-Control Plane with Single Layer 2 Node Behavior Link Aggregation Group Identifier System ID in a vpc System Primary and Secondary vpc Roles Spanning Tree Cisco Fabric Services over Ethernet Synchronization Protocol vpc Configuration Changes When the Peer Link Fails vpc Reload Restore vpc Configuration Consistency vpc Consistency Check vpc Configuration Synchronization Duplicate Frames Prevention in vpc vpc and Object Tracking In-Service Software Upgrade and vpc Interactions Between vpc and Routing HSRP Gateway Considerations HSRP Configuration and Best Practices for vpc ARP Synchronization Peer Gateway Layer 3 Link Between vpc Peers Layer 3 Link to the Core... 49

5 5 IGMP Snooping and vpc Multicast Traffic Forwarding with vpc Protocol Independent Multicast and vpc vpc with Fabric Extender Active-Active Design vpc Configuration Best Practices vpc Domain Configuration vpc Role and Priority Reload Restore Peer Gateway vpc Peer Link vpc Peer Keepalive vpc Ports LACP vpc Deployment Scenarios Best Practices for Attaching a Device to vpc Domain How to Attach Devices to a vpc Domain Access Device Dual-Attached to vpc Domain Single-Sided vpc with 16-Way Port-Channel Double-Sided vpc with 32-Way Port-Channel Double-Sided vpc Sample Configuration: Access Device Single-Attached to vpc Domain Access Device STP-Attached to vpc Domain vpc Failure Scenarios vpc Member Port Failure vpc Complete Dual-Active Failure (Double Failure) vpc Peer-Link Failure vpc Peer-Keepalive Failure Examples Troubleshooting Virtual Port Channel Issues Unable to configure vpc vpc in blocking state vpc domain ids Connectivity issues Peer-link issues vpc Consistency parameter issues Design, Implement and Troubleshoot Highly Available and redundant topologies for example FabricPath Cisco FabricPath Benefits FabricPath Technology Overview Spine and Edge Design Considerations for the Edge and Access Layer FabricPath VLANs... 92

6 6 MAC Learning at the Access Layer Learning the HSRP MAC Address Direct Connectivity between Edge Switches vpc+ at the Edge FabricPath Convergence Times Design, Implement and Troubleshoot Highly Available and redundant topologies for example STP Rules of Operation Spanning Tree Protocol Failure Design, Implement and Troubleshoot Data Center Layer 2 overlays for example VXLAN, OTV, EVPN Configuring a Layer 2 VXLAN Gateway on a Cisco Nexus Cisco Nexus 5600 Deployment Scenarios Cisco Nexus 5600 as a Layer 3 VXLAN Gateway Hypervisor-Originated VXLAN Underlay Requirements for VXLAN VXLAN Designs Centralized Gateway with Inter-VXLAN Routing in the Core/Aggregation Single and Dual Attached Physical Machines to Hardware VTEPs Single and Dual Attached Virtual Machines to Hardware VTEPs Single and Dual Attached Physical Machines to FEX Single and Dual Attached Virtual Machines to FEX Single and Dual Attached (Distributed Gateway) Hypervisor-Originated VXLAN to the Top of Rack (ToR) Single Attached Hypervisor-Originated VXLAN to FEX Hypervisor-Originated VXLAN with no VTEPs on Physical Switches Bud-Node VTEP VXLAN and Virtual PortChannel (vpc) Overlay Interfaces MAC Address Learning Multicast Group Addresses and IGMP Snooping High Availability and ISSU Virtualization Support Guidelines and Limitations for OTV MP-BGP EVPN Control Plane Overview Software and Hardware Support for the MP-BGP EVPN Control Plane IP Transport Devices Running MP-BGP EVPN VTEPs Running MP-BGP EVPN Inter-VXLAN Routing Multitenancy in MP-BGP EVPN MP-BGP EVPN NLRI and L2VPN EVPN Address Family Integrated Routing and Bridging with the MP-BGP EVPN Control Plane Local-Host Learning EVPN Route Advertisement and Remote-Host Learning

7 7 MP-BGP EVPN VTEP Configuration Design, Implement and Troubleshoot Network management protocols for example PTP, NTP, DNS, DHCP PTP Overview PTP Device Types Configuration Steps Name Servers DHCP Relay Agent Design, Implement and Troubleshoot Traffic management for example queuing, policing, shaping Configuring Queuing and Scheduling Configuring Color-Aware Policing Design, Implement and Troubleshoot layer 2 security features for example port security, MACsec, ACL, private vlans Port Security and Port Types Virtualization Support Configuration Steps Cisco TrustSec and Authentication SGACLs and SGTs Exam Essentials Further Reading Chapter 2: Design Implement and Troubleshoot Complex Layer 3 Technologies Design, Implement and Troubleshoot Routing Protocols and features for example OSPF, ISIS, MBGP, IBGP, PBR, BFD LSA types (1, 2, 3, 4, 5, 7, 9) Table below summarizes various LSA types and their description Table below, shows various OSPF network types and traffic that are allowed Route types (N1, N2, E1, E2) Implement and troubleshoot neighbor relationship Enabling OSPFv Configuration Steps Describe basic ISIS network Enabling IS-IS Configuration Steps Creating an IS-IS Instance Configuration Steps BGP Autonomous Systems Byte AS Number Support Administrative Distance BGP Peers BGP Sessions BGP Router Identifier BGP Path Selection

8 8 BGP and the Unicast RIB BGP Prefix Independent Convergence Core BGP Virtualization Multiprotocol BGP Enabling BGP Configuration Steps Creating a BGP Instance Configuration Steps Policy-Based Routing (PBR) Policy Route Maps Enabling the Policy-Based Routing Feature Configuring a Route Policy BFD Asynchronous Mode BFD Detection of Failures BFD Echo Function Enabling the BFD Feature Configuring Global BFD Parameters Further Reading Design, Implement and Troubleshoot Layer 3 Overlay Protocols for example OTV, LISP Table below shows the OTV entities/roles and their description Control Plane Considerations Multicast Enabled Transport Infrastructure OTV Neighbor Discovery Unicast-Only Transport Infrastructure (Adjacency-Server Mode) OTV Scalability Considerations OTV Deployment Options OTV in the DC Core Layer 2-Layer 3 Boundary at Aggregation Layer 2-Layer 3 Boundary at the Core OTV at the DC Aggregation OTV in Point-to-Point Deployments Deploying OTV at the DC Aggregation Further Reading LISP LISP Message Definitions User Data Control Messages EID-to-RLOC Mapping Database Connecting the Databases Together Database Lookup Procedures EID-to-RLOC Mapping Database

9 9 Summary of LISP Network Element Functions Ingress Tunnel Router Egress Tunnel Router Map-Request message Map Server Map Resolver ALT Router Proxy Ingress Tunnel Router Proxy Egress Tunnel Router Applicability and Use Cases Further Reading Design, Implement and Troubleshoot Multicast protocols for example PIM, IGMP, RP, BSR PIM Dense Mode (PIM-DM) PIM Sparse Mode (PIM-SM) Bidirectional PIM (Bidir-PIM) IGMP Table below shows IGMPv2 intervals and their default values Further Reading IGMP Snooping IGMP Querier Further Reading IGMP Filter Further Reading IGMP proxy Further Reading Rendezvous Point (RP) BSR Design, Implement and Troubleshoot layer 3 security features for example CoPP, ACL's, DAI, IP Source Guard ACLs ACL Types and Applications Dynamic ARP Inspection (DAI) Source Guard Further Reading Design, Implement and Troubleshoot Monitoring services for example Netflow Netflow v5, v Table below describes various NetFlow versions and their description Further Reading Nx-OS Features Configuration Guidelines Configuration Examples Exam Essentials

10 10 Chapter 3: Design, Implement and Troubleshoot External Fabric Connectivity Design, Implement and Troubleshoot policy driven External Fabric Connectivity for example L2/L3 connectivity Configuring a BGP Layer 3 Outside Network Connection Configuring BGP External Routed Network Using the NX-OS Style CLI Configuring a Tenant Layer 3 Outside Network Connection Shared Services Contracts Usage Shared Layer 3 Out ACI Transit Routing Transit Routing Overview Route Distribution Within the ACI Fabric External Layer 3 Outside Connection Types Further Reading ACI Virtual Port Channel Workflow Configure the Virtual Port Channel Configure the Application Profile Networking Domains Attachable Entity Profile Exam Essentials Part 2 Cisco Data Center Network Services Chapter 4: Design, Implement and Troubleshoot Service Insertion and Redirection Design, Implement and Troubleshoot Service Insertion and Redirection for example LB, vpath, ITD, RISE Topology and Design Principles Connecting Endpoint Groups with a Service Graph Extension to Virtualized Servers Management Model Service Graphs, Functions, and Rendering Hardware and Software Support Service Graph Definition Concrete Devices and Logical Devices Logical Device Selector (or Context) Splitting Bridge Domains Configuration Steps Further Reading Cisco vpath Cisco vpath Service Chaining Architecture vpath Service Chaining Architecture Components Management Plane Control Plane Virtual Service Nodes (VSN)

11 11 Service Data Plane vpath Service Chaining Example 1 with VSG and ASA Cisco vpath Service Chaining Features Dynamic Service Provisioning Multitenancy Administrative Boundaries Mobility VXLAN Enabling vpath Service Chaining in the Virtual Data Center Cisco Nexus 1000V Series Infrastructure Service Chaining Components Cisco ASA Further Reading Cisco ITD Intelligent Traffic High Scalability Operational Simplicity Investment Protection Deployment Modes One-Arm Deployment Mode One-Arm Deployment Mode with VPC Sandwich Deployment Mode Server Load-Balancing Deployment Mode Configuring Cisco RISE in a Direct Mode Deployment Configuring RISE in an Indirect Mode Deployment Configuring RISE on the Cisco Nexus Switch Configuring RISE in vpc Mode (Recommended Deployment Mode) Configuring RISE in a vpc Direct Mode Deployment Configuring RISE in a vpc Indirect Mode Deployment Route Health Injection Intelligent Service Card Manager Intelligent Service Card Client Universal Routing Information Base Exam Essentials Further Reading Chapter 5: Design, Implement and Troubleshoot network services Design, Implement and Troubleshoot network services for example policy driven L4-L7 services Further Reading Exam Essentials Part 3 Data Center Storage Networking and Compute Chapter 6: Configure and Troubleshoot Data Center Storage Protocols

12 12 Describe, Configure and Troubleshoot infrastructure to support Block Storage Protocols for example FC, FCoE, iscsi, DCB Describe SCSI standards and protocols Describe iscsi standards and protocols SCSI initiator SCSI target SCSI Flow Statistics Describe iscsi and its features Initiator Presentation Modes Transparent Initiator Mode Proxy Initiator Mode iscsi Routing Modes iscsi Immediate Data and Unsolicited Data Features iscsi Interface Advanced Features Parallel SCSI SCSI SCSI Fibre Channel over Ethernet Overview Design and implement data center bridging protocol and lossless Ethernet Lossless Ethernet Logical Link Up/Down Converged Network Adapters FCoE Topologies Directly Connected CNA Topology Remotely Connected CNA Topology Configuring FCoE Enabling FCoE Configuration Steps Disabling LAN Traffic on an FCoE Link Configuration Steps Configuring the FC-Map Configuration Steps Configure and Troubleshoot infrastructure to support File Storage Protocols for example NFS, CIFS. 240 Further Reading Exam Essentials Chapter 7: Design, Implement and Troubleshoot Data Center Storage Networking Features Design, Implement and Troubleshoot Data Center Storage Networking Features for example zoning, multihop technologies Design basic and enhanced zoning Describe multihop FCoE Design Consideration for Multihop FCoE with Edge-Core-Edge Design Access Layer and SAN Edge Fabric

13 13 Aggregation Layer and SAN Core Fabric Storage Edge Fabric Further Reading Exam Essentials Chapter 8: Implement and Troubleshoot Compute resources Implement and Troubleshoot Compute policies and pools for example SAN/LAN policies, boot policies, profiles Storage Policies for Multiple VM Disks Adding and Configuring a Storage Policy Virtual Storage Catalogs Network Policies Adding a Static IP Pool Policy Adding a Network Policy Networking Provisioning Policies Boot Policy Creating a Boot Policy Further Reading Service profile Overview Service profile types Analyze and Troubleshoot Compute resources using tools such as UCS Central, UCSM Faults Events Audit Log System Event Log Syslog Technical Support Files Troubleshooting Compute Resources with UCS Central Further Reading Exam Essentials UUID duplication across two service profiles can take place if a duplicate UUID has not yet been allocated as part of a pool however it is assigned statically Chapter 9: Design, Implement and Troubleshoot Data Center Compute Connectivity Design, Implement and Troubleshoot Data Center Compute Connectivity for example SAN/LAN uplinks, rack server integration, fabric ports, appliance ports Direct Connect Mode Management Connection Policy and Connection Mode Why Appliance Port VLANs Should be Allowed on Uplinks Definition of a Unified Storage Port When to Use Trunk or Access Mode Situations to Avoid Appliance Port Failover

14 14 Network Uplink Failure Appliance Port Troubleshooting Further Reading Exam Essentials Chapter 10: Design, Implement and Troubleshoot Virtual and Physical End Point Connectivity Design, Implement and Troubleshoot Virtual and Physical End Point Connectivity Cisco UCS 6100 Series Fabric Interconnects Gigabit Ethernet connectivity Cisco UCS Cluster Ethernet Switching Mode Switch Mode End-Host Mode Further Reading Part 4 Data Center Automation and Orchestration Chapter 11: Implement and Troubleshoot Data Center Automation Implement and Troubleshoot Data Center Automation using methods such as REST API implemented by scripting languages including Python Infrastructure Provisioning and Operations Development and Operations Models and Continuous Integration Monitoring and Advanced Analytics Standard Network Manageability Features Advanced Automation Features Extensible Messaging and Presence Protocol Support OpenStack Integration OpenDayLight Integration and OpenFlow Support Comprehensive Programmability Support Cisco NX-API Support Python Scripting Cisco onepk Further Reading Chapter 12: Implement and Troubleshoot Data Center Orchestration Tools Implement and Troubleshoot Data Center Orchestration using Tools such as UCSD, UCS Central, CPO Cisco UCS Director Orchestrator Tasks Workflows Service Requests Approvals Rollback Further Reading Cisco Process Orchestrator

15 15 Cisco Process Orchestrator Features That Support Service-Oriented Orchestration Target Types Inheritance Target Properties Target Relationships Target Operations Views Interobject Messaging and Policy Provided by Events for a Target Type Automatic and Consistent API Support Automation Packs TOSCA and Cisco Process Orchestrator Comparison Further Reading Chapter 13: Integrate Cisco Cloud offerings into existing Data Center infrastructure Describe and Integrate Cisco Cloud technologies into existing Data Center infrastructure using solutions such as Cisco ONE Enterprise cloud Suite Benefits of Cisco ONE Data Center for Compute Table below shows Features and Capabilities of Cisco ONE for Data Center Compute Further Reading Part 5 Data Center Fabric Infrastructure Chapter 14: Design, Implement and Troubleshoot policy and non-policy driven Internal Fabric Connectivity Design, Implement and Troubleshoot policy and non-policy driven Internal Fabric connectivity for example ECMP, ISIS, inband mgmt, VXLAN Policy Identification and Enforcement Encapsulation Normalization Extension to Virtualized Servers and VLAN and VxLAN Normalization Routed Design with VxLAN Overlays Load Balancing Configuring Static In-Band Management Access Using the NX-OS Style CLI Further Reading Chapter 15: Design, Implement and Troubleshoot Infrastructure policies and elements Design, Implement and Troubleshoot policy driven Infrastructure for example interface policies, MGMT policies Pre Provision Interface Policy Groups The Cisco ACI Policy Management Information Model Tenants Contexts Application Profiles Endpoint Groups Microsegmentation Intra-EPG Endpoint Isolation Contracts

16 16 Labels, Filters, and Subjects Govern EPG Communications What vzany Is Default Policies Further Reading Configure and Troubleshoot physical infrastructure components for example controllers, switches Setting up the APIC Accessing the CLI Switch Discovery with the APIC Switch Discovery Validation and Switch Management from the APIC Validating the Fabric Topology Unmanaged Switch Connectivity in VM Management Registering the Unregistered Switches Using the CLI Creating User Accounts Configuring a Local User Configuring a Remote User Adding Management Access Management Connectivity Modes Attach Entity Profiles Deploying an Application Policy Three-Tier Application Deployment Further Reading Analyze and Troubleshoot logical infrastructure elements using health indicators Viewing a Health Score Using the NX-OS-Style CLI Viewing a Health Score Using the ishell Understanding Faults How Health Scores Are Calculated Using Health Scores for Proactive Monitoring Using Health Scores for Reactive Monitoring Further Reading Part 6 Evolving Technologies Chapter 16: Cloud Compare and Contrast Cloud Deployment Models Private Cloud Community Cloud Public Cloud Hybrid Cloud Infrastructure, Platform, and Software Services (XaaS) Performance and Reliability Security and Privacy Scalability and Interoperability Describe Cloud implementations and Operations

17 17 Automation and Orchestration Workload Mobility Troubleshooting and Management OpenStack Components Compute Swift Cinder Keystone Dashboard Glance Ceilometer Heat Ironic Exam Essentials Further Reading Chapter 17: Network Programmability (SDN) Describe Functional Elements of Network Programmability (SDN) and How They Interact Controllers APIs Scripting Agents Northbound versus Southbound protocols Describe Aspects of Virtualization and Automation in Network Environments DevOps Methodologies, Tools and Workflows Network/Application Function Virtualization (NFV) Service Function Chaining Performance, Availability, and Scaling Considerations Exam Essentials Further Reading Chapter 18: Internet of Things Describe Architectural Framework and Deployment Considerations for Internet of Things (IoT) Performance, Reliability and Scalability Mobility Security and Privacy IoT Threats Security in IoT/M2M Authentication Authorization Network Enforced Policy Secure Analytics: Visibility and Control Standards and Compliance Environmental Impacts on the Network

18 18 Exam Essentials Further Reading

19 19 Preface Congratulations! You have taken your first step towards passing the CCIE Data Center V2.0 written exam. This book is dedicated to all those souls who will never settle for less than they can be, do, share, and give! What this study guide covers As you may already have noticed on the "Contents at a Glance" page that this guide has been formatted around the Cisco's official CCIE exam topics or curriculum. So as you read through parts and chapters, you know exactly where you're within your learning journey. All contents are carefully covered in enough details however still trimmed to exactly what's necessary to pass the exam. The result of this approach is that you have 300 odd pages on your hands (as opposed to 700 to 1,400 pages long reference manuals!). Each exam topic has "Further Reading" section, which you can refer to if you are looking for more in-depth details. How to use this study guide This guide is for everyone who's studying for CCIE DC written V2.0 exam, regardless if this is your first time sitting for a CCIE written exam or you're a pro who's recertifying. I strongly suggest to take a methodical approach for exam preparation, i.e. start with a target date when you would like to sit for the actual exam and then work backwards to see what kind of study plan would work for you. I believe you can cover the entire contents within eight weeks including the practice questions (refer to website). Lastly, the study guide in itself may not be enough for everyone to pass the exam so be sure to add other sources of learning and your hands-on experience on Cisco product portfolio and DC technologies. What's available on the CCIEin8Weeks website CCIEin8Weeks.com carries the extras (sold separately) that go hand in hand with this study guide to further ensure your exam success! Website includes:

20 20 6 Practice Exams (one for each section as per official curriculum), One final exam 4 Study Plans, to help you track your progress or help you come up with your own plan Study guide in printable PDF format Last but not least, CCIE Exam community forum, to help you interact with others, share knowledge, find study partners etc.

21 21 Part 1 Cisco Data Center L2/L3 Technologies Chapter 1: Design Implement and Troubleshoot Complex Layer 2 Technologies Chapter 2: Design, Implement and Troubleshoot Complex Layer 3 Technologies Chapter 3: Design, Implement and Troubleshoot External Fabric Connectivity

22 22 Chapter 1: Design Implement and Troubleshoot Complex Layer 2 Technologies This chapter covers the following exam topics from Cisco s official written exam curriculum. Design, Implement and Troubleshoot Highly Available and redundant topologies for example vpc, fabricpath, STP Design, Implement and Troubleshoot Data Center Layer 2 overlays for example VXLAN, OTV, EVPN Design, Implement and Troubleshoot Network management protocols for example PTP, NTP, DNS, DHCP Design, Implement and Troubleshoot Traffic management for example queuing, policing, shaping Design, Implement and Troubleshoot layer 2 security features for example port security, MACsec, ACL, private vlans

23 23 Chapter 1: Cisco Data Center Architecture, Products, Unified I/O and Platform Selection Design, Implement and Troubleshoot Highly Available and redundant topologies for example vpc vpc is a virtualization technology that presents both Cisco Nexus 7000 Series paired devices as a unique Layer 2 logical node to access layer devices or endpoints. vpc belongs to Multichassis EtherChannel [MCEC] family of technology. A virtual port channel (vpc) allows links that are physically connected to two different Cisco Nexus 7000 Series devices to appear as a single port channel to a third device. The third device can be a switch, server, or any other networking device that supports link aggregation technology. vpc provides the following technical benefits: Eliminates Spanning Tree Protocol (STP) blocked ports Uses all available uplink bandwidth Allows dual-homed servers to operate in active-active mode Provides fast convergence upon link or device failure Offers dual active/active default gateways for servers vpc also leverages native split horizon/loop management provided by port-channeling technology: a packet entering a port-channel cannot immediately exit that same port-channel. By using vpc, users get the immediate operational and architectural advantages: Simplifies network design Build highly resilient and robust Layer 2 network Enables seamless virtual machine mobility and server high-availability clusters Scales available Layer 2 bandwidth, increasing bisectional bandwidth Grows the size of the Layer 2 network

24 24 (Source: Cisco.com) vpc leverages both hardware and software redundancy aspects: vpc uses all port-channel member links available so that in case an individual link fails, hashing algorithm will redirect all flows to the remaining links. vpc domain is composed of two peer devices. Each peer device processes half of the traffic coming from the access layer. In case a peer device fails, the other peer device will absorb all the traffic with minimal convergence time impact. Each peer device in the vpc domain runs its own control plane, and both devices work independently. Any potential control plane issues stay local to the peer device and does not propagate or impact the other peer device. From a Spanning-Tree standpoint, vpc eliminates STP blocked ports and uses all available uplink bandwidth. Spanning-Tree is used as a fail safe mechanism and does not dictate L2 path for vpc-attached devices. Within a vpc domain, user can connect access devices in multiple ways: vpc-attached connections leveraging active/active behavior with port-channel, active/standby connectivity using spanning-tree, single attachment without spanning-tree running on the access device. NX-OS Version Requirement for vpc vpc technology is supported since NX-OS (i.e since the inception of Nexus 7000 platform). NX-OS appropriate version depends on line cards configuration (M1, F1 or F2), chassis type (7010, 7018 or 7009) and Fabric Module generation (FM generation 1 [46 Gbps per module] or generation 2 [110 Gbps per module]).

25 25 NX-OS License Requirement for vpc vpc feature is included in the base NX-OS software license. Hot Standby Router Protocol (HSRP), Virtual Router Redundancy Protocol (VRRP), Link Aggregation Control Protocol (LACP) are also included in this base license. Layer 3 features like Open Shortest Path First (OSPF) protocol or Intermediate-Systemto-Intermediate System (IS-IS) protocol require LAN_ENTERPRISE_SERVICES_PKG license. Virtual device contexts (VDCs) requires LAN_ADVANCED_SERVICES_PKG license. Components of vpc Table below lists important terms you need to know to understand vpc technology. These terms are used throughout this guide. Term Description vpc The combined port-channel between the vpc peers and the downstream device. A vpc is a L2 port type: switchport mode trunk or switchport mode access vpc peer device A vpc switch (one of a Cisco Nexus 7000 Series pair). vpc domain Domain containing the 2 peer devices. Only 2 peer devices max can be part of same vpc domain. vpc member port One of a set of ports (that is, port-channels) that form a vpc (or port-channel member of a vpc). vpc peer-link Link used to synchronize the state between vpc peer devices. It must be a 10-Gigabit Ethernet link. vpc peer-link is a L2 trunk carrying vpc VLAN. vpc peer-keepalive link The keepalive link between vpc peer devices; this link is used to monitor the liveness of the peer device.

26 26 vpc VLAN VLAN carried over the vpc peer-link and used to communicate via vpc with a third device. As soon as a VLAN is defined on vpc peer-link, it becomes a vpc VLAN non-vpc VLAN A VLAN that is not part of any vpc and not present on vpc peerlink. Orphan port A port that belong to a single attached device. vpc VLAN is typically used on this port. Cisco Fabric Services (CFS) protocol Underlying protocol running on top of vpc peer-link providing reliable synchronization and consistency check mechanisms between the 2 peer devices. Virtual PortChannels (vpcs) allow links that are physically connected to two different Cisco switches to appear to a third downstream device to be coming from a single device and as part of a single PortChannel. The third device can be a switch, a server, or any other networking device that supports IEEE 802.3ad PortChannels. vpc allows the creation of Layer 2 PortChannels that span two switches. At the time of this writing, vpc is implemented on the Cisco Nexus 7000 and 5000 Series platforms (with or without Cisco Nexus 2000 Series Fabric Extenders). vpcs consist of two vpc peer switches connected by a peer link. Of the vpc peers, one is primary and one is secondary. The system formed by the switches is referred to as a vpc domain.

27 27 (Source: Cisco.com) (Source: Cisco.com)

28 28 Following is a list of some possible Cisco Nexus vpc topologies, refer to diagrams above: vpc on the Cisco Nexus 7000 Series (topology A): This topology consists of access layer switches dual-homed to the Cisco Nexus 7000 Series with a switch PortChannel with Gigabit Ethernet or 10 Gigabit Ethernet links. This topology can also consist of hosts connected with virtual PortChannels to each Cisco Nexus 7000 Series Switch. vpc on Cisco Nexus 5000 Series (topology B): This topology consists of switches dualconnected to the Cisco Nexus 5000 Series with a switch PortChannel with 10 Gigabit Ethernet links, with one or more links to each Cisco Nexus 5000 Series Switch. Like topology A, topology B can consist of servers connected to each Cisco Nexus 5000 Series Switch via virtual PortChannels. vpc on the Cisco Nexus 5000 Series with a Cisco Nexus 2000 Series Fabric Extender singlehomed (also called straight-through mode) (topology C): This topology consists of a Cisco Nexus 2000 Series Fabric Extender single-homed with one to eight 10 Gigabit Ethernet links (depending on the fabric extender model) to a single Cisco Nexus 5000 Series Switch, and of Gigabit Ethernet or 10 Gigabit Ethernet-connected servers that form virtual PortChannels to the fabric extender devices. Note that each fabric extender connects to a single Cisco Nexus 5000 Series Switch and not to both, and that the virtual PortChannel can be formed only by connecting the server network interface cards (NICs) to two fabric extenders, where fabric extender 1 depends on Cisco Nexus 5000 Series Switch 1 and fabric extender 2 depends on Cisco Nexus 5000 Series Switch 2. If both fabric extender 1 and fabric extender 2 depend on switch 1 or both of them depend on switch2, the PortChannel cannot be formed. Dual-homing of the Cisco Nexus 2000 Series Fabric Extender (topology D): This topology is also called Cisco Nexus 2000 Series Fabric Extender (FEX for brief) Active/Active. In this topology each FEX is connected to each Cisco Nexus 5000 Series device with a virtual PortChannel. With this topology, the server cannot create a PortChannel split between two fabric extenders. The servers can still be dual-homed with active-standby or active-active transmit-load-balancing (TLB) teaming. The vpc peer switches are connected through a link called a peer link, also known as a multichassis EtherChannel trunk (MCT). vpc Peer Link The vpc peer link is the most important connectivity element in the vpc system. This link is used to create the illusion of a single control plane by forwarding Bridge Protocol data units (BPDUs) or Link Aggregation Control Protocol (LACP) packets to the primary vpc switch from the secondary vpc switch. The peer link is used to synchronize MAC addresses between aggregation groups 1 and 2, to synchronize IGMP entries for the purpose of IGMP snooping, it provides the necessary transport for multicast traffic and for the communication of orphaned ports. The term orphaned ports refers to switch ports connected to single-attached hosts, or vpc ports whose members are all connected to a single vpc peer.

29 29 In the case of a vpc device that is also a Layer 3 switch, the peer link also carries Hot Standby Router Protocol (HSRP) frames. For a vpc to forward a VLAN, that VLAN must exist on the peer link and on both vpc peers, and it must appear in the allowed list of the switch port trunk for the vpc itself. If either of these conditions is not met, the VLAN is not displayed when you enter the command show vpc brief, nor is it a vpc VLAN. When a PortChannel is defined as a vpc peer link, Bridge Assurance is automatically configured on the peer link. (Source: Cisco.com) vpc Peer-Keepalive A routed link (it is more accurate to say path ) is used to resolve dual-active scenarios in which the peer link connectivity is lost. This link is referred to as a vpc peer-keepalive or faulttolerant link. The peer-keepalive traffic is often transported over the management network through the management 0 port of the Cisco Nexus 5000 Series Switch or the management 0 ports on each Cisco Nexus 7000 Series supervisor. The peer-keepalive traffic is typically routed over a dedicated Virtual Routing and Forwarding (VRF) instance (which could be the management VRF, for example). The keepalive can be carried over a routed infrastructure; it does not need to be a direct point-to-point link, and, in fact, it is desirable to carry the peerkeepalive traffic on a different network instead of on a straight point-to-point link.

30 30 vpc Ports, and Orphaned Ports A vpc port is a port that is assigned to a vpc channel group. The ports that form the virtual PortChannel are split between the vpc peers and are referred to as vpc member ports. A non-vpc port, also known as an orphaned port, is a port that is not part of a vpc. To summarize, a vpc system consists of the following components: Two peer devices: the vpc peers, of which one is primary and one is secondary; both are part of a vpc domain A Layer 3 Gigabit Ethernet link called a peer-keepalive link to resolve dual-active scenarios A redundant 10 Gigabit Ethernet PortChannel called a peer link which is used to carry traffic from one system to the other when needed and to synchronize forwarding tables vpc member ports forming the virtual PortChannel Traffic Flows vpc configurations are optimized to help ensure that traffic through a vpc-capable system is symmetric. In Figure 6, for example, the flow on the left (in blue) reaching a Cisco Nexus switch (Agg1 in the figure) from the core is forwarded toward the access layer switch (Acc1 in the figure without traversing the peer Cisco Nexus switch device (Agg2). Similarly, traffic from the server directed to the core reaches a Cisco Nexus Switch (Agg1), and the receiving Cisco Nexus Switch routes this traffic directly to the core without unnecessarily passing it to the peer Cisco Nexus device. This process occurs regardless of which Cisco Nexus device is the primary HSRP device for a given VLAN.

31 31 (Source: Cisco.com) Dual-Control Plane with Single Layer 2 Node Behavior While still operating with two separate control planes, vpc helps ensure that the neighboring devices connected in vpc mode see the vpc peers as a single spanning-tree and LACP entity. For this to happen, the system has to perform IEEE 802.3ad control-plane operations in a slightly modified way (which is not noticeable to the neighbor switch). Link Aggregation Group Identifier IEEE 802.3ad specifies the standard implementation of PortChannels. PortChannel specifications provide LACP as a standard protocol, which enables negotiation of port bundling. LACP makes misconfiguration less likely, because if ports are mismatched, they will not form a PortChannel. LACP discerns that the only ports that can be bundled are port 1 going to port 4. According to the IEEE specifications, to allow LACP to determine whether a set of links connect to the same system and to determine whether those links are compatible with aggregation, you need to be able to establish two types of identification:

32 32 You need a globally unique identifier for each system that participates in link aggregation (that is, the switch itself needs to be unique. This number is referred to as the system ID and is composed of a priority and a MAC address that uniquely identifies the switch. Figure 7 illustrates the system ID. You need a means of identifying a link aggregation group. Dual-Control Plane with Single Layer 2 Node Behavior While still operating with two separate control planes, vpc helps ensure that the neighboring devices connected in vpc mode see the vpc peers as a single spanning-tree and LACP entity. For this to happen, the system has to perform IEEE 802.3ad control-plane operations in a slightly modified way (which is not noticeable to the neighbor switch). Link Aggregation Group Identifier IEEE 802.3ad specifies the standard implementation of PortChannels. PortChannel specifications provide LACP as a standard protocol, which enables negotiation of port bundling. LACP makes misconfiguration less likely, because if ports are mismatched, they will not form a PortChannel. ACP makes misconfiguration less likely, because if ports are mismatched, they will not form a PortChannel. Consider example A in the figure below, in which switch 1 connects to switch 2. Port 1 on switch 1 connects to port 4 on switch 2, and port 2 on switch 1 connects to port 6 on switch 2. Now imagine that the administrator configured a PortChannel on switch 1 between ports 1 and 2, while on switch 2 the PortChannel is configured between ports 5 and 3. Without LACP, the ports could be configured in channel-group mode, and you would not discover that this is a misconfiguration until you notice that traffic has dropped. LACP discerns that the only ports that can be bundled are port 1 going to port 4. According to the IEEE specifications, to allow LACP to determine whether a set of links connect to the same system and to determine whether those links are compatible with aggregation, you need to be able to establish two types of identification: You need a globally unique identifier for each system that participates in link aggregation (that is, the switch itself needs to be unique. This number is referred to as the system ID and is composed of a priority and a MAC address that uniquely identifies the switch. You need a means of identifying a link aggregation group.

33 33 (Source: Cisco.com) System ID in a vpc System Spanning tree and LACP use the switch MAC address for, respectively, the bridge ID field in the spanning-tree BPDU and as part of LACP LAGID. In a single chassis, they use the systemwide MAC address for this purpose. For systems that use vpcs, use of the systemwide MAC address would not work because the vpc peers needs to appear as a single entity as shown in example C in Figure 8. To meet this requirement, vpc offers both an automatic configuration and a manual configuration of the system ID for the vpc peers. The automatic solution implemented by vpc consists of generation of a system ID composed of a priority and a MAC address, with the MAC derived from a reserved pool of MAC addresses combined with the domain ID specified in the vpc configuration. The domain ID is encoded in the last octet and the trailing 2 bits of the previous octet of the MAC address. By configuring domain IDs to be different on adjacent vpcs complexes (and to be identical on each vpc peer complex), you will help ensure the uniqueness of the system ID for LACP negotiation purposes. You also help ensure that the spanning-tree BPDUs use a MAC address that is representative of the vpc complex. You can override the automatic generation of the system ID by using the command-line interface (CLI) and configuring the system ID on both vpc peers manually, as follows: (config-vpc-domain)#system-mac <mac> Primary and Secondary vpc Roles In a vpc system, one vpc switch is defined as primary and one is defined as secondary, based on defined priorities. The lower number has higher priority, so it wins. Also, these roles are

34 34 nonpreemptive, so a device may be operationally primary, but secondary from a configuration perspective. To understand the operational role of a vpc member, you need to consider the status of the peer-keepalive link and the peer link. When the two vpc systems are joined to form a vpc domain, the priority decides which device is the vpc primary and which is the vpc secondary. If the primary device were to reload, when the system comes back online and connectivity to the vpc secondary device (now the operational primary) is restored, the operational role of the secondary device (operational primary) will not change, to avoid unnecessary disruptions. This behavior is achieved with a sticky-bit method, whereby the sticky information is not saved in the startup configuration, thus making the device that is up and running win over the reloaded device. Hence, the vpc primary becomes the vpc operational secondary. If the peer link is disconnected but the vpc peers are still connected through the vpc peer keepalive link, the vpc operational roles stay unchanged. If both the peer link and peer-keepalive link are disconnected, both vpc peers become operational primary, but upon reconnection of the peer-keepalive link and the peer link, the vpc secondary device (operational primary) keeps the primary role, and the vpc primary becomes the operational secondary device. Spanning Tree vpc modifies the way in which spanning tree works on the switch to help ensure that the vpc peers in a vpc domain appear as a single spanning-tree entity on vpc ports. Also, vpc helps ensure that devices can connect to a vpc domain in a non-vpc fashion with classic spanningtree topology. vpc is designed to support hybrid topologies. Depending on the Cisco NX-OS Software release, this can be achieved in slightly different ways. In all Cisco NX-OS releases, the peer link is always forwarding because of the need to maintain the MAC address tables and Internet Group Management Protocol (IGMP) entries synchronized. vpc by default ensures that only the primary switch forwards BPDUs on vpcs. This modification is strictly limited to vpc member ports. As a result, the BPDUs that may be received by the secondary vpc peer on a vpc port are forwarded to the primary vpc peer through the peer link for processing. Non-vPC ports operate like regular spanning-tree ports. The special behavior of the primary vpc member applies uniquely to ports that are part of a vpc. Starting from Cisco NX-OS Releases 4.2(6) and 5.0(2), vpc allows the user to choose the peerswitch option. This option optimizes the behavior of spanning tree with vpc as follows: The vpc primary and secondary are both root devices and both originate BPDUs The BPDUs originated by both the vpc primary and the vpc secondary have the same designated bridge ID on vpc ports The BPDUs originated by the vpc primary and secondary on non-vpc ports maintain the local bridge ID instead of the vpc bridge ID and advertise the Bridge ID of the vpc system as the root

35 35 The peer-switch option has the following advantages: It reduces the traffic loss upon restoration of the peer link after a failure. It reduces the disruption associated with a dual-active failure (whereby both vpc members become primary). Both devices keep sending BPDUs with the same bridge ID information on vpc member ports, which prevents errdisable from potentially disabling the PortChannel for an attached device. It reduces the potential loss of BPDUs if the primary and secondary roles change. Cisco Fabric Services over Ethernet Synchronization Protocol The vpc peers use the Cisco Fabric Services protocol to synchronize forwarding-plane information and implement necessary configuration checks. vpc peers must synchronize the Layer 2 forwarding table - that is, the MAC address information between the vpc peers. This way, if one vpc peer learns a new MAC address, that MAC address is also programmed on the Layer 2 forwarding table of the other peer device. The Cisco Fabric Services protocol travels on the peer link and does not require any configuration by the user. To help ensure that the peer link communication for the Cisco Fabric Services over Ethernet protocol is always available, spanning tree has been modified to keep the peer-link ports always forwarding. The Cisco Fabric Services over Ethernet protocol is also used to perform compatibility checks to validate the compatibility of vpc member ports to form the channel, to synchronize the IGMP snooping status, to monitor the status of the vpc member ports, to synchronize the Address Resolution Protocol (ARP) table (starting from Cisco NX-OS 4.2(6) and future Release 5.0 releases). If the peer link is disconnected between two vpc peers, the synchronization between vpc peers is interrupted, which may lead to traffic drop for multicast traffic and to flooding for unicast traffic. vpc Configuration Changes When the Peer Link Fails The correct sequence for setting up vpcs requires that the two participating vpc switches see each other over the peer link and that they can communicate over the vpc peer keepalive link. Figure below illustrates this fundamental concept: the configuration depicted in Figure 9 (2) requires starting from Figure (1). If you try to configure a vpc like in Figure (2) without having established vpc peer-link and vpc peer keepalive connectivity, vpc ports won t go into forwarding state. Once you are in the state depicted in Figure (1) (which is a fully formed vpc domain) the peer keepalive connectivity is not strictly required in order to create or modify vpcs. In other words, you can configure the vpc in Figure (2) even if there s a loss of vpc peer keepalive connectivity after the configuration depicted in Figure (1). It is necessary and recommended to have functional vpc peer keepalive connectivity for the correct behavior of vpc in presence of failures, but from a traffic forwarding and configuration purpose, the temporary failure of the peer keepalive link doesn t have any impact.

36 36 The vpc peer-link connectivity is more important for the correct traffic forwarding operations as well as for the ability to create or modify vpcs. (Source: Cisco.com) The initial implementation of vpc did not allow configuration changes when the peer link was disconnected to avoid, upon reconnection of the peer switch, inconsistencies that could bring these links down. Thus, if the peer-link was lost, the interfaces on the vpc primary could not be flapping (i.e. if they went down, they stayed down up until the peer-link was restored), and while the peer link was down, a new vpc member port could not be activated. Example A in Figure shows the case of a single vpc peer. The user cannot activate any vpc member port until a vpc peer switch is present (example B in Figure).

37 37 (Source: Cisco.com) Because of this behavior, if the peer-link connection is lost, by default the user cannot add any vpc ports and activate them, nor can an interface flap. If a vpc interface flaps, the port will stay down after flapping. For example, imagine a vpc setup with PortChannel 8 configured as vpc 8: vpc status id Port Status Consistency Reason Active vlans Po8 up success success 23,50 After the peer-link failure only the primary keeps the vpc interfaces up. If the interface associated with PortChannel 8 flaps, it never goes up again. vpc status id Port Status Consistency Reason Active vlans Po8 down failed Peer-link is down To restore connectivity, you need to first restore the peer link, and then enter a shut/no shut command under the interface on the primary device. Similarly, if you want to create a new vpc interface and to activate it, a vpc peer needs to be present and connected through the peer link. While a peer link is not connected, vpc prevents activation of any new vpc member port. This vpc behavior may manifest itself in several scenarios: