HPE FlexFabric 7900 Switch Series

Transcription

1 HPE FlexFabric 7900 Switch Series VXLAN Configuration Guide Part number: R Software version: Release 213x Document version: 6W

2 Copyright 2015 Hewlett Packard Enterprise Development LP The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistent with FAR and , Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor s standard commercial license. Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website. Acknowledgments Intel, Itanium, Pentium, Intel Inside, and the Intel Inside logo are trademarks of Intel Corporation in the United States and other countries. Microsoft and Windows are trademarks of the Microsoft group of companies. Adobe and Acrobat are trademarks of Adobe Systems Incorporated. Java and Oracle are registered trademarks of Oracle and/or its affiliates. UNIX is a registered trademark of The Open Group.

3 Contents VXLAN overview 1 VXLAN network model 1 VXLAN packet format 2 Working mechanisms 3 Assignment of traffic to VXLANs 3 MAC learning 3 Traffic forwarding 4 Access modes of VSIs 7 ARP flood suppression 7 Protocols and standards 8 Configuring basic VXLAN features 9 VXLAN configuration task list 9 Prerequisites 9 Setting the forwarding mode for VXLANs 10 Creating a VXLAN on a VSI 10 Configuring a VXLAN tunnel 10 Assigning a VXLAN tunnel to a VXLAN 11 Mapping an Ethernet service instance to a VSI 12 Managing MAC address entries 13 Enabling VXLAN local MAC change logging 13 Configuring static remote-mac address entries 13 Enabling remote-mac address learning 14 Confining unknown-unicast floods to the local site 14 Configuring the destination UDP port number of VXLAN packets 15 Configuring VXLAN packet check 15 Enabling ARP flood suppression 15 Disabling remote ARP learning for VXLANs 16 Configuring VXLAN packet statistics 16 Enabling packet statistics for a VSI 16 Enabling packet statistics for an Ethernet service instance 17 Displaying and maintaining VXLANs 18 VXLAN configuration examples 19 Unicast-mode VXLAN configuration example 19 Multicast-mode VXLAN configuration example 23 Configuring VXLAN IP gateways 31 Overview 31 VXLAN IP gateways separated from VTEPs 31 Centralized VXLAN IP gateway deployment 32 Centralized VXLAN IP gateway group deployment 33 Hardware compatibility 34 Configuration prerequisites 34 Configuring a centralized VXLAN IP gateway on a VTEP 34 Configuring a centralized VXLAN IP gateway group 35 Configuring a VTEP group 35 Specifying a VTEP group as the gateway for an access layer VTEP 36 Configuring a VSI interface 36 Enabling packet statistics for VSI interfaces 37 Displaying and maintaining VXLAN IP gateway 37 VXLAN IP gateway configuration examples 38 Centralized VXLAN IP gateway configuration example 38 Centralized VXLAN IP gateway group configuration example 43 Configuring the VTEP as an OVSDB VTEP 47 Overview 47 Feature and software version compatibility 47 i

4 Protocols and standards 47 OVSDB VTEP configuration task list 47 Configuration prerequisites 48 Setting up an OVSDB connection to a controller 48 Configuration restrictions and guidelines 48 Configuring active SSL connection settings 48 Configuring passive SSL connection settings 49 Configuring active TCP connection settings 49 Configuring passive TCP connection settings 49 Enabling the OVSDB server 50 Enabling the OVSDB VTEP service 50 Specifying a global source address for VXLAN tunnels 50 Specifying a VTEP access port 51 Enabling flood proxy on multicast VXLAN tunnels 51 OVSDB VTEP configuration examples 51 Unicast-mode VXLAN configuration example 51 Flood proxy VXLAN configuration example 54 Document conventions and icons 59 Conventions 59 Network topology icons 60 Support and other resources 61 Accessing Hewlett Packard Enterprise Support 61 Accessing updates 61 Websites 62 Customer self repair 62 Remote support 62 Documentation feedback 62 Index 63 ii

5 VXLAN overview Virtual extensible LAN (VXLAN) is a MAC-in-UDP technology that provides Layer 2 connectivity between distant network sites across an IP network. VXLAN is typically used in data centers for multitenant services. VXLAN provides the following benefits: Support for more virtual switched domains than VLANs Each VXLAN is uniquely identified by a 24-bit VXLAN ID. The total number of VXLANs can reach (2 24 ). This specification makes VXLAN a better choice than 802.1Q VLAN to isolate traffic for VMs. Easy deployment and maintenance VXLAN requires deployment only on the edge devices of the transport network. Devices in the transport network perform typical Layer 3 forwarding. The device supports only IPv4-based VXLAN. VXLAN network model As shown in Figure 1, the transport edge devices assign VMs to different VXLANs, and then forward traffic between sites for VMs by using VXLAN tunnels. The transport edge devices are VXLAN tunnel endpoints (VTEP). They can be servers that host VMs or independent network devices. A Hewlett Packard Enterprise VTEP uses VSIs and VXLAN tunnels to provide VXLAN services. VSI A virtual switching instance is a virtual Layer 2 switched domain. Each VSI provides switching services only for one VXLAN. VSIs learn MAC addresses and forward frames independently of one another. VMs in different sites have Layer 2 connectivity if they are in the same VXLAN. VXLAN tunnel Logical point-to-point tunnels between VTEPs over the transport network. Each VXLAN tunnel can trunk multiple VXLANs. VTEPs encapsulate VXLAN traffic in the VXLAN, outer UDP, and outer IP headers. The devices in the transport network forward VXLAN traffic only based on the outer IP header. 1

6 Figure 1 VXLAN network model VXLAN packet format As shown in Figure 2, a VTEP encapsulates a frame in the following headers: 8-byte VXLAN header VXLAN information for the frame. Flags If the I bit is 1, the VXLAN ID is valid. If the I bit is 0, the VXLAN ID is invalid. All other bits are reserved and set to bit VXLAN ID Identifies the VXLAN of the frame. It is also called the virtual network identifier (VNI). 8-byte outer UDP header for VXLAN The default VXLAN UDP port number is byte outer IP header Valid addresses of VTEPs or VXLAN multicast groups on the transport network. Devices in the transport network forward VXLAN packets based on the outer IP header. Figure 2 VXLAN packet format 2

7 Working mechanisms The VTEP uses the following process to forward an inter-site frame: 1. Assigns the frame to its matching VXLAN if the frame is sent between sites. 2. Performs MAC learning on the VXLAN's VSI. 3. Forwards the frame. This section describes this process in detail. For intra-site frames in a VSI, the system performs typical Layer 2 forwarding and processes 802.1Q VLAN tags, as described in "Access modes of VSIs." Assignment of traffic to VXLANs Traffic from the local site to a remote site The VTEP uses an Ethernet service instance to match a list of VLANs on a site-facing interface. The VTEP assigns customer traffic from the VLANs to a VXLAN by mapping the Ethernet service instance to a VSI. An Ethernet service instance is identical to an attachment circuit (AC) in L2VPN. As shown in Figure 3, Ethernet service instance 1 matches VLAN 2 and is mapped to VSI A (VXLAN 10). When a frame from VLAN 2 arrives, the VTEP assigns the frame to VXLAN 10, and looks up VSI A's MAC address table for the outgoing interface. Figure 3 Identifying traffic from the local site Traffic from a remote site to the local site When a frame arrives at a VXLAN tunnel, the VTEP uses the VXLAN ID in the frame to identify its VXLAN. MAC learning The VTEP performs source MAC learning on the VSI as a Layer 2 switch. For traffic from the local site to the remote site, the VTEP learns the source MAC address before VXLAN encapsulation. For traffic from the remote site to the local site, the VTEP learns the source MAC address after removing the VXLAN header. A VSI's MAC address table includes the following types of MAC address entries: Local MAC Dynamic MAC entries learned from the local site. The outgoing interfaces are site-facing interfaces on which the MAC addresses are learned. VXLAN does not support manual local MAC entries. 3

8 Remote MAC MAC entries learned from a remote site, including static, dynamic, and OpenFlow MAC entries. The outgoing interfaces for the MAC addresses are VXLAN tunnel interfaces. Static Manually added MAC entries. Dynamic MAC entries learned in the data plane from incoming traffic on VXLAN tunnels. The learned MAC addresses are contained in the inner Ethernet header. OpenFlow MAC entries issued by a remote controller through OpenFlow. For a remote address, the manual static entry has higher priority than the dynamic entry. Traffic forwarding Unicast The VTEP performs Layer 2 or Layer 3 forwarding for VXLANs depending on your configuration. In Layer 3 forwarding mode, the VTEP uses the ARP table to forward traffic for VXLANs. In Layer 2 forwarding mode, the VTEP uses the MAC address table to forward traffic for VXLANs. Use Layer 3 forwarding mode if you want to use the VTEP as a VXLAN IP gateway. This section describes the Layer 2 forwarding processes. For information about Layer 3 forwarding, see "Configuring VXLAN IP gateways." The VTEP uses the following processes to forward traffic at Layer 2: Unicast process Applies to destination-known unicast traffic. Flood process Applies to multicast, broadcast, and unknown unicast traffic. When the VTEP forwards VXLAN traffic, it processes the 802.1q tag in the inner Ethernet header depending on the VSI access mode (VLAN or Ethernet mode). In VLAN access mode, sites can use different VLANs to provide the same service. For more information, see "Access modes of VSIs." The following process (see Figure 4) applies to a known unicast frame between sites: 1. The source VTEP encapsulates the Ethernet frame in the VXLAN/UDP/IP header. In the outer IP header, the source IP address is the source VTEP's VXLAN tunnel source IP address. The destination IP address is the VXLAN tunnel destination IP address. 2. The source VTEP forwards the encapsulated packet out of the outgoing VXLAN tunnel interface found in the VSI's MAC address table. 3. The intermediate transport devices (P devices) forward the frame to the destination VTEP by using the outer IP header. 4. The destination VTEP removes the headers on top of the inner Ethernet frame. It then performs MAC address table lookup in the VXLAN's VSI to forward the frame out of the matching outgoing interface. 4

9 Figure 4 Inter-site unicast VM 1 MAC Table on VTEP 1 VM 2 VXLAN/VSI MAC Interface VM 3 VXLAN 10/VSI A MAC 1 FGE1/0/1, VLAN 2 VXLAN 10/VSI A MAC 7 Tunnel 1 Server 1 VM 4 FGE1/0/1 FGE1/0/2 VXLAN tunnel 1 FGE1/0/1 VTEP 1 P VTEP 2 VM 7 VM 8 VM 9 VM 5 VM 6 Transport network MAC Table on VTEP 2 VXLAN/VSI MAC Interface Server 3 Server 2 VXLAN 10/VSI A MAC 1 Tunnel 1 VXLAN 10/VSI A MAC 7 FGE1/0/1, VLAN 20 Flood IMPORTANT: Flood proxy is available in Release 2137 and later versions. VXLAN supports the following modes for flood traffic: Unicast mode Also called head-end replication. The source VTEP replicates the flood frame, and then sends one replica to the destination IP address of each VXLAN tunnel in the VXLAN. See Figure 5. Multicast mode Also called tandem replication. The source VTEP sends the flood frame in a multicast VXLAN packet destined for a multicast group address. Transport network devices replicate and forward the packet to remote VTEPs based on their multicast forwarding entries. See Figure 6. Flood proxy mode The source VTEP sends the flood frame in a VXLAN packet over a VXLAN tunnel to a flood proxy server. The flood proxy server replicates and forwards the packet to each remote VTEP through its VXLAN tunnels. See Figure 7. The flood proxy mode applies to VXLANs that have many sites. This mode reduces flood traffic in the transport network without using a multicast protocol. To use a flood proxy server, you must set up a VXLAN tunnel to the server on each VTEP. The flood proxy mode is typically used in SDN transport networks that have a flood proxy server. For VTEPs to forward packets based on the MAC address table issued by an SDN controller, you must disable remote-mac address learning by using the vxlan tunnel mac-learning disable command. Each destination VTEP floods the inner Ethernet frame to all the site-facing interfaces in the VXLAN. To avoid loops, the destination VTEPs do not flood the frame to VXLAN tunnels. 5

10 Figure 5 Unicast mode VM 1 VM 2 VM 3 Server 1 VM 4 FGE1/0/1 FGE1/0/2 Replicate and encapsulate VXLAN tunnel Transport network VXLAN tunnel VTEP 1 P VTEP 2 FGE1/0/1 VM 7 VM 8 VM 9 VM 5 Server 3 VM 6 VTEP 3 Server 2 VM 10 VM 11 VM 12 Server 4 Figure 6 Multicast mode VM 1 VM 2 VM 3 Server 1 VM 4 FGE1/0/1 FGE1/0/2 Encapsulate with multicast address Transport network VXLAN tunnel FGE1/0/1 VTEP 1 P VTEP 2 VXLAN tunnel Replicate VM 7 VM 8 VM 9 VM 5 Server 3 VM 6 VTEP 3 Server 2 VM 10 VM 11 VM 12 Server 4 6

11 Figure 7 Flood proxy mode VM 1 VM 2 VM 3 Server 1 VM 4 VM 5 FGE1/0/1 FGE1/0/2 Encapsulate with flood proxy server address VXLAN tunnel Replicate and forward packet Source: Flood proxy server Destination: Each remote VTEP VXLAN tunnel Transport network Flood proxy server VXLAN tunnel VTEP 1 VTEP 2 VXLAN tunnel FGE1/0/1 VM 7 VM 8 VM 9 Server 3 VM 6 Server 2 VM 10 VTEP 3 VM 11 VM 12 Server 4 Access modes of VSIs The access mode of a VSI determines how the VTEP processes the 802.1Q VLAN tags in the Ethernet frames. VLAN access mode Ethernet frames received from or sent to the local site must contain 802.1Q VLAN tags. For an Ethernet frame received from the local site, the VTEP removes all its 802.1Q VLAN tags before forwarding the frame. For an Ethernet frame destined for the local site, the VTEP adds 802.1Q VLAN tags to the frame before forwarding the frame. In VLAN access mode, VXLAN packets sent between sites do not contain 802.1Q VLAN tags. You can use different 802.1Q VLANs to provide the same service in different sites. Ethernet access mode The VTEP does not process the 802.1Q VLAN tags of Ethernet frames received from or sent to the local site. For an Ethernet frame received from the local site, the VTEP forwards the frame with the 802.1Q VLAN tags intact. For an Ethernet frame destined for the local site, the VTEP forwards the frame without adding 802.1Q VLAN tags. In Ethernet access mode, VXLAN packets sent between VXLAN sites contain 802.1Q VLAN tags. You must use the same VLAN to provide the same service between sites. ARP flood suppression ARP flood suppression reduces ARP request broadcasts by enabling the VTEP to reply to ARP requests on behalf of VMs. 7

12 As shown in Figure 8, this feature snoops ARP packets to populate the ARP flood suppression table with local and remote MAC addresses. If an ARP request has a matching entry, the VTEP replies to the request on behalf of the VM. If no match is found, the VTEP floods the request to both local and remote sites. Figure 8 ARP flood suppression ARP flood suppression uses the following workflow: 1. VM 1 sends an ARP request to obtain the MAC address of VM VTEP 1 creates a suppression entry for VM 1, and floods the ARP request in the VXLAN. 3. VTEP 2 and VTEP 3 de-encapsulate the ARP request. The VTEPs create a suppression entry for VM 1, and broadcast the request in the local site. 4. VM 7 sends an ARP reply. 5. VTEP 2 creates a suppression entry for VM 7 and forwards the ARP reply to VTEP VTEP 1 de-encapsulates the ARP reply, creates a suppression entry for VM 7, and forwards the ARP reply to VM VM 4 sends an ARP request to obtain the MAC address of VM 1 or VM VTEP 1 creates a suppression entry for VM 4 and replies to the ARP request. 9. VM 10 sends an ARP request to obtain the MAC address of VM VTEP 3 creates a suppression entry for VM 10 and replies to the ARP request. Protocols and standards IETF draft, draft-mahalingam-dutt-dcops-vxlan-04 8

13 Configuring basic VXLAN features VXLAN configuration task list Tasks at a glance (Required.) Setting the forwarding mode for VXLANs (Required.) Creating a VXLAN on a VSI (Required.) Configuring a VXLAN tunnel (Required.) Assigning a VXLAN tunnel to a VXLAN (Required.) Mapping an Ethernet service instance to a VSI (Optional.) Managing MAC address entries (Optional.) Confining unknown-unicast floods to the local site (Optional.) Configuring the destination UDP port number of VXLAN packets (Optional.) Configuring VXLAN packet check (Optional.) Enabling ARP flood suppression (Optional.) Disabling remote ARP learning for VXLANs (Optional.) Configuring VXLAN packet statistics Remarks N/A N/A N/A To extend a VXLAN to remote sites, you must assign VXLAN tunnels to the VXLAN. Perform this task to assign customer traffic to VXLANs. You can add static remote MAC addresses. When network attacks occur, disable remote-mac address learning to prevent the device from learning incorrect remote MAC addresses. Perform this task to suppress unknown-unicast floods to the transport network. N/A Perform this task to check incoming VXLAN packets, including the following items: UDP checksum Q VLAN tags in the inner Ethernet header. N/A This feature is available in Release 2137 and later versions. N/A Prerequisites Before you can configure VXLANs, you must perform the following tasks: Set the system operating mode: a. Set the system operating mode to standard by using the system-working-mode standard command. For more information about setting the system operating mode, see device management in Fundamentals Configuration Guide. b. Save the configuration. c. Delete the binary.mdb next-startup configuration file. d. Reboot the device. 9

14 Reserve one global-type VLAN interface resource for the VSI interface of each VXLAN before the VXLAN is created if you enable Layer 3 forwarding for VXLANs. For more information about reserving global-type VLAN interface resources, see VLAN configuration in Layer 2 LAN Switching Configuration Guide. Setting the forwarding mode for VXLANs Step Command Remarks 1. Enter system view. system-view N/A 2. Set the forwarding mode for VXLANs. To enable Layer 2 forwarding: undo vxlan ip-forwarding To enable Layer 3 forwarding: vxlan ip-forwarding [ tagged untagged ] Use one of the commands. If the device is a VTEP, enable Layer 2 forwarding for VXLANs. If the device is a VXLAN IP gateway, enable Layer 3 forwarding for VXLANs. For more information about VXLAN IP gateway, see "Configuring VXLAN IP gateways." You must delete all VXLAN tunnel interfaces before you can change the forwarding mode. The tagged and untagged keywords are available in Release 2137 and later versions. Creating a VXLAN on a VSI Step Command Remarks 1. Enter system view. system-view N/A 2. Enable L2VPN. l2vpn enable By default, L2VPN is disabled. 3. Create a VSI and enter VSI view. 4. (Optional.) Configure a VSI description. vsi vsi-name description text By default, no VSIs are created. By default, a VSI does not have description. 5. Enable the VSI. undo shutdown By default, a VSI is enabled. 6. Create a VXLAN and enter VXLAN view. vxlan vxlan-id By default, no VXLANs are created. You can create only one VXLAN on a VSI. The VXLAN ID must be unique for each VSI. Configuring a VXLAN tunnel For two sites to communicate through VXLAN, you must manually configure a VXLAN tunnel between the sites. To configure a VXLAN tunnel: Step Command Remarks 1. Enter system view. system-view N/A 10

15 Step Command Remarks 2. Specify a global source address for VXLAN tunnels. 3. Create a VXLAN tunnel interface and enter tunnel interface view. 4. Specify a source IP address or source interface for the tunnel. 5. Specify a destination IP address for the tunnel. tunnel global source-address ip-address interface tunnel tunnel-number mode vxlan source { ipv4-address interface-type interface-number } destination ipv4-address By default, no global source address is specified for VXLAN tunnels. A VXLAN tunnel uses the global source address if you do not specify a source interface or source address for the tunnel. This command is available in Release 2137 and later versions. By default, no tunnel interfaces exist. The endpoints of a tunnel must use the same tunnel mode. ECMP is supported only by VXLAN tunnels with an ID in the range of 0 to 511. By default, no source IP address or source interface is specified for a tunnel. This step specifies the source IP address in the outer IP header of tunneled VXLAN packets. If an interface is specified, its primary IP address is used. By default, no destination IP address is specified for a tunnel. Specify the remote VTEP's IP address. This IP address will be the destination IP address in the outer IP header of tunneled VXLAN packets. Assigning a VXLAN tunnel to a VXLAN To provide Layer 2 connectivity for a VXLAN between two sites, you must assign the VXLAN tunnel between the sites to the VXLAN. You can assign multiple VXLAN tunnels to a VXLAN, and configure a VXLAN tunnel to trunk multiple VXLANs. For a unicast-mode VXLAN, the system floods unknown unicast, multicast, and broadcast traffic to each tunnel associated with the VXLAN. To assign a VXLAN tunnel to a VXLAN: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSI view. vsi vsi-name N/A 3. Enter VXLAN view. vxlan vxlan-id N/A 4. Assign a VXLAN tunnel to the VXLAN. tunnel tunnel-number [ flooding-proxy ] By default, a VXLAN does not contain any VXLAN tunnels. For full Layer 2 connectivity in the VXLAN, make sure the VXLAN 11

16 Step Command Remarks contains the VXLAN tunnel between each pair of sites in the VXLAN. The flooding-proxy keyword is available in Release 2137 and later versions. Mapping an Ethernet service instance to a VSI An Ethernet service instance matches a list of VLANs on a site-facing interface. The VTEP assigns customer traffic from the VLANs to a VXLAN by mapping the Ethernet service instance to a VSI. When you configure an Ethernet service instance, follow these guidelines: The match criterion in each Ethernet service instance on an interface must be unique. For example, you cannot configure the encapsulation untagged command in one Ethernet service instance if another Ethernet service instance already contains this command. You cannot use the encapsulation s-vid vlan-id command to specify the same 802.1Q VLAN ID for any two Ethernet service instances on the interface. An Ethernet service instance can contain only one match criterion. To change the match criterion, you must remove the original criterion first. When you remove the match criterion in an Ethernet service instance, the mapping between the service instance and the VSI is removed automatically. To forward the multicast traffic from a VLAN on the interface, make sure an Ethernet service instance contains the VLAN ID. The interface cannot forward a multicast packet that does not match any Ethernet service instance. You must create a VLAN interface for each VLAN that matches an Ethernet service instance if ARP flood suppression is enabled. You do not need to assign IP addresses to the VLAN interfaces. However, you must make sure the VLANs each contain a minimum of one up physical interface. To map an Ethernet service instance to a VSI: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter Layer 2 Ethernet interface view or Layer 2 aggregate interface view. 3. Create an Ethernet service instance and enter Ethernet service instance view. interface interface-type interface-number interface bridge-aggregation interface-number service-instance instance-id N/A By default, no Ethernet service instances exist. 12

17 Step Command Remarks 4. Configure a frame match criterion. 5. Map the Ethernet service instance to a VSI. Match any frames: encapsulation default Match any 802.1Q tagged or untagged frames: encapsulation { tagged untagged } Match frames tagged with the specified outer 802.1Q VLAN ID: encapsulation s-vid vlan-id [ only-tagged ] Match frames tagged with the specified outer and inner 802.1Q VLAN IDs: encapsulation s-vid vlan-id c-vid vlan-id xconnect vsi vsi-name [ access-mode { ethernet vlan } ] By default, an Ethernet service instance does not contain frame match criteria. The current software version does not support the tagged keyword. By default, an Ethernet service instance is not mapped to any VSI. Managing MAC address entries With VXLAN, local MAC addresses can only be learned dynamically. You cannot manually add local MAC addresses. However, you can log the local MAC changes. Remote MAC address entries include the following types: Manually created static entries. Dynamic entries learned in the data plane. Entries issued by a remote controller through OpenFlow. Enabling VXLAN local MAC change logging Local-MAC change logging enables the VXLAN module to send a log message to the information center when a local MAC address is added or removed. With the information center, you can set log message filtering and output rules, including output destinations. For more information about configuring the information center, see Network Management and Monitoring Configuration Guide. To enable local MAC change logging: Step Command Remarks 1. Enter system view. system-view N/A 2. Enable local MAC change logging. vxlan local-mac report By default, VXLAN local MAC change logging is disabled. Configuring static remote-mac address entries Step Command Remarks 1. Enter system view. system-view N/A 13

18 Step Command Remarks 2. Add a static remote entry. mac-address static mac-address interface tunnel tunnel-number vsi vsi-name By default, VXLAN VSIs do not have static remote-mac address entries. For the setting to take effect, make sure the VSI's VXLAN has been created and specified on the VXLAN tunnel. Enabling remote-mac address learning Step Command Remarks 1. Enter system view. system-view N/A 2. Enable remote-mac address learning. undo vxlan tunnel mac-learning disable By default, remote-mac address learning is enabled. When network attacks occur, disable remote-mac address learning to prevent the device from learning incorrect remote MAC addresses. Confining unknown-unicast floods to the local site By default, the VTEP floods unknown unicast frames received from the local site to the following interfaces in the frame's VXLAN: All site-facing interfaces except for the incoming interface. All VXLAN tunnel interfaces. To confine unknown-unicast floods to site-facing interfaces for a VXLAN: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSI view. vsi vsi-name N/A 3. Disable the VSI to flood unknown unicast traffic to VXLAN tunnel interfaces. 4. (Optional.) Enable selective flood for a MAC address. flooding disable selective-flooding mac-address mac-address By default, unknown unicast traffic is flooded to all interfaces in the VXLAN, except for the incoming interface. By default, selective flood is disabled. Use this feature to exclude a remote MAC address from the flood suppression done by using the flooding disable command. The VTEP will flood the frames destined for the specified MAC address to remote sites when unknown-unicast floods are confined to the local site. 14

19 Configuring the destination UDP port number of VXLAN packets Step Command Remarks 1. Enter system view. system-view N/A 2. Configure a destination UDP port for VXLAN packets. vxlan udp-port port-number By default, the destination UDP port number is 4789 for VXLAN packets. You must configure the same destination UDP port number on all VTEPs in a VXLAN. Configuring VXLAN packet check The device can check the UDP checksum and 802.1Q VLAN tags of each received VXLAN packet. UDP checksum check The device always sets the UDP checksum of VXLAN packets to zero. For compatibility with third-party devices, a VXLAN packet can pass the check if its UDP checksum is zero or correct. If its UDP checksum is incorrect, the VXLAN packet fails the check and is dropped. VLAN tag check The device checks the inner Ethernet header of each VXLAN packet for 802.1Q VLAN tags. If the header contains 802.1Q VLAN tags, the device drops the packet. If a remote VTEP uses the Ethernet access mode for an Ethernet service instance, its VXLAN packets might contain 802.1Q VLAN tags. To prevent the local VTEP from dropping the VXLAN packets, do not execute the vxlan invalid-vlan-tag discard command on the local VTEP. The access mode of an Ethernet service instance is configurable by using the xconnect vsi command. To configure VXLAN packet check: Step Command Remarks 1. Enter system view. system-view N/A 2. Enable the VTEP to drop VXLAN packets that fail UDP checksum check. 3. Enable the VTEP to drop VXLAN packets that have 802.1Q VLAN tags in the inner Ethernet header. vxlan invalid-udp-checksum discard vxlan invalid-vlan-tag discard By default, the VTEP does not check the UDP checksum of VXLAN packets. By default, the VTEP does not check the inner Ethernet header for 802.1Q VLAN tags. Enabling ARP flood suppression Use ARP flood suppression to reduce ARP request broadcasts. The aging timer is fixed at 25 minutes for ARP flood suppression entries. If the suppression table is full, the VTEP stops learning new entries. For the VTEP to learn new entries, you must wait old entries to age out, or use the reset arp suppression command to clear the table. If the flooding disable command is configured, set the MAC aging timer to a higher value than the aging timer for ARP flood suppression entries on all VTEPs. This setting prevents the traffic 15

20 blackhole that occurs when a MAC address entry ages out before its ARP flood suppression entry ages out. To set the MAC aging timer, use the mac-address timer command. To enable ARP flood suppression: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSI view. vsi vsi-name N/A 3. Enable ARP flood suppression. arp suppression enable By default, ARP flood suppression is disabled. Disabling remote ARP learning for VXLANs IMPORTANT: This feature is available in Release 2137 and later versions. By default, the device learns ARP information of remote VMs from packets received on VXLAN tunnel interfaces. To save resources on VTEPs in an SDN transport network, you can temporarily disable remote ARP learning when the controller and VTEPs are synchronizing entries. After the entry synchronization is completed, use the undo vxlan tunnel arp-learning disable command to enable remote ARP learning. As a best practice, disable remote ARP learning for VXLANs only when the controller and VTEPs are synchronizing entries. To disable remote ARP learning for VXLANs: Step Command Remarks 1. Enter system view. system-view N/A 2. Disable remote ARP learning for VXLANs. vxlan tunnel arp-learning disable By default, remote ARP learning is enabled for VXLANs. Configuring VXLAN packet statistics Enabling packet statistics for a VSI Step Command Remarks 1. Enter system view. system-view N/A 2. Set the packet statistic collection mode to VSI. statistic mode vsi By default, the packet statistic collection mode is VSI. If you execute the statistic mode command multiple times, the most recent configuration takes effect. 3. Enter VSI view. vsi vsi-name N/A 16

21 Step Command Remarks 4. Enable the packet statistics feature for the VSI. 5. (Optional.) Display packet statistics for VSIs. statistics enable display l2vpn vsi [ name vsi-name ] [ verbose ] By default, the packet statistics feature is disabled for all VSIs. This command is available in any view. Enabling packet statistics for an Ethernet service instance IMPORTANT: This feature is available in Release 2137 and later versions. To enable packet statistics for an Ethernet service instance: Step Command Remarks 1. Enter system view. system-view N/A 2. Set the packet statistic collection mode to AC. 3. Enter interface view. 4. Enter Ethernet service instance view. 5. Enable packet statistics for the Ethernet service instance. In software versions earlier than Release 2138: statistic mode s-channel In Release 2138 and later versions: statistic mode ac Enter Layer 2 Ethernet interface view: interface interface-type interface-number Enter Layer 2 aggregate interface view: interface bridge-aggregation interface-number service-instance instance-id statistics enable 17 By default, the packet statistic collection mode is VSI. In software versions earlier than Release 2138, you can use the statistic mode s-channel, statistic mode queue, or statistic mode vsi command to set the packet statistic collection mode. These commands overwrite each other. In Release 2138 and later versions, you can use the statistic mode ac, statistic mode queue, or statistic mode vsi command to set the packet statistic collection mode. These commands overwrite each other. For more information about the statistic mode queue command, see QoS commands in ACL and QoS Command Reference. N/A N/A By default, the packet statistics feature is disabled for all Ethernet service instances. For the statistics enable

22 Step Command Remarks command to take effect, you must configure a frame match criterion for the Ethernet service instance and map it to a VSI. If you modify the frame match criterion or VSI mapping, packet statistics of the instance is cleared. 6. (Optional.) Display packet statistics for Ethernet service instances. display l2vpn service-instance [ interface interface-type interface-number [ service-instance instance-id ] ] [ verbose ] This command is available in any view. Displaying and maintaining VXLANs IMPORTANT: The reset l2vpn statistics ac command is available in Release 2137 and later versions. Execute display commands in any view and reset commands in user view. Task Display the ARP flood suppression table (in standalone mode). Display the ARP flood suppression table (in IRF mode). Command display arp suppression vsi [ name vsi-name ] [ slot slot-number ] [ count ] display arp suppression vsi [ name vsi-name ] [ chassis chassis-number slot slot-number ] [ count ] Display MAC address entries for VSIs. display l2vpn mac-address [ vsi vsi-name ] [ dynamic ] [ count ] Display information about Ethernet service instances. display l2vpn service-instance [ interface interface-type interface-number [ service-instance instance-id ] ] [ verbose ] Display information about VSIs. display l2vpn vsi [ name vsi-name ] [ verbose ] Display information about the multicast groups that contain IGMP host-enabled interfaces. Display information about tunnel interfaces. Display VXLAN tunnel information for VXLANs. Display the current packet statistic collection mode. Clear ARP flood suppression entries on VSIs. Clear dynamic address entries on VSIs. display igmp host group [ group-address interface interface-type interface-number ] [ verbose ] display interface [ tunnel [ number ] ] [ brief [ description down ] ] display vxlan tunnel [ vxlan vxlan-id ] display statistic mode reset arp suppression vsi [ name vsi-name ] reset l2vpn mac-address [ vsi vsi-name ] Clear packet statistics on VSIs. reset l2vpn statistics vsi [ name vsi-name ] Clear packet statistics on Ethernet service instances. reset l2vpn statistics ac [ interface interface-type interface-number service-instance instance-id ] 18

23 VXLAN configuration examples Unicast-mode VXLAN configuration example Network requirements As shown in Figure 9: Configure VXLAN 10 as a unicast-mode VXLAN on Switch A, Switch B, and Switch C to provide Layer 2 connectivity for the VMs across the network sites. Manually establish VXLAN tunnels and assign the tunnels to VXLAN 10. Enable remote-mac address learning. Figure 9 Network diagram Configuration procedure 1. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 9. (Details not shown.) # Configure OSPF on all transport network switches (Switches A through D). (Details not shown.) 2. Configure Switch A: # Enable L2VPN. <SwitchA> system-view [SwitchA] l2vpn enable # Enable Layer 2 forwarding for all VXLANs. [SwitchA] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch B and Switch C. [SwitchA] interface loopback0 [SwitchA-Loopback0] ip address [SwitchA-Loopback0] quit 19

24 # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 1. [SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source [SwitchA-Tunnel1] destination [SwitchA-Tunnel1] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 2. [SwitchA] interface tunnel 2 mode vxlan [SwitchA-Tunnel2] source [SwitchA-Tunnel2] destination [SwitchA-Tunnel2] quit # Assign Tunnel 1 and Tunnel 2 to VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] tunnel 1 [SwitchA-vsi-vpna-vxlan10] tunnel 2 [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchA] vlan 2 [SwitchA vlan2] port fortygige 1/0/1 [SwitchA vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match frames from VLAN 2. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] service-instance 1000 [SwitchA-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchA-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchA-FortyGigE1/0/1-srv1000] quit [SwitchA-FortyGigE1/0/1] quit 3. Configure Switch B: # Enable L2VPN. <SwitchB> system-view [SwitchB] l2vpn enable # Enable Layer 2 forwarding for all VXLANs. [SwitchB] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch C. [SwitchB] interface loopback0 [SwitchB-Loopback0] ip address [SwitchB-Loopback0] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 2. [SwitchB] interface tunnel 2 mode vxlan 20

25 [SwitchB-Tunnel2] source [SwitchB-Tunnel2] destination [SwitchB-Tunnel2] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source [SwitchB-Tunnel3] destination [SwitchB-Tunnel3] quit # Assign Tunnel 2 and Tunnel 3 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] tunnel 2 [SwitchB-vsi-vpna-vxlan10] tunnel 3 [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchB] vlan 2 [SwitchB vlan2] port fortygige 1/0/1 [SwitchB vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match frames from VLAN 2. [SwitchB] interface fortygige 1/0/1 [SwitchB-FortyGigE1/0/1] service-instance 1000 [SwitchB-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchB-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchB-FortyGigE1/0/1-srv1000] quit [SwitchB-FortyGigE1/0/1] quit 4. Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Enable Layer 2 forwarding for all VXLANs. [SwitchC] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch B. [SwitchC] interface loopback0 [SwitchC-Loopback0] ip address [SwitchC-Loopback0] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 1. [SwitchC] interface tunnel 1 mode vxlan [SwitchC-Tunnel1] source [SwitchC-Tunnel1] destination

26 [SwitchC-Tunnel1] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 3. [SwitchC] interface tunnel 3 mode vxlan [SwitchC-Tunnel3] source [SwitchC-Tunnel3] destination [SwitchC-Tunnel3] quit # Assign Tunnel 1 and Tunnel 3 to VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] tunnel 1 [SwitchC-vsi-vpna-vxlan10] tunnel 3 [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchC] vlan 2 [SwitchC vlan2] port fortygige 1/0/1 [SwitchC vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match frames from VLAN 2. [SwitchC] interface fortygige 1/0/1 [SwitchC-FortyGigE1/0/1] service-instance 1000 [SwitchC-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchC-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchC-FortyGigE1/0/1-srv1000] quit [SwitchC-FortyGigE1/0/1] quit Verifying the configuration 1. Verify the VXLAN settings on the VTEPs. This example uses Switch A. # Verify that the VXLAN tunnel interfaces on the VTEP are up. [SwitchA] display interface tunnel 1 Tunnel1 Current state: UP Line protocol state: UP Description: Tunnel1 Interface Bandwidth: 64kbps Maximum transmission unit: Internet protocol processing: disabled Output queue - Urgent queuing: Size/Length/Discards 0/100/0 Output queue - Protocol queuing: Size/Length/Discards 0/500/0 Output queue - FIFO queuing: Size/Length/Discards 0/75/0 Last clearing of counters: Never Tunnel source , destination Tunnel protocol/transport UDP_VXLAN/IP # Verify that the VXLAN tunnels have been assigned to the VXLAN. [SwitchA] display l2vpn vsi verbose VSI Name: vpna VSI Index : 0 VSI State : Up 22

27 MTU : 1500 Bandwidth : - Broadcast Restrain : - Multicast Restrain : - Unknown Unicast Restrain: - MAC Learning : Enabled MAC Table Limit : - Drop Unknown : - Flooding : Enabled VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flooding proxy Tunnel1 0x Up Manual Disabled Tunnel2 0x Up Manual Disabled ACs: AC Link ID State FGE1/0/1 srv Up # Verify that the VTEP has learned the MAC addresses of remote VMs. <SwitchA> display l2vpn mac-address MAC Address State VSI Name Link ID/Name Aging cc3e-5f9c-6cdb Dynamic vpna Tunnel1 Aging cc3e-5f9c-23dc Dynamic vpna Tunnel2 Aging mac address(es) found Verify that VM 1, VM 2, and VM 3 can ping each other. (Details not shown.) Multicast-mode VXLAN configuration example Network requirements As shown in Figure 10: Configure VXLAN 10 as a multicast-mode VXLAN on Switch A, Switch B, and Switch C to provide Layer 2 connectivity for the VMs across the network sites. Manually establish VXLAN tunnels and assign the tunnels to VXLAN 10. Use the default remote-mac address learning method (MAC address learning in the data plane). 23

28 Figure 10 Network diagram Table 1 IP address assignment Device Interface IP address Device Interface IP address Switch A: Switch C: VLAN-interface /24 VLAN-interface /24 Switch D: Switch E: VLAN-interface /24 VLAN-interface /24 VLAN-interface /24 VLAN-interface /24 Switch F: Switch G: VLAN-interface /24 VLAN-interface /24 VLAN-interface /24 VLAN-interface /24 VLAN-interface /24 Switch B: Loop /32 VLAN-interface /24 Configuration procedure 1. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 10. (Details not shown.) # Configure OSPF on all transport network switches (Switches A through G). (Details not shown.) 2. Configure Switch A: # Enable L2VPN. <SwitchA> system-view 24

29 [SwitchA] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchA] undo vxlan ip-forwarding # Enable IP multicast routing. [SwitchA] multicast routing [SwitchA-mrib] quit # Create the VSI vpna and VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit # Assign an IP address to VLAN-interface 11, and enable the IGMP host feature on the interface. This interface's IP address will be the source IP address of VXLAN packets sent by the VTEP. [SwitchA] interface vlan-interface 11 [SwitchA-Vlan-interface11] ip address [SwitchA-Vlan-interface11] igmp host enable [SwitchA-Vlan-interface11] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 1. [SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source [SwitchA-Tunnel1] destination [SwitchA-Tunnel1] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 2. [SwitchA] interface tunnel 2 mode vxlan [SwitchA-Tunnel2] source [SwitchA-Tunnel2] destination [SwitchA-Tunnel2] quit # Assign Tunnel 1 and Tunnel 2 to VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] tunnel 1 [SwitchA-vsi-vpna-vxlan10] tunnel 2 # Configure the multicast group address and source IP address for multicast VXLAN packets. [SwitchA-vsi-vpna-vxlan10] group source [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchA] vlan 2 [SwitchA vlan2] port fortygige 1/0/1 [SwitchA vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] service-instance 1000 [SwitchA-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchA-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchA-FortyGigE1/0/1-srv1000] quit 25

30 [SwitchA-FortyGigE1/0/1] quit 3. Configure Switch B: # Enable L2VPN. <SwitchB> system-view [SwitchB] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchB] undo vxlan ip-forwarding # Enable IP multicast routing. [SwitchB] multicast routing [SwitchB-mrib] quit # Create the VSI vpna and VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Assign an IP address to VLAN-interface 12, and enable the IGMP host feature on the interface. This interface's IP address will be the source IP address of VXLAN packets sent by the VTEP. [SwitchB] interface vlan-interface 12 [SwitchB-Vlan-interface12] ip address [SwitchB-Vlan-interface12] igmp host enable [SwitchB-Vlan-interface12] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 2. [SwitchB] interface tunnel 2 mode vxlan [SwitchB-Tunnel2] source [SwitchB-Tunnel2] destination [SwitchB-Tunnel2] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source [SwitchB-Tunnel3] destination [SwitchB-Tunnel3] quit # Assign Tunnel 2 and Tunnel 3 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] tunnel 2 [SwitchB-vsi-vpna-vxlan10] tunnel 3 # Configure the VXLAN multicast group address and the source IP address for VXLAN packets. [SwitchB-vsi-vpna-vxlan10] group source [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchB] vlan 2 [SwitchB vlan2] port fortygige 1/0/1 [SwitchB vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchB] interface fortygige 1/0/1 [SwitchB-FortyGigE1/0/1] service-instance

31 [SwitchB-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchB-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchB-FortyGigE1/0/1-srv1000] quit [SwitchB-FortyGigE1/0/1] quit 4. Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchC] undo vxlan ip-forwarding # Enable IP multicast routing. [SwitchC] multicast routing [SwitchC-mrib] quit # Create the VSI vpna and VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Assign an IP address to VLAN-interface 13, and enable the IGMP host feature on the interface. This interface's IP address will be the source IP address of VXLAN packets sent by the VTEP. [SwitchC] interface vlan-interface 13 [SwitchC-Vlan-interface13] ip address [SwitchC-Vlan-interface13] igmp host enable [SwitchC-Vlan-interface13] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 1. [SwitchC] interface tunnel 1 mode vxlan [SwitchC-Tunnel1] source [SwitchC-Tunnel1] destination [SwitchC-Tunnel1] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 3. [SwitchC] interface tunnel 3 mode vxlan [SwitchC-Tunnel3] source [SwitchC-Tunnel3] destination [SwitchC-Tunnel3] quit # Assign Tunnel 1 and Tunnel 3 to VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] tunnel 1 [SwitchC-vsi-vpna-vxlan10] tunnel 3 # Configure the multicast group address and source IP address for VXLAN multicast packets. [SwitchC-vsi-vpna-vxlan10] group source [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchC] vlan 2 [SwitchC vlan2] port fortygige 1/0/1 27

32 [SwitchC vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchC] interface fortygige 1/0/1 [SwitchC-FortyGigE1/0/1] service-instance 1000 [SwitchC-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchC-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchC-FortyGigE1/0/1-srv1000] quit [SwitchC-FortyGigE1/0/1] quit 5. Configure Switch D: # Enable IP multicast routing. <SwitchD> system-view [SwitchD] multicast routing [SwitchD-mrib] quit # Enable IGMP and PIM-SM on VLAN-interface 11. [SwitchD] interface vlan-interface 11 [SwitchD-Vlan-interface11] igmp enable [SwitchD-Vlan-interface11] pim sm [SwitchD-Vlan-interface11] quit # Enable PIM-SM on VLAN-interface 21. [SwitchD] interface vlan-interface 21 [SwitchD-Vlan-interface21] pim sm [SwitchD-Vlan-interface21] quit 6. Configure Switch E: # Enable IP multicast routing. <SwitchE> system-view [SwitchE] multicast routing [SwitchE-mrib] quit # Enable IGMP and PIM-SM on VLAN-interface 13. [SwitchE] interface vlan-interface 13 [SwitchE-Vlan-interface13] igmp enable [SwitchE-Vlan-interface13] pim sm [SwitchE-Vlan-interface13] quit # Enable PIM-SM on VLAN-interface 23. [SwitchE] interface vlan-interface 23 [SwitchE-Vlan-interface23] pim sm [SwitchE-Vlan-interface23] quit 7. Configure Switch F: # Enable IP multicast routing. <SwitchF> system-view [SwitchF] multicast routing [SwitchF-mrib] quit # Enable PIM-SM on VLAN-interface 21, VLAN-interface 22, and VLAN-interface 23. [SwitchF] interface vlan-interface 21 [SwitchF-Vlan-interface21] pim sm [SwitchF-Vlan-interface21] quit [SwitchF] interface vlan-interface 22 28

33 [SwitchF-Vlan-interface22] pim sm [SwitchF-Vlan-interface22] quit [SwitchF] interface vlan-interface 23 [SwitchF-Vlan-interface23] pim sm [SwitchF-Vlan-interface23] quit 8. Configure Switch G: # Enable IP multicast routing. <SwitchG> system-view [SwitchG] multicast routing [SwitchG-mrib] quit # Enable IGMP and PIM-SM on VLAN-interface 12. [SwitchG] interface vlan-interface 12 [SwitchG-Vlan-interface12] igmp enable [SwitchG-Vlan-interface12] pim sm [SwitchG-Vlan-interface12] quit # Enable PIM-SM on VLAN-interface 22. [SwitchG] interface vlan-interface 22 [SwitchG-Vlan-interface22] pim sm [SwitchG-Vlan-interface22] quit Verifying the configuration 1. Verify the VXLAN settings on the VTEPs. This example uses Switch A. # Verify that the VXLAN tunnel interfaces on the VTEP are up. [SwitchA] display interface tunnel 1 Tunnel1 Current state: UP Line protocol state: UP Description: Tunnel1 Interface Bandwidth: 64 kbps Maximum transmission unit: Internet protocol processing: Disabled Last clearing of counters: Never Tunnel source , destination Tunnel protocol/transport UDP_VXLAN/IP # Verify that the VXLAN tunnels have been assigned to the VXLAN. [SwitchA] display l2vpn vsi verbose VSI Name: vpna VSI Index : 0 VSI State : Up MTU : 1500 Bandwidth : - Broadcast Restrain : - Multicast Restrain : - Unknown Unicast Restrain: - MAC Learning : Enabled MAC Table Limit : - Drop Unknown : Disabled Flooding : Enabled 29

34 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flooding proxy Tunnel1 0x Up Manual Disabled Tunnel2 0x Up Manual Disabled MTunnel0 0x Up Auto Disabled ACs: AC Link ID State FGE1/0/1 srv Up # Verify that the VTEP has learned the MAC addresses of remote VMs. <SwitchA> display l2vpn mac-address MAC Address State VSI Name Link ID/Name Aging cc3e-5f9c-6cdb Dynamic vpna Tunnel1 Aging cc3e-5f9c-23dc Dynamic vpna Tunnel2 Aging mac address(es) found --- # Verify that the VTEP has joined the VXLAN multicast group on VLAN-interface 11. <SwitchA> display igmp host group IGMP host groups in total: 1 Vlan-interface11( ): IGMP host groups in total: 1 Group address Member state Expires Idle Off 2. Verify that VM 1, VM 2, and VM 3 can ping each other. (Details not shown.) 30

35 Configuring VXLAN IP gateways Overview The following are available IP gateway placement designs for VXLANs: VXLAN IP gateways separated from VTEPs Use a VXLAN-unaware device as a gateway to the external network for VXLANs. On the gateway, you do not need to configure VXLAN settings. VXLAN IP gateways collocated with VTEPs Include the following placement designs: Centralized VXLAN IP gateway deployment Use one VTEP to provide Layer 3 forwarding for VXLANs. Typically, the gateway-collocated VTEP connects to other VTEPs and the external network. To use this design, make sure the IP gateway has sufficient bandwidth and processing capability. Centralized VXLAN IP gateway group deployment Use one VTEP group that contains redundant centralized VXLAN IP gateways to provide reliable gateway services for VXLANs. In a collocation design, the VTEPs use virtual Layer 3 VSI interfaces as gateway interfaces to provide services for VXLANs. VXLAN IP gateways separated from VTEPs As shown in Figure 11, an independent VXLAN IP gateway connects a Layer 3 network to a VTEP. VMs send Layer 3 traffic to the gateway through VXLAN tunnels. When the Layer 3 traffic arrives, the VTEP terminates the VXLANs and forwards the inner Layer 2 frames to the gateway. In this solution, the VTEP does not perform Layer 3 forwarding for VXLANs. Figure 11 VXLAN IP gateway separated from VTEPs 31

36 Centralized VXLAN IP gateway deployment As shown in Figure 12, a VTEP acts as a gateway for VMs in the VXLANs. The VTEP both terminates the VXLANs and performs Layer 3 forwarding for the VMs. In this solution, the VTEP provides gateway services for VXLANs on virtual Layer 3 VSI interfaces. Figure 12 Centralized VXLAN IP gateway deployment As shown in Figure 13, the network uses the following process to forward Layer 3 traffic from VM to the Layer 3 network: 1. The VM sends an ARP request to obtain the MAC address of the gateway (VTEP 3) at VTEP 1 floods the ARP request to all remote VTEPs. 3. VTEP 3 de-encapsulates the ARP request, creates an ARP entry for the VM, and sends an ARP reply to the VM. 4. VTEP 1 forwards the ARP reply to the VM. 5. The VM learns the MAC address of the gateway, and sends the Layer 3 traffic to the gateway. 6. VTEP 3 removes the VXLAN encapsulation and inner Ethernet header for the traffic, and forwards the traffic to the destination node. 32

37 Figure 13 ARP learning on the VTEP that acts as a VXLAN IP gateway VM VSI/VXLAN 10 VSI/VXLAN VM VM VSI/VXLAN 20 VSI/VXLAN VM VM VSI/VXLAN 30 IP transport network VSI/VXLAN VM VXLAN tunnel Server VXLAN tunnel VTEP 1 P VTEP 2 VXLAN tunnel Server Site 1 Site 2 VTEP 3/VXLAN IP gateway VSI/VXLAN 10 VSI-interface /24 L3 network VSI/VXLAN 20 VSI/VXLAN 30 VSI-interface /24 VSI-interface /24 Centralized VXLAN IP gateway group deployment IMPORTANT: This feature is available in Release 2137 and later versions. As shown in Figure 14, a VTEP group uses redundant centralized VXLAN IP gateways to provide reliable gateway services for VMs in the VXLANs. All member VTEPs in the group participate in Layer 3 forwarding and load share traffic between the Layer 3 network and the VXLANs. This design distributes processing among multiple VTEPs and prevents single points of failure. 33

38 Figure 14 Example of centralized VXLAN IP gateway group deployment L3 network Centralized VXLAN IP gateway group VXLAN tunnel VXLAN tunnel P Access layer VXLAN tunnel Access layer Server VTEP Transport VTEP network Server Site 1 Site 2 The VTEP group is a virtual gateway that provides services at a group IP address. Access layer VTEPs set up VXLAN tunnels to the group IP address for data traffic forwarding. Each VTEP in the group automatically uses its member IP address to set up tunnels to the other member VTEPs and access layer VTEPs. The tunnels are used to transmit protocol packets and synchronize ARP entries. Hardware compatibility An FX card cannot act as a VTEP for its local site if the card is a centralized VXLAN IP gateway. Configuration prerequisites Before you configure a centralized VXLAN IP gateway, you must perform the following tasks on VTEPs: Enable Layer 3 forwarding for VXLANs. Create VSIs and VXLANs. Configure VXLAN tunnels and assign them to VXLANs. Configuring a centralized VXLAN IP gateway on a VTEP Step Command Remarks 1. Enter system view. system-view N/A 2. Create a VSI interface and enter VSI interface view. 3. Assign an IP address to the VSI interface. interface vsi-interface vsi-interface-id ip address ip-address { mask mask-length } [ sub ] 34 By default, no VSI interfaces are created on the device. By default, no IP address is assigned to a VSI interface.

39 Step Command Remarks 4. Return to system view. quit N/A 5. Enter VSI view. vsi vsi-name N/A 6. Specify a gateway interface for the VSI. gateway vsi-interface vsi-interface-id By default, no gateway interface is specified for a VSI. Configuring a centralized VXLAN IP gateway group IMPORTANT: This feature is available in Release 2137 and later versions. Configuring a VTEP group Make sure the member VTEPs use the same VXLAN settings. To configure a VTEP group on a member VTEP: Step Command Remarks 1. Enter system view. system-view N/A 2. Create a VSI interface and enter VSI interface view. 3. Assign an IP address to the VSI interface. 4. Assign a MAC address to the VSI interface. interface vsi-interface vsi-interface-id ip address ip-address { mask mask-length } [ sub ] mac-address mac-address By default, no VSI interfaces exist. You must create the same VSI interface on all VTEPs in the VTEP group. By default, no IP address is assigned to a VSI interface. You must assign the same IP address to the VSI interface on each VTEP in the VTEP group. By default, the MAC address of VLAN interfaces applies. You must assign the same MAC address to the VSI interface on each VTEP in the VTEP group. 5. Return to system view. quit N/A 6. Enter VSI view. vsi vsi-name N/A 7. Specify a gateway interface for the VSI. gateway vsi-interface vsi-interface-id By default, no gateway interface is specified for a VSI. 8. Return to system view. quit N/A 9. Assign the local VTEP to a VTEP group and specify the member IP address for the VTEP. vtep group group-ip member local member-ip 35 By default, a VTEP is not assigned to any VTEP group. Perform this task on all member VTEPs in the VTEP group. The IP address specified by the member-ip argument must already

40 Step Command Remarks exist on the local VTEP. You must configure a routing protocol to advertise the IP address in the transport network. Member VTEPs in a VTEP group cannot use the group IP address or share an IP address. 10. Specify all the other VTEPs in the VTEP group. vtep group group-ip member remote member-ip&<1-8> By default, no VTEP group is specified. Perform this task on all member VTEPs in the VTEP group. Specifying a VTEP group as the gateway for an access layer VTEP Before you specify a VTEP group on an access layer VTEP, perform the following tasks on the VTEP: Enable Layer 2 forwarding for VXLANs. Configure VSIs and VXLANs. Set up VXLAN tunnels to remote sites and the VTEP group, and assign the tunnels to VXLANs. To specify a VTEP group as the gateway for an access layer VTEP: Step Command Remarks 1. Enter system view. system-view N/A 2. Specify a VTEP group and all its member VTEPs. vtep group group-ip member remote member-ip&<1-8> By default, no VTEP group is specified. Configuring a VSI interface Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSI interface view. 3. (Optional.) Assign a MAC address to the VSI interface. interface vsi-interface vsi-interface-id mac-address mac-address N/A By default, the MAC address of a VSI interface is the same as the MAC address of VLAN interfaces. If the specified MAC address has the same higher 36 bits as the device's bridge MAC address, the specified MAC address is assigned to the VSI interface. If the specified MAC address does not meet this requirement, the default MAC address is assigned to the VSI interface. 4. Set an ARP packet sending arp send-rate pps By default, the ARP packet sending 36

41 Step Command Remarks rate limit for the VSI interface. rate is not limited for a VSI interface. This command is available in Release 2137 and later versions. 5. (Optional.) Configure a description for the VSI interface. 6. (Optional.) Set the MTU for the VSI interface. 7. (Optional.) Set the expected bandwidth for the VSI interface. 8. (Optional.) Restore the default settings on the interface 9. (Optional.) Bring up the interface. description text mtu mtu-value bandwidth bandwidth-value default undo shutdown The default description of a VSI interface is interface-name plus Interface (for example, Vsi-interface100 Interface). The default MTU is 1500 bytes. The default expected bandwidth is kbps. N/A By default, a VSI interface is up. Enabling packet statistics for VSI interfaces The statistic mode vsi command takes effect only if the VSI interface is associated with only one VSI. If you execute the statistic mode command multiple times, the most recent configuration takes effect. To enable packet statistics for VSI interfaces: Step Command Remarks 1. Enter system view. system-view N/A 2. Set the packet statistic collection mode to VSI. statistic mode vsi By default, the packet statistic collection mode is VSI. 3. Enter VSI view. vsi vsi-name N/A 4. Enable the packet statistics feature for the VSI. statistics enable By default, the packet statistics feature is disabled for all VSIs. Displaying and maintaining VXLAN IP gateway Execute display commands in any view and reset commands in user view. Task Display information about VSI interfaces. Command display interface [ vsi-interface [ vsi-interface-id ] ] [ brief [ description ] ] Clear statistics on VSI interfaces. reset counters interface [ vsi-interface [ vsi-interface-id ] ] 37

42 VXLAN IP gateway configuration examples Centralized VXLAN IP gateway configuration example Network requirements As shown in Figure 15: Configure VXLAN 10 on Switch A through Switch C to provide Layer 2 connectivity for the VMs across the network sites. Configure a VXLAN IP gateway on Switch B to provide Layer 3 forwarding services for VMs in VXLAN 10. Figure 15 Network diagram Configuration procedure 1. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 15. (Details not shown.) # Configure OSPF on all transport network switches (Switches A through D). (Details not shown.) # Configure OSPF to advertise routes to networks /24 and /24 on Switch B and Switch E. (Details not shown.) 2. Configure Switch A: # Enable L2VPN. <SwitchA> system-view [SwitchA] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchA] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit 38

43 # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch B and Switch C. [SwitchA] interface loopback0 [SwitchA-Loopback0] ip address [SwitchA-Loopback0] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 1. [SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source [SwitchA-Tunnel1] destination [SwitchA-Tunnel1] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 2. [SwitchA] interface tunnel 2 mode vxlan [SwitchA-Tunnel2] source [SwitchA-Tunnel2] destination [SwitchA-Tunnel2] quit # Assign Tunnel 1 and Tunnel 2 to VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan10] tunnel 1 [SwitchA-vsi-vpna-vxlan10] tunnel 2 [SwitchA-vsi-vpna-vxlan10] quit [SwitchA-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchA] vlan 2 [SwitchA vlan2] port fortygige 1/0/1 [SwitchA vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match frames from VLAN 2. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] service-instance 1000 [SwitchA-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchA-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchA-FortyGigE1/0/1-srv1000] quit [SwitchA-FortyGigE1/0/1] quit 3. Configure Switch B: # Enable L2VPN. <SwitchB> system-view [SwitchB] l2vpn enable # Reserve a global-type VLAN interface resource. In this example, VLAN-interface 3000 is reserved. [SwitchB] reserve-vlan-interface 3000 global # Create the VSI vpna and VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch C. 39

44 [SwitchB] interface loopback0 [SwitchB-Loopback0] ip address [SwitchB-Loopback0] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 2. [SwitchB] interface tunnel 2 mode vxlan [SwitchB-Tunnel2] source [SwitchB-Tunnel2] destination [SwitchB-Tunnel2] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source [SwitchB-Tunnel3] destination [SwitchB-Tunnel3] quit # Assign Tunnel 2 and Tunnel 3 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan10] tunnel 2 [SwitchB-vsi-vpna-vxlan10] tunnel 3 [SwitchB-vsi-vpna-vxlan10] quit [SwitchB-vsi-vpna] quit # Create VSI-interface 1 and assign the interface an IP address. The IP address will be used as the gateway of VXLAN 10. [SwitchB] interface vsi-interface 1 [SwitchB-Vsi-interface1] ip address [SwitchB-Vsi-interface1] quit # Specify VSI-interface 1 as the gateway interface for the VSI vpna. [SwitchB] vsi vpna [SwitchB-vsi-vpna] gateway vsi-interface 1 [SwitchB-vsi-vpna] quit 4. Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchC] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch B. [SwitchC] interface loopback0 [SwitchC-Loopback0] ip address [SwitchC-Loopback0] quit # Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 1. [SwitchC] interface tunnel 1 mode vxlan [SwitchC-Tunnel1] source

45 [SwitchC-Tunnel1] destination [SwitchC-Tunnel1] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 3. [SwitchC] interface tunnel 3 mode vxlan [SwitchC-Tunnel3] source [SwitchC-Tunnel3] destination [SwitchC-Tunnel3] quit # Assign Tunnel 1 and Tunnel 3 to VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan10] tunnel 1 [SwitchC-vsi-vpna-vxlan10] tunnel 3 [SwitchC-vsi-vpna-vxlan10] quit [SwitchC-vsi-vpna] quit # Create VLAN 2, and assign FortyGigE 1/0/1 to VLAN 2. [SwitchC] vlan 2 [SwitchC vlan2] port fortygige 1/0/1 [SwitchC vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match frames from VLAN 2. [SwitchC] interface fortygige 1/0/1 [SwitchC-FortyGigE1/0/1] service-instance 1000 [SwitchC-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchC-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchC-FortyGigE1/0/1-srv1000] quit [SwitchC-FortyGigE1/0/1] quit Verifying the configuration 1. Verify the VXLAN IP gateway settings on Switch B: # Verify that the VXLAN tunnel interfaces are up on Switch B. [SwitchB] display interface tunnel 2 Tunnel2 Current state: UP Line protocol state: UP Description: Tunnel1 Interface Bandwidth: 64kbps Maximum transmission unit: Internet protocol processing: disabled Output queue - Urgent queuing: Size/Length/Discards 0/100/0 Output queue - Protocol queuing: Size/Length/Discards 0/500/0 Output queue - FIFO queuing: Size/Length/Discards 0/75/0 Last clearing of counters: Never Tunnel source , destination Tunnel protocol/transport UDP_VXLAN/IP # Verify that VSI-interface 1 is up. [SwitchB] display interface vsi-interface 1 Vsi-interface1 Current state: UP 41

46 Line protocol state: UP Description: Vsi-interface100 Interface Bandwidth: kbps Maximum transmission unit: 1500 Internet Address is /24 (primary) IP packet frame type: Ethernet II, hardware address: IPv6 packet frame type: Ethernet II, hardware address: Physical: Unknown, baudrate: kbps Last clearing of counters: Never Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops # Verify that the VXLAN tunnels have been assigned to the VXLAN, and VSI-interface 1 is the gateway interface of the VSI vpna. [SwitchB] display l2vpn vsi verbose VSI Name: vpna VSI Index : 0 VSI State : Up MTU : 1500 Bandwidth : - Broadcast Restrain : - Multicast Restrain : - Unknown Unicast Restrain: - MAC Learning : Enabled MAC Table Limit : - Drop Unknown : - Flooding : Enabled Gateway interface : VSI-interface 1 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flooding proxy Tunnel2 0x Up Manual Disabled Tunnel3 0x Up Manual Disabled # Verify that Switch B has created ARP entries for the VMs. [SwitchB] display arp Type: S-Static D-Dynamic O-Openflow R-Rule M-Multiport I-Invalid IP Address MAC Address VID Interface/Link ID Aging Type c-29c1-5e46 N/A Vlan20 19 D N/A Vsi10 20 D N/A Vsi10 19 D # Verify that Switch B has created FIB entries for the VMs. [SwitchB] display fib Destination count: 1 FIB entry count: 1 Flag: U:Useable G:Gateway H:Host B:Blackhole D:Dynamic S:Static R:Relay F:FRR Destination/Mask Nexthop Flag OutInterface/Token Label 42

47 / UH Vsi100 Null 2. Verify that the VMs can access the WAN: # Verify that VM 1 and VM 2 can ping each other. (Details not shown.) # Verify that VM 1, VM 2, and VLAN-interface 20 ( ) on Switch E can ping each other. (Details not shown.) Centralized VXLAN IP gateway group configuration example IMPORTANT: This example is available in Release 2137 and later versions. Network requirements As shown in Figure 16: Configure VXLAN 10 as a unicast-mode VXLAN on Switch A, Switch B, and Switch C. Manually establish VXLAN tunnels and assign the tunnels to VXLAN 10. Assign Switch B and Switch C to a VTEP group to provide gateway services for VXLAN 10. Figure 16 Network diagram Configuration procedure 1. On VM 1, specify as the gateway address. (Details not shown.) 2. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 16. (Details not shown.) # Configure OSPF on all transport network switches (Switches A through D). (Details not shown.) 3. Configure Switch A: # Enable L2VPN. <SwitchA> system-view [SwitchA] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchA] undo vxlan ip-forwarding # Create the VSI vpna and VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 43

48 [SwitchA-vsi-vpna-vxlan-10] quit [SwitchA-vsi-vpna] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnel to the VTEP group. [SwitchA] interface loopback 0 [SwitchA-Loopback0] ip address [SwitchA-Loopback0] quit # Create a VXLAN tunnel to the VTEP group. The tunnel interface name is Tunnel 1. [SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source [SwitchA-Tunnel1] destination [SwitchA-Tunnel1] quit # Assign Tunnel 1 to VXLAN 10. [SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan-10] tunnel 1 [SwitchA-vsi-vpna-vxlan-10] quit [SwitchA-vsi-vpna] quit # Create VLAN 2 and assign FortyGigE 1/0/1 to VLAN 2. [SwitchA] vlan 2 [SwitchA vlan2] port fortygige 1/0/1 [SwitchA vlan2] quit # On FortyGigE 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] service-instance 1000 [SwitchA-FortyGigE1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to the VSI vpna. [SwitchA-FortyGigE1/0/1-srv1000] xconnect vsi vpna [SwitchA-FortyGigE1/0/1-srv1000] quit [SwitchA-FortyGigE1/0/1] quit # Specify the VTEP group and its member VTEPs at and [SwitchA] vtep group member remote Configure Switch B: # Enable L2VPN. <SwitchB> system-view [SwitchB] l2vpn enable # Reserve the global resource of VLAN-interface [SwitchB] reserve-vlan-interface 3000 global # Create the VSI vpna and VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] quit [SwitchB-vsi-vpna] quit # Assign IP address /32 to Loopback 0. The IP address will be used as the IP address of the VTEP group. [SwitchB] interface loopback 0 [SwitchB-Loopback0] ip address [SwitchB-Loopback0] quit 44

49 # Assign an IP address to Loopback 1. The IP address will be used as the member IP address of the VTEP. [SwitchB] interface loopback 1 [SwitchB-Loopback1] ip address [SwitchB-Loopback1] quit # Create a VXLAN tunnel to Switch A. The tunnel source IP address is , and the tunnel interface name is Tunnel 2. [SwitchB] interface tunnel 2 mode vxlan [SwitchB-Tunnel2] source [SwitchB-Tunnel2] destination [SwitchB-Tunnel2] quit # Assign Tunnel 2 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] tunnel 2 [SwitchB-vsi-vpna-vxlan-10] quit [SwitchB-vsi-vpna] quit # Create VSI-interface 1 and assign the interface an IP address. The IP address will be used as the gateway address for VXLAN 10. Assign a MAC address to the interface. [SwitchB] interface vsi-interface 1 [SwitchB-Vsi-interface1] ip address [SwitchB-Vsi-interface1] mac-address [SwitchB-Vsi-interface1] quit # Specify VSI-interface 1 as the gateway interface for the VSI vpna. [SwitchB] vsi vpna [SwitchB-vsi-vpna] gateway vsi-interface 1 [SwitchB-vsi-vpna] quit # Assign the local VTEP to the VTEP group , and specify the member IP address of the local VTEP. [SwitchB] vtep group member local # Specify the other member VTEP Switch C. [SwitchB] vtep group member remote Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Reserve the global resource of VLAN-interface [SwitchC] reserve-vlan-interface 3000 global # Create the VSI vpna and VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan-10] quit [SwitchC-vsi-vpna] quit # Assign IP address /32 to Loopback 0. The IP address will be used as the IP address of the VTEP group. [SwitchC] interface loopback 0 [SwitchC-Loopback0] ip address [SwitchC-Loopback0] quit 45

50 # Assign an IP address to Loopback 1. The IP address will be used as the member IP address of the VTEP. [SwitchC] interface loopback 1 [SwitchC-Loopback1] ip address [SwitchC-Loopback1] quit # Create a VXLAN tunnel to Switch A. The tunnel source IP address is , and the tunnel interface name is Tunnel 2. [SwitchC] interface tunnel 2 mode vxlan [SwitchC-Tunnel2] source [SwitchC-Tunnel2] destination [SwitchC-Tunnel2] quit # Assign Tunnel 2 to VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan-10] tunnel 2 [SwitchC-vsi-vpna-vxlan-10] quit [SwitchC-vsi-vpna] quit # Create VSI-interface 1 and assign the interface an IP address. The IP address will be used as the gateway address for VXLAN 10. Assign a MAC address to the interface. [SwitchC] interface vsi-interface 1 [SwitchC-Vsi-interface1] ip address [SwitchC-Vsi-interface1] mac-address [SwitchC-Vsi-interface1] quit # Specify VSI-interface 1 as the gateway interface for the VSI vpna. [SwitchC] vsi vpna [SwitchC-vsi-vpna] gateway vsi-interface 1 [SwitchC-vsi-vpna] quit # Assign the local VTEP to the VTEP group , and specify the member IP address of the local VTEP. [SwitchC] vtep group member local # Specify the other member VTEP Switch B. [SwitchC] vtep group member remote

51 Configuring the VTEP as an OVSDB VTEP Overview A Hewlett Packard Enterprise network virtualization controller can use the Open vswitch Database (OVSDB) management protocol to deploy and manage VXLANs on VTEPs. To work with a controller, you must configure the VTEP as an OVSDB VTEP. As shown in Figure 17, an OVSDB VTEP stores all of its VXLAN settings in the form of entries in an OVSDB database. The OVSDB database, OVSDB VTEP service, and the controller interact through the OVSDB server. The controller communicates with the OVSDB server through the OVSDB protocol to manage the OVSDB database. The OVSDB VTEP service reads and writes data in the OVSDB database through the OVSDB server. The OVSDB VTEP service performs the following operations to manage the VXLAN settings on the VTEP: Converts data in the OVSDB database into VXLAN configuration and deploys the configuration to the VTEP. For example, create or remove a VXLAN or VXLAN tunnel. Adds site-facing interface information and the global source address of VXLAN tunnels to the OVSDB database. The information is reported to the controller by the OVSDB server. You can configure a VTEP both at the CLI and through a controller. As a best practice, do not manually remove the VXLAN configuration issued by the controller. Figure 17 OVSDB network model Feature and software version compatibility The OVSDB VTEP feature is available in Release 2137 and later versions. Protocols and standards RFC 7047, The Open vswitch Database Management Protocol OVSDB VTEP configuration task list Tasks at a glance (Required.) Setting up an OVSDB connection to a controller: Configuring active SSL connection settings Configuring passive SSL connection settings Configuring active TCP connection settings 47

52 Tasks at a glance Configuring passive TCP connection settings (Required.) Enabling the OVSDB server (Required.) Enabling the OVSDB VTEP service (Required.) Specifying a global source address for VXLAN tunnels (Required.) Specifying a VTEP access port (Optional.) Enabling flood proxy on multicast VXLAN tunnels Configuration prerequisites Before you configure the VTEP as an OVSDB VTEP, enable L2VPN by using the l2vpn enable command. Before you set up SSL connections to controllers, you must configure SSL as described in Security Configuration Guide. Setting up an OVSDB connection to a controller The OVSDB server supports the following types of OVSDB connections: Active SSL connection The OVSDB server initiates an SSL connection to the controller. Passive SSL connection The OVSDB server accepts the SSL connection from the controller. Active TCP connection The OVSDB server initiates a TCP connection to the controller. Passive TCP connection The OVSDB server accepts the TCP connection from the controller. Configuration restrictions and guidelines When you set up OVSDB connections, follow these restrictions and guidelines: You can set up multiple OVSDB connections. For the device to establish the connections, you must enable the OVSDB server. You must disable and then re-enable the OVSDB server if it has been enabled. You must specify the same PKI domain and CA certificate file for all active and passive SSL connections. Make sure you have configured the PKI domain before specify it for SSL. For more information about configuring a PKI domain, see Security Configuration Guide. Configuring active SSL connection settings Step Command Remarks 1. Enter system view. system-view N/A 2. Specify a PKI domain for SSL. 3. (Optional.) Specify a CA certificate file for SSL. ovsdb server pki domain domain-name ovsdb server bootstrap ca-certificate ca-filename By default, no PKI domain is specified for SSL. By default, SSL uses the CA certificate file in the PKI domain. 48

53 Step Command Remarks If the specified CA certificate file does not exist, the device obtains a self-signed certificate from the controller. The obtained file uses the name specified for the ca-filename argument. 4. Set up an active SSL connection. ovsdb server ssl ip ip-address port port-number By default, the device does not have active OVSDB SSL connections. You can set up a maximum of eight OVSDB SSL connections. Configuring passive SSL connection settings Step Command Remarks 1. Enter system view. system-view N/A 2. Specify a PKI domain for SSL. 3. (Optional.) Specify a CA certificate file for SSL. 4. Enable the device to listen for SSL connection requests. ovsdb server pki domain domain-name ovsdb server bootstrap ca-certificate ca-filename ovsdb server pssl [ port port-number ] By default, no PKI domain is specified for SSL. By default, SSL uses the CA certificate file in the PKI domain. If the specified CA certificate file does not exist, the device obtains a self-signed certificate from the controller. The obtained file uses the name specified for the ca-filename argument. By default, the device does not listen for SSL connection requests. You can specify only one port to listen for OVSDB SSL connection requests. Port 6640 is used if you do specify a port when you execute the command. Configuring active TCP connection settings Step Command Remarks 1. Enter system view. system-view N/A 2. Set up an active TCP connection. ovsdb server tcp ip ip-address port port-number By default, the device does not have active OVSDB TCP connections. You can set up a maximum of eight active OVSDB TCP connections. Configuring passive TCP connection settings Step Command Remarks 1. Enter system view. system-view N/A 2. Enable the device to listen for TCP ovsdb server ptcp [ port port-number ] By default, the device does not listen for TCP connection requests. 49

54 Step Command Remarks connection requests. You can specify only one port to listen for OVSDB TCP connection requests. Port 6640 is used if you do specify a port when you execute the command. Enabling the OVSDB server Make sure you have complete OVSDB connection setup before you enable the OVSDB server. If you change OVSDB connection settings after the OVSDB server is enabled, you must disable and then re-enable the OVSDB server for the change to take effect. To enable the OVSDB server: Step Command Remarks 1. Enter system view. system-view N/A 2. Enable the OVSDB server. ovsdb server enable By default, the OVSDB server is disabled. Enabling the OVSDB VTEP service Step Command Remarks 1. Enter system view. system-view N/A 2. Enable the OVSDB VTEP service. vtep enable By default, the OVSDB VTEP service is disabled. Specifying a global source address for VXLAN tunnels IMPORTANT: For correct VXLAN deployment and VTEP management, do not manually specify tunnel-specific source addresses for VXLAN tunnels if OVSDB is used. The VTEP reports the global VXLAN tunnel source address to the controller for VXLAN tunnel setup. To specify a global source address for VXLAN tunnels: Step Command Remarks 1. Enter system view. system-view N/A 2. Specify a global source address for VXLAN tunnels. tunnel global source-address ip-address By default, no global source address is specified for VXLAN tunnels. 50

55 Specifying a VTEP access port For the controller to manage a site-facing interface, you must specify the interface as a VTEP access port. To specify a VTEP access port: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter interface view. 3. Specify the interface as a VTEP access port. Enter Layer 2 Ethernet interface view: interface interface-type interface-number Enter Layer 2 aggregate interface view: interface bridge-aggregation interface-number vtep access port N/A By default, an interface is not a VTEP access port. Enabling flood proxy on multicast VXLAN tunnels If you use a flood proxy server, you must enable flood proxy globally on multicast tunnels. Then the multicast tunnels are converted into flood proxy tunnels. The VTEP sends broadcast, multicast, and unknown unicast traffic for a VXLAN to the flood proxy server through the tunnels. The flood proxy server then replicates and forwards flood traffic to remote VTEPs. To enable flood proxy on multicast VXLAN tunnels: Step Command Remarks 1. Enter system view. system-view N/A 2. Enable flood proxy on multicast VXLAN tunnels. vxlan tunnel service node By default, flood proxy is disabled on multicast VXLAN tunnels. OVSDB VTEP configuration examples Unicast-mode VXLAN configuration example Network requirements As shown in Figure 18, configure the controller cluster to deploy unicast-mode VXLAN 10 to Switch A, Switch B, and Switch C to provide Layer 2 connectivity for the VMs across the network sites. 51

56 Figure 18 Network diagram Controller cluster ( x) Transport network Configuration procedure 1. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 18. (Details not shown.) # Configure a unicast routing protocol on all transport network switches (Switches A through D). (Details not shown.) 2. Deploy a VXLAN IP gateway on the transport network. (Details not shown.) 3. Configure Switch A: # Enable L2VPN. <SwitchA> system-view [SwitchA] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchA] undo vxlan ip-forwarding # Configure active TCP connection settings. [SwitchA] ovsdb server tcp ip port 6632 # Enable the OVSDB server. [SwitchA] ovsdb server enable # Enable the OVSDB VTEP service. [SwitchA] vtep enable # Specify the site-facing interface FortyGigE 1/0/1 as a VTEP access port. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] vtep access port [SwitchA-FortyGigE1/0/1] quit 4. Configure Switch B: # Enable L2VPN. <SwitchB> system-view [SwitchB] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchB] undo vxlan ip-forwarding # Configure active TCP connection settings. [SwitchB] ovsdb server tcp ip port

57 # Enable the OVSDB server. [SwitchB] ovsdb server enable # Enable the OVSDB VTEP service. [SwitchB] vtep enable # Specify the site-facing interface FortyGigE 1/0/1 as a VTEP access port. [SwitchB] interface fortygige 1/0/1 [SwitchB-FortyGigE1/0/1] vtep access port [SwitchB-FortyGigE1/0/1] quit 5. Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchC] undo vxlan ip-forwarding # Configure active TCP connection settings. [SwitchC] ovsdb server tcp ip port 6632 # Enable the OVSDB server. [SwitchC] ovsdb server enable # Enable the OVSDB VTEP service. [SwitchC] vtep enable # Specify the site-facing interface FortyGigE 1/0/1 as a VTEP access port. [SwitchC] interface fortygige 1/0/1 [SwitchC-FortyGigE1/0/1] vtep access port [SwitchC-FortyGigE1/0/1] quit 6. Configure VXLAN settings on the controller. (Details not shown.) Verifying the configuration 1. Verify the VXLAN settings on the VTEPs. This example uses Switch A. # Verify that the VXLAN tunnel interfaces on the VTEP are up. [SwitchA] display interface tunnel Tunnel1 Current state: UP Line protocol state: UP Description: Tunnel1 Interface Bandwidth: 64kbps Maximum transmission unit: 1464 Internet protocol processing: Disabled Last clearing of counters: Never Tunnel source , destination Tunnel protocol/transport UDP_VXLAN/IP Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops # Verify that the VXLAN tunnels have been assigned to the VXLAN. [SwitchA] display l2vpn vsi verbose VSI Name: evpn2014 VSI Index : 0 53

58 VSI State : Up MTU : 1500 Bandwidth : - Broadcast Restrain : - Multicast Restrain : - Unknown Unicast Restrain: - MAC Learning : Enabled MAC Table Limit : - Drop Unknown : - Flooding : Enabled VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flooding proxy Tunnel1 0x Up Manual Disabled Tunnel2 0x Up Manual Disabled ACs: AC Link ID State FGE1/0/1 srv2 0 Up # Verify that the VTEP has learned the MAC addresses of remote VMs. <SwitchA> display l2vpn mac-address MAC Address State VSI Name Link ID/Name Aging 00ea Dynamic SDN_VSI_ Aging aa-2f0a Dynamic SDN_VSI_8008 Tunnel257 Aging 3c8c-404e-dd46 Dynamic SDN_VSI_8008 Tunnel257 Aging mac address(es) found Verify that VM 1, VM 2, and VM 3 can ping each other. (Details not shown.) Flood proxy VXLAN configuration example Network requirements As shown in Figure 19: Configure the controller cluster to deploy VXLAN 10 to Switch A, Switch B, and Switch C to provide Layer 2 connectivity for the VMs across the network sites. Enable flood proxy for VXLAN 10. Use the MAC address entries issued by the controller to direct traffic forwarding on Switch A, Switch B, and Switch C. 54

59 Figure 19 Network diagram Configuration procedure 1. Configure IP addresses and unicast routing settings: # Assign IP addresses to interfaces, as shown in Figure 19. (Details not shown.) # Configure OSPF on all transport network switches (Switches A through D). (Details not shown.) 2. Configure Switch A: # Enable L2VPN. <SwitchA> system-view [SwitchA] l2vpn enable # Configure active TCP connection settings. [SwitchA] ovsdb server tcp ip port 6632 # Enable the OVSDB server. [SwitchA] ovsdb server enable # Enable the OVSDB VTEP service. [SwitchA] vtep enable # Assign an IP address to Loopback 0. [SwitchA] interface loopback 0 [SwitchA-LoopBack0] ip address [SwitchA-LoopBack0] quit # Specify the IP address of Loopback 0 as the global source address for VXLAN tunnels. [SwitchA] tunnel global source-address # Specify the site-facing interface FortyGigE 1/0/1 as a VTEP access port. [SwitchA] interface fortygige 1/0/1 [SwitchA-FortyGigE1/0/1] vtep access port [SwitchA-FortyGigE1/0/1] quit # Disable remote-mac address learning. [SwitchA] vxlan tunnel mac-learning disable # Enable flood proxy on multicast VXLAN tunnels. 55