Understanding PVLANs in UCS Introduction This document explains the PVLAN support in UCS - a feature which is introduced in the Balboa (1.4) release. This document explains the feature, the caveats and configuration when using PVLANs with bare metal OS and/or in conjunction with hypervisor switch like Nexus 1000v which supports PVLANs too. Background Theory A private VLAN is a VLAN you configure to have Layer 2 isolation from other ports within the same private VLAN. Ports belonging to a private VLAN are associated with a common set of supporting VLANs that are used to create the private VLAN structure. There are three types of private VLAN ports: promiscuous, isolated, and community. A promiscuous port communicates with all other private VLAN ports and is the port you use to communicate with routers, backup servers, and administrative workstations. An isolated port has complete Layer 2 separation, including broadcasts, from other ports within the same private VLAN with the exception of the promiscuous port. Community ports communicate among themselves and with their promiscuous ports. These ports are isolated at Layer 2 from all other ports in other communities or isolated ports within their private VLAN.
Broadcasts propagate only between associated community ports and the promiscuous port. Privacy is granted at the Layer 2 level because the switch blocks outgoing traffic to all isolated ports. You assign all isolated ports to an isolated VLAN where this hardware function occurs. Traffic received from an isolated port is forwarded to all promiscuous ports only. Within a private VLAN are three distinct classifications of VLANs: a single primary VLAN, a single isolated VLAN, and a series of community VLANs. RFC 5517 defines PVLANs theory and operations and is suggested for reading to get a good understanding of the concepts behind PVLANs - http://tools.ietf.org/html/rfc5517 PVLAN implementation in UCS The important points are a) Only Isolated ports are supported in UCS. b) A server vnic in UCS cannot carry both regular and isolated VLANs. c) No support for Promiscuous ports/trunks, Community ports/trunks or Isolated trunks. d) Promiscuous ports need to be outside the UCS domain i.e upstream switch/router.
Network Topology and Configuration The configuration example in this document is for the topology described in Figure 1.
The desired behavior is that Blade 1, VM1, VM2 and Blade 3 cannot communicate with each other as they will part of the same isolated VLAN while all of them should be able to communicate to the L3 port on the Catalyst 6500 upstream which is configured as a promiscuous port. Configuration For this example, the following VLAN s will be used.
Primary VLAN 40 Secondary (Isolated) VLAN 400 vnic0 to the ESX host will carry the isolated VLAN. UCS Configuration Create the Primary VLAN (VLAN 40 in this example) in the VLAN tab Similarly create the Secondary VLAN (VLAN 400 in this example) and associate it with the Primary VLAN
Creating vnic for a blade running bare metal OS (Linux/Windows) is straight forward as the isolated VLAN needs to be chosen and set as the Native VLAN. Fabric Failover can be enabled if required and supported by the adapter in the blade to which the Service Profile will be assigned. Creating vnics for an ESX host is different and usually requires trunks extended to the blade. As mentioned earlier, PVLANs and regular VLANs cannot be extended on the same vnic and also a VNIC can only have one isolated VLAN. This implies that a vnic needs to be defined just to carry the isolated VLAN to the ESX blade running Nexus 1000v. In case of M81KR (Palo) adapter, it can be accomplished by creating vnics as per requirement. In case of the M71KR E/Q (Menlo) adapters which are characterized by a maximum of 2 vnics, one vnic can be defined as trunk to carry traffic for Service Console, VMotion, Control, Packet etc (which can have Fabric Failover enabled for redundancy) and one for carrying the isolated VLAN. Note: In case of Menlo s only 1 isolated VLAN is possible. In case of the other adapters (82598KR, M61KR, M72KR E/Q and M51KR) which do not support Fabric Failover, private VLANs are not feasible with
the Nexus 1000v if redundancy is required as the 2 available vnics need to be configured to back each other up. Catalyst 6500 Configuration Define the Primary and Secondary VLANs vlan 40 private-vlan primary private-vlan association 400 vlan 400 private-vlan isolated Configuration of L3 interface which is configured as promiscuous interface Vlan40 ip address 40.40.40.250 255.255.255.0 private-vlan mapping 400
Interface configuration of the trunk connecting to the FI interface TenGigabitEthernet3/2 switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 10,20,40,180,400 switchport mode trunk end Nexus 1000v Configuration Define the Primary and Secondary VLANs vlan 40 private-vlan primary private-vlan association 400 vlan 400 private-vlan isolated Define the uplink port profile which will be assigned to the pnic port-profile type ethernet pv-lan vmware port-group switchport mode trunk switchport trunk native vlan 40 switchport trunk allowed vlan 40,400 channel-group auto mode on mac-pinning
no shutdown state enabled Define the veth port profile which the VM s will consume port-profile type vethernet vms vmware port-group switchport mode private-vlan host switchport private-vlan host-association 40 400 no shutdown state enabled PVLANs with VMware DVS As seen with the Nexus 1000v configuration, the uplink port-profile defined on the VEM is a trunk with native set as the primary VLAN for the vnic which carries the isolated VLAN. As all traffic on that vnic as sent by the FI is untagged, it is processed by the Nexus 1000v as coming in on the primary (native VLAN) and is fowarded to the VM isolated ports. VMware DVS does not give the option to configure native VLAN on uplinks and hence PVLANs with DVS and UCS are currently not supported.