Deploying VMware vsphere 6.0 with HPE BladeSystem, HPE OneView, Cisco Nexus, and NetApp FAS

Transcription

1 Deploying VMware vsphere 6.0 with HPE BladeSystem, HPE OneView, Cisco Nexus, and NetApp FAS Last Updated: 5/17/2016 Technical white paper

2 Contents Introduction... 3 Solution Overview... 3 Data Core Networking... 3 Table 1 Network Connection Details 4 Connections Verification HPE Virtual Connect FlexFabric Modules 5 Cisco Nexus 5500 Series 6 Configuration Verification (from show run) 6 NetApp FAS 3220 with Clustered ONTAP... 8 Physical Infrastructure for FAS... 8 Storage Controller Configuration Testing with NetApp Config Advisor 8 NAS Storage for Data ONTAP Operating in Clustered mode Concepts and terms to be familiar with: 11 Setup, Configure and Manage NetApp Storage: Configure networking 12 Configuration of Clustered ONTAP Data Network 13 Data aggregates Create data Storage Virtual Machines (SVMs) 19 Create Network Create Logical Interconnect Group Create Enclosure Group and add Enclosure to the group Verify Interconnect Virtual Infrastructure Building the VMware Virtual Infrastructure - Clusters & Distributed Virtual Switches Mounting NFS shared Datastore, Map and testing CIFS share Storage Connectivity Verification Implementation of NetApp Virtual Storage Console (VSC) Register VSC with vcenter Server 41 Discover and Add Storage Resources 42 Virtual storage console Guest OS tools Optimal Storage Settings for ESX/ESXi Hosts 43 Install NetApp NFS VAAI Plug-in HPE OneView Integration with VMware vcenter vsphere Web Client Configuration HP Infrastructure Management Host Networking Management Conclusion Reference (Links)... 54

3 Introduction Adoption of server virtualization technology has driven an increased demand for a non-disruptive, scalable IP-based Network Attached Filesystem (NAS) storage in the modem data center and the cloud. This document covers the steps necessary to integrate HPE BladeSystem C7000 with Cisco Nexus switches, and NetApp NAS datastores in a VMware vsphere environment and how to leverage HPE OneView to simplify datacenter management. Target audience: This document is intended for IT virtualization administrators, data center architects, and implementation engineers that have a solid understanding of Compute, Network, Storage, and VMware vsphere. This whitepaper describes a testing that was built performed in December Solution Overview The solution that is covered in this document represents a virtualization environment for enterprise IT organizations seeking to implement a flexible private cloud. It is assumed that readers will already have reviewed their specific design requirements and completed physical installation and cabling. This solution deployment steps were: 1. Configure Cisco Core Data Network 2. Setup and Configure NetApp 3. Setup and Configure HPE ProLiant BladeSystem and OneView Appliance 4. Setup VMware vsphere Virtual Infrastructure and NetApp Plug-ins 5. Install and Configure OneView for VMware vcenter (OV4VC) Key Components tested and deployed in the solution: NX-OS version 7.0(2)N1(1). Data Core Networking Prepare the network before deploying VMware vsphere. To connect the network infrastructure, cable the Data IP network and Cluster network as shown in Figure 1 and verify the connections as shown in table 1.

4 CISCO NEXUS N5548UP ID STAT LNK X2065A 0a A 0b B LNK C c0a c0b D F A N 1 E n c l o s u r e I n t e r l i n k U I D U I D O A 1 O A 2 i L O i L O R e s e t A c t i v e E n c l o s u r e R e s e t A c t i v e U I D F A N 6 0c 0d P S e0a e0b H P V C F l e x F a b r i c / 4 0 F 8 M o d u l e U I D Q 1 Q 2 Q 3 Q P S P I D L / A X 1 X 5 P S 4 X 2 X 6 X 3 X 7 X 4 X 8 R e m o v e m a n a g e m e n t m o d u l e s b e f o r e e j e c t i n g s l e e v e SAN LAN PORT 2 SAN LAN PORT 2 P a i r e d PORT 1 SAN LAN PORT 1 SAN LAN 10GbEE CNA 10GbEE CNA P a i r e d R e s e t H P V C F l e x F a b r i c / 4 0 F 8 M o d u l e U I D Q 1 Q 2 Q 3 Q P I D L / A X 1 X 5 P S 3 X 2 X 6 X 3 X 7 X 4 X 8 P a i r e d P S 2 P a i r e d R e s e t P S 1 F A N 5 F A N B a y 1 B a y 9 D S M o d u l e P S 1 UID S P D L N K / S P D L N K / S P D L N K / S P D L N K / ProLiant BL460c Gen9 UID ProLiant BL460c Gen UID ProLiant BL460c Gen9 UID ProLiant BL460c Gen9 UID UID ProLiant BL460c Gen9 ProLiant BL460c Gen9 H P B l a d e S y s t e m c E n c l o s u r e B a y 8 B a y 1 6 P S 6 CISCO NEXUS N5548UP ID STAT LNK X2065A 0a A 0b B LNK C c0a c0b D 0c 0d 3 4 e0a e0b SAN LAN PORT 2 SAN LAN PORT 2 PORT 1 SAN LAN PORT 1 SAN LAN 10GbEE CNA 10GbEE CNA Technical white paper 1 X5 X6 X5 X HP BladeSystem c7000 With Virtual Connect FlexFabric 20/40 F8 Module E1/11 E1/12 E1/17-18 E1/21 Nexus 5548-A E1/22 vpc Peer-Link E1/17-18 E1/11 E1/21 E1/12 E1/22 Nexus 5548-B e2b e1b e2b e1b e1a X1140A e1a e2a e2a NetApp FAS Clustered Data ON-TAP 8.3 NetApp FAS Clustered Data ON-TAP 8.3 cdot Cluster Network Figure 1. Physical network connectivity. Table 1 Network Connection Details Device Name Port Device Name Port Note N5K-A E1/11 HPE VC FlexFabric 20/40 F8 - VC1 X5 Nexus vpc 11 N5K-A E1/12 HPE VC FlexFabric 20/40 F8 - VC2 X5 Nexus vpc 12 N5K-A E1/21 NetApp FAS e1b NetApp ifgrp a11a Nexus vpc 21 N5K-A E1/22 NetApp FAS e1b NetApp ifgrp a12a Nexus vpc 22 N5K-A E1/17 N5K-B E1/17 vpc - Peering Link N5K-A E1/18 N5K-B E1/18 vpc - Peering Link N5K-B E1/11 HPE VC FlexFabric 20/40 F8 - VC1 X6 Nexus vpc 11 N5K-B E1/12 HPE VC FlexFabric 20/40 F8 - VC2 X6 Nexus vpc 12 N5K-B E1/21 NetApp FAS e2b NetApp ifgrp a11a Nexus vpc 21 N5K-B E1/22 NetApp FAS e2b NetApp ifgrp a12a Nexus vpc 22 N5K-B E1/17 N5K-A E1/17 vpc Peer Link N5K-B E1/18 N5K-A E1/18 vpc Peer Link

5 Connections Verification HPE Virtual Connect FlexFabric Modules N5K-A# show lldp nei in VC Device ID Local Intf Hold-time Capability Port ID VC2040F8-7C J Eth1/ X5 VC2040F8-7C K Eth1/ X5 N5K-A# N5K-B# show lldp nei in VC Device ID Local Intf Hold-time Capability Port ID VC2040F8-7C J Eth1/ X6 VC2040F8-7C K Eth1/ X6 DIA-N5K-06# Connections Verification Connections - NetApp Controller Data-Plane N5K-A# sh cdp neighbors in e1b Device-ID Local Intrfce Hldtme Capability Platform Port ID cluster Eth1/ H FAS3220 e1b cluster Eth1/ H FAS3220 e1b N5K-A# N5K-B# sh cdp neighbors in e2 Device-ID Local Intrfce Hldtme Capability Platform Port ID cluster Eth1/ H FAS3220 e2b cluster Eth1/ H FAS3220 e2b N5K-B#

6 CISCO NEXUS N5548UP Technical white paper Cisco Nexus 5500 Series A pair of Cisco Nexus 5548 switches were used to terminate 10G Ethernet network connections from both Virtual Connect FlexFabric 20/40 F8 modules and both NetApp FAS 3220 network modules. Cisco Nexus 5548 prerequisites. Description The Cisco Nexus 5548 switch was running Cisco Nexus NX-OS version 7.0(2)N1(1). HP VC LACPs vpc Peerlink vpcs HP VC vpc-peer link vpcs - NetApp ID STAT FAS ifgrps vpcs HP VC Data (Server Facing) vpcs NetApp Data (Storage Facing) vpc Peer link - Control Figure 2. Nexus 5548 Switches Physical Diagram Configuration Verification (from show run) version 7.0(2)N1(1) vlan 10 name NFS-VLAN vlan 140 name MGMT-VLAN vlan 141 name vmotion-vlan vlan 142 name VM-Network1-VLAN Note: Run the commands below on both the Cisco Nexus 5548 A switch and the Cisco Nexus 5548 B switch. Four VLANs used in the paper are VLAN 10, VLAN vpc domain 1 peer-keepalive destination source delay restore 150 peer-gateway auto-recovery ip arp synchronize interface port-channel1 description vpc peer-link switchport mode trunk spanning-tree port type network vpc peer-link interface port-channel11 switchport mode trunk switchport trunk allowed vlan 10, spanning-tree port type edge trunk speed vpc 11

7 interface port-channel12 switchport mode trunk switchport trunk allowed vlan 10, spanning-tree port type edge trunk speed vpc 12 interface port-channel21 switchport mode trunk switchport trunk native vlan 10 switchport trunk allowed vlan 10, spanning-tree port type edge trunk speed vpc 21 interface port-channel22 switchport mode trunk switchport trunk native vlan 10 switchport trunk allowed vlan 10, spanning-tree port type edge trunk speed vpc 22 N5K-A(config)# sh vpc Legend: (*) - local vpc is down, forwarding via vpc peer-link vpc domain id : 1 Peer status : peer adjacency formed ok vpc keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vpc role : primary Number of vpcs configured : 4 Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) vpc Peer-link status id Port Status Active vlans Po1 up 1,10, vpc status id Port Status Consistency Reason Active vlans Po11 up success success 10, Po12 up success success 10, Po21 up success success 10, Po22 up success success 10, DIA-N5K-05(config)# N5K-B(config)# sh vpc Legend: (*) - local vpc is down, forwarding via vpc peer-link vpc domain id : 1 Peer status : peer adjacency formed ok

8 vpc keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vpc role : secondary Number of vpcs configured : 4 Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) vpc Peer-link status id Port Status Active vlans Po1 up 1,10, vpc status id Port Status Consistency Reason Active vlans Po11 up success success 10, Po12 up success success 10, Po21 up success success 10, Po22 up success success 10, N5K-B(config)# NetApp FAS 3220 with Clustered ONTAP Physical Infrastructure for FAS A pair of FAS3220 nodes with a DS2246 disk enclosure are used in this deployment. All disks are HDD, there are no SSD or any flash disks. Two dual port 10 GbE network cards are installed on each FAS3220 node. A NAS (CIFS & NFS) data storage system is deployed on dual FAS3200 nodes with a single DS4220 disk enclosure with Clustered ONATP RC operating system. Storage Controller Configuration Testing with NetApp Config Advisor Config Advisor, a tool for professional services, partners, and customers, was used to collect information from a NetApp controller and evaluate the cabling to the disk shelves. Download Config Advisor from the NetApp Support site.

9 Figure 2 Initial Sign-In Screen of Config Advisor

10 Figure 3 - Example of using Config Advisor to verify SAS cabling Note: The Config Advisor can be used different stages of the project to verify different parts of the configuration in a NetApp infrastructure. Output of the Config Advisor includes Network Interface, Cabling (as shown in above), System Summary and Aggregate Info. NAS Storage for Data ONTAP Operating in Clustered mode Clustered Data ONTAP is enterprise-capable, unified scale-out storage. It is the basis for virtualized, shared storage infrastructures. Clustered ONTAP is architected for non-disruptive operations, storage and operational efficiency, and scalability over the lifetime of the system. This document examines how to enable NAS (CIFS & NFS) on NetApp Clustered Data ONTAP with a virtual port interface group (ifgrp) uplink to Cisco Nexus which provides scalable and redundant access to the NAS datastore. Note: There are two Storage Virtual Machines (SVMs) used in this environment: Onepod is a NFS enabled SVM

11 X2065A A B C D Technical white paper SVM-DEMO is a CIFS & NFS enabled SVM. Concepts and terms to be familiar with: NetApp Data ONTAP Cluster 1. A cluster consists of one or more HA pairs. HA pairs are two NetApp controller nodes which must be the same type/model). 2. The maximum number of HA pairs in a cluster depends on the controller model and version of Data ONTAP and also the protocol (e.g. CIFS/NFS or FCP) in use. 3. The NetApp cluster enables the nodes to pool resources and distribute work across the cluster, while presenting administrators with a single entity to manage. Clustering also enables non-interrupted service to end users if a single node goes offline or is rebooted. Difference between Cluster and SVM administrator roles: Cluster administrators administer the entire cluster AND the SVMs (formerly known as Vservers) it contains. SVM administrators administer only their own data SVMs. Cluster administrators can administer the entire cluster and its resources. They can also set up data SVMs and delegate SVM administration to SVM administrators. The specific capabilities that cluster administrators have depend on their access-control roles. By default, a cluster administrator with the admin account name or role name has all capabilities for managing the cluster and SVMs. SVM administrators can administer only their own SVM storage and network resources, such as volumes, protocols, logical interfaces (LIFs), and services. The specific capabilities that SVM administrators have depend on the access-control roles that are assigned by cluster administrators. For more information about SVM administrator capabilities, see the Clustered Data ONTAP System Administration Guide for SVM Administrators. Ifgrp c-net c0a 0c e0a 1 e1b SAN LAN PORT 2 PORT 1 SAN LAN e1a 10GbEE CNA LNK 0a 0b LNK 2 e2b SAN LAN PORT 2 PORT 1 SAN LAN e2a 10GbEE CNA c0b 0d e0b X1140 A Ifgrp Data IP based NAS (CIFS & NFS) network c-net Control Cluster Network Figure 4 - NetApp FAS

12 Setup, Configure and Manage NetApp Storage: Initial cluster setup can only be performed at the system console port. After a cluster is successfully built and the cluster management IP can be accessed, then additional configuration tasks can be performed via SSH or NetApp OnCommand System manager web GUI. For more details on initial console configuration tasks, refer to cdot 8.3 Software Setup Guide After the initial setup, the following configuration steps to create a NAS datastore include: 1. Configure networking 2. Define and configure data aggregate 3. Create and configure SVMs with NAS (CIFS & NFS) protocol 4. Verify datastore creation Configure networking Clustered ONTAP networking NetApp Data ONTAP Cluster mode requires two distinct networks: 1. Cluster Network which handles all the cluster control traffic, can be implemented via an external switch or by cabling the nodes directly to each other. For information on supported external switches, refer to Cluster Management and Interconnect Switches 2. Data Network handles all of the IP and/or FCoE data traffic. Configuration of Cluster Network To verify cluster network from OnCommand System Manager, go to Cluster >> Configuration >> Network >> Network Interfaces or Config Advisor Output >> Network Interfaces (tab) Figure 4 - OnCommand Network View

13 Figure 5 - OnCommand Cluster Network Details Port e1a and e2a both from each node are assigned as part of default SVM called Cluster Configuration of Clustered ONTAP Data Network Clustered Data ONTAP IP Data network consists of Ethernet switches and NetApp Ethernet/CNA Controller card to provide IP based data connectivity for iscsi, NFS or CIFS. Following is a logical topology used for this paper. Figure 6- Logical Clustered ONTAP Data Network Connections Details NX5K-A PO 21 vpc 21 PO 22 vpc 22 vpc Peer PO 21 vpc 21 NX5K-B E1/17-18 E1/21 E1/22 E1/21 E1/22 PO 1 PO 22 vpc 22 Physical Ports e1b e2b e1b e2b NetApp - 01 NetApp - 02 Network interface cards (NICs) and host bus adapters (HBAs) provide physical (Ethernet and Fibre Channel) connections from each node to the physical networks (management and data networks). In earlier releases of Data ONTAP, default roles were assigned to each Ethernet network port. The roles included data, cluster, cluster management, intercluster, and node management. In Data ONTAP 8.3 these roles are assigned when LIFs are created. From Web GUI: Under Cluster >> Configuration >>Network >> Ethernet Ports

14 Figure 7 - FAS Ethernet Port Status Virtual Ports VLANs and interface groups (ifgrps) constitute the virtual ports. Although interface groups treat several physical ports as a single port, VLANs subdivide a physical port into multiple separate ports. Ifgrp a11a (e1b,e2b on node 1) and Ifgrp a12a (e1b,e2b on node 2) are used in this deployment. Interface groups provide a mechanism to bundle multiple physical interfaces (links) into one logical interface. NetApp Data ONTAP supports 3 types of interface groups: Single-mode, static multimode, and dynamic multimode (LACP). The following commands creates a dynamic multimode interface group that bundles physical port e1b and e2b with IP hashing. The two physical ports were connected with Nexus5500 pairs running vpc. cluster-01::> network port ifgrp create -node cluster ifgrp a11a -distr-func ip -mode multimode cluster-01::> network port ifgrp create -node cluster ifgrp a12a -distr-func ip -mode multimode To enabled CDP cluster-01::> system node run -node cluster options cdpd cdpd.enable on (value might be overwritten in takeover) cdpd.holdtime 180 (value might be overwritten in takeover) cdpd.interval 60 (value might be overwritten in takeover cluster-01::> system node run -node cluster cdpd show-neighbors Local Remote Remote Remote Hold Remote Port Device Interface Platform Time Capability e1b DIA-N5K-05(SSI. Ethernet1/21 N5K-C5548UP 170 RSI e2b DIA-N5K-06(SSI. Ethernet1/21 N5K-C5548UP 154 RSI cluster-01::> system node run -node cluster options cdpd cdpd.enable on (value might be overwritten in takeover) cdpd.holdtime 180 (value might be overwritten in takeover)

15 cdpd.interval 60 (value might be overwritten in takeover) cluster-01::> system node run -node cluster cdpd show-neighbors Local Remote Remote Remote Hold Remote Port Device Interface Platform Time Capability cluster-01::> e1b DIA-N5K-05(SSI. Ethernet1/22 N5K-C5548UP 121 RSI e2b DIA-N5K-06(SSI. Ethernet1/22 N5K-C5548UP 139 RSI cluster-01::> network port show -ifgrp-node cluster Speed (Mbps) Node Port IPspace Broadcast Domain Link MTU Admin/Oper cluster e1b Default - up 9000 auto/10000 e2b Default - up 9000 auto/ entries were displayed. cluster-01::> network port show -ifgrp-node cluster Speed (Mbps) Node Port IPspace Broadcast Domain Link MTU Admin/Oper cluster e1b Default - up 9000 auto/10000 e2b Default - up 9000 auto/ entries were displayed. Logical Interfaces A logical interface (LIF) is an IP address with associated characteristics, such as a role, a home port, a home node, a routing group, fail over ports, and a firewall policy. A LIF communicates over the network through the port (physical or virtual) to which it is currently bound. The different types of LIFs in a cluster are: data LIFs, cluster management LIFs, node management LIFs, intercluster LIFs, and cluster LIFs. The ownership of the LIFs depends on the Storage Virtual Machine (SVM) where the LIF resides. Data LIFs are owned by data SVMs. Node management, cluster management, and intercluster LIFs are owned by the admin SVMs. Cluster LIFs are owned by the cluster SVM. A Data LIF is created as part of Storage Virtual Machine creation. Cluster / management LIFs are created when clusters/nodes are initially set up. The Data LIF used in this paper is called SVM_Onepod_nfs_lif1 and has a home port of a11a (ifgrp). From Web GUI create / manage lifs: Cluster >> Network >> Network Interfaces From the LIFs listed, there are four LIFs for Cluster and four LIFs for Data. In last colum show Role To verify failover-group membership, type network interface failover-groups show from the CLI.

16 cluster-01::> network interface failover-groups show Failover Vserver Group Targets Cluster Cluster cluster-01-01:e1a, cluster-01-02:e1a cluster /22 cluster-01-01:e0m, cluster-01-02:e0m Data /24 cluster-01-01:a11a, cluster-01-02:a12a cluster-01-01_node_mgmt cluster-01-01:e0m cluster-01-02_node_mgmt cluster-01-02:e0m 5 entries were displayed. IPspaces IPspaces configure a single Data ONTAP cluster so that it can be accessed by clients from more than one administratively separate network domain, even if those clients are using the same IP address subnet range. This allows for separation of client traffic for privacy and security. An IPspace defines a distinct IP address space in which Storage Virtual Machines (SVMs) reside. Ports and IP addresses defined for an IPspace are applicable only within that IPspace. A distinct routing table is maintained for each SVM within an IPspace; therefore, no cross-svm or cross-ipspace traffic routing occurs. Note: IPspaces support both IPv4 and IPv6 addresses on their routing domains. To create IPspace, enter following command: cluster-01::> network ipspace create ipspace NAS-IPspace To verify IPspace creation, enter the following: cluster-01::> network ipspace show IPspace Vserver List Broadcast Domains cluster-01::cluster> network ipspace show IPspace Vserver List Broadcast Domains Cluster Cluster Cluster Default SVM-DEMO, SVM_Onepod, /22, Data cluster-01, node_01_svm /24 NAS-IPspace NAS-IPspace - 3 entries were displayed.

17 Broadcast Domains Broadcast domains group network ports that belong to the same layer 2 network. The ports in the group can then be used by a Storage Virtual Machine (SVM) for data or management traffic. cluster-01::> network port broadcast-domain create -broadcast-domain data /24 -mtu ipspace default cluster-01::> network port broadcast-domain show IPspace Broadcast Update Name Domain Name MTU Port List Status Details Cluster Cluster 9000 cluster-01-01:e1a complete cluster-01-02:e1a complete Default / cluster-01-01:e0m complete cluster-01-02:e0m complete Data / cluster-01-01:a11a complete cluster-01-02:a12a complete entries were displayed. Subnets Subnets allocate specific blocks, or pools, of IP addresses for Data ONTAP network configuration. This creates LIFs more easily when using the network interface create command, by specifying a subnet name instead of having to specify IP address and network mask values. It shows how many were set and how many are being used.

18 cluster-01::> network subnet show IPspace: Default Subnet Broadcast Avail/ Name Subnet Domain Gateway Total Ranges IP-Storage-Net /24 Data / / , For more information, please refer to the resource at NetApp Support site Clustered Data ONTAP 8.3 Network Management Guide Once Data Network is up, the next step is lay out Data Aggregates. Data aggregates To support the different security, backup, performance, and data sharing needs of users, administrators can group the physical data storage resources on the storage system into one or more aggregates. Aggregates can then be designed and configured to provide the appropriate level of performance and redundancy. For more detailed information please refer to NetApp documentation Clustered Data ONTAP 8.3 Physical Storage Management Guide NOTE: In the paper, no Flash Pool aggregates are used. All Hard Disk Drives with same type and size are used. To Create a Data Aggregate, Click Create Create Aggregate Wizard will be launched,

19 After create Data Aggregates on both nodes, verify configuration. Create data Storage Virtual Machines (SVMs) As mentioned earlier, SVM is the most important concept in Clustered Data ONTAP. The client accesses the cluster Data ONTAP NAS datastore via an SVM s LIFs. The following are the steps to create a CIFS & NFS enabled SVM called SVM-DEMO. When defining a new SVM, from the left-hand navigate panel, click on Storage Virtual Machines (SVM) and highlight cluster-01, on the top control pane windows Click create and setup Wizard will start.

20 Note: Only licensed protocols with be shown on the setup screen. NFS and CIFS can share same a LIF

21 After successfully creating an SVM, review and verify its name and other properties. SVM-DEMO is the SVM created for this paper and NetApp NAS (CIFS & NFS) protocols are licensed for both nodes and enabled on this SVM (as shown by the green highlight in the figure below). The next Step is to create Export policies or add rule(s). These are security features to set the scope for accessing Volumes within an SVM. For this paper, the default Export policy was used. Create Volumes and Shares To create volumes for an SVM, select Create from the GUI menu.

22 The following screen capture shows several volumes created within the SVM named SVM-DEMO. Namespaces in SVMs: A namespace is a logical grouping of volumes that are joined together at junction points create a single, logical file system that derives from the SVM root volume. CIFS and NFS servers on a data SVM can store and access data across the namespace. Each client can access the entire namespace by mounting an export or accessing a single CIFS share at the top of the namespace. The top of a file-system tree is always the root volume, which is represented by a slash (/). A junction leads from a directory in one volume to the root directory of another volume. ADD A BRIEF DESCRIPTION OF NAME Following screen is showing - Namespace for nas-data volume - Mounted with default Export Policy and use NTFS security style

23 Note: Please note the name (and case) of the Namespace for each volume within an SVM. It will be used on the NFS client side when a NFS mount point is created and is a case sensitive operation. In the following screenshot a CIFS share is created and junction point for volume to mount is right off the / or root of the SVM

24 After an SVM s volume and shares are created, verify volumes and shares are up running. Instead of GUI, alternatively use ssh cli to verify. Here are the examples: cluster-01::> volume show Vserver Volume Aggregate State Type Size Available Used% SVM-DEMO SVMDEMO_root node_01_aggr1 online RW 1GB 970.6MB 5% SVM-DEMO cifs node_01_aggr1 SVM-DEMO nas_data SVM_Onepod Data SVM_Onepod Home SVM_Onepod Images online RW 10GB 9.50GB 5% node_01_aggr1 online RW 50GB 47.50GB 5% node_01_aggr1 online RW 1000GB 912.6GB 8% node_01_aggr1 online RW 150GB 142.4GB 5% node_01_aggr1 online RW 1000GB 999.8GB 0% SVM_Onepod SVM_Onepod_root node_01_aggr1 online RW 1GB 972.5MB 5% SVM_Onepod vol1 node_01_aggr1 online RW 100GB 94.94GB 5% volume show command showed all the volume configured. cluster-01::> cifs show Server Status Domain/Workgroup Authentication Vserver Name Admin Name Style SVM-DEMO SVM-DEMO up HPDIA domain cifs show shows the active CIFS shares are running on which SVM and belong to which Active Directory domain Blade servers in the HPE BladeSystem c7000 enclosure provide VMware server infrastructure. The infrastructure management is provided by HPE OneView which enable server provisioning, monitoring, LAN, and SAN networking configuration tasks. OneView 2.0 includes many new features such as server profile templates, driver management, and more.

25 HPE OneView integrates functions of the majority of traditional HPE management software such as Onboard Administrator, Virtual Connect Manager, Virtual Connect Enterprise Manager and Insight Control software. HPE OneView server profiles define the personality assigned to a server that includes firmware baseline, BIOS settings, network connectivity, boot configuration, Integrated Lights-Out (ilo) settings, and unique IDs. HPE OneView server profile templates define configurations once, in minutes, and then provision or update the configuration many times consistently and reliably. This paper demonstrates how to configure network tunneling and LACP in HPE OneView with HPE Virtual Connect FlexFabric 20/40 F8 modules to simplify network configuration, while still providing great redundancy. We will also show how HPE OneView server profile connection configuration ties all network pieces together. The HPE OneView configuration steps are: 1. Create network 2. Define Logical interconnect group 3. Define Enclosure group / Add Enclosure to group 4. Interconnect - Verify Configurations and Connectivity Figure 9 - HPE Virtual Connect FlexFabric 20/40 F8 Module Detailed View Note: The two Virtual Connect modules are installed in C7000 interconnect bay 1 and bay 2. The two uplink ports used for this deployment are 10 GE at X5 and X6. Create Network The first step is to define which mode will be used for the HPE Virtual Connect networks. In this case, Tunnel mode is used. Select Create Network from OneView dropdown

26 Define network type as Ethernet and VLAN option as tunnel, and leave others fields at the defaults. Repeat to create network Tunnel-B.

27 Create Logical Interconnect Group The second step is to create a Logical Interconnect Group. A logical interconnect group is associated with an enclosure group and is used to define the logical interconnect configuration for every enclosure that is added to that enclosure group. When a new enclosure is imported into OneView, a logical Interconnect can be auto-instantiated consistently. In this paper, a pair of HPE Virtual Connect FlexFabric 20/40 F8 modules (bay 1 & 2) are defined in the Logical Interconnect Group. The Virtual Connect module in Bay 1 has the Tunnel-A network included in its local uplink set. The Virtual Connect module in Bay 2 has the Tunnel-B network included in its local uplink set. Both modules include their local X5 and X6 ports as physical uplink ports to connect with Cisco Nexus switches.

28 Create Enclosure Group and add Enclosure to the group An Enclosure Group represents a set of similarly configured c7000 enclosures and references a logical interconnect group used for defining and maintaining logical interconnects. When an enclosure is imported and assigned to an enclosure group, OneView creates a logical interconnect for that enclosure. The logical interconnect inherits the configuration specified by the logical interconnect group that is associated with the enclosure group. When the import operation completes, the physical interconnects are fully configured and the networks are available for use by server profiles. In this paper the enclosure named C was imported as a part of an enclosure group named DIA-C , which in turn created a logical interconnect instance and automatically configured the physical Virtual Connect modules..

29 To verify all networks definition inherited from Logical Interconnect Group and uplinks LACP status and Link connectivity. Verify all links status as Linked (Active) indicating successful LACP negotiation with upstream Cisco Nexus5500 switches running vpc. LAG ID is VC internal LACP bundle ID. Connected to indicates Nexus5500 port MAC address and interface number.

30 The Map view displays the relationship and dependencies among different components. The above diagram illustrates a powerful and unique feature of OneView - a real-time visual mapped diagram that displays all configured elements, license information and end to end connectivity. Verify Interconnect To check information on physical counters and transceivers, users can check Interconnects from the top level networking menu. The difference between Logical Interconnects and interconnects is that the former focus mainly on layer 2 configuration such as VLAN, VLAN tagging, and LACP parameters. The latter focus on low level counters like packet statistics and transceiver status.

31 Select Interconnects from top list menu. Select Interconnect 1 which refers to VC module in I/O bay 1, users can check Ethernet uplink port counters and traffic information.

32 Users can also select Ethernet uplink ports physical transceivers info. Users can also check detailed LLDP neighbor information to show VC is connected with Nexus5500 eth1/11 interface.

33 After successfully configuring and verifying Virtual Connect module uplink status, users can check server hardware information and start to provision server profiles. HPE OneView server profile and server profile template The following is an example of a server profile (ESX60u1-Host-Bay16) which was instantiated from a server profile template (ESX6.0). The blade s two 10 GE FlexLOM ports use the tunnel network defined in the logical interconnect group to connect to the upstream Nexus switches. Following are the screen shots for create server profile named ESXi60u1-Host-Bay16 from a server profile template called ESXi60.

34

35

36 After creating the Server Profile users can attach the server profile to an available blade and install VMware vsphere 6.0. The following is a screen shot of a successfully created server profile called ESXi6u1-Host-Bay16 from HPE OneView. The following is a sample ILO remote console window; a blade Server associated with server profile called ESXi6u1-Host-Bay16 after VMware vsphere 6.0 is installed.

37 Virtual Infrastructure In this paper, VMware vsphere ESXi 6.0 and vcenter server appliance 6.0 were tested. NetApp Virtual Storage Console (VSC) for vsphere 6.0 and NetApp NFS VAAI 1.1 were deployed and tested as vcenter server Plug-ins to provide better OS level control and enhance performance via hardware acceleration. HPE OneView for vcenter v was also implemented to enable manageability of the HPE infrastructure from within VMware vcenter. Building the VMware Virtual Infrastructure - Clusters & Distributed Virtual Switches There is one virtual distributed switch configured in the vsphere 6.0 cluster. This logical switch will handle virtual environment network needs for vsphere 6.0. After a successfully implementation, following is the screenshot.

38 Mounting NFS shared Datastore, Map and testing CIFS share To verify NFS shared Datastores from vsphere Web Client: Alternatively using command line from ssh into the ESXi Host to display configured NFS Datastores esxcli storage nfs list Volume Name Host Share Accessible Mounted Read-Only ispe Hardware Acceleration Onepod_Data /Data true true false false Supported nas_data /nas_data true true false false Supported Onepod_Home /Home true true false false Supported Onepod_Vol /vol1 true true false false Supported nfs /nfs true true false false Supported Onepod_Image /Images true true false false Supported To test the CIFS share client access, follow are the steps and resources: 1. Log on to a windows client Log on to Windows client by entering user credential (user is member of administrator group) A Windows 2012R2 virtual machine IP address at is used for testing. 2. Test access using the IP address In Windows Explorer, map a drive using the IP address of the data LIF for SVM and name of the share T: \\ \cifs Mapped drive (letter T), SVM LIF IP address at and name of the share is called cifs are used 3. Verify access rights After successful map to the CIFS share, create folders.

39 Following is the screen capture of CIFS share access testing Storage Connectivity Verification IO generator tools could be used to verify the host correctly sends and receives the storage traffic on the expected path. The following screenshot details the 4K 100% sequential read testing result. Note: The IOmeter tools used here is NOT for any benchmark or load test. It is used ONLY to illustrate storage traffic path and for error detection purposes.

40 Implementation of NetApp Virtual Storage Console (VSC) 6.0 Virtual Storage Console for VMware vsphere software is a single vcenter Server plug-in that provides end-to-end lifecycle management for virtual machines in VMware environments using NetApp storage systems. VSC integrates smoothly with the VMware vsphere Web Client and enables single sign-on (SSO) services. In addition, the VSC Summary page provides an overall status of the vsphere environment. Table 2 listed some VSC 6.0 new features VSC 6.0 Features Manage storage and configure the ESXi host VSC 6.0 allows to add, remove, assign credentials, and set up permissions for storage controllers in a VMware environment and manage ESXi servers connected to NetApp storage. VSC 6.0 allows users to set values for host timeouts, NAS, and multipathing as well as view storage details and collect diagnostic information. Create storage capability profiles and set alarms - When VASA Provider for clustered Data ONTAP is installed and registered with VSC 6.0, storage capability profiles and VMware virtual volumes (VVOLs) can be created and used. Alarms can be set to alert users when the thresholds for volumes and aggregates are approaching full capacity. Provision datastores and clone virtual machines VSC 6.0 uses FlexClone technology to efficiently create, deploy, and manage the lifecycle of virtual machines from an interface that has been integrated into the VMware environment. Perform online alignments and migrate virtual machines singularly and in groups into new or existing datastores - VSC 6.0 enables to quickly check the alignment status of virtual machines. If there are alignment issues with the virtual machines, in most cases, the alignment issues can be resolved without having to power down the virtual machines Backup and restore virtual machines and datastores VSC 6.0 backup and restore option can rapidly backup and restore virtual entities such as virtual machines on NetApp storage if licensed and installed. ** (Separate license is required) To install the VSC 6.0 software, complete the following steps: 1. Download the VSC 6.0 installer: Virtual Storage Console 6.0 for VMware vsphere Download 2. Double-click the installer icon, and click Run to start the installation wizard.

41 3. Follow the instructions in the installation wizard to install the software. Note: To install backup and restore features, select Backup and Recovery option. 4. Click Finish to complete the installation. 5. At the web page that appears after the installation completes, register VSC with the vcenter Server. vcenter Server host name or IP address and the administrative credentials are needed Note: Registering VSC with the vcenter Server requires Windows administrator privileges. Register VSC with vcenter Server To register the VSC with the vcenter Server, complete the following steps: 6. Open a Web browser. 7. A browser window with the registration URL automatically opens when the installation phase is complete. However, some browser settings may interfere with this function. If the browser window does not open automatically, open a browser window manually and enter the URL.

42 8. When the browser is running on the computer where VSC is installed, enter the URL provided by the installation wizard ( Otherwise, replace localhost with the host name or IP address (<<var_vsc_server_ip>>) of the VSC server. 9. In the Plug-in Service Information section, select the IP address that the vcenter Server uses to access the VSC server from the drop-down list. 10. In the vcenter Server Information section, enter the host name or IP address, port, user name, and user password. Click Register to complete the registration. Discover and Add Storage Resources To discover storage resources for the monitoring and host configuration and the provisioning and cloning capabilities, complete the following steps: 11. Log in to the vcenter Server by using the vsphere Web Client. 12. Click On the Virtual Storage Console from home screen under inventores 13. Summary of the VSC is displayed

43 14. In the navigation pane, select Storage Systems. 15. To add a new Controller simply click on the plus sign and enter the credentials for the controller. 16. Enter required credentials in the pop up and click OK. 17. After the system is added a Success Message is shown. After a successful VSC install, NetApp storage system tools are enabled. Virtual storage console Guest OS tools Optimal Storage Settings for ESX/ESXi Hosts VSC Guest OS tools allow for the automated configuration of storage-related settings for all ESXi 4.x, 5.x, and 6.x hosts that are connected to NetApp storage controllers. To use these settings, complete the following steps: 18. Select individual or multiple ESXi hosts in the VSphere Web Client and Right-Click. 19. Select Set Recommended Values for these hosts. 20. Check the settings to apply to the selected vsphere hosts. Note: This functionality sets values for HBAs and converged network adapters (CNAs), sets appropriate paths and path-selection plug-ins, and verifies appropriate settings for software-based I/O (NFS and iscsi).

44 Note: Depending on what changes have been made, servers may require a restart for network-related parameter changes to take effect. If no reboot is required, the Status value is set to Normal. If a reboot is required, the Status value is set to Pending Reboot. If a reboot is required, the ESXi servers should be placed into maintenance mode, evacuated (if necessary), and restarted before proceeding. Install NetApp NFS VAAI Plug-in Installing the NetApp NFS Plug-in for VMware VAAI Installing the NetApp NFS Plug-in for VAAI (VMware vstorage APIs for Array Integration) involves enabling VMware vstorage for NFS on the NetApp storage system, verifying that VAAI is enabled on the VMware host, downloading and installing the plug-in on the VMware ESXi host, and ensuring that the installation is successful. NFSv3 must be enabled on the storage system. VMware vsphere 6.0 must be available. Prepare Install steps: Download the NetApp NFS Plug-in for VMware VAAI: a. Go to the Software Download page at b. Scroll down and go to NetApp NFS Plug-in for VMware VAAI in the Products section. c. Select the appropriate ESXi host from the <Select Platform> drop-down menu and download either the offline bundle (.zip) or online bundle (.vib) of the most recent plug-in. Installing the NetApp NFS Plug-in for VMware VAAI Via VSC steps: a. Extract the offline bundle in.zip format. b. Copy the plug-in.vib file, NetAppNasPlugin.vib, to the VSC installation directory. The path name is VSC_installation_directory\etc\vsc\web Note: Ensure that the file name of the.vib file downloaded matches the predefined name that VSC uses: NetAppNasPlugin.vib If the file name does not match the predefined name, rename the.vib file. c. Go to the NFS Plug-in for VMware VAAI section of the Tools panel of Monitoring and Host Configuration, and select vcenter Server to install the plug-in. This option is available for both single and multi vcenter environment. d. Click Install to Host.

45 e. In the dialog box, specify the ESXi hosts to install the plug-in. f. Click OK. The Monitoring and Host Configuration capability automatically installs the plug-in on the hosts are specified. Reboot the ESXi host to complete the installation. However, the storage system does not need to be rebooted. To Verify that the plug-in installation for the ESXi host, ssh into the host command line shell:

46 ~ # esxcli software vib list more Verify NetApp NFS VAAI hardware Acceleration plugin enabled on the ESX host

47 HPE OneView Integration with VMware vcenter HPE OneView for vcenter (OV4VC) enable VMware admins to manage and provisioning HPE server and HPE storage infrastructure without leaving vcenter console. HPE OneView for VMware vcenter integrates the manageability features of HPE OneView, HPE ProLiant servers, HPE BladeSystem and HPE Storage into the VMware vcenter console. HPE OneView for VMware vcenter Includes support for the following components: HPE OneView for VMware vcenter Server module Adds HPE ProLiant and HPE BladeSystem hardware monitoring into the HPE OneView for vcenter console. Provides server hardware management capabilities, including comprehensive monitoring, firmware update, vsphere 5.x/ESXi 5.x image, and vsphere 6.x/ESXi image deployment, remote control, end-toend monitoring for Virtual Connect, and power optimization for HPE servers in the VMware environment. HPE OneView for VMware vcenter StoreFront Module Provides storage configuration and status information for mapping VMs, datastores, and hosts to LUNs on HPE storage arrays. The Storage Module enables you to register HPE Storage arrays and use the VASA provider. Supports provisioning on HPE 3PAR StoreServ, HPE StoreVirtual, HPE MSA, and HPE EVA storage systems. Supported provisioning tasks include creating, expanding, or deleting a datastore, and creating or cloning a VM. Displays view-only information for the HPE StoreOnce Backup systems. HPE 3PAR Recovery Manager for vcenter Installs RMV plugin with vsphere Client and vsphere Web Client integration. Note: HPE OneView for VMware vcenter StoreFront Module and HPE 3PAR Recovery Manager for vcenter are not installed, only HPE OneView for VMware vcenter Server module is installed for this paper. The OneView for vcenter software can be downloaded at HPE Software Depot For detailed documentations, please check HPE Enterprise Information Library HPE Enterprise Information Library. Following are the documentations related to HPE OneView for VMware vcenter. HPE OneView for VMware vcenter Release Notes HP OneView for VMware vcenter Installation Guide (7.8) HPE OneView for VMware vcenter User Guide (7.8) Software Installation A physical or VM Windows server should be reserved for the software installation and please check the above installation guide for the details. The software is downloaded as an.exe file format. Note: Only vsphere web client is supported. Before software installation is started, make sure all active vsphere web client are closed. During the installation process, enter the vcenter server information and login credential and leave storage Administrator Portal Credential fields blank.

48 A pop-up screen showed after a successful installation.

49 vsphere Web Client Configuration After finishing the software installation. Launch vsphere vcenter web client and login to vcenter. There are two blue HPE OV4VC plugin icons are appeared at vsphere web client home page, HP Infrastructure under inventory section, and HP Management Administration under administration section. Click HP Management Administration icon to launch OV4VC configuration Getting Started page. From the left navigation panel, click Server Module Credential and provide ESXi, ilo and OA login credential under Global Device Credentials section. Since OneView is used to manage Virtual Connect, Virtual Connect Username and Virtual Connect Password fields can be left empty. Virtual connect User Name and Virtual Connect Password on the page is for legacy Virtual Connect Manager, OneView credential is configured in the following steps.

50 Click the tab of Credentials for HP Management Software Integration, Click green + sign to add HP OneView Controller, enter OneView Appliance Information and login Credentials.

51 HP Infrastructure Management After finishing setting up all credentials for inter-communications between HPE OneView, ESXi hosts, Server ilo, Enclosure On-Board Administrator, and vcenter, go to vcenter home page and click HP Infrastructure icon at the Inventory section. HPE enclosure C Enclosure detail info is being displayed from vcenter, Host Networking Management With HPE OV4VC integration, Network diagram tab under Networking provides great end to end network visibility for a vsphere infrastructure environment and there is save icon which allows the displayed network diagram to be captured in a.png image format. To see the visual mapping between ESXi vswitch/dvs, Virtual Connect and TOR switch level, select any ESXi host and then select Manage->HP Management->Networking->Network Diagram tab

52 The Network Details tab provides hard to find VMware vsphere host network detail information on a single page.

53 Host Network Configuration HPE OV4VC Reference Host concept is similar to the Host Profile concept in a native VMware vsphere environment. The Reference Host sets one host inside a cluster as a baseline host in regard to OneView network profiles and vsphere host networking configuration Network Interface Controller Provides properties of physical NICs installed in a host, port numbering, MAC addresses, link speed, and virtual switch mapping. Standard Virtual Switch / vsphere Distributed Switches - Displays name of the vswitch, defined downlink port groups or distributed port groups, and uplink port groups. Virtual Connect Information displays properties of the Virtual Connect, enclosure location, Interconnect bay location, module model and more. External Switches Displays information about the uplink switches, device name, OS info, and interface MAC addresses.

54 Conclusion This paper is a technical deployment document showing how to deploy VMware vsphere 6.0 on a HPE BladeSystem c7000, integrated with Cisco Nexus switches and NetApp running cluster Data ONTAP in a multi-vendor data center environment seamlessly and how HPE OneView allows customers, from a central management GUI, to conduct standardized server infrastructure configuration, leverage configure-once capabilities (Server Profile Template), all with end to end visibility of their Server infrastructure. Reference (Links) HPE BladeSystem Documentation HPE Virtual Connect FleFabric-20/40 F8 Module for c-class BladeSystem Documentation HPE OneView Documentation HPE OneView for VMware vcenter (OV4VC) Documentation VMware vsphere 6.0 Documentation NetApp Cluster Data ONTAP Documentation NetApp Support Download Documentation Cisco Nexus 5500 Switch Documentation VMware vsphere 5.5 with HPE BladeSystem, HPE OneView, Cisco Nexus, and NetApp Deployment Guide