Table of Contents Lab Overview - - Virtual SAN 6.2: Challenge Lab... 2 Lab Guidance... 3 Module Switcher... 8 Challenge 1 - Set Up a Virtual SAN Cluster (15 Mins)... 10 Introduction... 11 Module Switcher... 12 Create A Virtual SAN Cluster... 13 Challenge 2 - Recover Host Failure (15 Mins)... 16 Introduction... 17 Module Switcher... 18 Key Concepts For This Challenge... 19 Challenge 2: Recover From Host Failure... 20 Challenge 3 - Troubleshoot VSAN Multicast Layer 2 Issues (15 Mins)... 23 Introduction... 24 Module Switcher... 25 Challenge 3: Recover From Network Partition... 26 Challenge 4 - VSAN Stretched Cluster Creation & Optimization (15 Mins)...28 Introduction... 29 Module Switcher... 30 Create A Stretched Cluster... 31 Appendix - Hints and Solutions... 33 Challenge 1 - Hints and Solutions... 34 Challenge 2 - Hints and Solutions... 49 Challenge 3 - Hints and Solutions... 64 Challenge 4 - Hints and Solution... 68 Page 1
Lab Overview - - Virtual SAN 6.2: Challenge Lab Page 2
Lab Guidance Note: It will take more than 60 minutes to complete this lab, This is a VSAN challenge lab and it consists of 4 challenges. Each challenge requires working knowledge of VMware Virtual SAN. We highly recommend taking HOL-1708-SDC-1 in case you are new to Virtual SAN. Challenge 1 is mandatory and need to finished to be able to move to Challenge 2 or Challenge 3. Challenge 2 or 3 can be taken in any order as you prefer. Challenge 4 should be attempted at the last. You can use the Table of Contents to access any module of your choosing. The Table of Contents can be accessed in the upper right-hand corner of the Lab Manual. Lab Abstract: VSAN is "radically simple" to setup and get going, but how about configuring it for advanced use-cases, troubleshooting performance and availability issues? In this lab you will be challenged to optimize VSAN for stretch-cluster use cases and to recover from misconfiguration issues - learn how to troubleshoot VSAN and restore it to full health. Lab Module List: Challenge 1 - Set Up a VSAN Cluster (15 minutes) (Basic) Challenge 2 - Recover Host Failure (15 minutes) (Beginner) Challenge 3 - Troubleshoot VSAN Multicast Layer 2 Issues (15 minutes) (Advanced) Challenge 4 - VSAN Stretched Cluster Creation & Optimization(15 minutes) (Beginner) Lab Captains: Module 1 - Mousumi Mullick, Sr MTS, Palo Alto Module 2 - Mousumi Mullick, Sr MTS, Palo Alto Module 3 - Mousumi Mullick, Sr MTS, Palo Alto Module 4 - Mousumi Mullick, Sr MTS, Palo Alto This lab manual can be downloaded from the Hands-on Labs Document site found here: [http://docs.hol.pub/hol-2017] Page 3
This lab may be available in other languages. To set your language preference and have a localized manual deployed with your lab, you may utilize this document to help guide you through the process: http://docs.hol.vmware.com/announcements/nee-default-language.pdf Location of the Main Console 1. The area in the RED box contains the Main Console. The Lab Manual is on the tab to the Right of the Main Console. 2. A particular lab may have additional consoles found on separate tabs in the upper left. You will be directed to open another specific console if needed. 3. Your lab starts with 90 minutes on the timer. The lab can not be saved. All your work must be done during the lab session. But you can click the EXTEND to increase your time. If you are at a VMware event, you can extend your lab time twice, for up to 30 minutes. Each click gives you an additional 15 minutes. Outside of VMware events, you can extend your lab time up to 9 hours and 30 minutes. Each click gives you an additional hour. Activation Prompt or Watermark When you first start your lab, you may notice a watermark on the desktop indicating that Windows is not activated. One of the major benefits of virtualization is that virtual machines can be moved and run on any platform. The Hands-on Labs utilizes this benefit and we are able to run the Page 4
labs out of multiple datacenters. However, these datacenters may not have identical processors, which triggers a Microsoft activation check through the Internet. Rest assured, VMware and the Hands-on Labs are in full compliance with Microsoft licensing requirements. The lab that you are using is a self-contained pod and does not have full access to the Internet, which is required for Windows to verify the activation. Without full access to the Internet, this automated process fails and you see this watermark. This cosmetic issue has no effect on your lab. Alternate Methods of Keyboard Data Entry During this module, you will input text into the Main Console. Besides directly typing it in, there are two very helpful methods of entering data which make it easier to enter complex data. Click and Drag Lab Manual Content Into Console Active Window Page 5
You can also click and drag text and Command Line Interface (CLI) commands directly from the Lab Manual into the active window in the Main Console. Accessing the Online International Keyboard You can also use the Online International Keyboard found in the Main Console. 1. Click on the Keyboard Icon found on the Windows Quick Launch Task Bar. Click once in active console window In this example, you will use the Online Keyboard to enter the "@" sign used in email addresses. The "@" sign is Shift-2 on US keyboard layouts. 1. Click once in the active console window. 2. Click on the Shift key. Page 6
Click on the @ key 1. Click on the "@" key. Notice the @ sign entered in the active console window. Look at the lower right portion of the screen Please check to see that your lab is finished all the startup routines and is ready for you to start. If you see anything other than "Ready", please wait a few minutes. If after 5 minutes you lab has not changed to "Ready", please ask for assistance. Page 7
Module Switcher In the VSAN Challenge lab, Module switcher application is vital in preparing the successive challenges presented in this lab. 1. VSAN Challenge lab consists of 4 Challenges. 2. Challenge 1 is mandatory and a pre-requisite for the successive challenges. That is a student MUST finish Challenge 1 in order to accept the sussesive challenges in this lab. 3. Challenge 2 and Challenge 3 can be accepted in any order. 4. Challenge 4 is recommended to be attempted at last. Once Challenge 4 is completed, student should not go back to earlier challenges. Start Module Switcher Open the folder named "Module Switchers" on the desktop Double click the module named "3 HOL-1708-SDC-3 - VSAN Challenge" Page 8
Working with Module Switcher It is a must that the student takes Module 1 aka Challenge 1 at the beginning and then switch to Module 2 or 3. Switching to a specific module is as simple as clicking on "Start" button for that module. Page 9
Challenge 1 - Set Up a Virtual SAN Cluster (15 Mins) Page 10
Introduction Virtual SAN is a hyper-converged, software-defined storage solution for the softwaredefined data center. It is a distributed storage solution that enables rapid provisioning of storage within VMware vcenter Server as part of virtual machine creation and deployment operations. Virtual SAN uses the concept of disk groups to pool together locally attached flash devices and magnetic disks as management constructs. Disk groups are composed of at least one cache device and one or more magnetic or flash capacity devices. In Hybrid architectures, flash devices are used as read cache and write buffer in front of magnetic disks to optimize virtual machine and application performance. In all flash Virtual SAN high endurance flash is used as cache device while leveraging lower cost SSD as capacity devices. Virtual SAN datastore aggregates disk groups across all hosts in the cluster to form a single shared datastore for all hosts in the cluster. Prerequisites This challenge assumes a certain level of VSAN familiarity. Specifically basic experience in the following is suggested. VSAN configuration VSAN fault domains VSAN metro cluster VSAN networking fundamentals Storage policy based management Page 11
Module Switcher Start Module 1 Start Module Switcher Open the folder named "Module Switchers" on the desktop Double click the module named "3 HOL-1708-SDC-3- VSAN Challenge" Working with Module Switcher You MUST take Module 1 before you proceed with Module 2, 3 or 4. It is a mandatory that the student takes Module 1 aka Challenge 1 at the beginning and then switch to Module 2, 3 or 4. Switching to a specific module is as simple as clicking on "Start" button for that module. Page 12
Create A Virtual SAN Cluster What you need to know about the lab environment This lab environment presents cluster RegionA01-COMP01 with 3 hosts (esx01a, esx02a & esx03a) equipped with all flash disks. All three hosts have VSAN VMkernel networking already configured, meaning the VMkernel interfaces for Virtual SAN traffic has already been configured. Log In To vsphere Web Client Click on the Chrome browser on the desktop and it will open vsphere Web Client. 1. Check the box "Use Windows session authentication". 2. Click on "Login". Page 13
Your Challenge: Create A Virtual SAN Cluster Challenge Perform the steps listed below in order to begin the challenge 1. Configure Virtual SAN ( Hint1HInt2) 2. Add disks to storage is set to Manual (Hint) 3. DO NOT enable Deduplication & Compression.(Hint) 4. Choose to configure Fault Domains (Hint) 5. Select disks to be claimed for cache and capacity in the VSAN cluster.(hint) 6. Create three fault domains namely FD1, FD2 & FD3 each consisting of esx-01a, esx02a, esx03a respectively.(hint1hint2) 7. Review your selection & complete the work flow to enable VSAN. Refresh Web client to monitor the changes in the Virtual SAN Admin page.(hint1hint2) 8. Check the vsandatastore size ( Hint:Datastores > ReginA01-VSAN-COMP01 - >Manage > Settings > VSANDatastore > General ) 9. Check the fault domains created in previous steps. ( Hint: Cluster RegionA01-COMP01 -> Manage -> Settings -> Fault Domains & Stretched Cluster) 10. Verify the vsandatastore (RegionA01-VSAN-COMP01) has been created. (Hint) Your Challenge: Enable Deduplication and Compression Perform the following steps to enable Deduplication and Compression in RegionA01-COMP01 VSAN Cluster. 1. Edit Virtual SAN Settings ( Hint: Click on the Cluster RegionA01-COMP01 -> Manage -> Settings -> Virtual SAN -> General) (Hint) 2. Enable Deduplication and Compression. (Hint) 3. DO NOT enable "Reduced Redundancy" (Hint) 4. Monitor progress in Cluster RegionA01-COMP01 -> Monitor -> Task tab.(hint) 5. Confirm that deduplication and compression has been enabled. (Hint) Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Perform the following list of steps: 1. Select VM linux-micro-01a registered in host esx05a.corp.local.(hint1) 2. Migrate the virtual machine "linux-micro-01ato host esx02a.corp.local under Cluster "RegionA01-COMP01" to VSAN datastore "RegionA01-VSAN- COMP01" ( Hint: Right click on the virtual machine-> Migrate). (Hint1) 3. Choose"VM Storage Policy" as "Virtual SAN Default Storage Policy" (Hint: Drop Down VM Storage Policy) (Hint1) 4. Choose Destination Network as VM-RegionA01-vDS-COMP (Hint) 5. Finish migration of the virtual machine. (Hint) 6. Power on virtual machine "linux-micro-01a". Page 14
7. Verify the VM Linux-micro-01a is compliant with "Virtual SAN Default Storage Policy" (Hint: VM linux-micr0-01a -> Manage -> Policies).(Hint) Page 15
Challenge 2 - Recover Host Failure (15 Mins) Page 16
Introduction The second challenge is a work flow divided into three steps, each highlighting very important steps in setting up a functioning VSAN cluster, troubleshooting basics of VSAN networking, and how to monitor and remediate a failure scenario. 1. Add a new host into the cluster 2. Troubleshoot VSAN networking issues. 3. Monitoring Virtual SAN health and Remediate. Page 17
Module Switcher Start Module 2. It is mandatory that the student finishes Module 1 aka Challenge 1 to proceed with Module 2. Working with Module Switcher Do not click on Start Module 2 in Module switcher, unless you have completed Module 1 aka Challenge 1. It is a must that the student takes Module 1 aka Challenge 1 at the beginning and then switch to Module 2, 3 or 4.. Switching to a specific module is as simple as clicking on "Start" button for that module. To start Challenge 2, click on Start for Module 2 button in Module Switcher. Page 18
Key Concepts For This Challenge Few essentials points to keep in mind before accepting this challenge to Recover A Cluster from Host Failure: Virtual SAN Needs minimum of 3 hosts, each host contributing at least 1 SSD and 1 MD in case of hybrid cluster ( For All Flash at least 2 SSDs for cache and capacity tier respectively). Minimum 3 hosts in a cluster is needed for the Virtual Machines to be compliant with a VSAN Default Storage Policy. If one of the 3 hosts goes down, VM will still remain accessible (HA could restart the VM if it is registered on the failed host). VM/VMs will be NONCOMPLIANT. To form a Virtual SAN cluster the hosts must be part of a Virtual SAN network and must be on the same subnet whether the hosts contribute storage or not. Virtual SAN requires a dedicated VMkernel port type for Virtual SAN traffic between the hosts. Need dedicated VMkernel NIC for Virtual SAN traffic. Page 19
Challenge 2: Recover From Host Failure Run Module 2 prep script in Module Switcher on Control Center Desktop Click on Start under Module 2 and wait for it to finish. Log Into vsphere Web Client Login to vsphere web client by checking "Use Windows session authentication". Page 20
Host esx01a Failure We can see that ESXi host esx01a.corp.local is not responding. As we have understood from previous slides that VSAN cluster is not in healthy state since it has only 2 live hosts out of the 3 hosts in the cluster. VM Compliance A VM in this cluster needs 3 active fault domains in order to be compliant with Virtual SAN Default Storage Policy. Since ESXi host esx01a is down, this leaves only two active fault domains in the cluster. Hence VM linux-micro-01a in a noncompliant state as seen in the screen shot. Page 21
Your Challenge Your challenge is to bring the VM back into compliance. Here are some of the actions you may try to make the VM compliant with the existing VSAN Default Storage Policy. Move ESXi host esx-04a into Cluster "RegionA01-COMP01". Hint1 Hint2 Hint3 Exit host esx-04a from maintenance mode. Hint1 Ensure VSAN networking on host esx04a.corp.local is configured correctly. Hint1 Add host esx-04a to fault domain FD1. Hint1 Verify host esx-04a has joined the VSAN cluster using Health UI under Cluster -> Monitor -> Virtual SAN. Hint1 Hint2 The same can be verified using RVC command vsan.cluster_info. Hint1 Take remedial action as dictated by Health UI, if not already done. Hint1 Run "Retest" in Health UI to confirm network connectivity. Hint1 Monitor the data health in Health UI and take action to fix the Virtual SAN object health. Hint1 Create a Disk Group on esx04a. Hint1 Retest Health UI. Hint1 Check VM compliance and ensure it is compliant with VSAN Default Storage Policy. Hint1 Power on host esx01-a.corp.local from the console access given in the lab environment.hint1 Refresh vsphere web client until host esx01a.corp.local is live. Hint1 You might want to retest the VSAN Health UI and confirm the health of the cluster. Congratulations!!! You have successfully completed Challenge 2. Note: DO NOT forget to power on host esx-01a.corp.local at the end of this challenge. Page 22
Challenge 3 - Troubleshoot VSAN Multicast Layer 2 Issues (15 Mins) Page 23
Introduction This VSAN challenge lab module will focus on one of our most popular troubleshooting topics -- Layer 2 Multicast feature. Many Virtual SAN errors can be traced back to improperly configured multicast settings and IP Multicast is a fundamental requirement of Virtual SAN. Virtual SAN networking requirement of Layer 2 Multicast leverages multicast forwarding for a portion of its network traffic. The goal of this challenge is to learn the basics of Layer 2 Multicast functionality in Virtual SAN, how they are tightly integrated with Layer 2 networking. Key Concepts Virtual SAN employs a highly available clustered metadata database and monitoring service (CMMDS). Keeping this database in sync with all the hosts in the cluster would require a considerable amount of unicast network bandwidth. By utilizing IP multicast Virtual SAN significantly reduces network bandwidth needed for sending metadata updates to all the hosts in the cluster. Virtual SAN depends on IP multicast communication for the process of joining and leaving cluster groups as well as other intra-cluster communication services. IP multicast must be enabled and configured in the IP Network segments that will carry the Virtual SAN traffic service. Since multicasting allows the data to be sent only once, this has the benefit of freeing up computing resources from the source ESXi host. In order to efficiently make use of IP multicast IGMP Snooping and an IGMP Querier can be leveraged to send multicast traffic to only a subset of switch ports that have devices attached that request it. This will avoid causing unnecessary load on other network devices as they will not have to process packets that they have not solicited. Page 24
Module Switcher It is mandatory that the student finishes Module 1 aka Challenge 1 to proceed with Module 3. Working with Module Switcher Do not click on Start Module 3 in Module switcher, unless you have completed Module 1 aka Challenge 1. It is a must that the student takes Module 1 aka Challenge 1 at the beginning and then switch to Module 2, 3 or 4.. To accept Challenge 3, click on Start button for Module 3. Page 25
Challenge 3: Recover From Network Partition If all ESXi hosts in the cluster cannot communicate with each other, a Virtual SAN cluster will split into multiple network partitions. Hosts belonging to one partition will not be able to communicate with the hosts in another partition. When this occurs, Virtual SAN objects may become unavailable until the network misconfiguration is resolved. For smooth operations of production Virtual SAN clusters, it is very important to have a stable network with no extra network partitions Run Module 3 prep script in Module Switcher on Control Center Desktop Click on Start under Module 2 and wait for it to finish. Log Into vsphere Web Client Login to vsphere web client by checking "Use Windows session authentication". Page 26
Your Challenge Your challenge is to resolve network partition and bring the cluster into a healthy state. Here are some of the actions you may explore in order to resolve the issue: 1. Run "Retest" under VSAN Health UI to check cluster health status. Hint1 2. From the network health status, check if the cluster has encountered network partition. Hint1 3. You may want to check multicast IP settings in VSAN Health UI. Hint1 4. Verify multicast IP settings on the problematic host(s) using esxcli commands and correct it if necessary. Hint1 5. Run "Retest" in VSAN Health UI and verify that the cluster is healthy. Hint1 Key Concepts Gained From Challenge 3 Requirement of different multicast master/agent IP addresses in the same layer 2 domain if it contains multiple VSAN clusters The Master Group Multicast address is used for updating CMMDS between the master and the backup nodes via port 12345. The Agent Group Multicast Address is used for sending heartbeats to all the nodes in the cluster via port 23451 Page 27
Challenge 4 - VSAN Stretched Cluster Creation & Optimization (15 Mins) Page 28
Introduction This challenge focuses on the concept of Virtual SAN stretched cluster concept. Stretched cluster is similar to fault domain in some respects but the nodes are placed in two fault domains that lie in two different sites. This ensures that Virtual Machines are available even in the event of a catastrophic outage in one of the sites. In this challenge you will be responsible for creating a stretched cluster using 4 ESXi nodes and a fifth witness node that lies outside the VSAN cluster. Page 29
Module Switcher Mandatory to have finished Module 1 aka Challenge 1 to proceed with this module Working with Module Switcher Do not proceed with this module, unless you have completed Module 1 aka Challenge 1. It is a must that the student takes Module 1 aka Challenge 1 at the beginning to proceed further. To accept Challenge 3, click on Start button for Module 3. Page 30
Create A Stretched Cluster Before we start off to create a Virtual SAN stretched cluster let us quickly understand a few concepts. Key Concepts Virtual SAN Stretched Cluster is a specific configuration implemented in environments where disaster/downtime avoidance is a key requirement. Stretched clusters build on fault domains and provide the capability of data center awareness. Virtual SAN Stretched Clusters can now provide availability for virtual machines even if a data center suffers a catastrophic outage. Virtual SAN stretched clusters need a dedicated ESXi host for the role of a "witness" to host witness component of virtual machine objects. The witness must have connection to both the master Virtual SAN node and the backup Virtual SAN node to join the cluster. Login To vsphere Web Client Click on "Use Windows session authentication" Click on "Login" Your Challenge Your challenge is to convert the existing All-Flash standard Virtual SAN cluster to a stretched cluster with two hosts each in "Preferred Site" (esx-01a & esx-02a) and "Secondary" sites (esx-03a & esx-04a). You would need another host outside the VSAN cluster RegionA01-COMP01 which can be used as a witness node. You can choose esx-07a.corp.local under cluster "VSAN-Witness-Cluster" as witness host. ( Hint.) Page 31
Start the challenge with navigating to cluster "RegionA01-COMP01" -> Manage -> Settings. ( Hint ) Expand Virtual San drop down from the left pane and select "Fault Domain & Stretched Cluster" ( Hint ) Create Preferred and Secondary fault domains ( Hint1& Hint2 ) Select witness node and create disk group in witness node ( Hint1& Hint2 ) Create stretch cluster. ( Hint ) Please ensure that the existing VM "linux-micro-01a" is compliant after creating the stretch Virtual SAN cluster ( Hint1& Hint2 ) Check Virtual SAn cluster health status ( Hint ) Monitor the deduplication stats ( Hint ) Conclusion Congrats!! You have successfully completed the challenge. You should now be proficient at implementing Virtual SAN stretched cluster. Page 32
Appendix - Hints and Solutions Page 33
Challenge 1 - Hints and Solutions Here are the hints and solutions for the first challenge. Create A Virtual SAN Cluster Once our network adapters are in place we can configure Virtual SAN at the Cluster level. Select RegionA01-COMP01 then navigate to Manage > Settings > Virtual SAN > General > Configure Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 34
Enable Virtual SAN Add disks to storage is set to Manual Select Configure fault domains radio button Click Next Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 35
Verify Virtual SAN Network Settings Checks have been put in to verify that there are VMKernel adapters configured and the VSAN network service is Enabled. Click Next Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 36
Claim Disks For Disk Groups Select which disks should be claimed for cache and which for capacity in the VSAN cluster. The disks are grouped by model and size or by host. The recommended selection has been made based on the available devices in your environment. You can expand the lists of the disks for individual disk selection. Click Next Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 37
Fault Domains Click on the green + icon which will open a dialog box as shown in the screen. Create three fault domains namely FD1, FD2 & FD3 each consisting of esx-01a, esx02a, esx03a respectively. Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 38
Create Fault Domains Click Next Back to Your Challenge: Create A Virtual SAN Cluster Challenge Create Virtual SAN Cluster Review and verify your selection. Click Finish Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 39
Refresh Display Click the Refresh icon to see the changes. ( If you see Misconfiguration detected you may need to Refresh a couple of times ) After the refresh you should see all 3 hosts in the Virtual SAN cluster Note : In all-flash configurations, flash devices used as capacity must be manually tagged prior to use. This is detailed in the VSAN 6.0 Product Documentation. Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 40
Virtual SAN - Disk Management Select Virtual SAN ->Disk Management The Virtual SAN Disk Groups on each of the ESXi hosts are listed. Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 41
Fault Domains Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 42
vsandatastore Properties A vsandatastore has also been created. To see the capacity navigate to Datastores > ReginA01-VSAN-COMP01 ->Manage > Settings > VSANDatastore > General The capacity shown is an aggregate of the capacity devices taken from each of the ESXi hosts in the cluster (less some vsandatastore overhead). The flash devices used as cache are not considered when the capacity calculation is made. Back to Your Challenge: Create A Virtual SAN Cluster Challenge Page 43
Enable Deduplication & Compression Back to Your Challenge: Enable Deduplication and Compression Enable Deduplication Back to Your Challenge: Enable Deduplication and Compression Page 44
Monitor Progress Back to Your Challenge: Enable Deduplication and Compression Deduplication Status Back to Your Challenge: Enable Deduplication and Compression Page 45
Migrate Virtual Machine With VSAN Default Storage Policy Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Choose Migration Type Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Page 46
Choose Destination Host Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Choose Destination Datastore Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Page 47
Choose VM Network Click Next & Finish the Operation. Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Verify Virtual SAN Default Storage Policy Back to Your Challenge: Migrate Virtual Machine With "HOL Default Storage Policy" Page 48
Challenge 2 - Hints and Solutions Add A New Host to cluster RegionA01-COMP01 Right click on host esx04a.corp.local and click on "Move To..." Back To Challenge 2 Page 49
Cluster RegionA01-COMP01 Click OK. Back To Challenge 2 Page 50
Move Host Into VSAN Cluster Click Ok. Back To Challenge 2 Page 51
Exit Maintenance Mode Right click on host esx04a.corp.local Navigate to "Maintenance Mode" Click on "Exit Maintenance Mode" Back To Challenge 2 Page 52
Add Host to Fault Domain Click on cluster RegionA01-COMP01 -> Manage -> Virtual SAN -> Fault Domains & Stretched Cluster 1. Click on fault domain FD1 2. Hover on icon tagged number 2, which shows "Move hosts into selected fault domain". Click on the icon. 3. Check the box of esx04a.corp.local to add it into fault domain FD1 4. Click Ok Back To Challenge 2 Page 53
Health UI To Detect Failures Click on cluster RegionA01-COMP01 -> Monitor -> Virtual SAN -> Health Expand "Network" health Take note of the failures Back To Challenge 2 Page 54
Troubleshooting Network Health Although host esx04a has joined the VSAN cluster but it in a network partitioned state, which is evident from the "Health UI" tab under Cluster-> Monitor -> Virtual SAN -> Health. esx04a is in "Partition 1", where as the rest of the 2 hosts are in "Partition 2". Back To Challenge 2 Page 55
RVC Command We can also see from the RVC command (/localhost/regiona01> vsan.cluster_info computers/regiona01-comp01/) output that esx04a networking is not configured. Back To Challenge 2 Page 56
Fix VMkernel Port Settings 1. Click on host esx04a.corp.local 2. Click on Manage tab 3. Click on Networking tab 4. Click on VMkernel adapters 5. Select vmk2 6. Click on pencil icon to edit VMkernel port settings 7. Check the box corresponding to "Virtual SAN traffic" 8. Click "OK" to finish Back To Challenge 2 Page 57
Virtual SAN Health UI Click "Retest" to see if the previously seen errors are fixed. Back To Challenge 2 Page 58
Virtual SAN Data Health Though we have added host esx04a into the VSAN cluster, the VSAN data health errors are still prevailing. Next step shows what it takes to fix it. Back To Challenge 2 Page 59
Create DIsk Group In esx04a Virtual SAN Cluster RegionA01-COMP01 is in "Manual Mode" Hence we need to create a disk group on host esx04a 1. Click on cluster RegionA01-COMP01 -> Manage tab 2. Click on Settings 3. Select Disk Management from left panel 4. Select host esx04a & click on icon with green + ""create a new disk group" 5. Choose cache tier disk (5 GB) 6. Choose capacity tier disk (40 GB) 7. Click OK to finish Back To Challenge 2 Page 60
Retest Health UI Navigate to Virtual SAN Health UI Click "Retest" Back To Challenge 2 Check VM Compliance Select VM linux-micro-01a Navigate to Manage tab Select Policies tab Click on the highlighted icon to check compliance of all VMs Back To Challenge 2 Page 61
Power On Host esx01a Click on the console on the left upper most corner in your lab console Power On host esx01a Back To Challenge 2 Page 62
Refresh vsphere Web Client Wait for host esx01a to boot and connect in vcenter Server Refresh until it shows connected Back To Challenge 2 Page 63
Challenge 3 - Hints and Solutions Check VSAN Health Status 1. Click on cluster RegionA01-COMP01 2. Click on Monitor tab 3. Choose Virtual SAN tab 4. Health 5. Network Back To Challenge 3 Page 64
Check Network Health 1. All hosts do not have matching multicast settings. 2. Host esx01a multicast IP address is different from the rest of the hosts in the Virtual Cluster. Back To Challenge 3 Fix The Problem Open a putty session to esx01a.corp.local ( Use Putty application on desktop). 1. Run "esxcli vsan network list". This will list the multicast settings in the host. 2. Run "esxcli vsan network remove -i vmk2" Page 65
3. Change the master/agent multicast IP addresses by running "esxcli vsan network ipv4 add -i vmk2 -d 224.2.3.4 -u 224.1.2.3 Back To Challenge 3 Page 66
Verify VSAN Health Retest Virtual SAN Health Expand Network health to verify Back To Challenge 3 Page 67
Challenge 4 - Hints and Solution Configure Stretched Cluster 1. Select Manage tab under RegionA01-COMP01 cluster 2. Click on Settings tab 3. Choose Fault Domain & Stretched Cluster from Virtual SAN drop down 4. Click on Configure Back To Challenge 4 Page 68
Configure Fault Domain 1. Selct host esx-04a. 2. Move host to Secondary fault domain. 3. Select host esx-03a and move to Secondary fault domain as well. Back to Challenge 4 Page 69
Preferred & Secondary Site 1. Ensure esx-01a, esx-02a are in preferred fault domain. 2. esx-03a and esx-04a are in secondary fault domain. 3. Click Next. Back to Challenge 4 Page 70
Witness Node 1. Select host esx-07a as witness node 2. Click Next. Back to Challenge 4 Page 71
Create DIsk Group In Witness Node 1. Select cache tier flash disk. 2. Select capacity ties magnetic disk. 3. Click Next Back to Challenge 4 Page 72
Create Stretched Cluster Click Finish. Back to Challenge 4 Noncompliant VMs 1. Select VM linux-micro-01a and navigate to Manage -> Policies. 2. Check compliance of VM linux-micro-01a. Back to Challenge 4 Page 73
Compliant Back to Challenge 4 Virtual SAN Health UI 1. Retest Virtual SAN Health Back to Challenge 4 Page 74
Deduplication Take a note of the deduplication and Compression Back to Challenge 4 Page 75
Conclusion Thank you for participating in the VMware Hands-on Labs. Be sure to visit http://hol.vmware.com/ to continue your lab experience online. Lab SKU: Version: 20161106-051226 Page 76