vsan Health Check Improvements First Published On: Last Updated On:

Transcription

1 vsan Health Check Improvements First Published On: Last Updated On:

2 Table of Contents 1. Health Check and Performance Improvements 1.1.Online Health Check 1.2.Performance Diagnostics 1.3.vSAN performance service improvements 1.4.Support Improvements 1.5.vSAN 6.6 Performance Improvements 2

3 1. Health Check and Performance Improvements vsan cloud powered health and performance monitoring improvements help administrators get the most out of vsan introduced in version 6.6 3

4 1.1 Online Health Check vsan 6.6 Health Check Improvements Online Health Check The Native vsan health UI was introduced with VSAN 6.1. This was an external plugin that could run a number of basics checks on a cluster. With vsan 6.2 this feature and functionality became a core art of the product in the form of the Health UI which can be found within the Monitor Tab, under VSAN and Health. Here dozens of proactive configuration and state checks are made and with each version of vsan the list of checks has expanded to help proactively alert users to potential risks in their environment. While customers have been enthusiastic about this functionality, a key limitation is that it requires an update to vsan and ESXi in order to deploy new health checks. In vsan 6.6 we look to solve this problem with the Online Health Check feature. These checks will be dynamically updated from VMware s environment as new potential issues and KB s are identified. An example is shown below where VMware has detected vsan and non-vsan disks attached to the same storage controller. An Ask VMware button is supplied in the user interface, which guides administrators to the relevant VMware knowledge base article. In our example above, this action would open the VMware Knowledge Base (KB) article Best practices when using vsan and non-vsan disks with the same storage controller ( ). New health checks appear as new KB articles are created and published. This benefit is delivered without the need to upgrade vsphere and vsan." 4

5 This process has a number of advantages over traditional notifications and release notes. It will identify specifically if a KB is relanvent to your environment for you. This will provide a feedback loop for both configuration information to engineering, as well as for engineering to customers on how to get the most out of their vsan environment. Requirements These features will CIEP enabled to function. For more information on the configuration, performance, and feature usage data that is collected please see the CEIP homepage. The vsan performance diagnostics will also require the performance service to be enabled in order to function. 1.2 Performance Diagnostics Performance Diagnostics Cloud-based performance Diagnostics in vsan enables administrators to easily upload performance data for diagnostic purposes of benchmarks. VMware will analyze the data and provide recommendations for improvement. This feature is intended to be used when running benchmarks. This feature requires that the Performance Service has been enabled, and the Customer Experience Improvement Program (CEIP) has been enabled. In the following example, a benchmark was performed on a hybrid cluster with a very small amount of SSD configured per host. By reviewing the specific time range (last 90 days kept in the performance database by default) phone home the data to be processed for recommendations. 5

6 The interface in this example presents with disk groups reporting that the cache size may not correctly be sized. Clicking into a disk group we can see that our read cache hit rate is low. From this, we can identify that if we would like to improve our latency response we need to increase the cache size. The Ask VMware option allows us to get more information about the performance counters we are looking at. HCI Bench Integration For running storage benchmarks on vsan HCI bench is VMware's recommended solution. It can be downloaded from the VMware flings website. in HCI bench version vsan performance diagnostic integration is built in when using it with vsan Within a project folder, the performance_diag_result.html will allow you to directly check improvement suggestions, as well as redirect to the vcenter Server UI for that benchmark run's time 6

7 frame. HCI bench run time frame will populate as fixed selection ranges to make it simpler check previous run tests. 1.3 vsan performance service improvements Performance Service Improvements Previously vsan observer was used for an number of very low level information counters related to internal queues, and network interface statistics. This product was highly useful for advanced admin performance diagnostics, but it had several draw backs. 1. It required to be manually run and should not be left on for long periods of time. 2. It required use of the ruby interface and was not native to the existing VMware tool set. 3. It did not have built in explanations or ask VMware functionality to describe what data points meant. It required advanced training to use. The VSAN performance service takes on several new critical functions in the 6.6 feature release. As part of deprecating of vsan observer it has to fill in the gap in collecting detailed information that allow for diagnosing of performance or configuration issues. Improved data collection New Data collection counters such as pnic and VMkernel adapter information is now being fed into the performance database to help identify networking issues that are causing vsan performance issues. 7

8 In addition to these counters internal vsan queues, resync traffic and other potential contention points are now exposed showing where all potential IO contention or issues are coming from. For production clusters, VMware strongly recommends the vsan performance service be enabled. In the next section will will discuss how the VMware performance service has been improved to assist with working with VMware GSS. 1.4 Support Improvements vsan 6.6 Phone Home and Support Improvements VMware Has made several improvements to vsan 6.6 to improve the experience in working with VMware support. vsan observer as of vsan 6.6 is being deprecated, but still included. As a result for Vmware support operations some key improvements will replace their need to rely on this tool. Verbose Mode A new option exists to directly phone home performance data using the performance service to VMware support. Cluster -> Configure -> Health & Performance -> Edit Settings of Performance Service. Hit the "verbose mode" checkbox. You will want to disable this once support confirms receipt 8

9 of vsan performance information. Additional Improvements for supportability by VMware GSS A key limitation of vsan observer was that it required a customer enable it for the purposes of troubleshooting. If an issue was intermitant, or had previously occurred prior to turning on vsan observer it was more difficult to understand the root of the issue. VMware support has added significant additional infrastructure to process the performance and configuration data that is coming in as part of the CEIP program. This will allow VMware GSS staff to be able to much more rapidly understand the cluster configuration, and performance characteristics of the entire environment not only from a single point in time but also over several days. For customers not using the CEIP program creating a support bundle will also export two days worth of performance data from the vsan performance database to be included in the review. The VSAN observer database will retain 90 days of performance data, and with the ability of GSS to export from it or leverage phoned home data, customers will no longer need to turn on and off vsan observer for supports purpose in diagnosing performance or configuration issues. 1.5 vsan 6.6 Performance Improvements vsan performance and performance Service Improvements vsan 6.6 has the following improvements 9

10 Much more logs in less space In vsan every IO operation is logged before been processed. Without efficient log management in place, the size of the log entries can grow very quickly. When the log size reaches close to the limit of their max allocated size this triggers unnecessary IO operations to process the entries. This may impact the performance of the cluster. Various factors play in determining the actual impact such as what is the current health of cluster pending IOs and such others vsan has developed efficient log management strategies that allows more logs to be packed per byte of storage. This prevents the log from reaching its assigned limit too fast and too frequently. In addition it provides enough time for vsan to process the log entries before it reaches it s assigned limit thereby avoiding unnecessary IO operations Pre-emptive destaging for large file deletes in scenarios where dataservices are turned on Deletes in vsan cluster where datservices are turned on need to be managed to maintain object consistency.the IO processes involved in processing the delete are maintained in logs and as such the log can quickly grow very large (even with enhanced log packing). This is specially true in scenarios where we are deleting multiple files several TBs in size. vsan has built in algorithms that destages data on periodic basis. The destaging operations coupled with efficient log management significantly improves performance for large file deletes including performance for write intensive workloads Checksum improvements Last but not the least are checksum improvements. Made several enhancements that made the checksum code path more efficient. It is expected the enhancements for all flash scenario shall be even further optimized since there is no additional read cache look up (all reads happen in capacity so avoids additional look ups). These enhancements are expected to provide significant performance benefits for both sequential and random workloads 10