Field Update Expanded Deduplication Sizing Guidelines. Oct 2015

Transcription

1 Field Update Expanded Deduplication Sizing Guidelines Oct 2015

2 As part of our regular service pack updates in version 10, we have been making incremental improvements to our media and storage management capabilities. The next generation of Commvault software includes significant enhancements that allows significantly improved scalability and resiliency. In order to take advantage of these improvements as well as to ensure that new designs will continue to be valid for the next generation of the platform we have expanded the sizing guidelines for deduplication. These new guidelines are available in Books Online at the following locations tm d_mode.htm d_ext_mode.htm Key changes to the guidelines are A new configuration (Extra Large) with significantly larger capacity per node. Increased capacity for existing configurations. Additional detail for each configuration including a suggested workload section that details the typical data usage profiles. Explicit sizing and suggested workloads for Partitioned Deduplication grids and extended mode deduplication nodes. When do these changes take effect? Immediately. All new and existing designs that are in progress can use these guidelines right away. These can also be used to respond to RFP questions and high level designs where specific details about the environment are not available. These designs have already been used and validated with few select customers in the field. What is the impact on existing customer environments? Existing customers will see no impact on their environments. We have not changed any hardware requirements on the existing configurations. We did increase supported capacities and suggested workloads on the same hardware. It is recommended that customers verify their current deduplication deployments do meet the hardware requirements and that they are on the latest service pack levels to fully benefit from the expanded capacity. Many existing customers are already finding capacity utilization and performance to exceed the previous guidelines. These expanded guidelines simply extend the benefit to all deployments. Do these guidelines apply to the next release of Commvault software? Yes, these expanded guidelines have been devised keeping in mind the needs of the next generation software. Solutions designed using these guidelines for version 10 will continue to work after upgrade to the next generation. However, we require all new implementations in v10 to meet the hardware requirements even if the upgrade to next generation software is not going to occur anytime soon. Service Packs in V10 includes many improvements that benefit from the additional hardware for the extra large configurations. Simpana Field Communications Update Commvault Products Team Page 2 of 5 2

3 Does this affect the sizing of other Simpana components? For example, does the VSA sizing change because of this? No, the other software components have their own sizing requirements that are detailed in separate sections in BOL. We will be bringing all of these sizing sections under a single sizing topic in the next few iterations of BOL. For example, the VSA sizing remains at 40 TB FET per node (for SAN mode) However a dedupe node that also runs the VSA agent is not limited to 40 TB. It only means that the VSA agent on that host can handle about 40 TB FET. The remaining capacity can be utilized by other network based agents including other network VSA agents. We will be publishing a revised sizing guideline for VSA in the next BOL update. When does the Index cache need to be on SSD? As we manage increasingly large datasets, we are recommending a shift to an incremental forever and infrequent Synthetic Full approach for data types that support it (mostly Files, VMs and s). This requires the index cache be located on fast disk to handle the backend processing required to support that approach. Consider using SSD for index cache in those scenarios as the cost differential is minimal these days between high performance spinning drives and value SSDs. However, certain data types, like applications, do not need large or high performing index cache volumes. If it can be guaranteed that the deduplication nodes will be used only for these workloads, then some of the Index cache requirements may be relaxed. However, before deviating from the prescribed requirements, please validate with your local solution architects or the Corporate SE team. Most customers however, prefer to use a standardized architecture since they cannot know with certainty all future workloads a particular configuration will be required to handle. It is important to note that as we build out new solutions that take better advantage of native access and native copy capabilities built into ContentStore, this cache space will be utilized for more than just the Index Cache. Do we really need all that SSD storage? Yes, SSD based metadata storage is required to build scalable, resilient architectures. While SSDs are more expensive than spinning drives, the use of SSDs, especially for deduplication databases, allows us to provide significantly higher capacity per node thus limiting the cost of acquiring additional nodes. The incremental cost of adding SSDs is more than offset with the higher capacity (50% or more) per node. Also the cost of SSDs continues to drop at a rapid rate. One way to position the use of SSDs is to describe this as cache layer for our engine. It is common for storage vendors to position controller based cache as one key factor that determines the storage system s scalability. Typically, this cache is a small percentage of the system capacity. We require just over 1% of SSD cache for usable active capacity which is a fairly low proportion. Note that we do not mandate expensive enterprise class SSDs. For most workloads the lower end (value or endurance value) SSDs validated for the particular server model will suffice. Look for drives write endurance (DWPD) between (0.3 and 1 or better) for guidance. Can I run deduplication nodes on blade servers/virtual machines if I meet the hardware requirements? As a software solution, we are agnostic to the underlying hardware, however, we are not indifferent to it. The biggest challenge with running on blade servers/virtual machines is the sharing of resources, specifically SSD disks. Most blade servers are constrained by how many internal drives they can have. As a result, most of the SSD cache must be stored on shared SAN storage devices. However, since the SSD resources must be dedicated, it becomes increasingly Simpana Field Communications Update Commvault Products Team Page 3 of 5 3

4 expensive to have dedicated DDB volumes on SAN controllers that support the concurrent IOPs necessary for all DDBs residing on it. The same is true for virtual machines. This soon becomes a cost/value decision, not a technical decision. Only a handful of very large storage consumers have the economies of scale to provide dedicated SAN controllers for multiple DDB workloads. So while it may be technically feasible, for most typical customers, the hardware spend could be high enough to create the perception that Commvault solution is expensive. Even if customer plans to provide dedicated hardware on day 1, often SAN storage and its utilization is not managed by the backup team. Other workloads could come online on the SAN after the Commvault deployment that are not visible to the backup team, reducing the dedicated available IOPS. In the scenario where a customer is insisting on running on shared infrastructure and claims to have the required dedicated hardware: Make sure the customer understands the real cost implications (dedicated SAN storage and blades with sufficient IOPS and blade infrastructure vs. cheaper dedicated internal SSD disks on rack mount servers) Validate the design with your local solution architects or the Corporate SE team. Before deployment require the validation of the environment using sizing tools to ensure the right resources are available. Ensure customer has metrics deployed to proactively track deduplication operating parameters. What is the relation between the usable backend capacity (BET) and suggested FET? Backend Terabyte (BET) refers to the actual storage capacity consumed and includes all retained copies of backup/archive data. The BET is best reflected by the size on disk property on the global dedupe policy. FET on the other hand, is the total size of the clients that backup/archive to a dedupe node. The FET for a client is best reflected by the application size of the last full backup for that client. As a rule of thumb for day retention, BET is 1.2X to 1.5X FET, depending on data type. For retention of up to 1 year BET is usually 2X FET. The capacities in the sizing guideline reflect the upper suggested values for both BET and FET. Note that there is nothing in the software that limits the BET or FET per dedupe node. The only limiting criteria is the performance and the capacity of the DDB volumes. We have many customers that have exceeded the suggested BET and FET limits very easily. How do I use the sizing guidelines to respond to RFPs? Can I use it for high level sizing when I don t have too many details? It is very common to have limited information about customer environment when doing a preliminary sizing exercise or responding to RFPs about hardware requirements. However, for most large environments you can assume certain data type distribution and retention. You can then apply these assumptions to the sizing guidelines to arrive at preliminary hardware recommendations. Typical data breakout in large environments is as follows Files/NAS: 50%-60% VMs: 20% - 30% Applications/Databases: 20%-30% Environments such as these typically need 2 copies and have disk retention ranging from 2 weeks to 3 months. Given those assumptions a 1 PB environment can be sized as using the Extra Large configuration as follows Simpana Field Communications Update Commvault Products Team Page 4 of 5 4

5 Example: Sizing for 1 PB environment Data Type Data Size # of nodes Primary # of nodes Secondary Comments Files/NAS 500TB 600TB 4 4 Use Extended mode on each MA. Covers files plus enough head room for typical growth. VMs 200 TB 300 TB 2 2 Assuming extended mode where each node is a primary backup target as well as secondary copy target from other site Applications/ DBs 200 TB 300 TB 2 2 Assuming extended mode where each node is a primary backup target as well as secondary copy target from other site Total 1 PB 8 8 Assumes data is split across 2 sites and each site creates secondary copy on the other site Table assumes data is split across two sites with cross site secondary copy. Note, that the above sizing leaves more than enough room for growth as well as longer retention. The above configuration allows total of 6.4 PB BET capacity across the 16 nodes and 2 sites. (Assuming extra large nodes and 400 TB BET per node in extended mode) For certain data types that require multi-year retention, size for up to a year to determine the storage capacity required. For retention longer than one year, we recommend automatically sealing the dedupe store each year. Just before the store is automatically sealed add the same amount of storage capacity to the disk library each year till the retained data starts aging out. This is explained in more detail in the Deduplication Sizing and Best Practices deck on CIC. Summary In anticipation of the next generation Commvault software release and to take advantage of the many enhancements in the deduplication engine, we have an enhanced deduplication sizing guideline that introduces new configurations and increased suggested capacities per node. These enhanced guidelines include a new extra large configuration as well as explicit description and guidelines for Extended Mode, Partitioned Dedupe and Extended Mode Partitioned Dedupe. Existing customers are not affected by these new recommendations. As a result of these changes Commvault can now support around 800 TB of usable space in an Extended Mode Partitioned Grid. The enhanced sizing guidelines, along with certain assumptions on data type distribution, allows one to quickly estimate hardware requirements for multi-pb opportunities using limited number of deduplication nodes. These guidelines have been created keeping in mind the next generation of Commvault software and should be used for all new designs going forward. Simpana Field Communications Update Commvault Products Team Page 5 of 5 5