OpEx Drivers in the Enterprise

Size: px

Start display at page:

Download "OpEx Drivers in the Enterprise"

Arline Fleming
5 years ago
Views:

1 OpEx Drivers in the Enterprise A Plexxi Perspective PLEXXI WHITE PAPER Affinity Networking for Data Centers and Clouds

2 IDC research indicates that capital costs account for nearly two-thirds of the purchasing decision for networking equipment. i But over the life of the gear, the total cost of ownership is dominated by ongoing operational costs those costs associated with the ongoing administration and maintenance of the device. Given their contribution to total cost of ownership, it is important to understand the drivers behind OpEx. This paper is intended not to advocate any specific solution but rather to highlight the major cost drivers in explicit terms. Cost Drivers vs. Cost Sources Many TCO models exhaustively look at all sources of expense to build a predictive cost model. While there are many sources, there are relatively few cost drivers. To manage the environmental, architectural, and operational contributions to overall TCO, these drivers are the highestorder elements to consider. The primary networking drivers of cost in the data center include: Number of devices under management Number of ports under management Number of administrative touch points Number of integration points Number of management models Devices Under Management Perhaps the most significant contributor to overall cost, the number of devices deployed in a data center serves as a proxy for most environmental costs. Each device will drive space costs based on its size and the price-perrack-unit for a particular environment. Similarly, each device will contribute to power and cooling costs. These costs are typically treated as annual fixed costs over the life of a device. Architectures that require fewer devices to meet capacity and connectivity requirements will perform better from a cost perspective over time. This is one of the drivers behind fabric architectures that collapse what has typically been a three-tier data center architecture into flatter, two-tier architectures. It is worth noting that environmental costs such as space, power, and cooling are a function of both the number of devices and the size of those devices. A smaller number of large chassis might consume the same resources as a larger number of small chassis. Architects should be mindful of the mix when developing detailed cost models. The total number of devices in a chosen architecture also impacts ongoing operational costs. For example, the total number of spare chassis and line cards required on-hand will scale linearly with the number of devices deployed, as will the carrying costs for these spares. These carrying costs will increase with the number of different platforms within an architecture, as each platform family requires its own spares. Additionally, administrative overhead has a correlation with the number of devices under management. Each device represents an element that must be ultimately provisioned, monitored, troubleshot, audited, and secured. As the number of devices in a data center goes up, so too do the costs associated with managing those devices. To some extent, these management costs can be partially offset by provisioning tools (as with DevOps-type tools), automation frameworks, and network controllers that reduce the total number of administrative touch points. Beyond the easily quantifiable drivers, there is an overarching complexity contribution to ongoing costs. It is impossible to model, but complexity is positively correlated with the number of devices under management. As the number increases, so too will complexity along with the costs required to manage it. Getting beyond traditional costs The traditional cost drivers in the data center are well known, but there are additional contributors to overall cost: utilization and real estate. The base requirement that drives the total number of devices in a particular deployment is capacity. The extent to which solutions can drive higher network utilization will greatly impact the total number of devices required to meet capacity requirements. If a data center expects to run at 30% capacity, it means only 30% of any capacity added is effectively used. Alternatively, it means 70% of any capacity added has no direct line of sight to being utilized. For every 100Gb of capacity needed, the data center would need to add 333Gb. This amounts to higher capital expenditures as well as more devices under management, each incurring an operational burden. In addition to utilization, a significant point of consideration for data center architects should be real estate. In most environments, the physical location is fixed. The square footage is limited by land ownership, and expansion into adjacent spaces is often not an option. This effectively creates a space budget. Every additional network device (or port) required for interconnect is space that cannot be used for servers and storage. This represents a potentially significant opportunity cost. Perhaps even more important than a space budget is the power and cooling budget. Large data centers are frequently limited by the total amount of power that can be drawn from the grid. Each additional device and port draws power, reducing the overall power allotment to servers and storage. i The Impact of a Total Cost of Ownership Model, 2009, idc_0928.pdf 1 Plexxi White Paper

3 Ports Under Management While the number of devices is a good proxy for environmental costs and administrative overhead, it is the number of ports under management that drives cabling and provisioning costs. The most basic cost tied to ports is the physical cabling required to interconnect the ports. For architectures that utilize many fabric ports, this additional cabling provides connectivity through the fabric but does not increase the total number of servers attached to the network. Essentially, the cost is pure overhead that does not increase the overall workload processing capability for the data center. How networking gear is cabled also impacts long-term operational costs. In some architectures, the interconnect ports are taken from the same pool as the server ports. For every interconnect port, the server capacity of the switch is reduced by one. For large, distributed architectures that require non-blocking paths through a core switch layer, this can represent a significant percentage of available serverfacing ports. The result is a larger number of devices to meet overall server port requirements, which is tied to ongoing environmental and operational expenses. The network architecture will also determine what build-out plans look like. When new servers require new network capacity, the resulting cabling effort can be significant. This is typically solved in two ways: preserving core-facing ports on the top-of-rack switches, or manually re-cabling the data center to insert new core switches. In the former, the impact is to overall device count as each ToR has fewer server-facing ports. In the latter, the impact is to operational costs tied to adding new capacity. Worth considering relative to the number of ports under management is the type of ports. Some port types will consume more power than others. For example, 10GBASE-TX ports will tend to be more power-hungry than DACS or optical ports. Architectures that favor lower-power ports will tend to drive lower operational expense. Beyond the cabling and power costs, the total number of ports under management servers as a reasonable proxy for provisioning, monitoring, and maintenance costs. Each port represents another entity that must be managed. Number of Administrative Touch Points The largest primary driver of cost is the total number of devices under management. While the environmental costs will grow linearly with the number of devices, the ongoing operational costs can be mitigated somewhat by reducing the number of administrative touch points in the network. To some extent, the rise of software-defined networking is a response to the rising operational costs tied to network growth. Controller-based architectures are designed to provide central points of control through which entire networks can be managed. Effectively, these controller-based solutions are reducing the number of administrative touch points in the network. By providing a single point from which all devices can be provisioned, monitored, and troubleshot, the overall effort required to do so is greatly reduced. What might have been thousands of lines of configuration can be reduced by several orders of magnitude if the provisioning point is centralized. This has the added benefit of driving down human error the single largest source of network downtime in most networks. A single point of administration also lends itself well to providing better network visibility. By collecting distributed data and presenting it from a single point, the network is better documented, making troubleshooting tasks shorter and more straightforward. This ultimately impacts metrics like Mean Time to Insight (the time it takes to correctly diagnose and triage new issues) and Mean Time to Repair. While network downtime is a significant cost driver, it is seldom built into TCO models because predicting downtime is difficult. But for many networks, even small amounts of unplanned downtime can dominate total operational costs. Number of Integration Points Capability in isolation is useless. Ultimately, data center networking gear must be integrated with surrounding infrastructure to provide any real value. That surrounding infrastructure certainly includes other networking devices, but integration extends well beyond network interoperability. Network infrastructure must be integrated with surrounding compute, storage, and application components. There are application lifecycle management systems, provisioning systems, monitoring infrastructure, capacity planning tools, billing systems, API portals, helpdesk ticketing systems, change management tools and processes, and the list goes on. Each integration requires time and money. Accordingly, the number of points at which these integrations must be executed will be a cost driver. For architectures requiring device-by-device integration, costs can be high. Those that handle integration through central points will contribute lower cost. These costs are incurred at both the time of integration as well as at any subsequent change. 2 Plexxi White Paper

4 More than the volume Beyond the sheer number, most of these integrations require some exchange of data (configuration, statistics, diagnostic information, and so on). If each supporting tool is responsible for harvesting this information separately, the effort to integrate will be higher. To the extent that architectures can provide a common means of extracting data from the system, these costs can be lowered. Number of Management Models Beyond just the number of devices that must be managed, the number of disparate ways in which those devices are managed is a cost driver. Where architectures are standardized around a single device type or family of devices, there is typically one management model. The single operating system environment lends itself well to developing and leveraging a single set of training materials, provisioning models and templates, standard operating procedures, and supporting processes like auditing and change management. Costs associated with these types of tasks will tend to scale linearly with the number of management models within a data center. Companies who reduce management complexity will see reduced ongoing operational costs. Architectural Decisions Points Given these drivers of operating expense, there are a few architectural decision points to consider. Start with fewer devices Given the role that the number of devices plays in driving long-term operational expense, the most important decision a data center architect can make is the foundational architectural approach. To control costs, architects should favor designs that require fewer devices. Legacy three-tier architectures are already being replaced by more modern two-tier approaches. As technology continues to evolve, two-tier architectures are being supplanted by completely flat designs. To the extent that these flat designs can reduce the total number of devices in the network, they can dramatically improve long-term cost models. Utilization matters Capacity costs can be measured in simple price-per-unit terms. If the usable capacity is only a fraction of what is available, the price-perunit increases. Low utilization also requires higher capacity overhead. Architects should consider how to drive higher network utilization so that they can take advantage of better economics. This is likely to be a function of solution capability, so buyers will need to augment purchasing criteria appropriately. Architect for uptime Architecture has a profound impact on network downtime (scheduled or otherwise). Architects should pay careful attention to failure and maintenance domains, resilience features, and upgrade procedures. Further, customers should consider how the short-term capital costs amortized over the life of the equipment compare to longer-term downtime trends. These evaluations are particularly important where fees and penalties are concerned (as with managed or cloud services). Additionally, the cost of downtime for certain applications can exceed capital costs for even small amounts of downtime (ecommerce, financial services, and so on). SDN should provide relief The central control model that SDN promotes provides a single point of administration, which will drive maintenance costs down. Accordingly, buyers should consider SDN controller-based capabilities as a top-tier purchasing criterion for data centers where cost is important. DevOps is still in its formative stages Automation is clearly the future for most large-scale data centers. The transition to a fully automated environment will have a dependence on a management discipline and a tooling ecosystem that are both still emerging. This is generally referred to as Development Operations (or DevOps). Put simply, DevOps provides a tailored glue layer between management models maintained by a combination of management frameworks (Chef, Puppet, Ansible, and so on) and in-house programming staff. Because DevOps is in its formative stages, it is impossible to predict with any kind of precision which tools will ultimately win. It is highly likely that companies will operate with some mix of commercial and homegrown tools designed to meet their specific requirements. This has already resulted in a fractured operational tooling landscape. The impact on networking vendors will be that point tool integrations will be handled case-by-case, typically driven by significant revenue opportunities. This will cause a scattered DevOps tool support matrix that will not perfectly match most customer environments. Until DevOps frameworks become richer and natively support more management models, expect to see higher in-house development costs to maintain a fully DevOps-automated environment. Accordingly, data center architects will need to consider not just operational tool support but also ongoing integratability of tools within the architecture. This should favor DevOps-friendly solutions that have built underlying data service infrastructures that allow for repeated integration with new tools. 3 Plexxi White Paper

5 Conclusion While there are many individual sources of cost, the major driving factors are few: Number of devices under management Number of ports under management Number of administrative touch points Number of integration points Number of management models Controlling these cost drivers should be a primary objective when designing all data centers. While specific device characteristics and capabilities can mitigate costs to some extent, the most significant contributor to ongoing operational costs is the underlying data center architecture. Accordingly, data center architects should consider long-term cost impacts of architectural designs. Every data center architecture begins with minimizing complexity. The ultimate measure of effective design, though, is whether complexity (and associated cost) remains low as applications place additional capacity and management requirements on the infrastructure. 4 Plexxi White Paper

RED HAT ENTERPRISE LINUX. STANDARDIZE & SAVE.

RED HAT ENTERPRISE LINUX. STANDARDIZE & SAVE. Is putting Contact us INTRODUCTION You know the headaches of managing an infrastructure that is stretched to its limit. Too little staff. Too many users. Not