OPEN COMPUTE PLATFORMS POWER SOFTWARE-DRIVEN PACKET FLOW VISIBILITY, PART 2 EXECUTIVE SUMMARY. Key Takeaways

Transcription

1 OPEN COMPUTE PLATFORMS POWER SOFTWARE-DRIVEN PACKET FLOW VISIBILITY, PART 2 EXECUTIVE SUMMARY This is the second of two white papers that describe how the shift from monolithic, purpose-built, network packet brokers (NPB) to software-driven, off-the-shelf hardware platforms based on merchant silicon is enabling packet flow visibility solutions that are more flexible, scalable and cost-effective to deploy. This paper describes how the latest generation of multicore processor x86 server platforms can further reduce the cost and increase the flexibility of deploying visibility overlay networks by performing memory- and CPU-intensive enhanced packet processing functions that typical Open Compute white box switches are not able to support due to memory restrictions and the limitations of merchant silicon network processors. At the same time, software-defined packet flow visibility enables software innovation not just in the data plane, but also in the control plane, allowing NPB vendors to simplify the configuration and management of the visibility overlay, streamline and automate operator workflows and reduce the time needed to perform many routine tasks. Note: This paper uses the term Open Compute generically but is referring to the types of commodity hardware platforms certified by the Open Compute Project ( Key Takeaways Market and technology trends are driving the need for pervasive packet flow visibility in enterprise and service provider networks Off-the-shelf, merchant silicon-based hardware platforms are cost-effective to deploy and enable softwaredriven packet flow visibility solutions that easily scale on demand Powerful multicore processor x86 server platforms offer a cost-effective alternative to purpose-built NBPs for performing enhanced packet processing functions Software-driven packet flow visibility unleashes innovation in both the control and data planes, streamlining the control and management of the visibility overlay network

2 PERVASIVE NETWORK VISIBILITY FOR MONITORING AND SECURITY The increasing demands on enterprise and service provider networks are driven by several powerful trends that are reshaping the modern data center: Growing adoption of DevOps methods to enable enterprise digital transformation Migration of enterprise IT applications to public, hybrid and multi-cloud environments Engagement with customers and consumers via cloud-based applications and services Support for IoT applications and services across a wide range of vertical markets These trends are forcing network operators to scale their data center networks to accommodate nonstop growth in users, applications, connectivity and bandwidth, as well as an increasingly diverse mix of devices: desktop PCs, laptops, tablets, mobile phones and IoT sensors. Data centers are moving to hyperscale leaf and spine switching architectures that require taps in every rack for visibility into not only north/south traffic to and from the data center, but also into the explosive growth of east/west traffic within the data center. These networks readily scale horizontally using pod-based designs in which compute and switching capacity are simply added in rack-based increments, so packet flow visibility infrastructure should scale just as easily. As enterprises embrace virtualization of network and computing infrastructure, softwaredefined networking, SD-WANs and Internet connectivity to public clouds and software as a service (SaaS) applications, network operators need to extend visibility beyond the confines of the traditional data center, across the WAN, out to remote sites and into the cloud. At the same time, DevOps, network, security and IT operations teams are using a growing and diverse set of tools for monitoring network and application performance across this entire expanse, which also presents a much larger attack surface for a myriad of security threats. As the number of monitoring and security tools deployed grows, the visibility overlay can help ensure that these tools are utilized more efficiently by preprocessing packets before delivering them to the tools, so that tool capacity is concentrated on critical processing functions and not consumed by processing duplicate packets, slicing excess packet payload, stripping headers, terminating tunnels, etc. ENHANCED PACKET PROCESSING FUNCTIONS Open Compute white box switches are not well suited to performing various kinds of enhanced packet processing functions before delivering packet flows to centralized monitoring and security tools due to memory restrictions and the limitations of current generation merchant silicon network processors. The following are the stateful, memory- and CPU-intensive enhanced packet processing functions typically implemented in monolithic NPBs using a combination of FPGAs, network processors and/or special-purpose chipsets, along with 2

3 sufficient packet buffer capacity and memory to perform the necessary stateful processing: IP tunnel termination MPLS/GRE header stripping Regular expression-based filters Packet slicing/masking Packet de-duplication Flow metadata (NetFlow, sflow, etc.) L4-L7 application-aware filtering GTP correlation MULTICORE PROCESSOR X86 SERVER PLATFORMS Over the past decade, as CPU performance has been leveling off, Intel has increased the total processing capacity of x86 servers by dramatically increasing the density of CPU cores per processor from two to four cores up to 28 cores in the latest Xeon server-class processor. Using multicore processors with large cache memories and high-performance NICs, it is now economically feasible to perform enhanced packet processing functions using an off-the-shelf x86 server platform that provides the CPU power and memory capacity to perform complex packet processing functions at line rate. Advanced packet processing in x86 server platforms is typically accelerated by incorporating special-purpose software for streamlining the packet processing data path, such as the open source Data Plane Development Kit (DPDK) software originally developed by Intel but now a Linux Foundation project. Vendors have delivered x86 server platforms supporting total system throughput in excess of 100 Gbps on current generation hardware. A high-end x86 server platform running packet flow visibility software and configured with sufficient packet buffer and cache memory can handle the complex, stateful enhanced packet processing functions previously listed. DEPLOYING X86 SERVER NETWORK PACKET BROKERS Figure 1 shows x86 server NPBs deployed to perform enhanced processing functions on packets aggregated from across a data center network using Open Compute NPB switches with the x86 server NBPs performing various advanced packet processing functions prior to the flows being delivered to centrally located monitoring and security tools. This is a cost-effective, pervasive visibility solution that fully leverages off-the-shelf, merchant silicon-based hardware platforms for packet flow visibility, from basic tap/aggregate and advanced brokering functions in white box switches to enhanced packet processing in high-performance x86 servers. 3

4 Figure 1. Packet Flow Visibility Using Open Compute NPB Switches and x86 Server NPBs Another key benefit of using off-the-shelf servers is the ease of scaling packet processing capacity by simply deploying additional x86 servers and distributing packets from these NPBs across multiple instances of the same monitoring and security tools (shown in Figure 2). Figure 2. Scaling by Distributing Flows across Multiple NPB x86 Servers Note that although these x86 servers are typically stacked adjacent to centrally located monitoring and security tools and interconnected via the underlying Open Compute NPB switches, the servers could just as easily be distributed across the network and deployed anywhere enhanced packet processing capacity is required, at a remote site or a private cloud location where certain traffic flows that need to be monitored traverse the network. BENEFITS OF X86 SERVER NETWORK PACKET BROKERS The benefits of NPBs based on x86 server platforms mirror those for NPBs based on Open 4

5 Compute white box switches: Software-defined packet flow visibility Enhanced packet processing is performed entirely in software Easy to integrate open source software, such as DPDK Software-driven features not gated by hardware development cycles Cost-effective, off-the-shelf hardware platforms Leverage high-volume manufacturing economies of scale Ride the merchant silicon (Intel x86) performance curve Scalable on-demand: Adding packet processing capacity is as simple as adding another server Servers can be stacked in a centralized pool or distributed as needed SOFTWARE-DRIVEN INNOVATION EASES OPERATIONAL COMPLEXITY Operational complexity is the number one challenge network operators face when deploying and managing a large-scale visibility overlay network. Pervasive visibility requires tapping into many links across the network using an array of NPB switches that are not only interconnected at the physical layer but must also be configured to support a packet flow topology from each tap point to the destination monitoring and security tools. The configuration process involves first defining how switches and ports are interconnected physically and then specifying how packet flows are to be selected and aggregated by defining a set of filters to be applied to traffic arriving on each switch port. These filters determine how each packet is processed and switched across the visibility overlay for delivery to the monitoring and security tools, which may involve passing them through an x86 server NPB for enhanced packet processing. This configuration process can be quite tedious, time consuming and error prone if the operator is required to do all this manually box by box, especially in a large-scale visibility overlay with many NPBs. Fortunately, disaggregation of packet flow visibility software from the underlying hardware enables software-driven innovation not just in the data plane but in the control plane as well, enabling NPB vendors to create new tools that ease operational complexity by simplifying, streamlining and automating the configuration and management of the overlay visibility network, dramatically reducing the time to perform routine operational tasks. MANAGEMENT INNOVATION STREAMLINES CONFIGURATION AND CONTROL For managing the network visibility overlay, network operators need easy-to-use GUI tools that allow them to visualize and configure the overlay network through a single pane of glass. 5

6 Operators need drag-and-drop tools for configuring switch ports, setting filters and defining how flows should be processed and where they should be delivered. The overlay network management system should then automatically send the necessary configuration parameters to the individual switches. Ideally, the operator should never have to configure or manage an individual NPB, except when performing low-level troubleshooting tasks that require examining the configuration or network state of the switch. Network operators also need visualization-based tools that simplify visibility overlay network monitoring and anomaly detection. Operators need simple dashboards with at-a-glance monitoring of graphically displayed KPIs. They should be able to drill down to the switch, port and filter level to gain visibility into packet flow topology and network state. It is also important to provide methods for processing and examining overlay network alarms. The sole purpose of the visibility overlay network is to deliver packet flows to monitoring and security tools. A software-driven visibility overlay should also support open APIs that provide tools with direct control over packet flow visibility, enabling programmatic control of filter settings so that the tools can automatically gain the desired visibility into packet flows on the fly without intervention by the operator. This could be done programmatically via the operator s management interface, but a true software-defined solution would allow this to be done directly via a control plane API. SUMMARY AND FUTURE DIRECTIONS Since the start of the decade, leading chip vendors, most prominently Broadcom and Intel, have driven remarkable advances in the processing power and throughput of commercially available, off-the-shelf switching and compute platforms based on merchant silicon. Open Compute white box switches now help power some of the largest global data centers. The core density of multicore Intel x86 processors increases with each new generation, further driving down the cost per MFLOP. Powerful x86 servers can now be used for advanced packet processing with system throughput exceeding 100 Gbps, extending their use beyond traditional computing workloads. Merchant silicon-based networking and compute platforms are key building blocks in the next generation of hyperscale data centers, powering software-defined infrastructure that scales linearly while unleashing an unprecedented wave of software innovation that is fundamentally changing how applications and services are delivered. The cloud-scale giants, leading SaaS providers and network service providers are all leveraging the disaggregation of monolithic, purpose-built hardware and software platforms into software-defined infrastructure based on off-the-shelf hardware. It should come as no surprise that these same off-the-shelf platforms can be used to provide 6

7 pervasive packet flow visibility across this next-generation infrastructure (in addition to existing data centers) while allowing network operators to take advantage of the enormous economies of scale and the same economic and operational benefits. Look for the leading NPB vendors to leverage this as their product lines evolve to keep pace with networking industry trends. Although off-the-shelf hardware is certainly more cost-effective than purpose-built hardware and can easily scale on demand, ultimately the greatest benefit of platform disaggregation will be software innovation in both the data plane and the control plane. NPB vendors can readily incorporate open source components for advanced packet processing, software-defined networking and API-driven control plane automation for streamlined operation and tighter integration with monitoring and security tools. Network operators will also benefit from better tools for simplified configuration and management of the visibility overlay. Looking further down the road, it seems inevitable that a network built exclusively using open, off-the-shelf, merchant silicon-based platforms could support a software-driven, entirely virtual packet flow visibility overlay network running on the same physical hardware as the production network. Continued advances in merchant silicon will enable switch implementations in which packet flow visibility becomes a native capability of data center switching architectures. ACG Research is an industry analyst and consulting firm focused on the networking and telecom markets, providing business consulting and syndicated research services. Copyright 2017 ACG Research. Sponsorship: NETSCOUT, September