THE MISSING LAYER: SDN ANALYTICS AND AUTOMATION FOR MULTI-SERVICE NETWORKS

Transcription

1 THE MISSING LAYER: SDN ANALYTICS AND AUTOMATION FOR MULTI-SERVICE NETWORKS WHITE PAPER

2 Table of Contents Introduction 3 The Challenges of Running Multi-Service Networks 3 Supporting Unique Service Requirements 3 Accelerating Service Activation 4 Delivering Intelligent Management and Automation 5 The Role of Analytics in SDN Automation 6 Packet Design s Explorer Suite 7 Packet Design Explorer SDN Platform 10 Packet Design SDN Traffic Engineering Application 12 Defining a Traffic Matrix 16 Specifying Policy 17 Recommendations 18 Packet Design SDN Path Provisioning Application 21 The Explorer SDN Platform s Open APIs 22 Summary 23 Page 2 of 24

3 Introduction Software Defined Networking (SDN) is revolutionizing data center networking by decoupling the control and data planes, and by enabling control plane programmability by applications. There has been tremendous innovation in data center networking applications, from virtual network overlays to security. The next step in the SDN evolution is to apply this architecture, particularly programmability, to the wide area network (WAN) and address management challenges facing operators. In this white paper, we focus on service provider IP/MPLS WANs, although our arguments apply equally to other WANs, including transport networks. We will describe how network virtualization and SDN technologies are making it possible to create dynamic, flexible networks that can be reconfigured quickly to accommodate different business requirements but lack the management intelligence to fulfill the promise of autonomous networking. Packet Design asserts that another layer of management is required to create truly adaptive networks. This analytics and automation layer uses real-time telemetry, analytics, path computation, optimization and policy to enable intelligent provisioning of network services for increased business agility, optimal use of capital investments, and increased operational efficiency. The Challenges of Running Multi-Service Networks Today s service provider networks are complex because they must support many applications ranging from Internet access, streaming video, voice over IP, layer 2 and layer 3 VPNs, 3GPP mobile backhaul and core transport, and cloud services. This is in contrast to networks of the past, which typically were dedicated for different applications. For example, a service provider may have operated a Frame Relay network for enterprises, a SONET ring-based metro-area network for mobile backhaul traffic, and a best-effort IP network for Internet access, etc. Obviously, running multiple networks required higher capex and opex investments than a single multiservices network. In addition, many of the traditional networks used circuit-switching technologies and were expensive to scale, since they wasted allocated but unused capacity. Many applications now run as a service on top of converged IP/MPLS packet-switching networks, which are much more efficient, scalable and fault tolerant. However, their performance is less predictable and requires closer monitoring of service paths. Supporting Unique Service Requirements Running multiple applications on a converged network presents management challenges because each application has unique performance requirements, growth rates, and fault-tolerance characteristics. For example, a financial services enterprise may be willing to pay premium prices for very short delay paths to support its time-sensitive trading application. To provide this service, the service provider may need to find these short delay paths, segregate the application traffic from other traffic, and fully protect it from link and router failures. Page 3 of 24

4 In contrast, services offered to residential customers for over-the-top streaming video (such as Netflix and YouTube) have quality requirements that, if not met, will cause pixilation and playback pauses that may lead to customer churn. So low jitter paths are most suitable for this type of traffic. As streaming video is adaptive to available bandwidth within limits, an operator may provision for optimum video quality under normal conditions but allow it to degrade within acceptable levels during a demand surge or under failure conditions. Optimizing the network for any service is hard and requires significant time investment by expensive engineers. Typically, a traffic demand matrix is created, mapping the volume of external traffic entering the network at each ingress router to each egress router where that traffic exits the network. An optimization algorithm uses this matrix and the network topology to recommend paths for each ingress-egress pair in the traffic matrix to minimize the maximum link utilization in the network 1. Once the paths are computed, they are provisioned in the network devices. This last step can be simplified and automated by an SDN controller or orchestrator that abstracts vendor-specific provisioning mechanisms. Optimizing a multi-services network is much harder and probably not achievable without automation due to each service s specific requirements. Each service requires a traffic matrix and an appropriate optimization algorithm, which must run concurrently with those for all other services. Again, an SDN controller can simplify and speed provisioning, but topology, traffic and performance analytics are needed to make intelligent, automated traffic engineering possible. Accelerating Service Activation In addition to supporting multiple services, another challenge that service providers face is an increase in the rate of service activation and deactivation requests combined with a reduction in acceptable service provisioning times from weeks to hours, and even seconds. For example, many service providers offer self-service portals that allow customers to request more bandwidth. Here again, these requests can be automated by an SDN controller that issues commands to the network devices without human intervention. These SDN uses cases are just the beginning. Before long, automated network provisioning will be done by machine to machine communications, using API calls. For example, when virtual machines must be migrated to a different data center, a data center orchestrator will communicate with the WAN orchestrator requesting paths with sufficient capacity to support the move and the communications patterns after the move. Many Internet of Things devices will operate this way, and the volume and rate at which services are provisioned will skyrocket. As we see, SDN can help address these challenges and streamline network provisioning. Figure 1 illustrates a simple two-tier architecture in which SDN applications control network behavior. Network 1 Page 4 of 24

5 devices (both physical and virtual) are not configured, rather they are programmed via southbound APIs by one or more SDN controllers. The controllers provide access to applications via northbound APIs, enabling the applications to modify network behavior to meet their needs. WAN SDN applications are only now being developed; examples include traffic engineering, demand placement, bandwidth calendaring, and risk mitigation and remediation. Figure 1: Two-Tier SDN Architecture Delivering Intelligent Management and Automation Although SDN controllers provide the means to change network configurations through software, it is clear that they lack management intelligence. In other words, to build networks that can adapt automatically to business demands requires another tier in the SDN architecture one for analytics and automation. The rest of this paper will explore SDN Analytics and Automation. The next section discusses the analytics required, and the subsequent sections describe Packet Design s Explorer SDN Platform technologies: Foundational telemetry, analytics and algorithms in the Explorer product suite Functional details of the first vendor-agnostic SDN Analytics and Automation Platform SDN Analytics and Automation use cases Page 5 of 24

6 The Role of Analytics in SDN Automation If applications can change network behavior without human intervention, what governs whether or not these programmatic changes should be made and what their impact on the network will be? Answering these questions is the role of analytics software. SDN analytics can determine the impact of requested changes and govern whether or not they should be allowed. There are two important functions for SDN analytics. The first is to maintain management visibility into the network even while changes are being made programmatically. With no traditional work order process to follow, operators may be unaware of changes being made. When programmatic changes work and have the desired results, this lack of visibility may be acceptable, but when changes are problematic, how does the operator diagnose the problem or find the application culprit, buggy controller or failing network device that is causing the issue? SDN analytics provide visibility into the network -- the devices and the controllers -- by recording real-time telemetry from the network s control plane protocols, including the topology, performance metrics, and traffic flow data. The recorded data can be very helpful for back-in-time forensics to identify the root cause of issues. The second and more important function of SDN analytics is to provide management intelligence. Traditionally, when an operator needs to make a significant change to the network, a planning group is responsible for assessing the network s readiness for the change. For example, when a service provider acquires a new enterprise customer, or an enterprise turns on a new service, the planning Figure 2: Three-Tier SDN Architecture, including an SDN Analytics and Automation Layer Page 6 of 24

7 group must determine if the network has sufficient capacity to accommodate it. If not, they will try to find paths in the network that may satisfy the needs. For viable, programmatic SDN automation, this planning know-how must be replicated in analytics software, the basis of which is a combination of real-time and historical telemetry, projections and algorithms. Given the importance of SDN analytics, therefore, the two-tier SDN architecture shown in Figure 1 needs to be expanded to include an analytics and automation layer, as shown in Figure 2. This new layer delivers both management visibility and intelligence to SDN applications, and is described in The Heavy Reading white paper authored by Caroline Chappell, Combining Management Intelligence & SDN Programmability to Meet Hyperfast Enterprise Needs 2. Packet Design s Explorer Suite The Explorer product suite provides the necessary telemetry and algorithms for the analytics and automation layer. For more than a decade the analytics in the Explorer products have enabled network teams to run their networks more efficiently. The same analytics are now the foundation for SDN applications. As shown in Figure 3, the Explorer suite combines routing, traffic and performance analytics. Figure 3: Explorer Suite - Route Explorer, Traffic Explorer and Performance Explorer Route Explorer provides a very detailed model of the WAN s routing, including the IGP topologies, RSVP-TE and Segment Routing tunnels, MP-BGP routes, as well as the services running on the network, such as layer 2 and layer 3 VPNs, and multicast. Indeed, Route Explorer s model is more 2 Page 7 of 24

8 detailed than the one found in routers; for example, Route Explorer s model includes topologies from all IGP areas and ASs, whereas a router only knows of the topologies of the IGP areas to which it is connected. Route Explorer maintains this model in real time by tapping into routing protocols to capture every routing event. It computes both shortest and constrained paths across networks, spanning multiple IGP areas and BGP ASs (see Figure 4). Figure 4: Route Analytics showing the path from Seattle to New York on a network topology map Traffic Explorer collects flow records from the network and provides Internet as well as layer 2 and layer 3 VPN traffic visibility. It taps into Route Explorer s network model so it can associate each flow with a service, identifying where it enters the network (ingress router, VRF, interface and autonomoussystem), where it exits the network (egress router, VRF, interface and autonomous-system), and what path it takes along the way, including RSVP-TE and Segment Routing tunnels, if any. From this, it builds traffic reports for each link, tunnel, service, ingress and egress router, as well as traffic matrices between any two routers. These traffic matrices, as we discussed in the introduction, are a key component of the SDN Traffic Engineering application described later in this paper. Performance Explorer collects delay, loss, jitter, device CPU and memory usage, and interface statistics, such as octets and packets in, out and dropped from network devices. It is made path- Page 8 of 24

9 aware by leveraging Route Explorer s topology model. For example, when a delay on an important path increases, Performance Explorer can determine if this was the result of a route change in the underlying topology. By correlating the end-to-end to per-hop measurements along the service path, the links or devices causing a delay surge are pinpointed quickly (see Figure 5). Performance Explorer s data also identifies low-delay or low-jitter paths in the network often sought by financial enterprises and others with low-latency application requirements. Figure 5: Path-aware analytics correlate end-to-end to per-hop performance measurements The Explorer suite includes interactive modeling capabilities for simulating modifications to the network topology, service traffic and performance measurements, and analyzing their impact. For example, it is straightforward to simulate moving a service running on VMs from one data center to another and see the impact in the WAN whether or not the service s workload would cause any congestion and impact on other services. In another example use case adding a new customer to the network the Explorer products would calculate the performance the new customer would receive and any negative impacts to other services. It is not surprising that this modeling capability is one of the fundamental building blocks for SDN Analytics and Automation and a key component of the Packet Design Explorer SDN Platform. Page 9 of 24

10 Packet Design Explorer SDN Platform Given the new management challenges that SDN created and described earlier in this paper, traditional management processes and tools must be reevaluated and updated. For example, if programmatic changes are being made to network configurations every few minutes or seconds, polling devices periodically to collect metrics or relying solely on flow data is inadequate. Dynamic networks require real-time telemetry and analytics. The Packet Design Explorer SDN Platform described below is designed for conventional IP/MPLS networks and next-generation SDN. Figure 6: Explorer SDN Platform Components Figure 6 illustrates the building blocks of Packet Design s Explorer SDN Platform. At the bottom of the figure are the network devices, both physical and virtual. These include routers, switches and network functions (e.g., firewalls, load balancers, etc.). Above this infrastructure layer is a set of SDN controllers that program these devices using southbound protocols. Several controllers may be deployed for different regions of the network and functionalities. For example, a service provider may have different controllers on each continent, and may use different controllers to provision services at the network edge versus provisioning paths in the core. These controllers may be supplied by different vendors and/or open source organizations; the Explorer SDN Platform is controller agnostic. Page 10 of 24

11 The Explorer SDN Platform is built on the Explorer products. At the heart of the platform is Route Explorer, capturing real-time network telemetry from the devices in the network as well as the SDN controllers. As a result, the platform knows the exact network topology and paths used by the services in the network, as well as each service s traffic flow and performance metrics. A realtime model of the network is maintained, which is important for adapting to unforeseen events in the network. For example, if a link fails and causes congestion, the model will reflect the change immediately, and a risk mitigation application could automatically change the paths for some flows to alleviate the issue. The platform also records all telemetry in its database, enabling network events to be replayed for back-in-time forensics, and also for establishing historical baselines. However, a real-time model alone is not sufficient, since many applications need predictive network models. For example, if a large amount of data is to be moved across the network, a time slot must be identified when the network will have sufficient available capacity. Predictive models of network behavior use historical baselines of network traffic and performance to calculate expected network behavior in the near future. The path computation and optimization components find shortest as well as constrained paths in the network. Multiple algorithms are used depending on the service s requirements. For instance, for a high-revenue premium service, an algorithm would be used to find shortest delay paths. For streaming video, an algorithm that minimizes jitter would be appropriate. Also, when the network capacity starts to run out, network traffic may be spread to under-utilized links using an artificial intelligence-based algorithm. The policy component allows operators to set the business rules. In the SDN Traffic Engineering application described later, it is used to establish the path characteristics that may be recommended under normal and exception conditions. For example, the policy for a streaming video service might be to allow a percentage of under-provisioning during heavy load periods, because streaming video is adaptive and the video quality may still be acceptable. The provisioning connector interacts with service orchestrators and SDN controllers via northbound APIs to provide the configuration data for provisioning paths and services. SDN applications interact with the Explorer SDN Platform using open, RESTful APIs. These APIs are available for Packet Design and third-party applications to access the platform s components. For example, they enable service providers to develop custom applications that implement their own business logic. Effectively, the APIs create plug-and-play SDN analytics and automation components (sometimes called micro-services) that may be accessed and used with other third-party components. Page 11 of 24

12 Packet Design SDN Traffic Engineering Application The SDN Traffic Engineering (SDN-TE) application demonstrates the platform s analytics and automation capabilities. The goal of the application is to balance link utilization, reducing the load on heavily-congested links by shifting it to lightly-used links. To illustrate how the application works, a small lab network testbed was created as shown in Figure 7. This topology consists of four provider edge (PE) routers and four provider core (P) routers. All of the edge routers are virtual Cisco XRv routers and all of the core routers are virtual Cisco IOSv routers. Another VM runs the OpenDaylight SDN controller which uses BGP-LS and PCEP protocols to interact with the routers. The Explorer SDN Platform is connected to both the routers and the OpenDaylight controller, as illustrated in Figure 8. Figure 7: SDN Traffic Engineering Testbed Topology Page 12 of 24

13 Figure 8: SDN Platform Deployment The SDN-TE application presents a network health view as shown in Figure 9. This view summarizes network performance with the current set of paths and traffic demands. Shown at the top left of the screen is the maximum link utilization which, in this example, is a congested 93 percent. Next to that are the maximum delay and maximum number of hops taken by the current paths in the network. In this small lab topology, the Explorer SDN Platform is reporting 12ms delay and three hops. To the right, the bar chart shows the link utilization distribution. Fifteen links are not congested at all, three are in the medium range, one is congested and two are extremely congested. The goal of the SDN-TE application is to reduce the utilization of these overly congested links. Page 13 of 24

14 Figure 9: SDN-TE Application Network Health View Below these charts is the traffic demand matrix. It shows how much traffic is flowing from each ingress router to each egress router. The cell colors indicate the utilization of the most congested link along the path from the ingress router to the egress router. Clicking on one of the cells shows the path for that ingress-egress pair and the congested link along the path (see Figure 10). In this case, even though there is only 50Mbps of traffic from the north-edge router to the east-edge router, it is going over a link that is congested to 92.5% link utilization by other traffic. The SDN-TE application s network health view may present a network with little or no congestion, in which case no further action would be necessary. In this testbed network, however, that is clearly not the case. Page 14 of 24

15 The SDN-TE application permits full-mesh traffic engineering as well as tactical traffic engineering to manage isolated congestion in the network by defining a suitable traffic matrix for each use case. Policies may be created to define allowable path characteristics; for example, if non-shortest paths may be created and whether or not traffic may be split across multiple paths. Traffic matrices and policies are explained in more detail below. Figure 10: The path between the north-edge and east-edge has a bottleneck link with 92.5% utilization Page 15 of 24

16 Defining a Traffic Matrix Traffic engineering calculations are often made using utilization levels when the network is most congested. The SDN-TE application allows operators to select either the peak or the 95-percentile traffic levels and whether daily, weekly or monthly statistics are preferred. It also allows a specific time to be selected. Whichever option is used, Traffic Explorer scans the time range for the traffic level requested and creates a traffic demand matrix for that time. The SDN-TE application lists router groups, which may be selected individually or in groups to define the traffic matrix. For example, choosing the Edge Routers group as the source and destination router groups will create a full-mesh traffic demand matrix between all edge routers in the network. Using other groups, it is possible to create a demand matrix between just the core routers, or geospecific traffic matrices. Multiple traffic matrices can be created simultaneously, and they can either be optimized at the same time or at different times of the day to satisfy the needs of the multiple services running on the network. It is possible to exclude small demands from the traffic matrix by specifying a bandwidth value as a filter. This causes only the large demands in the matrix to be rerouted. However, even though they are not rerouted, the smaller demands are factored into link utilization predictions when making new path recommendations. It is necessary to define different traffic matrices for different services. The traffic matrix definition process allows specification of either a traffic class-of-service (i.e., IP DiffServ or MPLS Exp bits) or a traffic group definition. The latter allows finer granularity in specifying the service, including the IP addresses and ports of the service end points. Tactical traffic engineering, used to move traffic away from specific links to alleviate congestion, is perhaps more widely deployed. In this case, a full-mesh of traffic matrices is not required. In this situation, the operator must determine the ingress and egress of the traffic flows traversing the few congested links. The traffic needs to be rerouted at the ingress, not at the point of congestion, otherwise, the flows would incur additional delay and network resources preceding the congested link would be wasted. This is very hard to do manually, but Traffic Explorer provides this information to the SDN-TE application. The user specifies a Maximum utilization per link value and any link above this threshold is automatically selected. The ingress and egress routers, whose flows use these links, are automatically inferred. A traffic matrix is then generated for this set of ingress-egress router pairs. Page 16 of 24

17 Specifying Policy Policies are used to specify the path characteristics that the Explorer SDN Platform can recommend. Foremost, a policy specifies an algorithm to use (see Figure 11). The default is a fast, global constraintbased optimization algorithm. Others include propagation delay-based and artificial intelligencebased algorithms. Figure 11: Controlling recommendations via policy Secondly, the policy controls whether traffic is allowed to be split among multiple paths between ingress and egress routers. This is accomplished by specifying a bandwidth value. In the example shown in Figure 11, traffic will be split into 450Mbps tunnels. So, if the traffic volume is 950Mbps, up to three tunnels will be used. However, if the resulting recommendations yield the same path for each of these tunnels, they will be merged to the minimum number of tunnels. Splitting the traffic into multiple Page 17 of 24

18 paths is useful when no single path has sufficient capacity to satisfy the demand. When setting policy, it is also possible to request these paths to be link-diverse. It is also possible to specify over- or under-provisioning of a service s demand by a percentage amount. For example, a service provider may opt to over-provision a business customer and underprovision video streaming traffic. Recommendations After defining the traffic matrices and policies, and affirming that the network needs to be traffic engineered, the SDN-TE application makes tunnel recommendations with a click of the mouse. The user is presented with a list of recommendations and a network health impact analysis, as seen in Figure 12. Figure 12: Recommendations and Impact Analysis Page 18 of 24

19 In this example, the SDN-TE application recommends 14 tunnels to be created. It has calculated that the new set of paths will reduce the maximum link utilization in the network from 93 to 55 percent, as shown at the top left of this view. Next to that, the delay and hop count show small to no adverse effects. The worst-case delay has increased by 5ms due to the use of non-shortest paths. The resulting link utilization distribution at the top right of the screen shows no congested links. It is possible to examine the individual paths as shown in Figure 13. Figure 13: A recommended tunnel path Once the operator is satisfied that the recommendations will have the desired impact, clicking the SDN-TE application s Provision button signals the Explorer SDN Platform to communicate these paths to the SDN controller via an open API. The SDN controller uses a southbound protocol (typically, but not limited to, the Path Computation Element Protocol) to set up the paths in the network devices and ensures these paths are persistent (the SDN-TE application may update or un-provision them upon the user s request). Page 19 of 24

20 The next step is to monitor these newly provisioned tunnels in operation. Any errors, perhaps due to anomalies in the network devices that SDN controllers cannot detect and analyze, will be detected and displayed, because Route Explorer directly monitors all network devices (see Figures 14A and 14B). We see that there are 14 tunnels and all of them are up. However, we also see that none of these tunnels is currently protected using fast re-route (FRR) techniques. The tunnel interface names on the right side of the figure are assigned by the XRv routers and are not available via the controller. Route Explorer s SDN analytics correlate what the SDN-TE application provisioned with what is actually running in the network. Figure 14A: Provisioned Tunnels Summary Page 20 of 24

21 Figure 14B: A provisioned tunnel details Packet Design SDN Path Provisioning Application Another application that demonstrates the Explorer SDN Platform s analytics and automation capabilities is SDN Path Provisioning. Today, if a subscriber requests a new transport path between a source and destination with pre-defined constraints, the current method involves analyzing the up-to-date network topology, gathering performance metrics, and then using an external planning tool to compute the possible paths that meet the constraints defined by the subscriber. This can take days or even weeks, and customers are not always willing to wait that long. The Explorer SDN Path Provisioning application, built on the Explorer SDN Platform, overcomes these challenges by accessing the analytics data collected by the Explorer Suite to compute new paths and provision them in minutes through third-party SDN controllers. A video demo of the SDN Path Provisioning application can be viewed here: Page 21 of 24

22 The Explorer SDN Platform s Open APIs The Explorer SDN applications accesses the Explorer SDN Platform components via open REST APIs. The APIs are also intended for third party use and use JSON for ease of development. A sample API is shown in Figure 15 with sample parameters and output. Figure 15: Sample REST API Page 22 of 24

23 Summary Service providers must operate increasingly complex networks that support multiple services with varying and often conflicting requirements. They must also process a higher volume and rate of requests for network resources that need to be provisioned rapidly, often within seconds. SDN architecture, once applied to the WAN, can help address these challenges. However, running a software defined network presents its own challenges, including loss of visibility into changes taking place in the network and the need to capture engineering know-how in SDN applications. Fortunately, both of these challenges can be addressed by analytics, consisting of network topology, traffic and performance telemetry, projections and algorithms. The Packet Design Explorer SDN Analytics and Automation Platform provides the critical analyticsbased automation layer needed between the SDN controller(s), service orchestrators and SDN applications. The SDN Traffic Engineering application leverages these analytics via RESTful APIs to demonstrate the platform s automation capabilities. For more than a decade, Packet Design s Explorer Suite has been empowering network engineers to run their networks more efficiently. Now, adapted for next-generation networks, the Explorer SDN Platform provides the same analytics for SDN applications and makes it possible for developers to quickly code SDN applications with rich analytics that leverage their network know-how. Page 23 of 24

24 To learn more about Packet Design and the Explorer Suite, please visit Page 24 of 24