The Road to SDN: An Intellectual History of Programmable Networks Network Virtualization and Data Center Networks SDN - Introduction

Transcription

1 The Road to SDN: An Intellectual History of Programmable Networks Network Virtualization and Data Center Networks SDN - Introduction Qin Yin Fall Semester Data, Control, and Planes Data, Control, and Planes Time-scale Tasks Location Data Control Packet (nsec) Forwarding, buffering, filtering, scheduling Line-card hardware Event (10 msec to sec) Routing, signaling (circuit setup) Router software Human (min to hours) Analysis, configuration Humans or scripts 3 4 Data vs. Control Plane Data plane: packets Streaming algorithms that act on packets Handling individual packets as they arrive Matching on some bits, taking a simple action at behest of control and management plane Wide range of functionality Forwarding, access control, traffic monitoring, buffering, marking, Shaping and scheduling, etc. Control plane: events Distributed routing algorithms Compute paths through the network Disseminate the addresses of end hosts Track changes in network topology Reserve resources along a path Routing vs. Forwarding Routing: control plane Computing paths the packets will follow Routers talking amongst themselves Individual router creating a forwarding table Forwarding: data plane Directing a data packet to an outgoing link Individual router using a forwarding table 5 6

2 Data and Control Planes Adding Plane data plane Line card Processor control plane Line card Making the network run well Traffic reaches the right destination Traffic flows over short, uncongested paths Unwanted traffic is discarded Failure recovery happens quickly Routers don t run out of resources Line card Line card Switching Fabric Line card Line card A control loop with the network Measure (sense): topology, traffic, performance, Control (actuate): configure control and data planes 7 8 Software Defined Networks In the SDN architecture, The control and data planes are decoupled Network intelligence and state are logically centralized The underlying network infrastructure is abstracted from the applications Open Networking Foundation white paper A Clean Slate 4D Approach to Network Control and Albert Greenberg, Gisli Hjalmtysson, David A. Maltz, Andy Myers, Jennifer Rexford, Geoffrey Xie, Hong Yan, Jibin Zhan, Hui Zhang SIGCOMM Comput. Commun. Rev A Conventional View of a Network A Conventional View of a Network A C E H A C E H F I F I B D J B D J G Configuration File Physical topology is a graph of nodes and links Run Dijkstra to find route to each node EIRGP Access Control Table NAT Table Tunnel Table BGP OSPF Forwarding Table

3 A Conventional View of a Network E A H C A Study of Operational Production Networks How complicated/simple are real control planes? What is the structure of the distributed system? B D EIRGP Access Control Table NAT Table Tunnel Table F Configuration File BGP J OSPF Forwarding Table I Use reverse-engineering methodology There are few or no documents The ones that exist are out-of-date Anonymized configuration files for 31 active networks (>8,000 configuration files) 6 Tier-1 and Tier-2 Internet backbone networks 25 enterprise networks Sizes between 10 and 1,200 routers 4 enterprise networks significantly larger than the backbone networks 14 Configuration State for One Network Limitation of Today s Control & Complex System of Systems Lines in config file Router ID (sorted by file size) Systems are designed as components to be used in larger systems in different contexts, for different purposes, interacting with different components Example: OSPF and BGP are complex systems in its own right, they are components in a routing system of a network, interacting with each other and packet filters Complex configuration to enable flexibility Lack of intellectual framework to understand global behavior 16 Complexity IP Control Plane function overloading Reachability Policy control Resiliency and survivability Traffic Engineering, load balancing VPN Ethernet control plane overloading Spanning Tree, RSP, MSTP, vlan, Control & management complexity works against robustness, dependability, security Limitation of Today s Control & Difficult to Implement Interesting Policies Data Center Infrastructure Servers Network designers want simple things, but achieving them is incredibly hard 18

4 Limitation of Today s Control & Difficult to Implement Interesting Policies Limitation of Today s Control & Difficult to Implement Interesting Policies Difficult for designers to express desired behaviors Difficult for designers to express desired behaviors Limitation of Today s Control & Difficult to Implement Interesting Policies Difficult for designers to express desired behaviors Limitation of Today s Control & Hard to Coordinate Multiple Low Level Mechanisms To Achieve Network-wide Goals Lack of higher level specification of network wide goals Load balancing objectives vs. per link OSFP weight Reachability matrix vs. per interface access control list Difficult to dynamically coordinate multiple mechanisms Forwarding, access control, NAT, tunnel management Limitation of Today s Control & Lack of Robust Communication Channel Circular dependencies between management plane, control plane, data plane & control traffic travels along the paths that they intend to maintain Operational networks usually require a separate control/management network Summary Limitations of Today s Control & Planes High complexity Difficult to implement non-trivial control policies Difficult to dynamically coordinate control logics Lack of robust control communication channel Root cause: Wrong partition of functionalities in control/management plane, which leads to lack of flexibility and high complexity 23 24

5 4D: A Clean-slate Design Propose a clean-slate repartitioning of functionality, rather than exploring incremental extensions Completely separate decision logic (network issues) from underlying protocols (distributed systems issues) Three Principles for Network Control & Network-level Objectives: Express goals explicitly Security policies, QoS, egress point selection Do not bury goals in box-specific configuration Reachability matrix Traffic engineering rules Logic Three Principles for Network Control & Network-wide Views: Design network to provide timely, accurate info Topology, traffic, resource limitations Give logic the inputs it needs Three Principles for Network Control & Direct Control: Allow logic to directly set forwarding state FIB entries, packet filters, queuing parameters Logic computes desired network state, let it implement it Reachability matrix Traffic engineering rules Reachability matrix Traffic engineering rules Write state Logic Logic Read state info Good Abstractions Reduce Complexity Plane Control Plane Data Plane Configs FIBs, ACLs FIBs, ACLs Decision Plane Dissemination Data Plane A Clean-Slate Approach: The 4D Architecture Decision Plane Dissemination Plane Generate table entries Routing Table Access Control Table NAT Table Tunnel Table Install table entries All decision making logic lifted out of control plane Routers no longer makes decisions on routing Dissemination plane provides robust communication to/from data plane switches Discovery Plane Data Plane Modeled as a set of tables 29

6 Overview of the 4D Architecture Decision Network-level objectives Overview of the 4D Architecture Decision Network-level objectives Network-wide views Dissemination Direct control Network-wide views Dissemination Direct control Discovery Discovery Data Data Decision Plane: All control and management logic implemented on centralized servers making all decisions Decision Elements use views to compute data plane state that meets objectives, then directly writes this state to routers Dissemination Plane: Provides a robust communication channel to each router and robustness is the only goal! May run over same links as user data, but logically separate and independently controlled Overview of the 4D Architecture Decision Network-level objectives Overview of the 4D Architecture Decision Network-level objectives Network-wide views Dissemination Direct control Network-wide views Dissemination Direct control Discovery Discovery Data Data Discovery Plane: Each router discovers its own resources and its local environment E.g. Topology and traffic monitoring Data Plane: Packet handling in spatially distributed routers/switches Can deploy with today s technology D Separates Distributed Computing Issues from Networking Issues Distributed computing issues: protocols and network architecture Overhead Resiliency Scalability Networking issues: decision logic Traffic engineering and service provisioning Egress point selection Tunnel management Reachability control (VPNs) Precomputation of backup paths Example: 4D Approach to Reachability Control Path Computation Traffic Matrix Topology Load info Reachability matrix FIBs, ACLs Decision Plane Discovery/Dissemination Plane Data Plane Reachability matrix directly expresses intended goal Path computation can jointly balance load and obey reachability constraints Packet filters installed only where needed, and changed when routing changes 35 36

7 Example: 4D Enables Simpler and Better Traffic Engineering Using the 4D Architecture D D Evaluated using Emulab ( Example network with 49 switches and 5 DEs OSPF normally calculates a single path to each destination D OSPF allows load-balancing only for equal-cost paths to avoid loops Using ECMP (Equal-cost multipath routing ) requires careful engineering of link weights Decision Plane with networkwide view can do more sophisticated optimization Linux PCs used as routers ( MHz) Tested on 9 enterprise network topologies ( routers each) A prototype of the 4D Architecture 4D Architecture permits many designs Prototype is a single, simple design point Decision plane Contains logic to simultaneously compute routes and enforce reachability matrix Multiple Decision Elements per network, using simple election protocol to pick master Dissemination plane Uses source routes to direct control messages Extremely simple, but can route around failed data links Does it work? Yes. 4D designed so performance can be predicted Recovers from single link failure in < 120 ms < 1 s response considered excellent Faster forwarding reconvergence possible Survives failure of master Decision Element New DE takes control within 170 ms No disruption unless second fault occurs Gracefully handles complete network partitions Less than 170 ms of outage At no point did two DEs attempt to master the same switch One Size Fits All? State-of-Art Different set of protocols for different data planes STP for Ethernet OSPF/BGP for IP Same protocols (logic) for different environments Data center, campus, ISP Hard to customize, hard to extend 4D Common dissemination (protocol) IPv4, IPv6, Ethernet Customizable decision plane (algorithm) Data center, enterprise, access, metro, backbone Reachability, traffic engineering, robustness, etc. Control Plane: The Key Leverage Point Great Potential: control plane determines the behavior of the network Reaction to events, reachability, services Great Opportunities A radical clean-slate control plane can be deployed Agnostic to packet format: IPv4/v6, ethernet No changes to end-system software Control plane is the nexus of network evolution Changing the control plane logic can smooth transitions in network technologies and architectures 41 42

8 Goal Ethane: Taking Control of the Enterprise Martìn Casado, Michael J. Freedman, Justin Pettit, Jianying Luo, Nick McKeown, Scott Shenker SIGCOMM Comput. Commun. Rev Design network where connectivity is governed by high-level, global policy Nick can talk to Martin using IM marketing can use http via web proxy Administrator can access everything Traffic from secret access point cannot share infrastructure with traffic from open access point Three Fundamental Principles How could we change the enterprise network architecture to make it more manageable? The network should be governed by policies declared over high-level names. Policy should determine the path that packets follow. The network should enforce a strong binding between a packet and its origin. Today s Namespace Lots of names in network namespace today Hosts Users Services Protocols Names are generally bound to network realities (e.g. DNS names are bound to IP addresses) Often are multiple bindings that map a name to the entity it represents (DNS -> IP -> MAC -> Physical Machine) Problem With Bindings Today Need Secure Bindings Host Name Goal: map hostname to physical host Bindings are authenticated IP MAC Physical Interface Host MAC But!!! What if attacker can interpose between any of the bindings? (e.g. change IP/MAC binding) What if bindings change dynamically? (e.g. DHCP lease is up) Or physical network changes? Cached bindings are appropriately invalidated Address reallocation Topology change Permissions changes/revocation Physical Interface Host 47 48

9 Policy Language Declare connectivity constraints over Users/groups Hosts/Nodes Access points Protocols Services Connectivity constraints are Permit/deny Require middlebox interposition Isolation Etc. Ethane Flow-based network Central Domain Controller (DC) Implements secure bindings Authenticates users, hosts, services, Contains global security policy Checks every new flow against security policy Decides the route for each flow Access is granted to a flow Can enforce permit/deny Can enforce middle-box interposition constraints Can enforce isolation constraints Design Overview Central domain controller Contains the global network policy and topology view. Performs route computation for permitted flows, and decides whether each flow is allowed and how it is routed. Ethane switches Consists of simple flow tables and a secure channel to Controller Simply forwards packets under the direction of the controller Names and policy language All users, hosts, switches, protocols etc. have names, that are used when writing rules for the controller. Five Basic Activities in an Ethane Network Registration: All switches, hosts, and users register with the controller. Bootstrapping: Switches maintain secure channels with controller. Minimum spanning tree (MST) rooted at controller. Authentication: A host joining the network is redirected by switch to the controller for authentication (by MAC) when it does DHCP. Controller records bindings host->ip, IP->MAC, MAC- >switch port. User is authenticated (e.g. password) via browser. Controller records binding user->host Five Basic Activities in an Ethane Network Bootstrapping Flow setup: A initiates connection to B. Switch1 has no matching entry in flow table -> forwards to controller. If controller accepts, computes path and updates all switches along path. Forwarding: Controller sends packet back to switch1, which forwards it and adds new entry in table to allow subsequent packets from this flow without asking the controller. When the network starts, the Switches must connect to and authenticate with the Controller On startup, the network creates a minimum spanning tree with the Controller advertising itself as the root. Each Switch has been configured with the Controller s credentials If a Switch finds a shorter path to the Controller, it attempts two way authentication with it before advertising that path as a valid route

10 Establishing Topology Example Deployment Switches generate neighbor lists during MST algorithm Send encrypted neighbor-list to DC DC aggregates to full topology Note: no switch knows full topology Ethane Switch A wired Ethane Switch is like a simplified Ethernet switch. Does not need to learn addresses, support VLANs, check for source-address spoofing, or keep flow-level statistics, maintain forwarding tables, run routing protocols such as OSPF, ISIS, and RIP. The flow table can be several orders-of-magnitude smaller than the forwarding table in an equivalent Ethernet switch Flow Table and Flow Entries Contain a Header (to match packets against), an Action (to tell the switch what to do with the packet), and Per-Flow Data. Local Switch Manager Monitor link status Establish and maintain the secure channel to the Controller Two ways a Switch can talk to the Controller. Within the same broadcast domain Using our modified Minimum Spanning Tree Secure channel stretching through these intermediate Switches all the way to the Controller. Not within the same broadcast domain IP tunnel DC Fault Tolerance and Scalability Cold standby: secondary controllers participate in same global MST. After primary controller goes down, will take over when MST converges. Simple, but slow recovery: hosts/users have to re-authenticate. Warm standby: a separate MST for each secondary controller. Controllers monitor one another s liveness. Bindings are replicated across controllers. Complex, but faster recovery. Multiple active controllers: Switches need to authenticate with only one controller. Spread flow decision queries across multiple controllers. Complex consistency issues etc. Multicast and broadcast traffic In theory: Switch: keeps for each flow a bitmap of ports to forward. Controller: from computed broad/multicast tree, assigns appropriate bits during path setup. Broadcast are mostly discovery protocols, e.g. ARP, which the controller can reply without creating a new flow or broadcasting. In practice: ARP causes a significant load on the controller. Might setup a dedicated ARP server, and controller directs ARP traffic there. But what about other disc protocols? Tradeoff: controller implements common protocols, and broadcasts unknown ones with rate-limit. Doesn t scale well, but expecting discovery protocols to go away if Ethane is used widely

11 Network Policy Network policy is a set of rules: [<condition(s)>]:action; Conditions: conjunction of predicates. Actions: allow, deny, waypoints, and outbound-only. Rules are independent. Rule lookups have to be fast. Can t simply compile because of huge namespace of users, hosts, etc. So use compilation plus just-intime creation of search functions. Deployment 11 wired and 8 wireless Switches Wireless access points using WRTSL54GS. 4-port gigabit switches using FPGA. 4-port gigabit switches using desktop PCs. ~300 hosts, several hundred users Create a network policy that matches existing firewall configurations, NATs, router ACLs etc. The whole network is managed by single PC-based controller. Hosts connected to an Ethane switch port does not require user authentication Primary performance questions Evaluation: controller scalability How many Controllers are needed for a network of a given size? How big does the flow table need to be in the Switch? A 22,000-host network observed max 9,000 flow requests per second, suggesting that a single controller can handle 20,000 hosts with flow request setup time under 1.5ms Effect of controller failures Cold-standby failure recovery Effect of link failures Failures were simulated by physically unplugging a link The path reconverges in under 40ms, but a packet could be delayed up to a second while the Controller handles the flurry of requests

12 Ethane Higher-level policies Defined on principals, not network identifiers Language for specifying policies Policy should dictate the paths Controller should select paths based on policy Fine-grain control Controller handles first packet of a flow Data plane maintains per-flow state Practical Challenges Scalability Decision elements responsible for many routers Response time Delays between decision elements and routers Reliability Surviving failures of decision elements and routers Consistency Ensuring multiple decision elements behave consistently Security Network vulnerable to attacks on decision elements Interoperability Legacy routers and neighboring domains The Genesis of NOX NOX - A Network Operating System for OpenFlow Simple, Open Data-Plane API Prioritized list of rules Pattern: match packet header bits Actions: drop, forward, modify, send to controller Priority: disambiguate overlapping patterns Counters: #bytes and #packets 1. src=1.2.*.*, dest=3.4.5.* drop 2. src = *.*.*.*, dest=3.4.*.* forward(2) 3. src= , dest=*.*.*.* send to controller The History of Network Virtualization Reference A clean slate 4D approach to network control and management. Albert Greenberg, Gisli Hjalmtysson, David A. Maltz, Andy Myers, Jennifer Rexford, Geoffrey Xie, Hong Yan, Jibin Zhan, and Hui Zhang SIGCOMM Comput. Commun. Rev. 35, 5 (October 2005), Ethane: taking control of the enterprise. Martin Casado, Michael J. Freedman, Justin Pettit, Jianying Luo, Nick McKeown, and Scott Shenker In Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications (SIGCOMM '07). ACM, New York, NY, USA, (Optional)The Past, Present, and Future of Software Defined Networking. Nick Feamster, Jennifer Rexford, and Ellen Zegura