Lecture 10.1 A real SDN implementation: the Google B4 case. Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it

Transcription

1 Lecture 10.1 A real SDN implementation: the Google B4 case Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it

2 WAN WAN = Wide Area Network WAN features: Very expensive (specialized high-end routers and links with high capacity) High availability How to face link/router failures? Overprovisioning: links utilization about 30-40% Reliability at very high cost (3 times required bandwidth)

3 Google WAN: B4 B4 is the Google WAN interconnecting multiple data centers: 12 data centers Central control of applications, servers and network Applications: search, video, cloud computing, enterprise S. Jain et al., B4: experience with a globally-deployed software defined wan, SIGCOMM '13, New York (USA), 2013

4 The Google WANs Two distinct WANs*: User-facing network Data Centers interconnection network (B4) Why two distinct networks? Different requirements User-facing network: Connection with different providers must support many protocols Dense topology hard to manage in a centralized manner Communication with end-users high availability level * Networks are connected each other

5 Applications running on B4 User data copies , documents, audio/video For availability/durability Remote storage access Computation over distributed data sources Large-scale data Synchronization among data centers Volume Latency sensitivity Priority

6 The Google B4 WANs (1/2) B4 is an SDN network using OpenFlow Why an SDN network? Traditional WAN architectures could not achieve the scalability, the cost efficiency and the granularity of control required commodity hardware and network software can do it! SDN provides a separation among data and control planes: Software-based control plane running on commodity hardware with a global view Programmable data-plane upgrade servers independently than switches

7 The Google B4 WANs (2/2) Motivations: Quick implementation of new customized protocols Simplified test environment (in local) Improved capacity planning Simplified management (central view of the network)

8 The SDN architecture Logically composed of three layers Hardware Controller Global

9 The hardware layer Forwarding of traffic based on Open Flow Switches realized by means of multiple switch chips with an embedded processor running Linux An user-level process running an extended version of Open Flow (Open Flow Agent OFA): to configure the hardware

10 The controller layer (1/2) Network control Servers (NCSs) running OpenFlow Controller (OFC) Network Control Applications (NCA) (TE Agent and RAP) OFC maintains the network state based on NCA output For each site there are several OFC controllers (for fault tolerance) Each OF switch actively communicate with one OFC The OFC software used is a modified version of ONIX Backward compatibility = integration with existing protocols Support for IP routing protocols (BGP/IS-IS) Quagga open source software is used

11 The controller layer (2/2) Routing Application Proxy (RAP) to provide connectivity between Quagga and OF switches Routing protocol control packets (from switches to Quagga) Routing protocol entries updates (from Quagga to the switches) Interface status updates (from the switches to Quagga)

12 Perform the Traffic Engineering (TE) algorithm The global layer Max-Min fair allocation among different applications using the B4 network TE algorithm input The Network Topology Flow Group (FG): applications aggregated on the basis of {source, dest, QoS} TE algorithm output: Tunnels (T) (IP in IP encapsulation): sequence of sites Tunnel Groups (TG): maps FGs to a set of tunnels (with weights) A fast algorithm with performance very close to LP solution

13 Encapsulation A Tunnel Group for FG { , } with two tunnels

14 Main results Utilization close to 100% (in old WAN is about 30%- 40%) Traffic differentiation effectiveness

15 Lecture 10.2 Network Function Virtualization (NFV) Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it

16 Network Function Virtualization (NFV) Definition: an initiative to virtualize the network services that are now being carried out by proprietary dedicated hardware Same motivation of SDN! A network service is referred to as a Virtual Network Function (VNF): Router, Evolved Packet Core (EPC), Firewall, Intrusion Detection System (IDS)

17 NFV NFV aims at reducing OpEx by implementing network functions as virtual machine running on general purpose hardware NFV exploits advantages of virtualization technique for: Flexible provisioning, Mobility (migration), automation, hardware independence, pay-per-use, fault-tolerance,

18 Service Function Chaining Service Function Chaining (SFC): Multiple Virtual Functions (VF) linked together to provide a service ISP IET proposed NSH (Network Service Header) encapsulation mechanism: each NF has its own ID and the SFC is defined as a list of IDs to be crossed

19 NFV and SDN NFV and SDN are independent and complementary. NFV already implemented and tested by ISP (NSH): SDN is a network solution to realize NFV Service Chaining

20 Lecture 10.3 SDN for Big Data and Big Data for SDN Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it

21 SDN and Big Data

22 SDN and Big Data (1/2) Big Data: 5Vs (volume, variety, velocity, value, veracity) Big data applications: large volume and computing complexity Requires the underlying support of networking SDN advantages: much easier to develop and deploy advanced network applications global view of the network

23 SDN and Big Data (2/2) SDN for Big Data: dynamic allocation of resources to different big data applications: better performance Service Level Agreements (SLA) Big Data for SDN: controller with a global view of the network, network related data from the devices (all layers): Big data analytics Improve network operation (Traffic Engineering, Security)

24 SDN for Big Data (1/2) Scenario: Cloud Data Center (SDN) Big Data Application (resource requirements change dynamically) Service Level Agreement (SLA) of a big data application: agreement among the big data app provider and the user Quality of Service (QoS), penalties QoS parameters: processing time, amount of data processed, rate of failure, security

25 SDN for Big Data (2/2) In [1] different SLAs (for different apps) are implemented by the SDN controller: A bandwidth provisioning table is maintained at controller level The controller notifies the different SLAs to SDN switches An SDN switch implements a packet scheduling policy based on the bandwidth provisioning [1] W. Hong, K. Wang and Y. H. Hsu, "Application-Aware Resource Allocation for SDN-based Cloud Datacenters," International Conference on Cloud Computing and Big Data, 2013

26 MapReduce In [2] SDN is used to improve the performance of Hadoop. MapReduce is a programming model and an associated implementation (Hadoop) for processing big data sets with a parallel, distributed algorithm on a cluster Idea: moving the computation and not the data A job is split into tasks. Each task is executed on the basis of the MapReduce blocks: Map: from data to (key,values) Shuffle: Map results sent to Reducers Reduce: the same key are processed [2] P. Qin, B. Dai, B. Huang, G. Xu, Bandwidth-aware scheduling with sdn in hadoop: A new trend for big data, IEEE Systems Journal, 2015

27 MapReduce: Words count (2/2)

28 SDN for Big Data (2/2) Hadoop performance: minimization of the job completion time (Minimum make span problem NP complete) In [2] an SDN-based scheduler is proposed: the SDN controller has a global view of the network It manages the bandwidth and allocates it on a time slot manner It performs task assignment (locally or remotely) based on the completion time

29 Big Data for SDN Definition of ad-hoc operational and management solutions for an SDN network based on Big Data applications Many areas: Traffic Engineering Security Network monitoring - many others

30 Big Data for SDN: Traffic Engineering Traffic Engineering (TE): routing solution to optimize (network load balancing, network utilization maximization) the performance of the network TE with legacy solutions in a data center is complex: Thousand of hosts Different requirements Routing based on IP destination field (or MAC destination)

31 Big Data for SDN: Traffic Engineering The SDN controller obtain traffic and failure information from the switches It is possible to define routing policies for arbitrary granularity flows (multiple fields for matching in SDN Flow Tables) TE can be implementing dynamically inserting/modifying Flow Table in switches

32 Traffic Matrix computation with Hadoop Research work by ISP should know the Traffic Matrix (TM): amount of traffic from a each incoming node to each outgoing node

33 Traffic Matrix computation with Hadoop The ISP knows links load and routing matrix R x TM = Y R= routing matrix (MxN 2 ) Y= links load vector (M) TM= vector representation (N 2 ) Number of equations << number of equations highly underdetermined system It is complex to extract ingress/egress traffic relationships

34 Traffic Matrix computation with Hadoop Idea: collect traffic at each node and work on text files Each text file is composed of: A row for each captured packet Each row has three values: IP source, IP dest, length (byte) Proposed solution: Each node process its local captured files Each node send to the other nodes the list of IP addresses (Population set) reachable by itself. Local process is an Hadoop execution Main features: distributed, data exchange is highly reduced, no need of dedicated control protocols.