Scalable Constraint-based Virtual Data Center Allocation

Size: px

Start display at page:

Download "Scalable Constraint-based Virtual Data Center Allocation"

Benedict Kennedy
5 years ago
Views:

1 Scalable Constraint-based Virtual Data Center Allocation Sam Bayless Nodir Kodirov Ivan Beschastnikh Holger H. Hoos Alan J. Hu Computer Science University of British Columbia

2 Data centers, data centers, for all

3 Data centers, data centers, for all Flexible abstraction: Provider

4 Data centers, data centers, for all Flexible abstraction: Provider get() ip Customer

5 Data centers, data centers, for all Cisco: traffic flowing through data centers will triple between 014 and 019 (reaching 10.4 ZB/year) Wide variety of applications being hosted [Benson et al. IMC 10, Kanev et al. ISCA 15]

6 More capacity + capabilities lead to more complex workloads Complex workload examples Allocate a web-server, cache, database in a particular topology and with enough bandwidth to satisfy a certain QoS Deploy a distributed compute task in which some nodes communicate a lot, and others rarely Allocate a chain of NFV elements some of which require special hardware (GPUs)

7 allocation Customer

8 allocation Customer

9 allocation ToR Customer

10 allocation ToR Customer

11 allocation Aggregation Switch ToR ToR Customer

12 allocation: multi-tenancy Customer Customer Customer

13 allocation Observation 1: get()-style API is inappropriate Customer Customer Customer

14 allocation Observation 1: get()-style API is inappropriate Observation : It is more effective to allocate virtual data centers (VDCs), than virtual machines (s) Customer Customer Customer

15 As DCs evolve, so must the programming models and allocation mechanisms Allocation one at a time: get() Sub-optimal for the provider and the customer More info about an allocation: helps the provider plan and to effectively pack the data center Customers benefit since they get the properties that they ultimately need

As DCs evolve, so do the programming models and

Sub-optimal for the provider and the customer More info

effectively pack the data center Customers benefit since

16 As DCs evolve, so do the programming models and allocation mechanisms Allocation one at a time Sub-optimal for the provider and the customer More info about an allocation: helps the provider plan and to effectively pack the data center Customers benefit since they get the properties that they ultimately need AWS Lambda Amazon EMR (Hadoop)

17 In this talk Introduce virtual data center (VDC) allocation Discuss prior work (there is lots of it, mostly in the networking community) Describe NetSolver: our approach to solve VDC allocation (based on MonoSAT SMT-solver) Show how NetSolver compares to other approaches

18 What s the problem? Multi-path VDC allocation Input 1: a (directed/undirected) physical DC topology (DC) with edge capacities/latencies and per-host constraints (disk/memory/cpu/gpu/etc) Input : a virtual data center (VDC) that describe connectivity graph between s, and connectivity/ requirements Output: assignment of s to hosts, and virtual edges to physical paths (possibly multi-path) s.t. all constraints (end-to-end bandw, and ) are satisfied and respect DC

19 What s the problem? AggSw1 AggSw Physical DC topology: ToR ToR core [ 4 GB ] core [ 4 GB ] core [ 4 GB ]

20 What s the problem? AggSw1 AggSw Physical DC topology: ToR ToR core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

21 What s the problem? AggSw1 AggSw Physical DC topology: ToR ToR core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Output: Mapping Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

22 Example NetSolver solution AggSw1 AggSw Physical DC topology: ToR ToR core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Mapping 1: -> host Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

23 Example NetSolver solution AggSw1 AggSw Physical DC topology: ToR ToR Mapping : - edges -> paths core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Mapping 1: -> host Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

24 Example NetSolver solution AggSw1 AggSw Physical DC topology: ToR ToR Mapping : - edges -> paths core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Mapping 1: -> host Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

25 Example NetSolver solution AggSw1 AggSw Physical DC topology: ToR ToR Mapping : - edges -> paths core [ 4 GB ] core [ 4 GB ] core [ 4 GB ] Mapping 1: -> host Virtual data center (VDC): s1 m 1 core [ GB ] s 1 core [ GB ] 1 core 1 core [ GB ] GB s3 [ ]

26 Related work dimensions Sound: respect end-to-end bandw. guarantees VDC topology: Star/Hose/All DC topology: Tree/All Complete: finds a solution if a solution exists Multi-: can map more than one to a host Multi-path: supports multi-path allocations

27 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

28 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

29 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

30 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

31 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

32 Related work break-down 9: Guo et al. CoNEXT 10 48: Tantawi, MASCOTS 1 8: Ballani et al. CCR 11 54: Zhani et al. INM 13 43: Rost et al. CCR 15 50: Yu et al. SIGCOMM 08 15: Cheng et al. CCR 11 31: Huang et al., ICC : Chowdhury et al, INFOCOM 09 36: Lischika et al., VISA 09 51: Yuan, FMCAD 13

33 MonoSAT background Switch to other slide-deck

34 The MonoSAT constraint solver MonoSAT is an SMT solver for monotonic theories. MonoSAT supports: Graph constraints (shortest paths, maximum flows... ) Finite state machines & string acceptance Temporal logic (CTL) synthesis D polygonal geometry constraints Bounded integer & cardinality constraints Propositional logic (Boolean satisfiability) Has C++, Python, and Java bindings August 14, / 7

35 Finite Monotonic Predicates Apredicatep is positive monotonic i : : p(...,x,...), x apple y =) p(...,y,...) Apredicatep is negative monotonic i : : p(...,x,...), x apple y =) p(...,y,...) August 14, / 7

36 Monotonic theories Many useful predicates are monotonic: Graph Predicates: I Reachability I Shortest paths I Maximum s t flow I Minimum Spanning Tree I Acyclicity August 14, / 7

37 Monotonic predicates Reachability is monotonic with respect to edges: August 14, / 7

38 Monotonic predicates Reachability is monotonic with respect to edges: August 14, / 7

39 Monotonic predicates Reachability is monotonic with respect to edges: August 14, / 7

40 Monotonic predicates Reachability is monotonic with respect to edges: August 14, / 7

41 Graph constraints in MonoSAT MonoSAT supports constraints over one or more finite graphs: Combines arbitrary Boolean constraints with high performance graph constraints. Supported graph constraints I Reachability I Shortest paths I Maximum s-t flow I Minimum spanning tree I Acyclicity Graphs can be directed Edges can have bit vector weights/capacities Scales to 100,000s of nodes and edges August 14, / 7

42 Graph constraints in MonoSAT s_0 a b c s_1 d e s_3 s_ ( a _ b) ^ ( d _ e) ^ reaches(s 0, s 3 ) ^ reaches(s 1, s 3 ) Figure : A directed graph with edge inclusion controlled by Booleans {a, b, c, d, e}, and a formula constraining the graph. August 14, / 7

43 Graph constraints in MonoSAT s_0 a b c s_1 d e s_3 s_ ( a _ b) ^ ( d _ e) ^ reaches(s 0, s 3 ) ^ reaches(s 1, s 3 ) Figure : A directed graph with edge inclusion controlled by Booleans {a, b, c, d, e}, and a formula constraining the graph. s_0 b s_1 e s_3 s_0 a c s_1 e s_3 s_ s_ Figure : Satisfying (left) and unsatisfying (right) solutions. August 14, / 7

44 Weighted graph constraints in MonoSAT s_0 x s_1 z y s_ (x > 1)^(x < y)^(y < 4)^(z = y)^(shortestpath(s 0, s ) apple 3) Figure : A directed graph with variable edge weights, and a formula constraining those weights. August 14, / 7

45 Weighted graph constraints in MonoSAT s_0 x s_1 z y s_ (x > 1)^(x < y)^(y < 4)^(z = y)^(shortestpath(s 0, s ) apple 3) Figure : A directed graph with variable edge weights, and a formula constraining those weights. s_0 s_1 3 3 s_ s_0 1 s_1 s_ Figure : Satisfying (left) and unsatisfying (right) solutions. August 14, / 7

46 Maximum-flow graph constraints in MonoSAT s_0 x y w s_1 v z s_3 s_ (x apple z apple )^(x > y)^(z > v)^( apple maximumflow(s 0, s ) apple 3) Figure : A directed graph with variable edge weights, and a formula constraining those weights. August 14, / 7

47 Maximum-flow graph constraints in MonoSAT s_0 x y w s_1 v z s_3 s_ (x apple z apple )^(x > y)^(z > v)^( apple maximumflow(s 0, s ) apple 3) Figure : A directed graph with variable edge weights, and a formula constraining those weights. s_0 0 s_1 0 s_3 s_0 1 s_1 1 s_3 s_ s_ Figure : Two satisfying solutions. August 14, / 7

48 Combined graph constraints in MonoSAT s_0 a,x s_1 d,v b,y c,w e,z s_3 s_ reaches(s 0, s ) ^ shortestpath(s, s 3 )=maximumflow(s 0, s 3 ) Figure : A graph with edge inclusion controlled by Booleans {a, b, c, d, e}, and edge weights {v, w, x, y, z}. August 14, / 7

49 Combined graph constraints in MonoSAT s_0 a,x s_1 d,v b,y c,w e,z s_3 s_ reaches(s 0, s ) ^ shortestpath(s, s 3 )=maximumflow(s 0, s 3 ) Figure : A graph with edge inclusion controlled by Booleans {a, b, c, d, e}, and edge weights {v, w, x, y, z}. s_0 a,1 s_1 s_ d,1 e,1 s_3 Figure : A satisfying solution. August 14, / 7

50 NetSolver design Basic idea: encode VDC allocation as a MonoSAT query. Either outputs a solution or one does not exist Global constraints: connectivity and bandwidth Local constraints: s respect host resources Challenge: efficiency (e.g., each - path can be modeled as a max-flow constraint, these are expensive)

51 Global constraints Assume: that we know the -host assignments Given: G =(V,E) c(u, v) (u, v) E Directed graph and integer constraints on each edge K i K, i =(s i,t i,d i ) d i s i V t i V commodity demands, representing demand between and Graph (DC): Demands (VDC): s1 m s s3

52 Global constraints Assume: that we know the to host assignment Given: G =(V,E) c(u, v) (u, v) E Directed graph and integer constraints on each edge K i K, i =(s i,t i,d i ) d i s i V t i V commodity demands, representing demand between and Integral multi-commodity flow problem: Find feasible flow such that each satisfied For each edge total flow of all capacities is d i (u, v) apple c(u, v)

53 Commodity flow encoding Create graphs G 1... K : one per demand with same topology as G For each edge create a new symbolic capacity Assert: that (u, v) i G i c(u, v) i apple c(u, v) X i c(u, v) i apple c(u, v) Assert: for each demand, i =(s i,t i,d i ) max-flow(s i,t i ) d Solver s task: find partitioning of capacities across K graphs while satisfying lower-bounds across all demands

54 Modeling local constraints G Construct a graph that is the VDC and one node for each For each v and each server s G, create directed symbolic edge e vs with unlimited capacity; e vs controls allocation of v to servers v e vs Assert: for each, exactly on enabled s Assert: for each server, set of s assigned to s obey server s local resources G (s i,t i,d i ) Assert: satisfies flow for each commodity constraint

55 Further technical innovations Naive encoding slow: V max-flow constraints in worst case. Optimize by merging demands from same source So far assumed that VDC topology constant: only works for allocating sequence of identical VDCs To allocate diverse VDCs, encode superset of VDCs and use MonoSAT s assumption mechanism to disable parts of this superset during allocation

56 NetSolver Evaluation Key questions: Can a sound+complete scale to realistic topologies? Are there any practical benefits to being complete? How does NetSolver compare to related work? SecondNet (CoNEXT 10) Z3-based abstraction refinement technique (FMCAD 13)

57 Identical VDC packing and median alloc runtime (Tree) 000 servers, 16 cores; varying VDC sizes; Tree DC topologies

58 Identical VDC packing and median alloc runtime (BCube/FatTree) 51 servers, 16 cores; varying VDC sizes; BCube DC topologies 43 servers, 16 cores; varying VDC sizes; FatTree DC topologies

59 NFV chain allocation Intrusion Prevention System 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD Video Firewall Cache Optimizer 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD CPU core 4 GPU core 8 GB RAM 0 TB SSD CPU core 0 GPU core 4 GB RAM 10 TB SSD Parental Control 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD

60 NFV chain allocation Intrusion Prevention System 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD Video Firewall Cache Optimizer 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD CPU core 4 GPU core 8 GB RAM 0 TB SSD CPU core 0 GPU core 4 GB RAM 10 TB SSD Parental Control 4 CPU core 0 GPU core 8 GB RAM 0 TB SSD 100 servers; commercial DC topology; increasing chain bandwidth constraint

61 Extensibility NetSolver supports a variety of additional constraints A B C A B C 0/4 0/4 0/4 3/6 1/6 0/6 3/6 1/6 0/6 VDC Affinity Affinity Possible allocations No-hotspot: B, C C, C C, B VDC Possible allocations Minimize utilized servers: A, B B, A A, A B, B VDC Affinity No hotspots Minimize utilized severs

62 AggSw1 AggSw Contributions Physical Data Center ToR1 ToR Virtual Data Center Developed NetSolver, a new VDC allocator NetSolver encodes problem into MonoSAT. Can be reused for other problems: NFV placement, data migration, task distribution, etc Improves DC capacity utilization by 300% over prior work (but slower than incomplete approaches) Constraints-based approach flexibly extends to other kinds of constraints, such as (anti-)affinity

VNF Chain Allocation and Management at Data Center Scale

VNF Chain Allocation and Management at Data Center Scale Internet Cloud Provider Tenants Nodir Kodirov, Sam Bayless, Fabian Ruffy, Ivan Beschastnikh, Holger Hoos, Alan Hu Network Functions (NF) are useful