Load Sharing in Networking Systems

Transcription

1 Load Sharing in Networking Systems Lukas Kencl Intel Research Cambridge MFF UK Praha,, November 2005 (With lots of help from Weiguang Shi, University of Alberta, and citations from D. Thaler, R. Ravishankar and K. Ross)

2 Outline Intel Research Cambridge Intel Research Cambridge Load sharing in networking systems Problem statement Motivation Imbalance due to traffic properties. Break. Inspiration: distributed Web caching Solutions and properties: Adaptive HRW hashing Adaptive Burst Shifting. Break. Seminar: Q&A, Exercises and Homeworks IR Movies!!! 2

3 Intel Research Cambridge 3

4 Intel R&D Commitment Intel Research: 4 Lablets,, target 80 people worldwide Intel Research Cambridge California Washington Oregon Massachusetts Illinois Pennsylvania New Mexico Swindon Cambridge Shannon Glasgow Barcelona Copenhagen Stockholm Nice Gdansk Braunschweig, Ulm Nizhny Novgorod Sarov Beijing Shanghai China Arizona Haifa Delhi Bangalore Mumbai India Tsukuba Tokyo Over 75 labs & 7000 R&D professionals worldwide 4

5 Intel Research Mission Build the technical leadership, knowledge assets and systems perspective to make Intel a preeminent driver of disruptive information technologies David Tennenhouse Vice President, Corporate Technology Group Director of Research, Intel Corporation 5

6 What is unique about Intel Research? Research is largely exploratory Off roadmap, years out on timeline Innovative collaborative model engaging with key Universities Lablets (Berkeley, Seattle, Pittsburgh, Cambridge) Intel employee facilities co-located on University campus to facilitate collaboration Open Collaborative Agreements signed with Universities Strategic Research Projects (SRP) inside Intel Technology transfer mechanism from open collaborative research Proprietary exploratory research outside scope of any business unitu Alignment of University engagements Research Council, Academic Relations, Labs, Visiting Faculty, Internships!!! 6

7 Intel Research Cambridge (IRC) Established in March 2003 Currently 12 full-time time researchers Visiting faculty, student interns Make sure the smartest work with us: cooperation with Cambridge University, as well as others throughout UK, Europe and elsewhere 7

8 IRC Key Research Areas Optical Packet Switching Virtualisation Xen Para-virtualisation - run modified or new OS on specialised guest architecture Wired Networking CoMo Continuous Monitoring Adaptive methods Anticipate the wireless WIP: from internet to Intairnet wireless access network using multidirectional antennas HAGGLE: mobile ad-hoc networks Ubiquitous computing 8

9 Load Sharing in Networking Systems Problem Statement 9

10 Map packets to processors Incoming Packets Multiple (N) Inputs Multiple (M) Processors Assumptions: Data arrives in packets. Any processor can process any packet. Heterogenous processor capacity μ j. N M Task: Map packets to processors so that load within some measure of balance. 10

11 Packet flows: avoid remapping or out-of-sequence!!! Incoming Packets Multiple (N) Inputs Multiple (M) Processors Assumptions: Data arrives in packetized flows. Any processor can process any packet. Heterogenous processor capacity μ j. N M Task: Load on processors within some measure of balance. Same flow to same processor (reordering, context). Advantage: system optimization. Drawback: complexity, overhead. 11

12 Reduce state maintenance of Flow-to to-processor Mapping Incoming Packet Multiple (N) Inputs 1 Multiple (M) Processors 1 v 2 f ( v ) = flow identifier vector v 4 3 N M Upon packet arrival, a decision is made where to process the packet, based on the flow identifier. A flow-to to-processor mapping f is thus established. 12

13 Acceptable load sharing as a measure of balance Processing load on processor j j (t) Capacity on processor j Workload intensity on processor j j (t) = j (t) / j Total system workload intensity (t) = Σ j (t) / Σ j j Acceptable load sharing: if (t) [ 1 then j, j (t) [ 1, if (t) > 1 then j, j (t) > 1. No single processor is overutilized if the system in total is not overutilized, and vice versa. 13

14 Coefficient of Variation (CV) as a measure of balance Useful, as it takes the scale of measurements out of consideration. 14

15 Minimizing Disruption Possible Goal: Acceptable load sharing without maintaining flow state information and yet minimizing the probability of mapping disruption (flow remapping or reordering). Special Integer Programming Problem: max Σ v Δ v (t) ). Σ j (1{ f(t-δ t)(v) ) = j}. 1{ f(t)(v)= )= j}), while Σ v a v (t) ). 1{ f(t)(v)= )= j} = j (t) [ j, j. v - flow identifier vector in the packet header, f (t) (v) - function mapping flows to processors, changing over time, Δ v (t)c {0,1} - indicator if v has appeared in the intervals (t-2δ t, t t-δ t) and (t-δ t, t), a v (t) - how many times has v appeared in the interval (t-δ t, t), t - iteration interval. We suspect an NP-complete problem use heuristics. 15

16 Summarize Balance load Avoid remapping - or packets out-of of-sequence Minimize overhead 16

17 Load Sharing in Networking Systems Motivation 17

18 Practical Examples Distributed Router, or Parallel Forwarding Engine Server Farm Load Balancer Network Processor Multiple (M) Servers S 1 S 2 Incoming Packet Packet-toserver mapping computation Packet forwarding S M Server Farm Load Balancer 18

19 Network Processor NP Network processor: Pool of multi-threaded threaded forwarding engines Integrated or attached GPP Memories of varying bandwidth and capacity Large parallel programming problem Concurrency Packet order Resource allocation Load balancing At 10 Gbps,, packets arrive every 36 ns!!! Packets SRAM GPP DRAM Pool of Microengines (ME) 19

20 Load Sharing in Networking Systems Traffic properties make balancing difficult 20

21 Traffic Properties Address distribution 32bit address space very unevenly assigned and populated Burstiness Power law 21

22 TCP Traffic is Bursty TCP behaviour: ideal and typical Packets arrive in periodic bursts! 22

23 Networking Traffic Exhibits Power-Law (Zipf-law) Properties P(R)~1/R a Flow popularities in traffic traces Frequency of an event as a function of its rank obeys powerlaw. Popularity of network flows with a close to 1 (from above). 23

24 Power-law leads to imbalance if static mapping (Shi et al, 2004) m - number of processors, K - number of distinct object addresses or flow IDs, p i (0<i<K) - popularity of object i, q j (0<j<m) - number of distinct adresses or flow IDs mapped to processor j 24

25 Conclusion Part I Complex problem Traffic properties work against static solutions 25

26 Break 5 minutes 26

27 Load Sharing in Networking Systems Inspiration: Distributed Web Caching and Web Server Load Balancing 27

28 What is a Distributed Web Cache Internet Intranet Proxy Cache stores Web objects locally Collection of distributed Web caches 28

29 HRW Mapping Intel Research Cambridge Highest Random Weight (HRW) Mapping, Thaler, Ravishankar,, 1997, Ross, 1998,, CARP Protocol, Windows NT LB Def.: Object-to to-processor mapping function f, f (v): V d M : f (v)( ) = j w x j. g ( v, j ) = max k x k. g (v, k), where v is the object identifier vector, x = (x 1,..., x m ) is a weights' vector and g (v,(, j)c (0,1) is a pseudorandom function of uniform distribution. Minimal disruption of mapping in case of processor addition or removal/failure. Load balancing over heterogenous processors: : weights vector x is in a 1-to1 to-1 correspondence to p = (p 1,..., p m ), the vector of object fractions received at each processor. Pseudorandom function g (v,( j)c (0,1) can be implemented as a fast-computable hash function. Example: 3 processors x 3. g (v, 3) x 3 Map to max of: x 2 x 1. g (v, 2). g (v, 1) x 2 x

30 HRW Minimal Disruption Minimal disruption of mapping in case of processor addition: Partitioning into contiguous set HRW Example: Add processor No. 4, vectors mapped either (i) as before addition or (ii) to the newly added processor minimal number of vectors change mapping. Max of: 1 Max of: 1 g (v, 2) g (v, 4) g (v, 3) g (v, 1) g (v, 4) g (v, 2) g (v, 3) g (v, 1) 30

31 HRW Load Balancing Intel Research Cambridge m - number of servers, S 1,, S m K - number of distinct object addresses or flow IDs, from the set of objects O p i (0<i<K) - popularity of object i, q j (0<j<m) - number of distinct adresses or flow IDs mapped to processor j, e.g. sum of the popularities of objects mapped to S j 31

32 Load Sharing in Networking Systems Adaptive Methods - Adaptive HRW Hashing - Adaptive Burst Shifting 32

33 Adaptive HRW Hashing (Kencl, Le Boudec 2002) 33

34 Adaptive HRW mapping 34

35 Adaptation through Feedback Problem: incoming requests are packets, not flows! Packets not evenly distributed over flows -> not evenly distributed over the request object space -> > HRW mapping not sufficient for acceptable load sharing bounds > > need to adapt! 3. Compute new x (t) = (x 1 (t),,..., x m (t)). ). 4. Download new x := x(t). x Multiple (N) Input Cards 4 N Control Point CP 2. Evaluate (t) = ( ( 1 (t), 2 (t),,..., m (t)) (compare threshold). 1. Filtered workload intensity ( j) = j (t) M Multiple (M) processors Trigger definition targets preventing overload, if system in total not overloaded, and vice versa. A threshold triggers adaptation when close to load sharing bounds. Flow-to to-processor mapping f becomes a function of time f(t)(v) Adaptation may cause flow remapping! How to minimize the amount remapped? 35

36 Adaptation Algorithm Start Wait time t Compute filtered processor workload intensity (t) No Trigger Adaptation? Yes Adapt weights' vector x and upload Triggering Policy Adaptation Policy 36

37 Triggering Policy Intel Research Cambridge ρ(t), system in total workload intensity threshold ρ(t), system in total triggering threshold ρ(t), system in total max, min hysteresis bound Dynamic workload intensity threshold ' (t) = 1/2 (1+ (t)) (t)) Example: (t) = (0.8, 0.2, 0.2), (t) = 0.4 e (t) = 0.7, 1 (t) > (t) e adapt. Triggering threshold (t) = max( ' (t), upper) or vice versa Hysteresis bound upper: (1+ H (t)). (t) lower: (1 - H (t)). (t) Triggering policy (i) if (t) [ 1 and max j (t) > (t) then adapt; (ii) if (t) ) > 1 and min j (t) < (t) then adapt. 37

38 Adaptation Policy: Minimal Disruption A, B - mutually exclusive subsets of M={1,...,m}, M=A 4 B. α c (0, 1). f, f' two HRW mappings with the weights' vectors x, x': x' j = α. x j, j c A, x' j = x j, j c B. p j, p' j - fraction of objects mapped to node j using f, f'. Intel Research Cambridge p' j [ p j, j c A, 1) p' p' j m p j, j c B. 2) Fraction of objects mapped to a different node by each mapping is MINIMAL, that is, equal to p' j - p j. V at every node j. 38

39 Adaptation Policy: Minimal Disruption Example (3 proc.) Intel Research Cambridge x 3 x 3. g (v, 3) x 2 x Max of: x 2 x 1. g (v, 2). g (v, 1) x 3 x 3. g (v, 3) 2/3.x 2 Max of: 2/3. x 2. g (v, 2) 2/3.x 1 2/3. x 1. g (v, 1) reduced receives less, unaltered receives more, if reduction by a single, invariable multiplier. minimal disruption of the mapping. 39

40 Adaptation Policy Let (t) [ 1. Then: x j (t) : = c(t). x j (t- t) t), if j (t) > (t) ( j exceeds threshold (t)), x j (t) : = x j (t- t), t), if j (t) [ (t) ( j does not exceed threshold (t)). If (t) > 1, 1 the adaptation is carried out in a symmetrical manner. The weights' multiplier coefficient c(t) : c(t) = (t) ( ) min { j (t) j (t) > (t)} Factor c(t) is proportional to the minimal error and to the number of nodes. 1/m 40

41 Adaptive Burst Shifting (Shi, MacGregor, Gburzynski 2005) 41

42 Adaptive Burst Shifting Insight: the periodicity of bursts may allow shifting flows in-between bursts, without affecting order within flows. 42

43 Adaptive Burst Shifting Burst Distributor Algorithm 43

44 Performance evaluation 44

45 Adaptive HRW on generated traffic 45

46 Expectations Workload intensity on individual processors close to that of the system in total (acceptable load sharing); Packet loss probability lowered (acceptable load sharing); Persistent flows (appearing in two consecutive iterations) seldom remapped (minimize disruption). 46

47 AHH Keeps Per-processor Workload Intensity Close to Ideal 2.5 ρ(t), system in total 2.5 ρ(t), system in total 2 2 Workload intensity Workload intensity Time Naive, no LS Time Adaptive HRW ρ(t), system in total ρ(t), system in total Max, min ρj(t), no LS Max, min ρj(t), static LS Max, min ρj(t), adaptive LS Workload intensity Workload intensity Time Time Static HRW Max and min of all. 47

48 Packet Loss Significantly Reduced with the Adaptive Control Loop Packets droppeds x 10 4 System in total No LS Static LS Adaptive LS Packets dropped 3.5 x Static LS Adaptive LS Time Packet loss: Naive, Static, Adaptive, Ideal Time Packet loss in excess of Ideal: Static, Adaptive. Adaptive HRW saves on average 60% of packets dropped in by the static load sharing. 48

49 Minimal Disruption Property Ensures Few Flow Remappings Flows Flows appearing Flows persistent Flows remapped Time Flows, per iteration: appearing, persitent and remapped. Adaptive control loop leads on average to: less than 0.05% of the appearing flows remapped per iteration; less than 0.2% of the persistent flows remapped per iteration. 49

50 AHH and ABS on generated traffic 50

51 Adaptive HRW and Intel Research Cambridge Adaptive Burst Switching Packet drop rates. Packet reordering. Packet remapping. 51

52 Conclusion ABS better in preventing packet drop AHH better in preventing remapping and reordering, although larger table would improve ABS ABS works on much smaller time-scale can address packet bursts AHH converges to optimal allocation, but timescale too long for bursts Best solution probably hybrid under investigation must be careful to avoid conflicting feedback-control control mechanisms 52

53 The End Thank you! 10:40 53

54 Seminar 54

55 Seminar agenda Q & A Exercises 1, 2, 3 IRC Homework Q & A II. Cinema? 55

56 Exercise 1: Mapping Disruption Let there be a system with M servers of the same capacity, and an (M+1)th server be added to the system. Compute the amount of mapping disruption with the: a) Contiguous mapping b) Mod M mapping 56

57 Exercise 2: Prove the x2p relationship of HRW mapping 57

58 Homework: 1. Prove the mindisruption property of Adaptive HRW hashing 2. Try and design your own adaptive load balancing mapping that a) reduces packet reordering within flows, or b) reduces flow remapping. 58

59 Q & A 59

60 Internships? Films? Thank you! research.net/cambridge/ 60