INF5071 Performance in distributed systems Distribution Part II

Transcription

1 INF071 Performance in distributed systems Distribution Part II 27/

2 Type IV Distribution Systems Combine Types I, II or III Network of servers Server hierarchy Autonomous servers Cooperative servers Coordinated servers Proxy caches Not accurate Cache servers Keep copies on behalf of a remote server Proxy servers Perform actions on behalf of their clients INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

3 Type IV Distribution Systems Combine Types I, II or III Network of servers Server hierarchy Autonomous servers Cooperative servers Coordinated servers Proxy caches Not accurate Cache servers Keep copies on behalf of a remote server Proxy servers Perform actions on behalf of their clients INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

4 Type IV Distribution Systems Combine Types I, II or III Network of servers Server hierarchy Autonomous servers Cooperative servers Coordinated servers Proxy caches Not accurate Cache servers Keep copies on behalf of a remote server Proxy servers Perform actions on behalf of their clients INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

5 Type IV Distribution Systems Variations leaning Autonomous, coordinated possible In komssys Proxy prefix caching Coordinated, autonomous possible In Blue Coat (which was formerly Cacheflow, which was formerly Entera) Periodic multicasting with pre-storage Coordinated The theoretical optimum INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

6 leaning Webster s Dictionary: from Late Latin glennare, of Celtic origin to gather grain or other produce left by reapers to gather information or material bit by bit Combine patching with caching ideas non-conflicting benefits of caching and patching Caching reduce number of end-to-end transmissions distribute service access points no single point of failure true on-demand capabilities Patching shorten average streaming time per client true on-demand capabilities INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

7 leaning Combines Patching & Caching ideas Wide-area scalable Reduced server load Reduced network load Can support standard clients Join! Proxy cache multicast cyclic buffer Central server Unicast patch stream Proxy cache Unicast Unicast 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

8 Proxy Prefix Caching Split movie Prefix Suffix Operation Store prefix in prefix cache Coordination necessary! On demand Deliver prefix immediately Prefetch suffix from central server oal Reduce startup latency Hide bandwidth limitations, delay and/or jitter in backbone Reduce load in backbone Prefix cache Client Central server Unicast Unicast INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

9 MCache One of several Prefix Caching variations Combines Batching and Prefix Caching Can be optimized per movie server bandwidth network bandwidth cache space Uses multicast Needs non-standard clients Prefix cache Unicast Central server Batch (multicast) Prefix cache Unicast 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

10 Proxy Prefix Caching Basic version Practical No multicast Not optimized Aimed at large ISPs Wide-area scalable Reduced server load Reduced network load Can support standard clients Can partially hide jitter Optimized versions Theoretical Multicast Optimized Optimum is constantly unstable jitter and loss is experienced for each client! INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

11 Periodic Multicasting with Pre Storage Optimize storage and network Wide-area scalable Minimal server load achievable Reduced network load Can support standard clients Assumed start of the show Central server Specials Can optimize network load per subtree Negative Bad error behaviour 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

12 Periodic Multicasting with Pre Storage Optimize storage and network Wide-area scalable Minimal server load achievable Reduced network load Can support standard clients Central server Specials Can optimize network load per subtree Negative Bad error behaviour 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

13 Type IV Distribution Systems Autonomous servers Requires decision making on each proxy Some content must be discarded Caching strategies Coordinated servers Requires central decision making lobal optimization of the system Cooperative servers No quantitative research yet INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

14 Autonomous servers

15 Simulation Binary tree model allows Allows analytical comparison of Caching Patching leaning Considering optimal cache placement per movie basic server cost per-stream costs of storage, interface card, network link movie popularity according to Zipf distribution /1 /x ility b a b r o p e tiv la r e central server optional cache server network link INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

16 Simulation Example 00 different movies 220 concurrent active users basic server: $2000 interface cost: $100/stream network link cost: $30/stream storage cost: $1000/stream Analytical comparison demonstrates potential of the approach very simplified Caching λ-patching leaning Caching Unicast transmission No caching Client side buffer Multicast Caching Proxy client buffer Multicast 4664 Mio $ 37 Mio $ 276 Mio $ INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

17 Simulation Modeling User behaviour Movie popularity development Limited resources Hierarchical topology Individual user s Intention depends on user s time (model randomly) Selection depends on movies popularity Popularity development rity la u p o P Movie title age its H d e rv s e b O Movie title age INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

18 Caching Strategies FIFO: First-in-first-out Remove the oldest object in the cache in favor of new objects LRU: Least recently used strategy Maintain a list of objects Move to head of the list whenever accessed Remove the tail of the list in favor of new objects LFU: Least frequently used Maintain a list distance between last two requests per object Distance can be time or number of requests to other objects Sort list: shortest distance first Remove the tail of the list in favor of new objects INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

19 Caching Strategies Considerations limited uplink bandwidth quickly exhausted performance degrades immediately when working set is too large for storage space conditional overwrite strategies can be highly efficient ECT: Eternal, Conditional, Temporal LFU Forget object statistics when removed Cache all requested objects Log # requests or time between hits ECT Remember object statistics forever Compare requested object and replacement candidate Log times between hits INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

20 Simulation Movies 00 movies Zipf-distributed popularity 1. MBit/s 400 sec File size ~7.9 B Central server Limited uplink Cache Unlimited downlinks Clients INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

21 Effects of caching strategies on user hit rates Cache Hit Ratio for single server, 64 B cache 1 1 Cache Hit Ratio for single server, 96 B cache n nn n n n n n n n tio a R it H 0.7 ECT FIFO n LRU tio a R it H 0.7 better ECT FIFO n LRU Users Hit ratio dumb strategies (almost) do not profit from cache size increases intelligent strategies profit hugely from cache size increases strategies that use conditional overwrite outperform other strategies massively doesn t have to be ECT Users INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

22 Effects of caching strategies on throughput 1 MBit/s uplink usage for single server, 64 B cache 1 MBit/s uplink usage for single server, 96 B cache t u p h g u r o h T n n n n n n n n n n n n n n ECT FIFO n LRU better 0 n 0 nn Users t u p h g u r o h T n n n n n n n n n n n n n ECT FIFO n LRU 0 n Users Uplink usage profits from small cache increase greatly... if there is a strategy conditional overwrite reduces uplink usage INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

23 Effects of number of movies on uplink usage % t u p h g u ro h T ECT Cache uplink usage with 64 B, 1 MBit/s link 0 ; ; ; ; ; ; ; ; ; ; ; ; ; ; 000 users users ; Movies in system better % t u p h g u ro h T ECT Cache uplink usage with 64 B, 622 MBit/s link ; ; ; ; ; ; ; ; ; ; ; ; ; ; 000 users users ; In spite of 99% hit rates Increasing the number of users will congest the uplink Note scheduling techniques provide no savings on low-popularity movies identical to unicast scenario with minimally larger caches INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

24 Effects of number of movies on hit ratio ECT Cache hit ratio with 64 B, 1 MBit/s link ECT Cache hit ratio with 64 B, 622 MBit/s link 1 1 a tio R it H ; ; ; 000 users users ; ; ; ; ; ; ; ; a tio R it H ; ; ; ; ; ; ; ; ; ; ; 000 users users ; better ; ; ; Movies in system ; Movies in system Limited uplink bandwidth Prevents the exchange of titles with medium popularity Unproportional drop of efficiency for more users Strategy can not recognize medium popularity titles INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

25 Effects of user numbers on refusal probabilities 0.02 Refusal probability, 96 B cache, 1 Mbit/s uplink Caching strategy: ECT 0.02 Refusal probability, 64 B cache, 622 Mbit/s uplink Caching strategy: ECT Ref usal Probability Ref usal Probabilit y better Refusal in this simulation Network is admission controlled users have to wait for their turn Users wait up to minutes afterwards count one refusal Scenarios Left: cache ~12 movies, uplink capacity ~103 streams Right: cache ~8 movies, uplink capacity ~41 streams Uplink-bound scenario 0 Users No bottleneck between cache server and clients Moderately popular movies can exhausted uplink bandwidth quickly Unicast gains a lot from large caches Users INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

26 Effects of user numbers on refusal probabilities Ref usal Probability Refusal probability, 96 B cache, 1 Mbit/s uplink batching gleaning unicast Caching strategy: ECT better Caching strategy: ECT Users Users Uplink-bound scenario Shows that low popularity movies are accessed like unicast by all techniques Patching techniques with infinite window can exploit multicast Collecting requests does not work Cache size Is not very relevant for patching techniques Is very relevant for full title techniques Ref usal Probability Refusal probability, 64 B cache, 622 Mbit/s uplink batching gleaning unicast INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

27 Bandwidth effect of daytime variations Change popularity according to time-of-day Two tests Popularity peaks and valleys uniformly distributed Complete exchange of all titles Spread over the whole day Popularity peaks and valleys either at 10:00 or at 20:00 Complete exchange of all titles Within a short time-frame around peak-time Astonishing results For ECT with all mechanisms Hardly any influence on hit rate uplink congestion Traffic is hidden by delivery of low-popularity titles INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

28 Hint based Caching 0.9 Hit ratio development, increasing #hints, ECT history Hit ratio development, increasing #hints, ECT history hints 100 hints 1000 hints hints 10 hints 100 hints 1000 hints hints tio a R it H better tio a R it H Users Users Idea Caches consider requests to neighbour caches in their removal decisions Conclusion Instability due to uplink congestion can not be prevented Advantage exists and is logarithmic as expected Larger hint numbers maintain the advantage to the point of instability Intensity of instability is due to ECT problem ECT inherits IR drawback of fixed size histograms INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

29 Simulation: Summary High relevance of population sizes complex strategies require large customer bases Efficiency of small caches 90:10 rule of thumb reasonable unlike web caching Efficiency of distribution mechanisms considerable bandwidth savings for uncached titles Effects of removal strategies relevance of conditional overwrite unlike web caching, paging, swapping,... Irrelevance of popularity changes on short timescales few cache updates compared to many direct deliveries INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

30 Coordinated servers

31 Distribution Architectures Combined optimization Scheduling algorithm Proxy placement and dimensioning origin server d-1 st level cache d-2 nd level cache 2 nd level cache 1 st level cache client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

32 Distribution Architectures Combined optimization Scheduling algorithm Proxy placement and dimensioning No problems with simple scheduling mechanisms Examples Caching with unicast communication Caching with greedy patching Patching window in greedy patching is the movie length INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

33 Distribution Architectures Caching Caching and reedy Patching origin origin Movies move away from clients Cache Level Cache Level Network for free (not shown all at origin) 1 client Movie (0-300 of 00) Link Cost 1 client Increasing network costs Movie (0-300 of 00) Link Cost Decreasing popularity top movie INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

34 Distribution Architectures Combined optimization Scheduling algorithm Proxy placement and dimensioning Problems with complex scheduling mechanisms Examples Caching with λ patching Patching window is optimized for minimal server load Caching with gleaning A 1st level proxy cache maintains the client buffer for several clients Caching with MPatch The initial portion of the movie is cached in a 1st level proxy cache INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

35 Distribution Architectures: λ Patching 1st client multicast cyclic buffer Central server Unicast patch stream 2nd client position in movie (offset) time Number of concurrent streams! = 2 " F "! M U INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

36 Distribution Architectures: λ Patching Placement for λ patching Popular movies may be more distant to the client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

37 Distribution Architectures: λ Patching Failure of the optimization Implicitly assumes perfect delivery Has no notion of quality User satisfaction is ignored Disadvantages Popular movies further away from clients Longer distance Higher startup latency Higher loss rate More jitter Popular movies are requested more frequently Average delivery quality is lower INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

38 Distribution Architectures: leaning Placement for gleaning Combines Caching of the full movie Optimized patching Mandatory proxy cache 2 degrees of freedom Caching level Patch length Proxy cache Unicast multicast cyclic buffer Central server Unicast patch stream Proxy cache Unicast 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

39 Distribution Architectures: leaning Placement for gleaning INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

40 Distribution Architectures: MPatch Placement for MPatch Combines Caching of the full movie Partial caching in proxy servers Multicast in access networks Patching from the full copy Prefix cache Central server Batch (multicast) Prefix cache 3 degrees of freedom Caching level Patch length Prefix length Unicast Unicast 1 st client 2 nd client INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

41 Distribution Architectures: MPatch Placement for MPatch INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

42 Approaches Current approached does not consider quality Penalize distance in optimality calculation Sort Penalty approach Low penalties Doesn t achieve order because actual cost is higher High penalties Doesn t achieve order because optimizer gets confused Sorting Trivial Very low resource waste INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

43 Distribution Architectures Combined optimization Scheduling algorithm Proxy placement and dimensioning Impossible to achieve optimum with autonomous caching Solution for complex scheduling mechanisms A simple solution exists: Enforce order according to priorities (simple sorting) Increase in resource use is marginal INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen

44 References S.-H. ary Chan and Fourad A. Tobagi: "Distributed Server Architectures for Networked Video Services", IEEE/ACM Transactions on Networking 9(2), Apr 2001, pp Subhabrata Sen and Jennifer Rexford and Don Towsley: "Proxy Prefix Caxching for Multimedia Streams", Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), New York, NY, USA, Mar 1999, pp Sridhar Ramesh and Injong Rhee and Katherine uo: "Multicast with cache (mcache): An adaptive zero-delay video-on-demand service", Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), Anchorage, Alaska, USA, Apr 2001 Michael Bradshaw and Bing Wang and Subhabrata Sen and Lixin ao and Jim Kurose and Prashant J. Shenoy and Don Towsley: "Periodic Broadcast and Patching Services - Implementation, Measurement, and Analysis in an Internet Streaming Video Testbed", ACM Multimedia Conference (ACM MM), Ottawa, Canada, Sep 2001, pp Bing Wang and Subhabrata Sen and Micah Adler and Don Towsley: "Proxy-based Distribution of Streaming Video over Unicast/Multicast Connections", Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), New York, NY, USA, Jun 2002 Carsten riwodz and Michael Zink and Michael Liepert and iwon On and Ralf Steinmetz, "Multicast for Savings in Cache-based Video Distribution", Multimedia Computing and Networking (MMCN), San Jose, CA, USA, Jan 2000 Carsten riwodz and Michael Bär and Lars C. Wolf: "Long-term Movie Popularity in Video-on- Demand Systems", ACM Multimedia Conference (ACM MM), Seattle, WA, USA, Nov 1997, pp Carsten riwodz: "Wide-area True Video-on-Demand by a Decentralized Cache-based Distribution Infrastructure", PhD thesis, Darmstadt University of Technology, Darmstadt, ermany, Apr 2000 INF071, Autumn 2007, Carsten riwodz & Pål Halvorsen