Scalable overlay Networks

Transcription

1 Scalable overlay Networks Dr. Samu Varjonen Scalable overlay networks

2 Lectures MO C122 Introduction. Exercises. Motivation. TH DK117 Unstructured networks I MO C122 Unstructured networks II TH DK117 Bittorrent and evaluation MO C122 Privacy (Freenet etc.) and intro to power-law networks. TH DK117 Consistent hashing. Distributed Hash Tables (DHTs) MO C122 DHTs continued TH DK117 Power-law networks MO C122 Power-law networks and applications. TH DK117 Applications I MO C122 Applications I TH DK117 Advanced topics MO C122 Conclusions and summary TH DK117 Reserved Scalable overlay networks

3 Contents Today Structured Overlay Networks Content Delivery Networks Akamai Coral Amazon Dynamo Scalable overlay networks

4 Content Delivery Networks 4

5 Limitations of Web Proxies (Caching) Inability of cache all objects/content Dynamic Data Encrypted data Server Side Analytics Scalability Hit Metering (limiting and reporting RFC 2227), User Demographics, etc. Inability to support flash crowds 5

6 Content Delivery Networks Role Redirect content requests to an 'optimal site' Cache and Serve content from 'optimal site' Export logs and other information to origin servers Redirection mechanism DNS redirection URL rewriting 6

7 Critical Issues in Deploying CDNs Servers Placement Where to place the servers? How many in each location? Content Selection Which content to distribute in CDNs? Content Replication Proactive push from origin server Cooperative vs Uncooperative Pulls Pricing George Pallis et al. Insight and perspectives for content delivery networks. In Communicatoins of ACM 49, 1 (January 2006), 7

8 Server Placement Problem Given N possible locations at edge of the Internet, we are able to place K (K<N) surrogate servers, how to place them to minimize the total cost? Minimum K-median problem Given N points we need to select K centers Assign each input point j to a center 'closest' to it Minimize the sum of distances between each j and its center NP-Hard 8

9 Minimum K-median problem 9

10 Redirection Techniques Routing Strategy Anycast Load Balancing Application specific selection HTTP redirection (HTTP status codes 3xx) Naming based redirection DNS (example next slide) 10

11 DNS Based Redirection CDN DNS Server 5 Client ISP DNS Server 2 3 Content Provider DNS Server 11

12 Akamai CDN (overview) Client requests content from Original Server URLs for content in CDN modified in the original response Client resolves <content>.<akamai host> name Server from the region (best server) chosen Client fetches content from akamai server 12

13 Akamai Content Source (initial request) DNS Root Server Akamai (high level DNS server) Akamai (low level DNS server) Akamai (content server) Srinivasan Seshan Computer Networking Caching, CDN, Consistent Hashing, P2P 13

14 Akamai Content Source (subsequent request) DNS Root Server Akamai (high level DNS server) 1 Akamai (low level DNS server) Akamai (content server) Srinivasan Seshan Computer Networking Caching, CDN, Consistent Hashing, P2P 14

15 Democratizing Content Publication with Coral (Coral CDN) 15

16 Coral Objectives Pool resources to dissipate Flash Crowds Work with unmodified clients Fetch content only once from Origin No centralized management Browser Coral CDN Coral http prx dnssrv Coral http prx dnssrv Coral http prx dnssrv Origin Server Coral http prx dnssrv Browser Browser Browser Browser Browser 16

17 Using Coral Origin Server rewrites URLs abc.com abc.com.coralhost:coralport Redirect clients to Coral server Coral CDN Components DNS server Given address of resolver used by the client, return the address of proxy near the client HTTP proxy Given the URL find nearest proxy that has content Cache the content (DHT) Distributed Sloppy Hash Table (DSHT) 17

18 Coral System Overview 1) A client sends a DNS request for nyud.net to its local resolver. 2) The client s resolver attempts to resolve the hostname using some Coral DNS server(s) 3) Upon receiving a query, a Coral DNS server probes the client to determines its round-trip-time and last few network hops. 4) Based on the probe results, the DNS server checks Coral to see if there are any known nameservers and/or HTTP proxies near the client s resolver. 5) The DNS server replies, returning any servers (close ones) found through Coral in the previous step; if none were found, it returns a random set of nameservers and proxies. 6) The client s resolver returns the address of a Coral HTTP proxy for 7) The client sends the HTTP request x.com.nyud.net:8090/ to the specified proxy. If the proxy is caching the file locally, it returns the file and stops. Otherwise, this process continues. 8) The proxy looks up the web object s URL in Coral. 9) If Coral returns the address of a node caching the object, the proxy fetches the object from this node. Otherwise, the proxy downloads the object from the origin server, (not shown). 10) The proxy stores the web object and returns it to the client browser. 11) The proxy stores a reference to itself in Coral, recording the fact that is now caching the URL. Freedman, Michael et al. "Democratizing Content Publication with Coral." In NSDI

19 Distributed Sloppy Hash Table Sloppy hashing and self-organizing clusters Michael J. Freedman and David Mazières DHTs provide the wrong abstraction. DHTs have poor locality Overloading nodes caching frequent updates a node might need to send a query half way around the world to learn that its neighbor is caching a particular web page Coral s two principal goals are to avoid hot spots and to find nearby data without querying distant nodes The fundamental observation is that a node doesn t need to know every replicated location of a resource it only needs a single, valid, nearby copy On top of Chord but equally supports Kademlia 19

20 DSHT Coral was made up of concentric rings of distributed hash tables (DHTs) each ring representing a wider and wider geographic range (ping range) The DHTs are composed of nodes all within some latency of each other (for example, a ring of nodes within 20 milliseconds of each other) It avoids hot spots (the 'sloppy' part) by only continuing to query progressively larger sized rings if they are not overburdened If the two top-most rings are experiencing too much traffic, a node will just ping closer ones: when a node that is overloaded is reached, upward progression stops This minimises the occurrence of hot spots, with the disadvantage that knowledge of the system as a whole is reduced. 20

21 Hierarchical Indexing Diameter, Clusters, Levels Each Coral Node part of several DSHTs called clusters Each cluster characterized by max RTT (diameter) Fixed hierarchy of diameters called levels Group of nodes can form a level-i cluster if the pairwise RTT less than threshold for level-i Paper uses 3 (levels): 20ms (2), 60ms (1), SHA-1 for Coral Keys and Node-Ids Bitwise XOR is distance (Kademlia) Longer matching prefix numerically closer Key stored at node having ID close to key (0) 21

22 Routing and Sloppy Storage Routing Initiate from highest level cluster Sloppy Storage Cache key/value pairs at nodes whose IDs are close to the key being referenced Reduces hot-spot congestion for popular content More knowledge of nearby nodes Continue at lower levels on MISS Routing table size logarithmic in total number of nodes Freedman, Michael et al. "Democratizing Content Publication with Coral." In NSDI

23 Reduction in Server Load Initial minute, 15 requests hit the origin web server During this first minute, equal numbers of requests were handled by the level-1 and level-2 cluster caches Within 8-10 minutes, the files became replicated at nearly every server Freedman, Michael et al. "Democratizing Content Publication with Coral." In NSDI

24 Insights from 5 year Deployment A large majority of its traffic does not require any cooperative caching Handling of flash crowds relies on cooperative caching Flash Crowds Small fraction of CoralCDN s domains experience large rate increases within short time periods Flash crowd domains traffic accounts for a small fraction of the total requests Request rate increases very rarely occur on the order of seconds Content delivery via untrusted nodes requires the HTTP protocol to support end-to-end signatures for content integrity Freedman, Michael. "Experiences with CoralCDN: A Five Year Operational View." In NSDI

25 Other CDNs 27

26 Distributing content P2P applications may be oblivious to underlying network Lot of inter-domain traffic (Karagiannis et al. 2005) Approaches to address this problem ISP approaches Block P2P, Rate-limit P2P, Cache content, etc. Reduces traffic but hurts the overlay 28

27 Ono project (P2P) Summary: there are no good solutions that allow ISPs to reduce traffic generated by P2P applications. Proposed solution: use information provided by CDNs for biased peer selection. Features: no cooperation between users and ISPs, no infrastructure,and no network topology information required. Hypothesis: clients who are redirected to the same replica server by a CDN are likely to be good peers. BitTorrent plugin that Periodically, each peer does a DNS lookup for popular CDNs Builds and share ratio maps that how many times the peer has been directed to a certain CDN node. Peer selection biased toward similar ratio maps Taming the Torrent: A Practical Approach to Reducing Cross-ISP Traffic in P2P Systems, Choffnes and Bustamante, in SIGCOMM

28 itracker of P4P P4P (Provider Portals for Applications) Network provider runs an itracker itracker used by ISPs to provide additional information regarding network topology P2P networks may choose to utilize to optimize network data delivery 3. Query itrackers of the ISP for peer selection 1. ISPs run their own itrackers 2. Register (a & b) If trackerless then a and b can connect the itrackers themself Haiyong Xie et al. P4P: provider portal for applications. In SIGCOMM

29 Did this work? Principal result, however, is that this benefit is difficult to achieve in practice Limited impact Reduced performance and robustness Degrades the structural robustness of the overlay Local peers might not be the fastest ones Conflicting interests Too few peers inside a ISP Some of the tier-1 IPSs have incentives to create longer paths than necessary Improvements shown (although, not that big than the original papers) System designers should not expect universal cooperation from IPSs Piatek et al., Pitfalls for ISP-friendly P2P design, 31

30 Maygh P2P CDN P2P CDN on Browser a system that distributes the cost of serving content across the visitors to a web site. Maygh automatically recruits web visitors to help serve content to other visitors, thereby substantially reducing the costs for the web site Leverage on WebSockets, WebRTC, WebStorage API Liang Zhang et al. Maygh: building a CDN from client web browsers. In EuroSys '13. 32

31 Amazon Dynamo 34

32 ACID (Recap) Atomicity Consistency Successful transaction commits only legal results Isolation All or nothing Events within a transaction must be hidden from other transactions running concurrently Durability Once a transaction has been committed its results, the system must guarantee the results survive subsequent malfunctions Theo Haerder et al. "Principles of transaction oriented database recovery." ACM Computing Surveys

33 CAP Theorem C: Strong Consistency (only one consistent state can be observed among parallel processes) A: High Availability (available at all times) P: Partition Tolerance (survive partition between replicas) PICK ANY TWO This is usually relaxed a bit A. Fox et al. Harvest, yield, and scalable tolerant systems. HotOS

34 Two Phase Perspective of CAP Two phase commit P1: Coordinator asks databases to perform a pre-commit and asks them if commit is possible. If all DBs agree then proceed to P2 P2: Coordinator asks DBs to commit Two phase commit supports consistency and partitioning. How is availability violated? Availability of any system is the product of the availability of the components required for the operation i.e., two DBs (availability 99.9) taking part in 2PC results in 99.8 availability ACID provides Consistency. Partion Tolerance is essential. How do you achieve Availability? BASE 37

35 BASE Basically available, Soft state, Eventually consistent Strong vs Eventual (informal comparison) Strong: Every replica sees every update in the same order (atomic updates) Eventual: every replica will eventually see updates and eventually agree on all values (non-atomic updates) Eventual Consistency Database consistency will be in a state of flux but eventually it will be consistent Reads might not return the results of the latest update Dan Pritchett. BASE: An Acid Alternative. ACM Queue

36 Requirements from Dynamo Key-value store shopping carts, seller lists, preferences, product catalog System built using off-the-shelf hardware. Platform must scale to support continuous growth Address tradeoff of high-availability, guaranteed performance, cost-effectiveness, and performance The system needs to have scalable and robust solutions for load balancing, membership and failure detection, failure recovery, replica synchronization, overload handling, state transfer, concurrency and job scheduling, request marshalling, request routing, system monitoring and alarming, and configuration management G. DeCandia et al. Dynamo: Amazon s Highly Available Key value Store, In SOSP

37 Partitioning and Replication in Dynamo Consistent Hashing DHT Each physical node added as multiple virtual nodes Each data-item replicated in N nodes Each virtual node responsible for the region between it and its Nth predecessor Preference List: list of nodes (in multiple datacenters) storing a key G. DeCandia et al. Dynamo: Amazon s Highly Available Key value Store, In SOSP

38 API get (key) may return many versions of the same object put(key, context, object) Context: encodes system metadata and includes information such as the version of the object may return to its caller before the update has been applied at all the replicas An object may have different version sub-histories Vector clock based versioning One vector clock associated with every version of objects 41

39 Data Versioning Assume object is shopping cart. Requirements: additions to the cart don t get lost but deletions can be lost Node Clock Objects versions: D1, D2, D3,... all versions of the object committed to the system are returned when read Reconciliation is usually a merge of the versions so nothing is lost but deleted items may pop back The vector may get long over time but, Dynamo suppors truncation (remove oldest pairs, not foolproof) G. DeCandia et al. Dynamo: Amazon s Highly Available Key value Store, In SOSP

40 Sloppy Quorum Read + Write involves N nodes from the preference list R: minimum number of nodes for Read W: minimum number of nodes for Write R+W>N R = W = 5 high consistency but system is vulnerable to network partitions R = W = 1 weak consistency with failure Typical values of (N, R, W) = (3,2,2) balance between performance and consistency Critique: in the case of network partitions, writes could be made to nodes outside of the set of top N preferred nodes. In this case, there would be no guarantee that writes and reads over the N nodes would overlap since the nodes that constitute N are in flux. Therefore, the formula R+W>N has no meaning. 43

41 Read and Write Operations Coordinator Node responsible for read/writes First node in the preference list Write Operation New vector clock from coordinator Write locally and forward to N-1 nodes, if W-1 nodes respond then write was successful Read Operation Forward request to N-1 nodes, if R-1 nodes respond then forward to user User resolves conflicts and writes back result 44

42 Membership Changes Gossip-based Protocol to propagate membership changes Each node contacts a peer chosen at random every second and the two nodes efficiently reconcile their persisted membership change histories Each node is aware of the key ranges handled by its peers 45

43 Handling Failures: Hinted Handoff Imagine A goes down and N=3 Keys stored by A will now be stored by D D is hinted in the metadata that it is storing keys meant for A When A recovers, the keys at D are now copied to A 46

44 Handling Failures: Merkle Trees Each node maintains seperate Merkle tree for each key-range H(...) Merkle Tree: Leaves are hashes of keys Parents are hashes of children Minimize the amount of transferred data H(H(B3), H(H(B4), H(B5))) H(H(B1), H(B2)) H(B3) H(B1) H(B2) H(B4) Easy to check if a branch is the same (i.e. the hash is the same) H(H(B4), H(B5)) H(B5) 47