Comet for Highly- Scalable Applications

Transcription

1 Comet for Highly- Scalable Applications Michael Carter and Arthur Lee diagrams by Jacob Rus 24 October 2007 The Ajax Experience Boston

2 What is Comet? Comet describes a model of user interaction Pushing real-time updates to web browsers Renkoo Gmail Chat Meebo

3 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

4 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

5 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

6 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

7 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

8 Old Method: Polling Otherwise known as hammering the server App (Event Dispatcher (Event Queues inside)

9 Polling Overview Old and busted Pros: Easy to implement Client side Server side Cons: Wasted requests Additional latency

10 Enter Long-Polling The new hotness Pros: Latency gone! Client side still easy Cons: Difficult to scale server-side Conceptually more complicated

11 Vertical Scaling: How do you scale comet on a single server? photo by Michael (mx5tx)

12 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Supports a few dozen users CPU usage and latency go through the roof as users are added App Threaded Comet Dispatcher Event Queue Connect Queue

13 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Supports a few dozen users CPU usage and latency go through the roof as users are added App Threaded Comet Dispatcher Event Queue Connect Queue

14 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Supports a few dozen users CPU usage and latency go through the roof as users are added App Threaded Comet Dispatcher Event Queue Connect Queue

15 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Supports a few dozen users CPU usage and latency go through the roof as users are added App Threaded Comet Dispatcher Event Queue Connect Queue

16 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Supports a few dozen users CPU usage and latency go through the roof as users are added App Threaded Comet Dispatcher Event Queue Connect Queue

17 Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Compared to polling: Latency reduced for very light load Avg. latency depends on the order of events As much as 7 the CPU for moderate load CPU-bound at moderate load

18 Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Supports as many users as CPU and I/O can handle Event-Based Comet Dispatcher App Event Queue Connect Pool CPU usage scales linearly and latency remains low

19 Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Supports as many users as CPU and I/O can handle Event-Based Comet Dispatcher App Event Queue Connect Pool CPU usage scales linearly and latency remains low

20 Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Supports as many users as CPU and I/O can handle Event-Based Comet Dispatcher App Event Queue Connect Pool CPU usage scales linearly and latency remains low

21 Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Compared to polling: Decreased latency and CPU usage Compared to threaded Comet: Vastly decreased CPU usage Order of events irrelevant Performance degrades gracefully

22 Horizontal Scaling: How do you scale comet across many servers? photo by ulybug

23 Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Embed publish-subscribe architecture into comet server Keep application state within comet nodes Allow comet nodes to communicate with each-other

24 Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Design is inflexible, can t scale for some uses Comet Nodes Application Node Hard to scale to many nodes in the best case A B C D E F

25 Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Design is inflexible, can t scale for some uses Comet Nodes Application Node A sends message to E Hard to scale to many nodes in the best case A B C D E F

26 Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Design is inflexible, can t scale for some uses Comet Nodes Application Node E sends response to A Hard to scale to many nodes in the best case A B C D E F

27 Horizontal Scaling: How do you scale comet across many servers? Right way: Share nothing between nodes No CPU or I/O overhead of sharing state Application Node CPU usage scales linearly and latency remains low Comet Nodes

28 Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Pre-defined hash function Application Nodes Comet Nodes Hash Function Leave authentication to app layer A B C D E F

29 Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Pre-defined hash function Application Nodes Comet Nodes Hash Function B sends message to E Leave authentication to app layer A B C D E F

30 Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Pre-defined hash function Application Nodes Comet Nodes Hash Function E sends response to B Leave authentication to app layer A B C D E F

31 Publish / Subscribe Pushing published events to groups of browsers What s it good for? Applications for which multiple users need the same live updates

32 Publish / Subscribe Pushing published events to groups of browsers What s it good for? Applications for which multiple users need the same live updates real-time chat

33 Publish / Subscribe Pushing published events to groups of browsers What s it good for? Applications for which multiple users need the same live updates stock tickers real-time chat

34 Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single comet node, which dispatches events to multiple users in multiple groups then how do we scale up to multiple comet nodes?

35 Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single node, which dispatches events to multiple users in multiple groups Application Node Comet Dispatcher Node A single comet node serves all users and all groups

36 Scaling Pub/Sub How do you scale publish/subscribe horizontally? then how do we scale up to a pair of nodes? Application Node Comet Dispatcher Nodes Either we arrange nodes by group: All of a group s events are sent by one node

37 Scaling Pub/Sub How do you scale publish/subscribe horizontally? then how do we scale up to a pair of nodes? Application Node Comet Dispatcher Nodes Or we arrange nodes by user: A user receives all events from the same node

38 Scaling Pub/Sub How do you scale publish/subscribe horizontally? But how, specifically? Split the architecture into layers Publish/Subscribe Cloud Comet Dispatcher Nodes App Do not mix pub/sub layer with comet dispatcher Treat pub/sub layer as a black box

39 Pub/Sub Options Publish/subscribe already has solutions Don t reinvent the wheel: Many well-understood, scalable publish/subscribe solutions exist Internet Relay Chat: Nearly 20 years old. Scales well for large groups of users. XMPP (Jabber): Flexible protocol. Does presence well, in addition to messaging. Java Message Service: It s enterprise, baby.

40 Pub/Sub with Orbited Orbited can be easily added to a pub/sub stack Scaling through modularity: Its simplicity makes Orbited easy to add to any web app architecture Orbited handles comet dispatch IRC or Jabber handles publish/ subscribe App servers do everything else Internet Relay Chat Application / Web Server Orbited Pub/Sub Stack IRC Bridge Orbited Browser

41 Conclusion: Polling latency unacceptable Comet solves this problem Impossible to scale Comet with threaded design Event-based architecture is necessary Horizontal scaling requires share nothing design Publish/subscribe is a difficult problem Scaling pub/sub is easiest with layered architecture

42 Michael Carter 24 September 2007 AJAXWorld