Comet and WebSocket Web Applications How to Scale Server-Side Event-Driven Scenarios

Transcription

1 Comet and WebSocket Web Applications How to Scale Server-Side Event-Driven Scenarios Simone Bordet 1

2 Agenda What are Comet web applications? Impacts of Comet web applications WebSocket web applications The CometD project Questions & Answers 2

3 What are Comet & WebSocket Web Applications? 3

4 Web Classic Interaction Request Pattern Bursts of requests for HTML, images, CSS, JS Web 1.0 Applications Navigation Mode Full page based Interaction with Server Reactive, changes happen on user click Resources download 4

5 Ajax Web Applications Web Classic + XMLHttpRequest (XHR) Request Pattern: changed Bursts (for classic) + Concurrent (for XHR) Requires synchronization in server code Navigation Mode: radically changed Sometimes full page, most often single page with partial changes Interaction with Server: changed Reactive as before Data download 5

6 Comet Web Applications Web Classic + Ajax + WebSocket Request Pattern, Navigation Mode, Interaction with Server Same as with XHR More Efficient Now messages can be server-initiated Messages do not require a response Uses JavaScript Same as with XHR 6

7 Rich User Interfaces Traditional Web sacrificed Rich UI for: Ubiquitous access Zero Install Heavy usage of client-side JavaScript and XHR changed the way we create and develop webapps They become rich, and raise expectations Server side events are needed To meet user raised expectations To implement new types of web applications 7

8 Comet Examples 8

9 Polling for Server Side Events Polling Strategy for Server-side Events Notification Simple to implement ( I can do that! ) Sensible latency Can look like a denial of service (DoS) attack to the poor server When poll interval is short When the number of clients is large 9

10 Long Polling for Server-side events Comet Strategy More difficult to implement correctly Minimal latency The long poll request is held by the server until: A server-side event arrives A timeout expires The client disconnects 10

11 Websocket for Server-side events WebSocket strategy More efficient than long poll True bidirectional web communication WebSocket Messages No more Request & Response Not widely deployed yet Messages can be initiated by the server 11

12 WebSocket From HTML 5 API & Protocol Two way communication Developed by WHATWG Browser vendors Little server side input In IETF WG Developing standard Slow progress interface WebSocket { attribute Function onopen; attribute Function onmessage; attribute Function onerror; attribute Function onclose; boolean send(in String data); void close(); }; Experimentally Deployed Chrome, Opera, Firefox 12

13 Impacts on the Server 13

14 Scaling Polling Scaling Polling 1000 clients, each polling every 5 seconds Request Rate: 200 requests/s Latency: 2.5 seconds in average, too high How can polling achieve, say, 250 ms latency? Reduce the polling interval to 0.5 seconds Request Rate: 2000 requests/s, pretty high Limits Most likely, the server is CPU bound, then connection bound 14

15 Scaling Comet (long polling) Scaling Comet (Blocking) 1000 clients, long poll request held for 20 seconds Request Rate: 50 requests/s, good But yields 1000 concurrent requests! Running out of threads in the pool On 64-bit JVM the thread stack size is 1 MB Latency Average latency approaches network latency Maximal latency is at least 3x network latency Limits Most likely, the server is memory bound Note that threads stack memory is outside Java heap 15

16 Scaling Comet Comet has huge impacts on server-side You cannot just deploy your Comet web application in a standard configuration You cannot deploy your Comet application behind Apache Httpd & other intermediaries Do not scale for the same reasons You need a new generation of servers 16

17 Asynchronous Servers Jetty Server explored these problems in the Servlet(TM) space for JMS messaging. The Jetty Continuations allow the Jetty Server to threadlessly suspend requests The continuation concepts have been incorporated in the new Servlet(TM) 3.0 specification Jetty 6 and Jetty 7 successfully deployed Comet applications worldwide Jetty 8 implements the Servlet(TM) 3.0 specification 17

18 Blocking Comet Server-side Event-driven Comet Web Application 18

19 Asynchronous Comet Server-side Event-driven Comet Web Application 19

20 Scaling Comet (Asynchronous) Comet (Asynchronous) 1000 clients, long poll request held for 20 seconds Wait for server-side events is threadless Request is re-run on event arrival or on long poll timeout Request Rate: 100 requests/s, good Only few concurrent active (threaded) requests Good latency, good request rate, no thread pool problems Limits Most likely, the server is connection bound, then CPU bound Scales a lot better than normal polling and blocking Comet 20

21 Scaling Comet (WebSocket) WebSocket 1000 clients, keepalive message every 20 seconds No need to reply, decreased bandwidth Wait for server-side events is threadless Request Rate: 50 requests/s, good Only few concurrent active (threaded) requests Optimal latency, good request rate, no thread pool problems Limits Most likely, the server is connection bound, then CPU bound Scales a slightly better than Comet with continuations due to better data density and reduced request rate 21

22 The Need of a Framework With Comet, we have an asynchronous bidirectional web Looks like messaging to me Writing server-side code based on continuations (or, in the future, with Servlet 3.0) is difficult It really is, you don't want to do it How are failures handled? Timeouts, dropouts, retries, backoffs, cross domain, etc. Web applications should be easy to write Can we abstract the gory details into a library? 22

23 The CometD Project 23

24 The CometD Project We have now a scalable bidirectional web What do we need to write applications? We don't want to care about: Which transport: long-polling, websocket limits: same-origin policy, two connection limits We want: A clean way to publish data to the server A clean way to receive data from the server A clean way to emit server-side events to clients 24

25 CometD JavaScript API var cometd = $.cometd; // jquery style cometd.init(' cometd.subscribe('/my/channel', function(message) { var data = message.data; // Do something with the data }); cometd.publish('/my/channel', { chattext: 'hello!' }); cometd.disconnect(); 25

26 The CometD Project There are a lot of other details to take care of Transport Negotiation WebSocket, Long Polling over CORS, JSONP, etc. Authentication Network Failures With possible automatic retry Message Batching Message Acknowledgment Message serialization Etc. From these and other requirements and input, the CometD project was born 26

27 The CometD Project The CometD project delivers libraries that use the Comet technique (long poll) or WebSocket to transport Bayeux messages The Bayeux protocol specifies the format of the Bayeux messages It is based on JSON Libraries are available in JavaScript (client) Java (client and server) Perl & Python (less active) 27

28 Bayeux Bayeux is at its core a publish/subscribe messaging system Very similar to JMS publish/subscribe A Bayeux channel is similar to a JMS Topic You may subscribe to it You will be delivered messages published onto You can publish messages to a channel The server automatically delivers the messages to all channel subscribers 28

29 CometD JavaScript The JavaScript library features Common code with bindings for Dojo and jquery Highly configurable Message batching Automatic reconnection Pluggable transports WebSocket, Long Polling and Callback Polling available Supports Cross-Origin servers Extensible Many extensions already available 29

30 CometD Java The Java library features Highly scalable (the client is based on Jetty asynchronous HTTP Client) Message batching Lazy messages A variety of listeners to be notified of relevant events in client and server Data filters to automatically convert data Extensions SecurityPolicy 30

31 Performance 31

32 DEMO 32

33 WebSocket Redux Slightly better maximal latency But don't run your pacemaker on it Slightly better data density If only that was a limitation! No help with hard problems Connection management Handling Failures You still need a framework like CometD 33

34 References WebSocket CometD Jetty 34

35 Questions & Answers 35