State of the real-time web with Django

Transcription

1 State of the real-time web with Django Aymeric Augustin DjangoCon US - September 5th,

2 What are we talking about? 2

3 Real-time 1. Systems responding within deadlines 2. Simulations running at wall clock time 3. Processing events without perceivable delay 3

4 Real-time web The real-time web is a set of technologies and practices that enable users to receive information as soon as it is published by its authors, rather than requiring that they or their software check a source periodically for updates. 4

5 Use cases Chat Games VoIP Notifications Collaboration Live data Social feeds All these use cases require servers to push events to clients. 5

6 Problem Client Server GET / HTTP/1.1 HTTP/ OK??? information published! The request-response model doesn t allow servers to push events to clients. 6

7 From hacks to protocols 7

8 Early solutions Java applets (1996) Not quite the web Pushlets (2000) Call back from Java applications into DHTML Comet (2006) Long-lived HTTP connections to reduce latency A revolution in browser-based user interfaces 8

9 HTTP long polling Server keeps the request on hold and only sends a response when there is an event to deliver Client resends a request after each response Client request response request response request Server event event 9

10 HTTP streaming Server sends a series of events in a single HTTP response Client Server request response... Chunked EOF-terminated Client processes each incoming event response... response... event event 10

11 Chunked response HTTP/ OK Content-Type: text/plain Transfer-Encoding: chunked 25 This is the data in the first chunk Client Server request response... event response... 1C and this is the second one 0 response... event 11

12 RFC 6202 The authors acknowledge that both the HTTP long polling and HTTP streaming mechanisms stretch the original semantic of HTTP and that the HTTP protocol was not designed for bidirectional communication. 12

13 Server-sent events HTTP stream of events Format: text/ event-stream JavaScript API: EventSource interface and events : The data field is mandatory. data: This is the first message. : event and id are optional. event: message data: This is another message data: over several lines. event: flash data: This is a flash event! id:

14 Server-sent events 14

15 WebSocket Provides bidirectional communication in the context of the existing HTTP infrastructure RFC 6455 (supersedes hybi-xx and hixie-xx) Opening handshake to upgrade from HTTP Framing protocol and closing handshake Provisions for extensions and subprotocols JavaScript API: WebSocket interface and events

16 WebSocket > GET /endpoint HTTP/1.1 > Host: ws.example.com > Connection: Upgrade > Upgrade: websocket > Sec-WebSocket-Key: dghlihnhbxbszsbub25jzq== > Sec-WebSocket-Version: 13 < HTTP/ Switching Protocols < Connection: Upgrade < Upgrade: websocket < Sec-WebSocket-Accept: s3pplmbitxaq9kygzzhzrbk+xoo= 16

17 WebSocket > ec a0 1e 0c 81 f9 75 # TEXT [ M A S K ] m Y k < 81 0a c 6c 6f 20 6d 59 6b 21 # TEXT H e l l o m Y k! < e8 # CLOSE 1005 > a 29 aa # CLOSE [ M A S K ]

18 WebSocket (2009) 18

19 Transport frameworks Socket.IO SockJS LightStreamer (proprietary) 19

20 Let s try it! 20

21 An example of long polling Events Any source Web page JS Redis PUBLISH GET Long polling Pub/Sub Redis Redis SUBSCRIBE Application Django 21

22 Helpers # demo/models.py (1/2) import redis CHANNEL = 'demo' def send_message(message): client = redis.strictredis() message = message.encode('utf-8') return client.publish(channel, message) 22

23 Helpers # demo/models.py (2/2) def recv_message(): client = redis.strictredis() pubsub = client.pubsub() pubsub.subscribe(channel) for event in pubsub.listen(): if event['type'] == 'message': message = event['data'].decode('utf-8') break pubsub.unsubscribe() return message 23

24 Publisher # demo/send_msg.py #!/usr/bin/env python import sys from demo.models import send_message message = " ".join(sys.argv[1:]) num = send_message(message) if num == 1: log = "Sent to one subscriber" else: log = "Sent to {} subscribers".format(num) print("{}: {}".format(log, message)) 24

25 Subscriber # demo/views.py from django.http import HttpResponse from demo.models import recv_message def long_polling_endpoint(request): message = recv_message() return HttpResponse(message.encode('utf-8'), content_type='text/plain; charset=utf-8') 25

26 HTML # demo/templates/demo/long_polling.html <!DOCTYPE html>{% load static %} <html> <head> <title>long polling demo</title> </head> <body> <ul><!-- messages will be inserted here --></ul> <script src="//ajax.googleapis.com/ajax/libs/ jquery/2.0.3/jquery.min.js"></script> <script src="{% static 'demo/long_polling.js' %}"></script> </body> </html> 26

27 JavaScript # demo/static/demo/long_polling.js $(function () { function show_next_message() { $('<li><i>loading </i></li>').appendto($('ul')).load('endpoint/', show_next_message); } show_next_message(); }); 27

28 28 Demo

29 29 Demo

30 Deployment % gunicorn --timeout 10 --workers 2 dcus13rt.wsgi [68271] [INFO] Starting gunicorn 17.5 [68271] [INFO] Listening at: [68271] [INFO] Using worker: sync [68274] [INFO] Booting worker with pid: [68275] [INFO] Booting worker with pid: % ab -c 20 -n 20 \ Benchmarking (be patient)... % PYTHONPATH=. demo/send_msg.py spam Sent to 2 subscribers: spam 30

31 Deployment % ab -c 20 -n 20 \ Server Software: gunicorn/17.5 Server Hostname: Server Port: 8000 Document Path: Document Length: /long_polling/endpoint/ 4 bytes Concurrency Level: 20 Time taken for tests: seconds Complete requests: 20 Failed requests: 18 (Connect: 0, Receive: 0, Length: 18, Exceptions: 0) 31

32 Let s try again? 32

33 An example of WebSocket Events Any source Web page JS Redis PUBLISH WebSocket Pub/Sub Redis Redis SUBSCRIBE Real time Tulip 33

34 Subscriber # demo/handlers.py import tulip from demo.models import def simple_endpoint(websocket, uri): # Doesn't work! recv_message isn't a coroutine. message = yield from recv_message() websocket.send(message) 34

35 Subscriber # demo/handlers.py (1/2) import tulip import websockets from demo.models import recv_message subscribers = def endpoint(websocket, uri): global subscribers subscribers.add(websocket) yield from websocket.recv() subscribers.remove(websocket) 35

36 Subscriber # demo/handlers.py (2/2) def relay_messages(): while True: message = recv_message() for websocket in subscribers: if websocket.open: websocket.send(message) if name == ' main ': websockets.serve(endpoint, 'localhost', 7999) loop = tulip.get_event_loop() loop.run_in_executor(none, relay_messages) loop.run_forever() 36

37 HTML # demo/templates/demo/websocket.html <!DOCTYPE html>{% load static %} <html> <head> <title>websocket demo</title> </head> <body> <ul><!-- messages will be inserted here --></ul> <script src="//ajax.googleapis.com/ajax/libs/ jquery/2.0.3/jquery.min.js"></script> <script src="{% static 'demo/websocket.js' %}"></ script> </body> </html> 37

38 JavaScript # demo/static/demo/websocket.js $(function () { var ws = new WebSocket("ws://localhost:7999/"); ws.onmessage = function (event) { $('<li>' + event.data + '</li>').appendto($('ul')); } }); 38

39 By the way

40 Deployment with gevent % gunicorn --timeout 10 --workers 2 \ --worker-class gevent dcus13rt.wsgi [68261] [INFO] Starting gunicorn 17.5 [68261] [INFO] Listening at: [68261] [INFO] Using worker: gevent [68264] [INFO] Booting worker with pid: [68265] [INFO] Booting worker with pid: % ab -c 20 -n 20 \ Benchmarking (be patient)... % PYTHONPATH=. demo/send_msg.py spam Sent to 20 subscribers: spam 40

41 Deployment with gevent % ab -c 20 -n 20 \ Server Software: gunicorn/17.5 Server Hostname: Server Port: 8000 Document Path: Document Length: /long_polling/endpoint/ 4 bytes Concurrency Level: 20 Time taken for tests: seconds Complete requests: 20 Failed requests: 0 41

42 42 Asynchronous I/O

43 Execution model Based on an event loop Handle many socket connections in a single thread epoll (Linux), kqueue (BSD), IOCP (Windows) More efficient than one thread per connection Suitable for network programming 43

44 Programming model Based on explicit cooperative multi-threading Callbacks Coroutines In Python: yield (from) Suitable for concurrent applications 44

45 The sad truth Converting [a synchronous operation] to asynchronous requires modifying every point that calls it to yield control appropriately. 45

46 What about gevent? It modifies every function that performs I/O by monkey-patching the standard library so you don t have to change your own code. You get the benefits of the execution model for free! But you lose the benefits of the programming model. 46

47 Major Python frameworks gevent is a coroutine-based networking library Twisted is an event-driven networking engine Tornado is a web framework and asynchronous networking library 47

48 PEP 3156 Pluggable event loop API for interoperability Callbacks, transports, protocols, futures High-level scheduler based on coroutines Suspend execution with yield from... Reference implementation code-named Tulip Effort led by Guido van Rossum himself 48

49 49 It s too complicated!

50 django-c10k-demo Define WebSocket Wire them in URLconfs Use runserver in dev and gunicorn in prod Tulip under the hood Real-time made easy! 50

51 django-c10k-demo from c10ktools.http import def worker(ws): #... initial synchronization (not shown)... while True: msg = yield from ws.recv() if msg is None: break step, row, col, state = msg.split() for subscriber in subscribers[row][col]: if subscriber.open: subscriber.send(msg) for row, col in subscriptions: subscribers[row][col].remove(ws) 51

52 That was a lie. Markus Raetz, Silhouette (1993).

53 Django isn t asynchronous # Synchronous cache access def sync(request, key): value = cache.get(key) return HttpResponse(value) # Asynchronous cache def async(ws): value = yield from cache.get(key) ws.send(value) 53

54 Django isn t asynchronous # Synchronous ORM access def sync(request, user_id): user = User.objects.get(id=int(user_id)) bio_html = user.bio.html return HttpResponse(bio_html) # Asynchronous ORM def async(ws): user_id = yield from ws.recv() user = yield from User.objects.get(id=int(user_id)) bio_html = #??? no yield from attribute access! ws.send(bio_html) 54

55 Several lies, actually. Piotr Kowalski, Identité 4 (1993).

56 HTTP!= real-time Execution threads vs. events Programming preemptive vs. cooperative Protocol request / response vs. message streams Scalability stateless vs. stateful Workload CPU vs. I/O bound 56

57 Key learnings PEP 3156 will improve support and resources for asynchronous I/O in Python 3.4. Django isn t designed for explicit cooperative multi-threading and that s unlikely to change. Robust client and server stacks are emerging as well as deployment best-practices. Simplified development setups are possible at the cost of losing parity with production. 57

58 95% use case? Charles Ross, Brûlures solaires, 1993 (detail).

59 We re still learning. Charles Ross, Brûlures solaires, 1993.

60 Thank you! Questions? Photos were taken by the author at the Château d Oiron, France, on June 22 nd. 60 The Curios & Mirabilia exhibition is a modern tribute to the cabinet of curiosities gathered by Claude Gouffier circa 1550.