IERG 4080 Building Scalable Internet-based Services

Department of Information Engineering, CUHK MScIE 2 nd Semester, 2015/16 IERG 4080 Building Scalable Internet-based Services Lecture 9 Web Sockets for Real-time Communications Lecturer: Albert C. M. Au Yeung 2 nd & 3 rd November, 2016

Push Technology 2

Pull and Push All of the examples we have gone through in network programming so far can be regarded as using the pull method Communication is always initiated by the client Client pulls data or services from the server when necessary (e.g. when the user launches the app, or presses a button) 1 Client requests to be served by the server Server responds with the data or service requested by the client 2 Application Server (E.g. Web server, email server, translation server) 3

Pull and Push HTTP is a pull-based protocol Users browse the Web and actively decide which Website to browse, which link to follow A effective and economical way (every user chooses what he needs) However, if some resources are regularly requested, the pull model can put heavy load on the server 4

Pull and Push There are cases in which the server would like to initiate communication with the client(s) When new email arrives When a peer sends a message to the user through the server When the app/data needs to be updated In these cases, the server needs to push data or services to the client 5

Implementing Push The World Wide Web, and in particular the HTTP protocol, is designed for pull, and additional engineering is required to implement push on the Web. Some ways to emulate push on the Web Polling (Periodic pull) The Comet model BOSH WebSockets 6

Implementing Push Polling The client polls the server periodically to check if new messages or updates are available Client Send request Server Advantages: 1. Easy to implement 2. No extra development on the server-side Disadvantages: 1. Unnecessary network traffic generated 2. Extra workload on the server Wait for 10 seconds Wait for 10 seconds Response Wait for 10 seconds 7

Implementing Push Polling Examples The Post-Office Protocol (POP) for email Email clients using the POP3 protocol make regular requests to the mail server to check for new emails RSS Feed Readers RSS resources are served by HTTP, and thus are all pull-based RSS feed readers poll the RSS servers regularly and check for new updates of the feeds Note: polling can place heavy workload on the server, the client and the network 8

Implementing Push The Comet Model Comet is a model for implementing Web applications that allow servers to push data to clients (browsers) Implementations of Comet applications fall into two major categories 1. Streaming 2. Long-polling Reference: http://en.wikipedia.org/wiki/comet_(programming) 9

Implementing Push The Comet Model Streaming A persistent connection is established between the browser and the server Data is sent from the server a chunked block Events are incrementally sent to the browser (e.g. using <script> tags to execute JavaScript commands) Data is incrementally sent to the client (browser) Data sent from the server Another event happens, server sends another block... Reference: http://en.wikipedia.org/wiki/comet_(programming) http://www.ibm.com/developerworks/library/wa-reverseajax1/... 10

Implementing Push The Comet Model Client Server Long-polling A request is sent to from the client to the server The server holds the connection until some events happen, then response is sent back to the client The client, on receiving a response, issue another request immediately to the server (Usually implemented using Ajax) Send request Response Send another request Response Send another request Connection is held until some data needs to be sent Reference: http://en.wikipedia.org/wiki/comet_(programming) http://www.ibm.com/developerworks/library/wa-reverseajax1/... 11

Implementing Push BOSH BOSH stands for Bidirectional-streams over Synchronous HTTP It makes use of HTTP long-polling A single TCP connection is established to receive push data from server If no data needs to be pushed, server sends an empty <body/> message If client needs to send data to server, a second socket is used to send HTTP post requests The old and new connections will then switch roles (the new will be used for longpolling thereafter) Reference: http://xmpp.org/extensions/xep-0124.html 12

XMPP XMPP stands for Extensible Messaging and Presence Protocol Originally named Jabber, an open source project for real-time and instant messaging Using a client-server architecture (non-p2p) Decentralized and federated model: no central server, but servers for different domains Each user connects to a public XMPP server, which will relay his messages to other users in other domains 13

XMPP http://upload.wikimedia.org/wikipedia/commons/a/a8/jabbernetwork.svg 14

WebSockets 15

WebSockets A protocol providing full-duplex communications channels between two computers over a TCP connection Designed to be implemented in Web browsers (clients) and Web servers HTTP is half-duplex: only one side can send data at a time, like walkie-talkie Communications are done over TCP port 80 (can be used in secured computing environments) Socket.io to be introduced later Reference: http://tools.ietf.org/html/rfc6455 http://www.websocket.org/ 16

WebSockets A persistent connections between a Web browser (or a mobile app) and a server Both sides can send out data to the other side at any time Why WebSocket? Lower latency (avoid TCP handshaking) Smaller overhead (only 2 bytes per message) Less unnecessary communication (data is only sent whenever needed) 17

WebSockets WebSocket is part of the HTML5 standard Supported in latest versions of major Web browsers Simple API in JavaScript Libraries also available on ios and Android var host = 'ws://localhost:8000/example'; var socket = new WebSocket(host); socket.onopen = function() { console.log('socket opened'); socket.send('hello server!'); } socket.onmessage = function(msg) { console.log(msg); } socket.onclose = function() { console.log('socket closed'); } Try the game at http://chrome.com/supersyncsports/ 18

WebSockets Particularly useful when you would like to develop applications such as: Real-time multiplayer games Chat rooms Real-time news feed Collaborative apps (e.g. consider something like Google Documents) Live commenting 19

WebSockets Design principles of WebSockets An additional layer on top of TCP Enable bi-directional communication between client and servers Support low-latency apps without HTTP overhead Web origin-based security model for browsers Support multiple server-side endpoints 20

WebSockets How does WebSockets work? 1. WebSocket handshake Client sends a regular HTTP request to the server with an upgrade header field GET ws://websocket.example.com/ HTTP/1.1 Origin: http://example.com Connection: Upgrade Host: websocket.example.com Upgrade: websocket WebSockets uses the ws scheme (or the wss scheme for secure connections, equivalent to HTTPS) If server supports WebSockets, it sends back a response with the upgrade header field HTTP/1.1 101 WebSocket Protocol Handshake Date: Wed, 16 Oct 2013 10:07:34 GMT Connection: Upgrade Upgrade: WebSocket 21

WebSockets Once the handshake is completed: The initial HTTP connection will be replaced by the WebSockets connection (using the same underlying TCP/IP connection) Both the client and the server can now start sending data to the other side Data are transferred in frames Messages (payload) will be reconstructed once all frames are received Because of the established WebSockets connection, much less overhead will be incurred on the message being transmitted Check whether a browser supports WebSockets: http://caniuse.com/#search=websocket 22

socket.io 23

socket.io A library based on node.js for real time, bi-directional communication between a server and one or multiple clients Using WebSocket to perform data communication (fall back to older solutions when WebSocket is not supported) Official Website: http://socket.io/ Originally written for node.js on the sever side and JavaScript on the client side, there are now libraries for Python, Android and ios 24

socket.io socket.io has two parts: 1) Server and 2) Client Client libraries are available in JavaScript (Web), Android and ios Allow you to build real-time apps across multiple platforms socket.io-client (JavaScript) socket.io (Server Side) socket.io-client (Android) socket.io-client (ios) 25

socket.io Event-driven Once connected, the server and client can communicate with each other by triggering or emitting events Create callback functions to carry out different actions whenever some events happens 26

socket.io Flow of creating a server-side program Create Socket.io Server Listen for incoming connections Wait for events to happen Handle events 27

socket.io Flow of creating a client-side program Connect to a Socke.io Server Wait for events to happen Handle events 28

Flask-SocketIO 29

Flask-SocketIO A module that allows you to use socket.io in Flask applications http://flask-socketio.readthedocs.org/ Install using the following command: $ pip install flask-socketio Note: install the concurrent networking library Eventlet as well: $ pip install eventlet 30

Flask-SocketIO Initialization You need to provide a secret key for Flask to encrypt data for user sessions from flask import Flask, render_template from flask_socketio import SocketIO app = Flask( name ) app.config['secret_key'] = secretkey socketio = SocketIO(app) if name == main : socketio.run(app) Setup the app to use functions of socket.io This is for debugging purpose, we will discuss how to deploy applications that using socket.io soon 31

Flask-SocketIO Remember we mentioned that in Socket.io all communications are based on events We will have to define handlers of events in our Flask application For example: @socketio.on('message') def handle_message(message): # Actions to be performed when a message is received #... The name of the event (NOTE: some names cannot be used because they already represent some special events) 32

Events Both the server and the client can generate events, and if the other side has a handler of that event, the handler will be invoked to carry out some actions In Flask-SocketIO, there are several different types of evetns Special events ( connect, disconnect, join, leave ) Unnamed events ( message or json ) Custom events (a name of your choice, e.g. my event ) 33

Events Connection events # Will be invoked whenever a client is connected @socketio.on('connect') def connect_handler(): print "Client connected. # Will be invoked whenever a client is disconnected @socketio.on('disconnect') def disconnect_handler(): print "Client disconnected." 34

Events Unnamed events # Will be invoked whenever an unnamed event happens @socketio.on('message') def message_handler(message): print "message received: %s" % message # Will be invoked whenever an unnamed event happens (with JSON data) @socketio.on('json') def disconnect_handler(json): print "json received: %s" % str(json) 35

Events Custom events are events with custom-defined names # Will be invoked when the client emits an event of the type event_001 @socketio.on('event001') def event001_handler(json): print "json received: %s" % str(json) 36

Events The server can send message to clients by creating events using the send() function or the emit() function The send() function is for sending unnamed events The emit() function is for sending custom named events from flask_socketio import send, emit @socketio.on('message') def handle_message(message): send(message) @socketio.on('json') def handle_json(json): send(json, json=true) @socketio.on('event001') def handle_my_custom_event(json): emit('event001', json) 37

Broadcasting Normally, send and emit only send message to Broadcasting allows you to send a message to all clients who are connected to the server For example, in a multi-player game, one user performs an action, and you want this to be known to all other users @socketio.on('event001') def handle_event001(data): emit('event001', data, broadcast=true) Set broadcast=true when emitting a message 38

Rooms In some applications, users may interact with only a subset of other users Examples: Chat application with multiple rooms Multiplayer games (multiple game boards, game rooms, etc.) from flask_socketio import join_room, leave_room @socketio.on('join') def on_join(data): username = data['username'] room = data['room'] join_room(room) send('user_join', '%s joined' % username, room=room) @socketio.on('leave') def on_leave(data): username = data['username'] room = data['room'] leave_room(room) send('user_leave', '%s left' % username, room=room) 39

Deploying Flask-SocketIO If you have eventlet installed, you can use the embedded server which is production-ready Invoked by socketio.run(app, host="0.0.0.0", port=5000) Sample supervisor configuration files: [program:ierg4080] command = python app.py directory = /home/albert/ierg4080 user = albert autostart = true autorestart = true stdout_logfile = /home/albert/ierg4080/app.log redirect_stderr = true 40

Deploying Flask-SocketIO If you would like to deploy a single Flask app including both your APIs and the socket.io event handlers, use Gunicorn. For example: [program:iems5722] command = gunicorn --worker-class eventlet -w 1 app:app b localhost:8000 directory = /home/albert/ierg4080 user = albert autostart = true autorestart = true stdout_logfile = /home/albert/ierg4080/app.log redirect_stderr = true 41

What can you do with socket.io? Real-time chat rooms Real-time multiplayer games Real-time event broadcasting (e.g. real-time results of a competition or an event) And a snake game across multiple screens! (https://www.youtube.com/watch?v=yucrsu0curq) 42

Scaling Socket.io Applications 43

Scaling Socket.io Applications Horizontal scaling must be applied when there is a large number of concurrent users How can we scale socket.io applications? What are the requirements of scaling socket.io apps? Socket.io App Clients Socket.io App Load Balancer Socket.io App 44

Scaling Socket.io Applications Requirement 1 The same client must be served by the same worker (aka sticky sessions) If we use Nginx, we can use the ip_hash algorithm for load balancing The same client will have the same IP address, and thus will be routed to the same host/worker http { upstream myservers { ip_hash; server s1.myserver.com; server s2.myserver.com; server s3.myserver.com; } } server { listen 80; } location / { proxy_pass http://myservers; } 45

Scaling Socket.io Applications Requirement 2 If there are multiple hosts/workers running the same socket.io app, only a subset of the clients will be served by each worker Problem: how does the server application emit events to all connected clients? Solution: a message queue! 46

Scaling Socket.io Applications Requirement 2 If there are multiple hosts/workers running the same socket.io app, only a subset of the clients will be served by each worker Problem: how does the server application emit events to all connected clients? Solution? 47

Scaling Socket.io Applications Solution: using a message queue! When emitting an event, send it to its own clients, as well as submitting the event to the message queue, such that other hosts will be notified. Clients 1 Clients 2 Message Queue Clients 3 48

Scaling Socket.io Applications You can specify a message queue (Redis or RabbitMQ) when you create the Socket.io object in your Flask app For example: socketio = SocketIO(app, message_queue='redis://localhost:6379//') or socketio = SocketIO(app, message_queue='amqp://guest:guest@localhost:5672//') 49

Scaling Socket.io Applications With the message queue set up, you can actually emit events from any other processes (e.g. another Python program, an asynchronous task worker, etc.) Example:... socketio = SocketIO(message_queue='amqp://...') socketio.emit('event001', {'data': 123'})... 50

End of Lecture 11 51