From the event loop to the distributed system. Martyn 3rd November, 2011

Transcription

1

2 From the event loop to the distributed system Martyn 3rd November, 2011

3 From the event loop to the distributed system

4 From the event loop to the distributed system An introduction to Pusher

5 From the event loop to the distributed system An introduction to Pusher The event loop

6 From the event loop to the distributed system An introduction to Pusher The event loop Why you d use it

7 From the event loop to the distributed system An introduction to Pusher The event loop Why you d use it Managing complexity

8 From the event loop to the distributed system An introduction to Pusher The event loop Why you d use it Managing complexity The distributed system

9 From the event loop to the distributed system An introduction to Pusher The event loop Why you d use it Managing complexity The distributed system Some general considerations

10 From the event loop to the distributed system An introduction to Pusher The event loop Why you d use it Managing complexity The distributed system Some general considerations Some specific problems and how we solved them

11 Who am I?

12 Who am I? Martyn Loughran

13 Who am I? Martyn Loughran CTO of Pusher

14 Who am I? Martyn Loughran CTO of Pusher We re based in London, England

15 Who am I? Martyn Loughran CTO of Pusher We re based in London, England Rubyist and EventMachine enthusiast

16 Who am I? Martyn Loughran CTO of Pusher We re based in London, England Rubyist and EventMachine enthusiast Started building Pusher in January 2010

17 Who am I? Martyn Loughran CTO of Pusher We re based in London, England Rubyist and EventMachine enthusiast Started building Pusher in January 2010 Eu não falo Português

18 Part I An introduction to Pusher

19 So what is Pusher anyway? I have stickers ->

20 So what is Pusher anyway? A web service which helps developers add real-time functionality to their web applications I have stickers ->

21 So what is Pusher anyway? A web service which helps developers add real-time functionality to their web applications It makes scaling easy I have stickers ->

22 So what is Pusher anyway? A web service which helps developers add real-time functionality to their web applications It makes scaling easy Complex distributed system I have stickers ->

23 Notifications

24 Notifications

25 Notifications

26 Chat

27 Collaboration

28 Data Sync CloudApp

29 WebSocket, the basics:

30 WebSocket, the basics: A pretty silly logo

31 WebSocket, the basics: A pretty silly logo Sockets for the web

32 WebSocket, the basics: A pretty silly logo Sockets for the web Bidirectional

33 WebSocket, the basics: A pretty silly logo Sockets for the web Bidirectional Low latency

34 WebSocket, the basics: A pretty silly logo Sockets for the web Bidirectional Low latency Bandwidth efficient

35 WebSocket, the basics: A pretty silly logo Sockets for the web Bidirectional Low latency Bandwidth efficient Already supported in Safari, Chrome, and Firefox

36 WebSocket, the basics: A pretty silly logo Sockets for the web Bidirectional Low latency Bandwidth efficient Already supported in Safari, Chrome, and Firefox Coming to IE in version 10

37 We use EventMachine

38 We use EventMachine Evented IO for Ruby

39 We use EventMachine Evented IO for Ruby em-websocket

40 We use EventMachine Evented IO for Ruby em-websocket em-hiredis

41 Ruby doesn t scale!

42 13,969,264 Number of API requests made yesterday (There are 86,400 seconds in a day)

43 35,552,810,379 Total number of messages sent to clients since launch

44 < 10ms Mean end to end latency (excluding the internet)

45 Part II The Event Loop

46 Why use an event loop?

47 Why use an event loop? To handle massive numbers of connections

48 Why use an event loop? To handle massive numbers of connections

49 Why use an event loop? To handle massive numbers of connections To share data without Mutexes

50 Why use an event loop? To handle massive numbers of connections To share data without Mutexes Efficient scheduling of work

51 Why use an event loop? To handle massive numbers of connections To share data without Mutexes Efficient scheduling of work

52 It s really easy to use in Ruby require 'eventmachine' EM.run do # Start a server # Make some network connections # Create a timer # etc. end

53 The Pusher event loop Redis Pubsub Redis WebSocket WebSocket WebSocket WebSocket Timer Timer Timer ZeroMQ

54 Never block the reactor

55 Don t get caught up in callback spaghetti

56 Using callbacks and deferrable objects EM.run { stream = TwitterStream.new('yourtwitterusername', 'pass', 'term') stream.ontweet { tweet LanguageDetector.new(tweet).callback { lang puts "New tweet in #{lang}: #{tweet}" } } }

57 Return a deferrable from a function def do_something_complex df = EM::DefaultDeferrable.new use_lots_of_callbacks {... { df.succeed(result) }... df.fail(error) } return df end

58 Pass a deferrable to a strategy Juggler.juggle(:send_webhook, 100) do df, job_params http = EM::HttpRequest.new(job_params['url'].post({ :body => job_params["data"] }) http.callback do response df.success end http.errback do df.fail end end

59 Or try Fibers

60 Part III The distributed system

61 A distributed system is a collection of independent computers that appears to its users as a single coherent system Distributed Systems: Principles and Paradigms, Tanenbaum and Steen 2006

62 The distributed system

63 The distributed system Why would I build one?

64 The distributed system Why would I build one? Work doesn t fit on a single machine any more

65 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability

66 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability

67 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability How can I make one?

68 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability How can I make one? Decouple the application so that each function is handled by a separate component

69 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability How can I make one? Decouple the application so that each function is handled by a separate component Scale components horizontally, and independantly

70 The distributed system Why would I build one? Work doesn t fit on a single machine any more You need better availability How can I make one? Decouple the application so that each function is handled by a separate component Scale components horizontally, and independantly Make components tolerant to failure

71 State

72 Messaging

73 Do not communicate by sharing memory; instead, share memory by communicating. Effective Go, Google State Messaging

74 State: CAP theorem It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: - Consistency (all nodes see the same data at the same time) - Availability (a guarantee that every request receives a response about whether it was successful or failed) - Partition tolerance (the system continues to operate despite arbitrary message loss)

75 State: More questions

76 State: More questions What performance do you need?

77 State: More questions What performance do you need? How durable does it need to be?

78 State: More questions What performance do you need? How durable does it need to be? How much data do you need to store?

79 State: More questions What performance do you need? How durable does it need to be? How much data do you need to store? Does it need to be highly available?

80 State: More questions What performance do you need? How durable does it need to be? How much data do you need to store? Does it need to be highly available? Does it need to be consistent / eventually consistent?

81 State so what do we use?

82 MySQL ~ 20GB

83 MySQL ~ 20GB Consistent

84 MySQL ~ 20GB Consistent Durable

85 MySQL ~ 20GB Consistent Durable Not highly available - but this doesn t matter

86 MySQL ~ 20GB Consistent Durable Not highly available - but this doesn t matter

87 MySQL ~ 20GB Consistent Durable Not highly available - but this doesn t matter Rails models

88 MySQL ~ 20GB Consistent Durable Not highly available - but this doesn t matter Rails models Aggregated usage statistics

89 Redis ~ 500MB

90 Redis ~ 500MB Consistent

91 Redis ~ 500MB Consistent Very fast

92 Redis ~ 500MB Consistent Very fast

93 Redis ~ 500MB Consistent Very fast Shared memory for all processes

94 Redis ~ 500MB Consistent Very fast Shared memory for all processes Some current statistics, waiting to be aggregated

95 ZooKeeper ~ 1MB

96 ZooKeeper ~ 1MB Slow

97 ZooKeeper ~ 1MB Slow Consistent

98 ZooKeeper ~ 1MB Slow Consistent Highly available

99 ZooKeeper ~ 1MB Slow Consistent Highly available Not partition tolerant

100 ZooKeeper ~ 1MB Slow Consistent Highly available Not partition tolerant

101 ZooKeeper ~ 1MB Slow Consistent Highly available Not partition tolerant Processes state, and assignment of roles

102 Messaging

103 Messaging Central broker

104 Messaging Central broker AMQP - the SQL of messaging?

105 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box

106 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box Simple, but hard to scale

107 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box Simple, but hard to scale Custom messaging topologies

108 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box Simple, but hard to scale Custom messaging topologies ZeroMQ - point to point, fanout, pubsub, load balanced

109 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box Simple, but hard to scale Custom messaging topologies ZeroMQ - point to point, fanout, pubsub, load balanced Lots of choices, therefore complex

110 Messaging Central broker AMQP - the SQL of messaging? A single all powerful box Simple, but hard to scale Custom messaging topologies ZeroMQ - point to point, fanout, pubsub, load balanced Lots of choices, therefore complex This is the future, but we re not quite there yet

111 Messaging what do we use?

112 Messaging what do we use? Redis pub/sub

113 Messaging what do we use? Redis pub/sub ZeroMQ

114 Messaging what do we use? Redis pub/sub ZeroMQ Beanstalkd

115 Some examples

116 Usage statistics and latency metrics In memory Redis MySQL

117 Usage statistics and latency metrics Loads of events In memory Redis MySQL

118 Usage statistics and latency metrics Loads of events Collect incrementers and distributions in memory In memory Redis MySQL

119 Usage statistics and latency metrics Loads of events Collect incrementers and distributions in memory Flush to redis every minute In memory Redis MySQL

120 Usage statistics and latency metrics Loads of events Collect incrementers and distributions in memory Flush to redis every minute Eventually consistent state In memory Redis MySQL

121 Storing presence information

122 Storing presence information Need to know when a user joins or leaves a channel

123 Storing presence information Need to know when a user joins or leaves a channel Needs to be consistent across processes

124 Storing presence information Need to know when a user joins or leaves a channel Needs to be consistent across processes Use redis incrementers

125 Storing presence information Need to know when a user joins or leaves a channel Needs to be consistent across processes Use redis incrementers Needs to survive process failure Use a global hash, and a hash per process, with redis transactions

126 Storing presence information Need to know when a user joins or leaves a channel Needs to be consistent across processes Use redis incrementers Needs to survive process failure Use a global hash, and a hash per process, with redis transactions Consistent state

127 Optimising internal messaging

128 Optimising internal messaging Debug console shows all events for all connections

129 Optimising internal messaging Debug console shows all events for all connections Unnecessary messaging, most of the time

130 Optimising internal messaging Debug console shows all events for all connections Unnecessary messaging, most of the time Only publish data when it s needed

131 Optimising internal messaging Debug console shows all events for all connections Unnecessary messaging, most of the time Only publish data when it s needed Eventually consistent, distributed state

132 Redis caches, and live caches # (pseudo simplified version) set = RedisLiveSet.new("debug_open") set.add('42') # redis.sadd("debug_open", 42) # redis.publish("debug_open", ["sadd", "42"]) # On another process set.member?('42') # Checks the in memory set

133 Recovering from process failure

134 Recovering from process failure Store process UUIDs in ZooKeeper as ephemeral files

135 Recovering from process failure Store process UUIDs in ZooKeeper as ephemeral files Leader process notices process failure, and takes required action

136 Recovering from process failure Store process UUIDs in ZooKeeper as ephemeral files Leader process notices process failure, and takes required action Low volume, highly available, and consistent

137 Some other thoughts...

138 Avoid configuration

139 Distributed locking

140 Delay anything you can

141 Think about concurrency

142 In Conclusion

143 In Conclusion Consider an event loop for concurrency

144 In Conclusion Consider an event loop for concurrency EventMachine is great, you don t need to use node.js

145 In Conclusion Consider an event loop for concurrency EventMachine is great, you don t need to use node.js Think about state & messaging It s all about compromises; there are no right answers

146 In Conclusion Consider an event loop for concurrency EventMachine is great, you don t need to use node.js Think about state & messaging It s all about compromises; there are no right answers Find creative solutions to your problems

147 any questions? We re hiring (in London) Come and talk about WebSockets, EM or whatever Thanks for listening Martyn try pusher, it s great! I have stickers, come and get one! ->