IGL S AND MOUNT NS. Taking AKKA to Production

Transcription

1 IGL S AND MOUNT NS Taking AKKA to Production

2 THIS TALK Hello! I m Derek Wyatt, Senior Platform Developer at Auvik Networks! and author of Akka Concurrency Scala, Play and Akka form the foundational triad for Auvik Networks core technology. In this talk, we ll discuss how Akka enables the asynchronous design of the system and how it powers the production environment. However, it s not all cherries and moonbeams - we faced a number of challenges, including technological hurdles and more than a few spiky PEBKACs. You ll hear how we overcame these issues, as well as provide some insight into how we re growing our understanding and practices around the Typesafe platform.

3 AUVIK AND THE TYPESAFE STACK Auvik Networks Auvik simplifies monitoring, configuring and automating small and medium sized enterprise networks. We help IT professionals build Networks That Work. TM

4 AUVIK AND THE TYPESAFE STACK Auvik Networks Auvik simplifies monitoring, configuring and automating small and medium sized enterprise networks. We help IT professionals build Networks That Work. TM Auvik is all in with Akka, Scala and Play - our future success is tied to the success of the Typesafe Platform. Akka s design is what powers Auvik s design.

5 30K FOOT ARCHITECTURE Hybrid Cloud SaaS Auvik s solution is half in the cloud and half on site

6 30K FOOT ARCHITECTURE Hybrid Cloud SaaS Auvik s solution is half in the cloud and half on site The cloud provides the bulk of the operational logic and the on site software works on behalf of the cloud

7 30K FOOT ARCHITECTURE Hybrid Cloud SaaS Auvik s solution is half in the cloud and half on site The cloud provides the bulk of the operational logic and the on site software works on behalf of the cloud Technically, the on site software can operate disconnected from the cloud, but eventually it has to connect in, otherwise it s working for nothing

8 30K FOOT ARCHITECTURE Hybrid Cloud SaaS Auvik s solution is half in the cloud and half on site The cloud provides the bulk of the operational logic and the on site software works on behalf of the cloud Technically, the on site software can operate disconnected from the cloud, but eventually it has to connect in, otherwise it s working for nothing While the on site component is an Akka solution, it does not communicate with the cloud using Akka protocols, since these aren t design for the WAN

9 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors

10 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Those Actors ensure that our customer design is

11 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Those Actors ensure that our Isolated customer design is

12 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Those Actors ensure that our Isolated customer design is Resilient

13 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Asynchronous Isolated Those Actors ensure that our customer design is Resilient

14 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Asynchronous Isolated Those Actors ensure that our customer design is Non Blocking Resilient

15 AUVIK USES AKKA Auvik s core design is an Actor-based design Customers exist in distinct Actor subtrees. Tables, On Site connections, the Execution Engine, and many other components are all Actors Asynchronous Non Blocking Isolated Resilient Those Actors ensure that our customer design is Reasonable

16 CLOUD DEPLOYMENT The Cluster

17 CLOUD DEPLOYMENT The Cluster We use Akka Clustering to manage our node fabric

18 CLOUD DEPLOYMENT The Cluster We use Akka Clustering to manage our node fabric Everything runs in Amazon, defined by CloudFormation and configured by Chef

19 CLOUD DEPLOYMENT The Cluster We use Akka Clustering to manage our node fabric Everything runs in Amazon, defined by CloudFormation and configured by Chef Akka Clustering doesn t survive across a WAN, so communication with the onsite component is proprietary

20 CLOUD DEPLOYMENT The Cluster We use Akka Clustering to manage our node fabric Everything runs in Amazon, defined by CloudFormation and configured by Chef Akka Clustering doesn t survive across a WAN, so communication with the onsite component is proprietary Backends and Frontends are specified to scale independently

21 CLOUD DEPLOYMENT The Cluster We use Akka Clustering to manage our node fabric Everything runs in Amazon, defined by CloudFormation and configured by Chef Akka Clustering doesn t survive across a WAN, so communication with the onsite component is proprietary Backends and Frontends are specified to scale independently Cluster discovery happens through a well-known seed node (not shown)

22 CHALLENGES AND SURPRISES Cluster Singletons Customers would be great Cluster Singletons, except Cluster Singletons don t load balance

23 CHALLENGES AND SURPRISES Cluster Singletons Inner classes and closures Customers would be great Cluster Singletons, except Cluster Singletons don t load balance Inner classes and closures are awesome, unless they get caught up in messages and force us to serialize the planet

24 CHALLENGES AND SURPRISES Cluster Singletons Inner classes and closures Customers would be great Cluster Singletons, except Cluster Singletons don t load balance Inner classes and closures are awesome, unless they get caught up in messages and force us to serialize the planet Application Debugging asynchronous applications is still hard. Bulkhead your application to protect it from itself

25 CHALLENGES AND SURPRISES Cluster Singletons Inner classes and closures Customers would be great Cluster Singletons, except Cluster Singletons don t load balance Inner classes and closures are awesome, unless they get caught up in messages and force us to serialize the planet Application Performance issues Debugging asynchronous applications is still hard. Bulkhead your application to protect it from itself Tracking down performance issues can be tough - that ForkJoin pool may not be your friend

26 CHALLENGES AND SURPRISES Cluster Singletons Inner classes and closures Customers would be great Cluster Singletons, except Cluster Singletons don t load balance Inner classes and closures are awesome, unless they get caught up in messages and force us to serialize the planet Application Performance issues Debugging asynchronous applications is still hard. Bulkhead your application to protect it from itself Tracking down performance issues can be tough - that ForkJoin pool may not be your friend Play Framework Play doesn t play nicely with Actor applications

27 CHALLENGES AND SURPRISES Cluster Singletons Inner classes and closures Customers would be great Cluster Singletons, except Cluster Singletons don t load balance Inner classes and closures are awesome, unless they get caught up in messages and force us to serialize the planet Application Performance issues Debugging asynchronous applications is still hard. Bulkhead your application to protect it from itself Tracking down performance issues can be tough - that ForkJoin pool may not be your friend Play Framework Future Correctness & Await.result Play doesn t play nicely with Actor applications Await.result is just evil. It s not sorta evil it s the devil with the flu.

28 MANAGING CLUSTER CUSTOMERS Our own Cluster Singleton

29 MANAGING CLUSTER CUSTOMERS Our own Cluster Singleton We created our own way of managing customers as single instances

30 MANAGING CLUSTER CUSTOMERS Our own Cluster Singleton We created our own way of managing customers as single instances The guy that manages the customers is a proper Akka Cluster Singleton

31 MANAGING CLUSTER CUSTOMERS Our own Cluster Singleton We created our own way of managing customers as single instances The guy that manages the customers is a proper Akka Cluster Singleton He manages the instantiation of the customers on-demand

32 MANAGING CLUSTER CUSTOMERS Our own Cluster Singleton We created our own way of managing customers as single instances The guy that manages the customers is a proper Akka Cluster Singleton He manages the instantiation of the customers on-demand He also reacts to Akka Cluster events (new nodes, dead nodes, etc ) and can rebalance customers as required

33 CLOSING OVER THE WORLD trait MyApp { case class Question(s: String) case class Answer(s: String)! class Server extends Actor { def receive = { case Question(q) => sender! Answer(s Sup? $q ) } } class Client(server: ActorRef) extends Actor { server! Question( Dood! ) Scala makes this just too damn easy } } def receive = { case Answer(a) => println(s Got $a ) }

34 CLOSING OVER THE WORLD trait MyApp { case class Question(s: String) case class Answer(s: String)! class Server extends Actor { def receive = { case Question(q) => sender! Answer(s Sup? $q ) } } class Client(server: ActorRef) extends Actor { server! Question( Dood! ) Scala makes this just too damn easy Chances are that Question and } } def receive = { case Answer(a) => println(s Got $a ) } Answer won t serialize

35 CLOSING OVER THE WORLD trait MyApp { case class Question(s: String) case class Answer(s: String)! class Server extends Actor { def receive = { case Question(q) => sender! Answer(s Sup? $q ) } } class Client(server: ActorRef) extends Actor { server! Question( Dood! ) Scala makes this just too damn easy Chances are that Question and } } def receive = { case Answer(a) => println(s Got $a ) } Answer won t serialize And even if they did, they d be huge

36 PLAY DOESN T PLAY NICELY Modular Design

37 PLAY DOESN T PLAY NICELY Modular Design Play doesn t start like just any other app Special developer mode Special run mode Either way, it s special

38 PLAY DOESN T PLAY NICELY Modular Design Play doesn t start like just any other app Special developer mode Special run mode Either way, it s special We want to start like a normal app But not for developers Frontend needs to start backed when the Play Plugin starts But not when it s running in Frontend standalone mode

39 PLAY DOESN T PLAY NICELY Modular Design Play doesn t start like just any other app Special developer mode Special run mode Either way, it s special We want to start like a normal app But not for developers Frontend needs to start backed when the Play Plugin starts But not when it s running in Frontend standalone mode Bootstrapping the configuration differs depending on how we start it

40 PLAY DOESN T PLAY NICELY Modular Design Modes for SBT Play doesn t start like just any other app Special developer mode Special run mode Either way, it s special Main app for Production We want to start like a normal app But not for developers Frontend needs to start backed when the Play Plugin starts But not when it s running in Frontend IDE Configurations standalone mode Bootstrapping the configuration differs depending on how we start it

41 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug

42 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug We had tons of work being done in the ForkJoin scheduler, but why?

43 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why?

44 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? We had a lot of timeouts occurring, but why?

45 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? A chain of N futures can time out, but knowing which one timed out isn t obvious We had a lot of timeouts occurring, but why?

46 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? A chain of N futures can time out, but knowing which one timed out isn t obvious We had a lot of timeouts occurring, but why? We had race conditions that shouldn t be there, but why?

47 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? We had a lot of timeouts occurring, but why? We had race conditions that shouldn t be there, but why? A chain of N futures can time out, but knowing which one timed out isn t obvious Message ordering guarantees are only between two Actors (adding a Future in the middle can warp your perception)

48 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? We had a lot of timeouts occurring, but why? We had race conditions that shouldn t be there, but why? A chain of N futures can time out, but knowing which one timed out isn t obvious Message ordering guarantees are only between two Actors (adding a Future in the middle can warp your perception) We had Exceptions being thrown in messaging processors but messages sent by what?

49 DEBUGGING IS HARD Captain Obvious Akka makes writing concurrent software easier and clearer, but it doesn t necessarily make it easier to debug The overhead of the Future had better be worth it! We had tons of work being done in the ForkJoin scheduler, but why? We had a lot of timeouts occurring, but why? We had race conditions that shouldn t be there, but why? A chain of N futures can time out, but knowing which one timed out isn t obvious Message ordering guarantees are only between two Actors (adding a Future in the middle can warp your perception) We had Exceptions being thrown in messaging processors but messages sent by what? Dropping a crappy message in a mailbox is perfectly correct; processing it is disconnected with queueing it

50 DEBUGGING IS HARD Side effecting Futures

51 DEBUGGING IS HARD Side effecting Futures If you re plumbing Futures to Actors, avoid side-effects Horrendous! dosomedbthing() onsuccess { case result => someactor! result }

52 DEBUGGING IS HARD Side effecting Futures If you re plumbing Futures to Actors, avoid side-effects Horrendous! Errors are lost, and onsuccess has no dosomedbthing() onsuccess { } case result => someactor! result return value

53 DEBUGGING IS HARD Side effecting Futures If you re plumbing Futures to Actors, avoid side-effects Horrendous! Errors are lost, and onsuccess has no dosomedbthing() onsuccess { } case result => someactor! result return value Always treat Futures as message transformers. Make a clear application protocol and stick with it Absolutely Lovely dosomedbthing() map { result => DBResult(result) } recover { case t: Throwable => DBCallFailed(t) } pipeto someactor

54 DEBUGGING IS HARD Side effecting Futures If you re plumbing Futures to Actors, avoid side-effects Horrendous! Errors are lost, and onsuccess has no dosomedbthing() onsuccess { } case result => someactor! result return value Always treat Futures as message transformers. Make a clear application protocol and stick with it This is easy to understand, catches the errors, and composes as well as asynchronous code can Absolutely Lovely dosomedbthing() map { result => DBResult(result) } recover { case t: Throwable => DBCallFailed(t) } pipeto someactor

55 DEBUGGING IS HARD Broken Promises

56 DEBUGGING IS HARD Broken Promises Most of us shouldn t be creating a promise

57 DEBUGGING IS HARD Broken Promises Most of us shouldn t be creating a promise If you do, make damn sure you complete that promise Horrendous! val p = Promise[String]() dosomedbthing() foreach { result => val s = result.get( somestringvalue ) p.success(s) }

58 DEBUGGING IS HARD Broken Promises Most of us shouldn t be creating a promise If you do, make damn sure you complete that promise Horrendous! Using the side effect operations on Futures val p = Promise[String]() dosomedbthing() foreach { result => val s = result.get( somestringvalue ) p.success(s) } (for example) is a pretty bad way to do it

59 DEBUGGING IS HARD Broken Promises Most of us shouldn t be creating a promise If you do, make damn sure you complete that promise Horrendous! Using the side effect operations on Futures val p = Promise[String]() dosomedbthing() foreach { result => val s = result.get( somestringvalue ) p.success(s) } (for example) is a pretty bad way to do it If you want to plumb a Future to a Promise, this is a much better approach Catches Everything val p = Promise[String]() p.completewith(dosomedbthing() map { _.get( somestringvalue ) })

60 DEBUGGING IS HARD Broken Promises Most of us shouldn t be creating a promise If you do, make damn sure you complete that promise Horrendous! Using the side effect operations on Futures val p = Promise[String]() dosomedbthing() foreach { result => val s = result.get( somestringvalue ) p.success(s) } (for example) is a pretty bad way to do it If you want to plumb a Future to a Promise, this is a much better approach Catches Everything val p = Promise[String]() p.completewith(dosomedbthing() map { _.get( somestringvalue ) }) If you re not plumbing a Future to a Promise then you re on your own to make sure it completes don t mess it up

61 DEBUGGING IS HARD The Lost Sender

62 DEBUGGING IS HARD The Lost Sender Think of the sender as the recipient of your response, not the subscriber to your stream

63 DEBUGGING IS HARD The Lost Sender Think of the sender as the recipient of your def receive = { case Subscribe() => addsubscriber(sender) sender! Subscribed() } response, not the subscriber to your stream

64 DEBUGGING IS HARD The Lost Sender Think of the sender as the recipient of your def receive = { case Subscribe() => addsubscriber(sender) sender! Subscribed() } response, not the subscriber to your stream So long as you always call it this way, you re probably OK (although no guarantees) OK publisher! Subscribe() def receive = { case Subscribed() => println( Keen! ) }

65 DEBUGGING IS HARD The Lost Sender Think of the sender as the recipient of your def receive = { case Subscribe() => addsubscriber(sender) sender! Subscribed() } response, not the subscriber to your stream So long as you always call it this way, you re probably OK (although no guarantees) But what if you ask? You re going to subscribe the temporary ask s Actor! OK #Fail publisher! Subscribe() def receive = { case Subscribed() => println( Keen! ) } publisher.ask(subscribe()).mapto[subscribed].foreach(println)

66 DEBUGGING IS HARD The Lost Sender Think of the sender as the recipient of your def receive = { case Subscribe(subscriber) => addsubscriber(subscriber) sender! Subscribed(subscriber) } response, not the subscriber to your stream So long as you always call it this way, you re probably OK (although no guarantees) But what if you ask? You re going to subscribe the temporary ask s Actor! The sender is just too flexible to treat this way, so make it part of your messaging protocol and you re good to go OK OK publisher! Subscribe(self) def receive = { case Subscribed(`self`) => println( Keen! ) } publisher.ask(subscribe(guy)).mapto[subscribed].foreach(println)

67 LESS THAN AWESOME TOOLING Tools aren t quite there

68 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM 10% Effective

69 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit 10% Effective 30% Effective

70 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created

71 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created That helps us understand how our app is working, but it s also been quite effective to understand how Akka / Java is working

72 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created That helps us understand how our app is working, but it s also been quite effective to understand how Akka / Java is working Bottom line we d still like better tools! :)

73 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that Where did this Future come from? we ve created That helps us understand how our app is working, but it s also been quite effective to understand how Akka / Java is working Bottom line we d still like better tools! :)

74 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created Where did this Future come from? What Futures have yet to complete? That helps us understand how our app is working, but it s also been quite effective to understand how Akka / Java is working Bottom line we d still like better tools! :)

75 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created That helps us understand how our app is Where did this Future come from? What Futures have yet to complete? Is this Actor busy? Blocked? working, but it s also been quite effective to understand how Akka / Java is working Bottom line we d still like better tools! :)

76 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created That helps us understand how our app is Where did this Future come from? What Futures have yet to complete? Is this Actor busy? Blocked? working, but it s also been quite effective to understand how Akka / Java is working Which Future timed out? Bottom line we d still like better tools! :)

77 LESS THAN AWESOME TOOLING Tools aren t quite there We used JVisualVM We used YourKit We built our own tools 10% Effective 30% Effective 50% Effective Our tools help us debug the proprietary programming language and execution that we ve created That helps us understand how our app is Where did this Future come from? What Futures have yet to complete? Is this Actor busy? Blocked? working, but it s also been quite effective to understand how Akka / Java is working Bottom line we d still like better tools! :) Which Future timed out? Where did this message come from?

78 BULKHEAD! One Thread Pool never works

79 BULKHEAD! One Thread Pool never works When we start, we code quickly, and JDBC calls go on the same thread pool as everything else

80 BULKHEAD! One Thread Pool never works When we start, we code quickly, and JDBC calls go on the same thread pool as everything else Your blocking calls start chewing up threads that Actors need

81 BULKHEAD! One Thread Pool never works When we start, we code quickly, and JDBC calls go on the same thread pool as everything else Your blocking calls start chewing up threads that Actors need So you increase the number of threads, and if they re ForkJoin threads, well we ll get to that

82 BULKHEAD! One Thread Pool never works When we start, we code quickly, and JDBC calls go on the same thread pool as everything else Your blocking calls start chewing up threads that Actors need So you increase the number of threads, and if they re ForkJoin threads, well we ll get to that I think we did it right. Go fast, figure things out, then pull the foundations together once you know what you need done

83 BULKHEAD! One Thread Pool never works When we start, we code quickly, and JDBC calls go on the same thread pool as everything else Your blocking calls start chewing up threads that Actors need So you increase the number of threads, and if they re ForkJoin threads, well we ll get to that I think we did it right. Go fast, figure things out, then pull the foundations together once you know what you need done The ExecutionContext is your friend!

84 THREADS, FUTURES AND FORKJOIN Managing Asynchrony

85 THREADS, FUTURES AND FORKJOIN Managing Asynchrony Got Futures?

86 THREADS, FUTURES AND FORKJOIN Managing Asynchrony Got Futures? Got (lotsa) Threads?

87 THREADS, FUTURES AND FORKJOIN Managing Asynchrony Got Futures? Got (lotsa) Threads? Got ForkJoin?

88 THREADS, FUTURES AND FORKJOIN Managing Asynchrony Got Futures? Got (lotsa) Threads? Got ForkJoin?!!! Haz Problemz!

89 THREADS, FUTURES AND FORKJOIN Managing Asynchrony The ForkJoin Pool s scheduler is O(n) in the number of threads Got Futures? Got (lotsa) Threads? Got ForkJoin?!!! Haz Problemz!

90 THREADS, FUTURES AND FORKJOIN Managing Asynchrony The ForkJoin Pool s scheduler is O(n) in the number of threads Got Futures? If you need lots of threads, for whatever reason, then scheduling work on the FJ Pool Got (lotsa) Threads? becomes a non-trivial part of your computation Got ForkJoin?!!! Haz Problemz!

91 THREADS, FUTURES AND FORKJOIN Managing Asynchrony The ForkJoin Pool s scheduler is O(n) in the number of threads Got Futures? If you need lots of threads, for whatever reason, then scheduling work on the FJ Pool Got (lotsa) Threads? becomes a non-trivial part of your computation Got ForkJoin? Do yourself a favour and configure a standard thread pool executor!!! Haz Problemz!

92 THREADS, FUTURES AND FORKJOIN Managing Asynchrony The ForkJoin Pool s scheduler is O(n) in the number of threads Got Futures? If you need lots of threads, for whatever reason, then scheduling work on the FJ Pool Got (lotsa) Threads? becomes a non-trivial part of your computation Got ForkJoin? Do yourself a favour and configure a standard thread pool executor!!! Haz Problemz! Our throughput doubled when we did that

93 AWAIT AND DIE Await!= synchronized

94 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block

95 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :)

96 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :)

97 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :) If you re waiting for the response to Your Message, how long are you gonna wait?

98 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :) If you re waiting for the response to Your Message, how long are you gonna wait? You re waiting behind all the other messages in that mailbox, plus fighting for time from all the other Actors in the system

99 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :) If you re waiting for the response to Your Message, how long are you gonna wait? You re waiting behind all the other messages in that mailbox, plus fighting for time from all the other Actors in the system AND you re chewing up a much-needed thread during that time!

100 AWAIT AND DIE Await!= synchronized Some people seem to think that an Await.result (on an Actor response) is like a synchronized block Some people are silly :) If you re waiting for the response to Your Message, how long are you gonna wait? You re waiting behind all the other messages in Auvik Bear Judges you! that mailbox, plus fighting for time from all the other Actors in the system AND you re chewing up a much-needed thread during that time!

101 OUR RULES OF THUMB

102 OUR RULES OF THUMB Avoid nested classes to help serialization

103 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests

104 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects

105 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code

106 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code Choose your thread pool types carefully (ForkJoin?)

107 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code Choose your thread pool types carefully (ForkJoin?) sender is the message recipient, not the subscriber to a stream

108 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code Choose your thread pool types carefully (ForkJoin?) sender is the message recipient, not the subscriber to a stream Keep things simple and just let it crash

109 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code Choose your thread pool types carefully (ForkJoin?) sender is the message recipient, not the subscriber to a stream Keep things simple and just let it crash Ask, How does that work when we re clustered?

110 OUR RULES OF THUMB Avoid nested classes to help serialization Never use Await well, you can in tests Keep Futures as pure as possible; no side-effects Bulkhead your IO and other nasty code Choose your thread pool types carefully (ForkJoin?) sender is the message recipient, not the subscriber to a stream Keep things simple and just let it crash Ask, How does that work when we re clustered? Instrument your code and measure, measure, Measure!

111 WE RE HAPPY! Akka powers our world Our distributed, asynchronous design is an Akka design

112 WE RE HAPPY! Akka powers our world Our distributed, asynchronous design is an Akka design Clustering and scaling out isn t a simple side effect of coding in Akka, but it s heavily simplified

113 WE RE HAPPY! Akka powers our world Our distributed, asynchronous design is an Akka design Clustering and scaling out isn t a simple side effect of coding in Akka, but it s heavily simplified Message-based network-oriented programming is the killer paradigm for the cloud

114 WE RE HAPPY! Akka powers our world Our distributed, asynchronous design is an Akka design Clustering and scaling out isn t a simple side effect of coding in Akka, but it s heavily simplified Message-based network-oriented programming is the killer paradigm for the cloud Akka has allowed us to program defensively by simply adopting the let it crash philosophy

115 WE RE HAPPY! Akka powers our world Our distributed, asynchronous design is an Akka design Clustering and scaling out isn t a simple side effect of coding in Akka, but it s heavily simplified Message-based network-oriented programming is the killer paradigm for the cloud Akka has allowed us to program defensively by simply adopting the let it crash philosophy Basically we re quite happy :)

116 Networks That Work.