Above the Clouds: Introducing Akka. Jonas Bonér Scalable Solutions

Transcription

1 Above the Clouds: Introducing Akka Jonas Bonér Scalable Solutions

2 The problem It is way too hard to build: 1. correct highly concurrent systems 2. truly scalable systems 3. fault-tolerant systems that self-heals...using state-of-the-art tools

3 Introducing akka

4 Vision Simpler Concurrency Scalability Fault-tolerance

5 Vision...with a single unified Programming model Runtime service

6 Manage system overload

7 Scale up & Scale out

8 Replicate and distribute for fault-tolerance

9 Transparent load balancing

10 ARCHITECTURE CORE SERVICES

11 ARCHITECTURE ADD-ON MODULES

12 ARCHITECTURE CLOUDY AKKA

13 WHERE IS AKKA USED? SOME EXAMPLES: FINANCE Stock trend Analysis & Simulation Event-driven messaging systems BETTING & GAMING Massive multiplayer online gaming High throughput and transactional betting TELECOM Streaming media network gateways SIMULATION 3D simulation engines E-COMMERCE Social media community sites

14 What is an Actor?

15 Actor Behavior State

16 Event-driven Thread Actor Behavior State

17 Event-driven Thread Actor Behavior State

18 Event-driven Thread Actor Behavior State

19 Event-driven Thread Actor Behavior State

20 Event-driven Thread Actor Behavior State

21 Actor Behavior State

22 Event-driven Thread Actor Behavior State

23 Akka Actors one tool in the toolbox

24 Actors!"#$%&'($!)!"#$%!*"##!&'()*+,!$+)$,-#!-$*',!.!!."/!$'()*+,!/!0!!-$0!,+$+#1+!/!.!!!!!"#$!"#$%!/2!!!!!!!$'()*+,!3/!4!!!!!!5,#)*6)7$'()*+,8!!9 9

25 Create Actors."*!$'()*+,!/!:$*',;<=&'()*+,> $'()*+, is an -$*',?+<

26 Start

27 Stop

28 Send:! $'()*+,!B!"#$%! fire-forget

29 Send:!!! CC!,+*(,)A!:!<(*(,+."*!<(*(,+!/!:$*',!BBB!D+AA:E+ returns the Future directly

30 <(*(,+4!,+$+#1+!.!!!"#$!J''7R:,8!/2!S<''T

31

32 Future."*!<(*(,+4!/!0&/!.!!:X!Y)*!!!!Z[!:$*',!BBB!\]+66'\!CC!,+*(,)A!^!!RX!M*,#)E!Z[!:$*',!BBB!:!!!!!!!CC!,+*(,)A!\40\!!$X!M*,#)E!Z[!:$*',!BBB!_!!!!!!!CC!,+*(,)A!\4V\ 9!12$*-!R!3!\[\!3!$."*!<(*(,+H!/!0&/!.!!:!Z[!:$*',!BBB!?+`7\]+66'\8!$'66+$*!.!$:A+!?+A7QX!Y)*8!!!!/2!Q!9!!R!Z[!:$*',!BBB!?+`7:8!!!!!!!$'66+$*!.!$:A+!?+A7QX!M*,#)E8!/2!Q!9!!$!Z[!:$*',!BBB!?+`7_8!!!!!!!$'66+$*!.!$:A+!?+A7QX!M*,#)E8!/2!Q!9 9!12$*-!R!3!\[\!3!$

33 Dataflow <6'F!.!!W!ZZ!Q78!3!L78!!5,#)*6)7\W!/!\!3!W788 9 <6'F!.!Q!ZZ!V0!9 <6'F!.!L!ZZ!H!9

34 uses Future under the hood and blocks until timeout or completion

35 9

36 HotSwap 9

37 HotSwap 9

38 HotSwap

39 Set 9 :$*',@U#A5:*$O+,!/!U#A5:*$O+,!CC!R+<',+!A*:,*+U

40 Remote Actors

41 Remote Server CC!(A+!O'A*!f!5',*!#)!$')<#E Scalable implementation based on NIO (Netty) & Protobuf

42 Two types of remote actors Client initiated & managed Server initiated & managed

43 Client-managed supervision works across nodes A+,1#$+!B!K+AA:E+

44 Server-managed register and manage actor on server client gets dumb proxy handle server part

45 A+,1#$+!B!K+AA:E+ client part

46 Server-managed server part

47 Remoting in Akka 1.0 Remote Actors Client-managed Server-managed Problem Deployment (local vs remote) is a dev decision We get a fixed and hard-coded topology Can t change it dynamically and adaptively Needs to be a deployment & runtime decision

48 Clustered Actors (in development for upcoming Akka 2.0)

49 Address 1:6!:$*',!/!:$*',;<=DL-$*',>7S315#$/.2!$T8 Bind the actor to a virtual address

50 Deployment Actor address is virtual and decoupled from how it is deployed If no deployment configuration exists then actor is deployed as local The same system can be configured as distributed without code change (even change at runtime) Write as local but deploy as distributed in the cloud without code change Allows runtime to dynamically and adaptively change topology

51 Deployment configuration :%%:!.!!:$*',!.!!!!U+56'LK+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:A*[$5(\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!O'K+!!!!!!/!\)'U+X*+A*[)'U+[4\!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9

52 Deployment configuration :%%:!. Address!!:$*',!.!!!!U+56'LK+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:A*[$5(\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!O'K+!!!!!!/!\)'U+X*+A*[)'U+[4\!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9

53 Deployment configuration :%%:!. Address Type of!!:$*',!.!!!!u+56'lk+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:a*[$5(\!!!!!!!!$6(a*+,+u!.!!!!!!!!!!o'k+!!!!!!/!\)'u+x*+a*[)'u+[4\!!!!!!!!!!,+56#$:a!!/!i!!!!!!!!!!a*:*+6+aa!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 load-balancing

54 Deployment configuration Clustered or Local :%%:!. Address Type of!!:$*',!.!!!!u+56'lk+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:a*[$5(\!!!!!!!!$6(a*+,+u!.!!!!!!!!!!o'k+!!!!!!/!\)'u+x*+a*[)'u+[4\!!!!!!!!!!,+56#$:a!!/!i!!!!!!!!!!a*:*+6+aa!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 load-balancing

55 Deployment configuration Clustered or Local :%%:!. Address Type of!!:$*',!.!!!!u+56'lk+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:a*[$5(\!!!!!!!!$6(a*+,+u!.!!!!!!!!!!o'k+!!!!!!/!\)'u+x*+a*[)'u+[4\!!!!!!!!!!,+56#$:a!!/!i!!!!!!!!!!a*:*+6+aa!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 load-balancing Home node

56 Deployment configuration Clustered or Local :%%:!. Address Type of!!:$*',!.!!!!u+56'lk+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:a*[$5(\!!!!!!!!$6(a*+,+u!.!!!!!!!!!!o'k+!!!!!!/!\)'u+x*+a*[)'u+[4\!!!!!!!!!!,+56#$:a!!/!i!!!!!!!!!!a*:*+6+aa!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 in cluster load-balancing Home node Nr of replicas

57 Deployment configuration Clustered or Local :%%:!. Address Type of!!:$*',!.!!!!u+56'lk+)*!.!!!!!!315#$/.2!$!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\6+:a*[$5(\!!!!!!!!$6(a*+,+u!.!!!!!!!!!!o'k+!!!!!!/!\)'u+x*+a*[)'u+[4\!!!!!!!!!!,+56#$:a!!/!i!!!!!!!!!!a*:*+6+aa!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 in cluster Stateful or Stateless load-balancing Home node Nr of replicas

58 The runtime provides Subscription-based cluster membership service Highly available cluster registry for actors Automatic cluster-wide deployment Highly available centralized configuration service Automatic replication with automatic fail-over upon node crash Transparent and user-configurable load-balancing Transparent adaptive cluster rebalancing Leader election Durable mailboxes - guaranteed delivery

59 Upcoming features Publish/Subscribe (broker and broker-less) Compute Grid (MapReduce) Data Grid (querying etc) Distributed STM (Transactional Memory) Event Sourcing

60 Akka Node

61 Akka Node."*!5#)E!/!:$*',;<=a#)E>7S5#)ET8."*!5')E!/!:$*',;<=a')E>7S5')ET8 5#)E!B!e:6675')E8

62 Akka Node."*!5#)E!/!:$*',;<=a#)E>7S5#)ET8."*!5')E!/!:$*',;<=a')E>7S5')ET8 5#)E!B!e:6675')E8 Ping Pong

63 Akka Node Akka Cluster Node Ping Pong

64 Akka Cluster Node Akka Node Akka Cluster Node Akka Cluster Node Ping Pong Akka Cluster Node Akka Cluster Node

65 Akka Cluster Node Akka Cluster Node Akka Cluster Node Ping Pong Akka Cluster Node Akka Cluster Node

66 Akka Cluster Node Akka Cluster Node Ping :%%:!.!!:$*',!.!!!!U+56'LK+)*!.!!!!!!5#)E!.9!!!!!!5')E!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\,'()U[,'R#)\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 Pong Akka Cluster Node Akka Cluster Node Akka Cluster Node

67 Akka Cluster Node Akka Ping Cluster Node :%%:!.!!:$*',!.!!!!U+56'LK+)*!.!!!!!!5#)E!.9!!!!!!5')E!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\,'()U[,'R#)\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 Pong Akka Cluster Node Akka Cluster Node Akka Cluster Node

68 Akka Pong Cluster Node Akka Ping Cluster Node :%%:!.!!:$*',!.!!!!U+56'LK+)*!.!!!!!!5#)E!.9!!!!!!5')E!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\,'()U[,'R#)\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!')!!!!!!!!9!!!!!!9!!!!9!!9 9 Akka Pong Cluster Node Akka Pong Cluster Node Akka Cluster Node

69 Akka Pong Cluster Node Akka Ping Cluster Node :%%:!.!!:$*',!.!!!!U+56'LK+)*!.!!!!!!5#)E!.9!!!!!!5')E!.!!!!!!!!!!!!!!!!!!!!!!!!!,'(*+,!/!\,'()U[,'R#)\!!!!!!!!$6(A*+,+U!.!!!!!!!!!!,+56#$:A!!/!I!!!!!!!!!!A*:*+6+AA!/!') ZooKeeper!!!!!!!!9 Ensemble!!!!!!9!!!!9!!9 9 Akka Pong Cluster Node Akka Pong Cluster Node Akka Cluster Node

70 Let it crash fault-tolerance

71 The Erlang model

72 9 nines

73 ...let s take a standard OO application

74

75 Which components have critically important state and explicit error handling?

76

77 Classification of State Scratch data Static data Supplied at boot time Supplied by other components Dynamic data Data possible to recompute Input from other sources; data that is impossible to recompute

78 Classification of State Scratch data Static data Supplied at boot time Supplied by other components Dynamic data Data possible to recompute Input from other sources; data that is impossible to recompute

79 Classification of State Scratch data Static datamust be Supplied at boot time protected Supplied by other components Dynamic by datany means Data possible to recompute Input from other sources; data that is impossible to recompute

80 Fault-tolerant onion-layered Error Kernel

81 Error Kernel

82 Error Kernel

83 Error Kernel

84 Error Kernel

85 Error Kernel

86 Error Kernel

87 Error Kernel

88 Error Kernel

89 Error Kernel

90 Error Kernel

91 Error Kernel

92 Error Kernel

93 Error Kernel

94 Error Kernel

95 Error Kernel

96

97

98 Node 1 Node 2

99 Linking 6#)%7:$*',8 ()6#)%7:$*',8 A*:,*i#)%7:$*',8 A5:F)i#)%=DL-$*',>

100 Fault handlers -66J',;)+M*,:*+EL7!!+,,',AG!!K:Qc,;<?+*,#+AG!!!F#*O#)"#K+?:)E+8 ;)+J',;)+M*,:*+EL7!!+,,',AG!!K:Qc,;<?+*,#+AG!!!F#*O#)"#K+?:)E+8

101 Supervision!*"##!M(5+,1#A',!$+)$,-#!-$*',!.!!<:(6*]:)U6+,!/!-66J',;)+M*,:*+EL7!!!!i#A*7$6:AA;<=Y66+E:6M*:*+PQ$+5*#')>8!!!!!^G!^00088!!-$0!,+$+#1+!/!.!!!!!"#$!?+E#A*+,7:$*',8!/2!6#)%7:$*',8!!9 9

102 Manage 9

103 ...and much much more HTTP STM Camel FSM Microkernel Guice JTA OSGi Dataflow AMQP scalaz Spring Security

104 Akka 1.1 To be released today

105 Get it and learn more

106 EOF