Event-sourced architectures with Luminis Technologies

Transcription

1 Event-sourced architectures with Luminis Technologies

2 Today's journey Event-sourcing Actors Akka Persistence Design for ES

3 Event-sourcing Is all about getting the facts straight

4 Typical 3 layer architecture UI/Client Service layer fetch modify store Database

5 Typical 3 layer architecture UI/Client Service layer fetch modify store Database Databases are shared mutable state

6 Typical entity modelling Concert artist: String date: Date availabletickets: int price: int... 1 * TicketOrder nooftickets: int userid: String

7 Typical entity modelling Concert artist = Aerosmith availabletickets = 100 price = TicketOrder TicketOrder nooftickets = 3 TicketOrder userid = 1 nooftickets = 3 userid nooftickets = 1 = 3 userid = 1

8 Typical entity modelling Concert artist = Aerosmith availabletickets = 100 price = TicketOrder TicketOrder nooftickets = 3 TicketOrder userid = 1 nooftickets = 3 userid nooftickets = 1 = 3 userid = 1 Changing the price

9 Typical entity modelling Concert artist = Aerosmith availabletickets = 100 price = TicketOrder TicketOrder nooftickets = 3 TicketOrder userid = 1 nooftickets = 3 userid nooftickets = 1 = 3 userid = 1 Changing the price

10 Typical entity modelling Concert artist = Aerosmith availabletickets = 100 price = TicketOrder TicketOrder nooftickets = 3 TicketOrder userid = 1 nooftickets = 3 userid nooftickets = 1 = 3 userid = 1 Canceling an order

11 Typical entity modelling Concert artist = Aerosmith availabletickets = 100 price = TicketOrder TicketOrder nooftickets = 3 userid = 1 nooftickets = 3 userid = 1 Canceling an order

12 Update or delete statements in your app? Congratulations, you are LOSING DATA EVERY DAY

13 Event-sourced modelling time ConcertCreated TicketsOrdered TicketsOrdered TicketsOrdered artist = Aerosmith availabletickets = 100 price = 10 nooftickets nooftickets = 3 = 3 nooftickets = 3 userid userid = 1= 1 userid = 1...

14 Event-sourced modelling time ConcertCreated TicketsOrdered TicketsOrdered TicketsOrdered artist = Aerosmith availabletickets = 100 price = 10 nooftickets nooftickets = 3 = 3 nooftickets = 3 userid userid = 1= 1 userid = 1... Changing the price

15 Event-sourced modelling time ConcertCreated TicketsOrdered TicketsOrdered TicketsOrdered PriceChanged artist = Aerosmith availabletickets = 100 price = nooftickets nooftickets = 3 = 3 nooftickets = 3 userid userid = 1= 1 userid = 1 price = 100 Changing the price

16 Event-sourced modelling time ConcertCreated TicketsOrdered TicketsOrdered TicketsOrdered PriceChanged artist = Aerosmith availabletickets = 100 price = nooftickets nooftickets = 3 = 3 nooftickets = 3 userid userid = 1= 1 userid = 1 price = 100 Canceling an order

17 Event-sourced modelling time ConcertCreated TicketsOrdered TicketsOrdered PriceChanged OrderCancelled TicketsOrdered artist = Aerosmith nooftickets nooftickets = 3 = 3 price = 100 userid = 1 nooftickets = 3 availabletickets = 100 userid userid = 1= 1 userid = 1 price = Canceling an order

18 Event-sourced modelling Immutable events Append-only storage (scalable) Replay events: reconstruct historic state Events as audit mechanism Events as integration mechanism

19 Event-sourcing: capture all changes to application state as a sequence of events Events: where from?

20 Commands & Events Do something (active) Can be rejected (validation) Can be responded to It happened. Deal with it. (facts)

21 Querying & event-sourcing How do you query a log?

22 Querying & event-sourcing How do you query a log? Command Query Responsibility Segregation

23 CQRS without ES Command Query UI/Client Service layer Database

24 CQRS without ES Command Query Command Query UI/Client UI/Client Command Query Service layer Model Model(s)? Database Datastore Datastore Datastore Datastore

25 Event-sourced CQRS Command Query UI/Client Command Model Query Model(s) Events Journal Datastore Datastore Datastore

26 Actors Mature and open source Scala & Java API Akka Cluster

27 Actors "an island of consistency in a sea of concurrency" async message send Actor mailbox state behavior Process message: update state send messages change behavior Don't worry about concurrency

28 Actors A good fit for event-sourcing? Actor mailbox state is transient mailbox is non-durable state (lost messages) behavior

29 Actors Just store all incoming messages? async message send Actor mailbox state behavior Problems with command-sourcing: side-effects poisonous (failing) messages store in journal

30 Persistence Experimental Akka module Scala & Java API Actor state persistence based on event-sourcing

31 Persistent Actor async message send (command) PersistentActor actor-id state event event Derive events from commands Store events Update state Perform sideeffects journal (actor-id)

32 Persistent Actor PersistentActor actor-id state Recover by replaying events, that update the state (no side-effects) event event journal (actor-id)

33 Persistent Actor case object Increment case object Incremented // command // event class CounterActor extends PersistentActor { def persistenceid = "counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") val receiverecover: Receive = { case Incremented => state += 1

34 Persistent Actor case object Increment case object Incremented // command // event class CounterActor extends PersistentActor { def persistenceid = "counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") val receiverecover: Receive = { case Incremented => state += 1

35 Persistent Actor case object Increment case object Incremented // command // event class CounterActor extends PersistentActor { def persistenceid = "counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") async callback (but safe to close over state) val receiverecover: Receive = { case Incremented => state += 1

36 Persistent Actor case object Increment case object Incremented // command // event class CounterActor extends PersistentActor { def persistenceid = "counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") val receiverecover: Receive = { case Incremented => state += 1

37 Persistent Actor case object Increment case object Incremented // command // event class CounterActor extends PersistentActor { def persistenceid = "counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") Isn't recovery with lots of events slow? val receiverecover: Receive = { case Incremented => state += 1

38 Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceid = "snapshotting-counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") case "takesnapshot" => savesnapshot(state) val receiverecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotstate: Int) => state = snapshotstate

39 Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceid = "snapshotting-counter" var state = 0 val receivecommand: Receive = { case Increment => persist(incremented) { evt => state += 1 println("incremented") case "takesnapshot" => savesnapshot(state) val receiverecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotstate: Int) => state = snapshotstate

40 Plugins: Journal Cassandra & Snapshot Cassandra Kafka Kafka DynamoDB MongoDB MongoDB HBase HBase EventStore EventStore JDBC JDBC

41 Plugins: Serialization Default: Java serialization Pluggable through Akka: Protobuf Kryo Avro Your own

42 Persistent View Persistent Actor Persistent View Persistent View Views poll the journal Eventually consistent Polling configurable Actor may be inactive Views track single persistence-id journal Views can have own other snapshots snapshot store datastore

43 Persistent View case object ComplexQuery class CounterView extends PersistentView { override def persistenceid: String = "counter" override def viewid: String = "counter-view" var querystate = 0 def receive: Receive = { case Incremented if ispersistent => { querystate = someverycomplicatedcalculation(querystate) // Or update a document/graph/relational database case ComplexQuery => { sender()! querystate; // Or perform specialized query on datastore

44 Persistent View case object ComplexQuery class CounterView extends PersistentView { override def persistenceid: String = "counter" override def viewid: String = "counter-view" var querystate = 0 def receive: Receive = { case Incremented if ispersistent => { querystate = someverycomplicatedcalculation(querystate) // Or update a document/graph/relational database case ComplexQuery => { sender()! querystate; // Or perform specialized query on datastore

45 Persistent View case object ComplexQuery class CounterView extends PersistentView { override def persistenceid: String = "counter" override def viewid: String = "counter-view" var querystate = 0 def receive: Receive = { case Incremented if ispersistent => { querystate = someverycomplicatedcalculation(querystate) // Or update a document/graph/relational database case ComplexQuery => { sender()! querystate; // Or perform specialized query on datastore

46 Sell concert tickets bit.ly/akka-es Commands: CreateConcert BuyTickets ChangePrice AddCapacity ConcertActor price availabletickets starttime salesrecords ConcertHistoryView $50 $75 $100 journal

47 Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere Cluster node Cluster node Cluster node Persistent Persistent Actor Actor id = "1" id = "1" Persistent Persistent Actor Actor id = "1" id = "2" Persistent Persistent Actor Actor id = "1" id = "3" distributed journal

48 Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere How are persistent actors distributed over cluster? Cluster node Cluster node Cluster node Persistent Persistent Actor Actor id = "1" id = "1" Persistent Persistent Actor Actor id = "1" id = "2" Persistent Persistent Actor Actor id = "1" id = "3" distributed journal

49 Scaling out: Akka Cluster Sharding: Coordinator assigns ShardRegions to nodes (consistent hashing) Actors in shard can be activated/passivated, rebalanced Cluster node Cluster node Cluster node Coordinator Persistent Persistent Actor Actor id = "1" id = "1" Shard Region Persistent Persistent Actor Actor id = "1" id = "2" Shard Region Persistent Persistent Actor Actor id = "1" id = "3" Shard Region distributed journal

50 Design for event-sourcing

51 DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate Eventual Consistency Aggregate Root entity Root entity entity entity entity entity entity

52 DDD+CQRS+ES DDD: Domain Driven Design Persistent Fully consistent Aggregate Actor Eventual Consistency Persistent Fully consistent Aggregate Actor Root entity Message passing Root entity entity entity entity entity entity

53 DDD+CQRS+ES DDD: Domain Driven Design Persistent Fully consistent Aggregate Actor Eventual Consistency Persistent Fully consistent Aggregate Actor Root entity Message passing Root entity entity entity entity entity entity Akka Persistence is not a DDD/CQRS framework But it comes awfully close

54 Designing aggregates Focus on events Structural representation(s) follow ERD

55 Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views)

56 Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views) With CQRS read-your-writes is not the default When you need this, model it in a single Aggregate

57 Designing commands Self-contained Unit of atomic change Granularity and intent

58 Designing commands Self-contained Unit of atomic change Granularity and intent UpdateAddress street =... city =... vs ChangeStreet street =... ChangeCity street =...

59 Designing commands Self-contained Unit of atomic change Granularity and intent UpdateAddress street =... city =... vs ChangeStreet street =... ChangeCity street =... vs Move street =... city =...

60 Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1

61 Versioning Actor logic: v1 and v2 Actor logic: v2 event v2 event v2 event v2 event v2 event v1 event event v1 v2 event v1 event event v1 v2

62 Versioning Actor logic: v1 and v2 Actor logic: v2 Actor logic: v2 event v2 event v2 event v2 event v2 event v2 event v2 event v1 event event v1 v2 snapshot event v1 event v1 event event v1 v2 event v1

63 Versioning Actor logic: Actor logic: Actor logic: Be backwards v1 and v2 v2 v2 compatible Avro/Protobuf event v2 event v2 event v2 Serializer can do translation event v2 event v2 event v2 snapshot event v1 event event v1 v2 event v1 Snapshot versioning: event v1 event event v1 v2 event v1 harder

64 In conclusion Event-sourcing is... Powerful but unfamiliar

65 In conclusion Event-sourcing is... Powerful Combines well with UI/Client Command Model Journal Events Query Model Datastore Datastore Datastore but unfamiliar DDD/CQRS

66 In conclusion Event-sourcing is... Powerful Combines well with Not a UI/Client Command Model Journal Events Query Model Datastore Datastore Datastore but unfamiliar DDD/CQRS

67 In conclusion Akka Actors & Akka Persistence A good fit for event-sourcing Experimental, view improvements needed async message send (command) PersistentActor actor-id state event event journal (actor-id)

68 Thank you. Luminis Technologies