Building Flexible Systems

Transcription

1 Building Flexible Systems with Clojure and Datomic Stuart Sierra Cognitect

2 We don t want to paint ourselves into a corner

3

4

5 Clojure

6 Flexible Systems Fact-based Context-free Non-exclusive Observable

7 Fact Based

8 Fact Based Fact = statement about the world Cannot be invalidated

9 public class Person { private List<Person> friends; public void addfriend(person newfriend);

10 mutable? public class Person { private List<Person> friends; public void addfriend(person newfriend) { if (friends.length() < 500) friends.add(newfriend); else... race condition?

11

12 Universal Data Collection: Set #{ } Ordered: Vector ["Alice" "Bob"] Command or stack: List (println user) Structured: Map {:name "Bob" :age 42} Associative: Map {412 "a" 114 "b"}

13 map {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} vector of maps

14 (defn add-friend [person new-friend] (if (< (count (:friends person)) 500) (update person :friends conj new-friend) {:error :too-friendly}))

15 define function name parameters (defn add-friend [person new-friend] list symbol vector

16 condition (if (< (count (:friends person)) 500)

17 condition (if (< (count (:friends person)) 500) (update person :friends conj new-friend) then

18 {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} navigation (update person :friends conj new-friend)

19 [{:name "Alice"} {:name "Bob"}] conjoin vector conj new-friend) [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

20 {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} update map (update person :friends conj new-friend) {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

21 {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} universal data structures (update person :friends conj new-friend) universal operations {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

22 {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} (add-friend person {:name "Claire"}) domain rule in terms of universal data and operations {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

23 (A Few) Universal Operations nth get map concat keys filter cycle union vals remove interleave difference assoc reduce interpose intersection assoc-in replace distinct select update some flatten subset? update-in sort group-by superset? dissoc shuffle partition join merge reverse split-at project merge-with take split-with index select-keys drop frequencies rename

24 Immutable Values (defn add-friend [person new-friend] (if (< (count (:friends person)) 500) no race condition (update person :friends conj new-friend)...)) immutable value

25 value function value

26 value function value function value Time

27 identity value function value function value Time

28 stable live identity value function value function value Time

29 (let [current-user (atom {:name "Kelly Q." :friends [{:name "Alice" :name "Bob"}]})] current-user {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

30 dereference (let [friends (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:name friend)])])) current-user {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

31 dereference (let [friends (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:name friend)])])) immutable value {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

32 identity (swap! current-user add-friend {:name "Claire"}) change state pure function {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} current-user add-friend {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

33 connection storage service

34 (db connection) connection database value storage service

35 database value (let [db (db connection)] (query db...) (query db...) (query db...)

36 connection database value transact database value transact database value Time

37 Universal Data Entity Attribute Value kelly :name "Kelly Q." kelly :friends alice kelly :friends bob

38 Universal Data in Time Entity Attribute Value Op Tx kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000

39 Universal Data in Time Entity Attribute Value Op Tx kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000 kelly :friends bob retract 1023 kelly :friends claire add 1023

40 Current State Entity Attribute Value Op Tx kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 {:name "Kelly Q." kelly :friends bob add 1000 :friends [{:name "Alice"} {:name "Claire"}]} kelly :friends bob retract 1023 kelly :friends claire add 1023

41 State as-of Last Week Entity Attribute Value Op Tx kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000 {:name "Kelly Q." kelly :friends bob retract 1023 :friends [{:name "Alice"} {:name "Bob"}]} kelly :friends claire add 1023

42 (swap! current-user add-friend...) change state identity pure function (transact connection [[:add-friend...]])

43 stable live identity database value transact database value transact database value Time

44 Fact Based Fact = statement about the world Cannot be invalidated Immutable values Using uniform data structures Incorporating time

45 Context Free

46 Context Free Values are complete and self-describing No out-of-band knowledge required to handle correctly

47 {:id "1234abcd" :name "T.J."} {:id "abcd" :balance 42.00} {:id 7980 :region "AUS"}

48 {:customer/id "1234abcd" :customer/name "T.J."} {:billing.customer/id "abcd" :billing.customer/balance 42.00} {:widgetco.customer/id 7980 :widgetco/region "AUS"} namespaced keywords

49 (merge customer billing-customer widgetco-customer) {:customer/id "1234abcd" :customer/name "T.J." :billing.customer/id "abcd" :billing.customer/balance :widgetco.customer/id 7980 :widgetco/region "AUS"}

50 clojure.spec (s/def :customer/id (s/and string? #(re-matches #"[0-9a-e]{8}" %))) (s/def :customer/name (s/and string? #(not (str/blank? %)))) (s/def ::Customer (s/keys :req [:customer/id] :opt [:customer/name]]))

51 (let [user {:customer/name ""}] (s/explain ::Customer user)) val: {:customer/name ""} fails spec: ::Customer predicate: (contains? % :customer/id) In: [:customer/name] val: "" fails spec: :customer/name at: [:customer/name] predicate: (not (blank? %))

52 (pull db [:customer/name :customer/start-date {:customer/account [:account/id :account/balance]}] customer) {:customer/name "T.J." :customer/start-date #inst" " :customer/account {:account/id :account/balance 4200}}

53 (defui UserWidget static om/iquery om.next (query [this] [:user/name {:user/friends [:user/name]}]) Object (render [this] (let [friends (:user/friends (om/props this))] (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:user/name friend)])]))))

54 Context Free Values are complete and self-describing No out-of-band knowledge required to handle correctly Use namespaces for globally-unique labels Let consumers control interactions

55

56 Non-exclusive

57 Non-exclusive Do not obstruct data evolution Continue correct operation in the presence of unexpected input

58 input function input output

59 input function input extra extra output extra extra

60 condition (defn gold-status [customer] (if (< 100 (:customer/orders customer)) then: augment and return (assoc customer :loyalty/tier :gold) customer)) else: return unchanged

61 clojure.spec (s/fdef gold-status :args (s/cat :customer (s/keys :req [:customer/id :customer/orders]))) minimal required keys

62 (gold-status {:customer/id "abcd1234" :customer/balance 4200}) ExceptionInfo Call to #'user/gold-status did not conform to spec: In: [0] val: {:customer/id "abcd1234", :customer/balance 4200} fails at: [:args :customer] predicate: (contains? % :customer/orders)

63 (gold-status {:customer/id "abcd1234" :customer/orders 123 :customer/balance 4200}) {:customer/id "abcd1234" :customer/orders 123 :customer/balance 4200 :status :gold} extra data passed through

64 Non-exclusive Do not obstruct data evolution Continue correct operation in the presence of unexpected input Enforce only minimum input requirements Ignore and pass through extra input

65

66 <?xml version="1.0" encoding="utf-8"?> <description xmlns=" targetnamespace= " xmlns:tns= " xmlns:ghns = " xmlns:wsoap= " xmlns:soap=" xmlns:wsdlx= " <types> <xs:schema xmlns:xs=" targetnamespace=" ressvc" xmlns=" <xs:element name="checkavailability" type="tcheckavailability <xs:complextype name="tcheckavailability"> <xs:sequence> <xs:element name="checkindate" type="xs:date"/> <xs:element name="checkoutdate" type="xs:date"/> <xs:element name="roomtype" type="xs:string"/> </xs:sequence>

67 Observable

68 Observable Internal state of the system can be recreated at any point Output of the system is sufficient to recover its state

69 Internal state

70 value function

71 current state new state function new input output

72 (defn handle [state request] ;; compute new state ;; decide how to respond (-> (update state...) (assoc :response...))

73 current state new state function new input output log

74 {:uri "datomic:ddb:us-east-1..." :t 12345}

75 state {:uri "datomic:ddb:us-east-1..." :t 12345} input {:method :post :path "/users" :params {:name "Leslie"}} (let [conn (connect uri) db (as-of (db conn) t)] (handle db request))

76 {:request-id :method :post :path "/users" :query-params {:name "Leslie"}} [[:db/add user :user/name "Leslie"] [:db/add tx :tx/request-id ]]

77 Observable Internal state of the system can be recreated at any point Output of the system is sufficient to recover its state Minimize necessary information to completely describe internal state Store inputs at boundaries

78

79 Flexible Systems Fact-based Immutable values, in domain terms, with time Context-free Namespaces; consumer control Non-exclusive Enforce minimum, pass-through extra Observable Log input, point-in-time state, provenance

80 The New Normal blog series, Mike Nygard Case Studies: cognitect.com, datomic.com clojure.org ClojureTV on YouTube stuartsierra.com