Stuart

Transcription

1 Clojure Time Stuart Copyright Relevance, Inc. This presentation is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. See 1

2 the plan modeling the world total control model concurrency and parallelism an approach 2 2

3 time when things happen before/after later concurrency now relative 3 3

4 identity continuity over time built by minds sameness across a series of perceptions not a name, but can be named can be composite 4 4

5 perception becoming aware via the senses uncoordinated provides values (snapshots) contemplate values for as long as you like 5 5

6 values units of perception points in time of identities immutable possibly composite 6 6

7 action change to identity(ies) over time independent of other perceivers makes new values available to perceivers many possible semantics might be coordinated 7 7

8 total control model global, total control one processor one memory anything else is deep voodoo roll-your-own time model 8 8

9 difficulties with rolling your own time identity perception values action present unreliable, past nonexistent locking + convention ad hoc (copying?) class-level convention? side effects everywhere 9 9

10 the problems with convention 10 10

11 add concurrency and parallelism to the mix 11 11

12 where did this dangerous assumption that Parallelism == Concurrency come from?

13 we need to switch mental models 13 13

14 Clojure s approach 14 14

15 syntax 15 15

16 atomic data types type example java equivalent string "foo" String character \f Character regex #"fo*" Pattern integer 42 Long a.r. integer 42N BigInteger double Double a.p. double M BigDecimal boolean true Boolean nil nil null ratio 22/7 N/A symbol foo, + N/A 16 keyword :foo, ::foo N/A 16

17 data literals type properties example list vector map singly-linked, insert at front indexed, insert at rear key/value (1 2 3) [1 2 3] {:a 100 :b 90} set key #{:a :b} 17 17

18 function call semantics: fn call arg (println "Hello World") structure: symbol string list 18 18

19 function definition define a fn fn name docstring arguments (defn greet "Returns a friendly greeting" [your-name] (str "Hello, " your-name)) fn body 19 19

20 it's all data symbol symbol string vector (defn greet "Returns a friendly greeting" [your-name] (str "Hello, " your-name)) list 20 20

21 metadata prefix with ^ class name or arbitrary map (defn ^String greet "Returns a friendly greeting" [your-name] (str "Hello, " your-name)) 21 21

22 values 22 22

23 persistent data structures immutable change by function application maintain performance guarantees full-fidelity old versions 23 23

24 persistent example: linked list your my list list newt tern rabbit tiger 24 24

25 bit-partitioned tries your trie my trie 25 25

26 32-way tries

27 identities 27 27

28 refs explicit semantic ref value value ref value state 28 28

29 life with variables? variable??? variable??? 29 29

30 perceptions 30 30

31 Epochal Time Model Process events (pure functions) F F F v1 v2 v3 v4 States Identity (succession of states) (immutable values) Observers/perception/memory 31 31

32 actions 32 32

33 unified update model (change-state ref fn [args*]) fn gets current state of ref fn return becomes next state of ref snapshot always available no user locking no deadlocks writers never impede readers 33 33

34 unified update ;refs (dosync (alter foo assoc :a "lucy")) ;agents (send foo assoc :a "lucy") ;atoms (swap! foo assoc :a "lucy") 34 34

35 atoms 35 35

36 cas as time construct F vn vn+1 F F vns v2 v3 v4 AtomicReference 36 (swap! an-atom f args) (f vn args) becomes vn+1 - can automate spin 1:1 timeline/identity Point-in-time value Atomic state succession perception 36

37

38 board is just a value (defn new-board "Create a new board with about half the cells set to :on." ([] (apply new-board dim-board)) ([dim-x dim-y] (for [x (range dim-x)] (for [y (range dim-y)] (if (< 50 (rand-int 100)) :on :off)))))) distinct bodies by arity 38 38

39 update is just a function rules (defn step "Advance the automation by one step, updating all cells." [board] (doall (map (fn [window] (apply #(doall (apply map rules %&)) (doall (map torus-window window)))) (torus-window board)))) cursor over previous, me, next 39 39

40 state is trivial identity initial value (let [stage (atom (new-board))]...) (defn update-stage "Update the automaton." [stage] (swap! stage step)) apply a fn update fn 40 40

41 software transactional memory 41 41

42 stm as time construct v1 v2 v3 v4 v1 v2 v3 v4 v1 v2 v3 v4 v1 v2 v3 v4 F F F F Transactions F F F F F F F F 42 42

43 ref example: chat identity (def messages (ref [])) initial value 43 43

44 reading value (deref messages) => => [] 44 44

45 alter (alter r update-fn & args) oldval r (apply update-fn oldval args) newval 45 45

46 updating apply an... (defn add-message [msg] (dosync (alter messages conj msg))) scope a...update fn transaction 46 46

47 stms are not all created equal 47 47

48 Clojure s stm not lock free uses locks, latches, to avoid churn deadlock detection & barging no read tracking readers never impede writers nobody ever impedes readers commute ensure 48 48

49 agents 49 49

50 agents as time construct F F F F F F vn vn+1 (send aref f args) returns immediately vns queue enforces serialization 50 (f vn args) becomes vn+1 happens asynchronously in thread pool thread 1:1 timeline/identity Atomic state succession Point-in-time value perception 50

51 agents are not actors actors can distribute this queue F F F F F F vn vn+1 vns but at a price: 1. no snapshot observers X X 2. no deadlock prevention 51 51

52 locks stm actors is not a useful partition 52 52

53 richer taxonomy semantics leverage locality presume distance uncoordinated synchronous atoms N/A coordinated synchronous stm, pods N/A uncoordinated asynchronous agents actors 53 53

54 the devil is in the details 54 54

55 thread ready all Clojure constructs work from any thread fns are Callable and Runnable use (future...) to throw work to a thread pool (defn fight [term1 term2] (let [r1 (future (estimated-hits-for term1)) r2 (future (estimated-hits-for term2))] (future

56 transients 56 Persistent data structures are slower in sequential use (especially writing ) But - no one can see what happens inside F vn I.e. the birthing process of the next value can use our old (and new) performance tricks: F vn+1 Mutation and parallelism Parallel map on persistent vector same speed as loop on j.u.arraylist on quad-core Safe transient versions of PDS possible, with O(1) conversions between persistent/transient 56

57 use commute when update can happen anytime 57 57

58 not safe for commute (defn next-id "Get the next available id." [] (dosync (alter ids inc))) 58 58

59 safe! (defn increment-counter "Bump the internal count." [] (dosync (alter ids inc)) nil) 59 59

60 prefer send-off if agent ops might block 60 60

61 send caller (send a fn & args) agent fixed thread pool (apply fn oldval args) 61 61

62 send-off caller agent cached thread pool (send a fn & args) (apply fn oldval args) 62 62

63 use ref-set to set initial/base state 63 63

64 unified update, revisited update mechanism pure function application pure function (commutative) pure function (blocking) ref atom agent alter swap! send commute send-off setter ref-set reset!

65 validation 65 65

66 create a function that checks every item... (def validate-message-list (partial every? #(and (:sender %) (:text %)))) (def messages for some criteria (ref () :validator validate-message-list)) and associate fn with updates to a ref 66 66

67 sending to agents from within transactions 67 67

68 tying agent to a tx (defn add-message-with-backup [msg] (dosync (let [snapshot (alter messages conj msg)] (send-off backup-agent (fn [filename]!!! (spit filename snapshot)!!! filename)) snapshot))) exactly once if tx succeeds 68 68

69 where are we? time model beats control model resembles reality more closely makes design/code/test easier in general now is a good time to switch easier concurrency easier parallelism Clojure s provides an approach that is unified multi-faceted 69 69

70 thanks!