March 3, 2014
(DP) what is declarativeness lists, trees iterative comutation recursive computation (DC) DP and DC in Haskell and other languages 2 / 32
Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) 3 / 32
Some quotes Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) Recent Effective Scala from Twitter developers http://twitter.github.com/effectivescala/ Use Futures to manage. Futures allow the programmer to express concurrent computation in a declarative style If an immutable collection will do, use it... reasoning about them in a concurrent context is simple. The Java memory model is a subtle beast, but luckily we can avoid all of these pitfalls by using the declarative style What is declarative style, future and immutable collection and how are they special? 4 / 32
What is declarativeness? Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) An operation with input and output is declarative Same input always gives the same output Examples independent (of outside state) stateless (no inside state) deterministic ( fixed control-flow) typical algebraic operations (+,-,*,/) if-then-else pure functions 5 / 32
ness is important Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) Compositional plug declarative component operation arguments results Reasoning is simple Rest of computation understand component behaviour alone no need to consider external/internal state 6 / 32
Classification of declarative Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) descriptive (XML, HTML) programmable (Turing complete) observational (just declarative interface) definitional (declarative internals ) functional logic declarative model (DP in Oz) (not every functional language is declarative, e.g. Scheme) 7 / 32
In Clojure (and other FP) Some quotes What is declarativeness? ness is important Classification of declarative In Clojure (and other FP) variables are immutable function arguments, loop variables, local (let) variables except def, defn creates global mutable variables standard data are immutable (persistent) list, vector, map, set pure functions no side effects: i.e. no IO/shared state read or write 8 / 32
Immutable linked lists Appending lists Trees with DV 9 / 32
Immutable linked lists Immutable linked lists Appending lists Trees with DV (list 3 4 7) or with quoting (3 4 7) actual representation (cons 3 (cons 4 (cons 7 ()))) empty list () ends the list cons pair (cons H T) consists of head H references one list value tail T references the rest of the list first/rest accessing head and tail destructuring: extract head and tail (let [[H & T] (3 4 7)] (println T) ) prints (4 7) changing an element in the list is impossible adding head (cons 11 lst), also (conj lst 11) 10 / 32
Appending lists Immutable linked lists Appending lists Trees with DV Appending a=(1 2 3) b=(6 7) must result in copying the first list (cons 1 (cons 2 (cons 3 b))) 1 2 3 6 7 1 2 3 Append one list to another by just reassigning the tail? No! the first list must stay immutable whoever has reference to it should not see changes the second list may be appended to any other whoever has reference to it sees no changes 11 / 32
Trees Immutable linked lists Appending lists Trees with DV (defrecord tree [value left right]) Java class with 3 immutable fields Having (tree. 3 :leaf (tree. 33 :leaf :leaf)) tree 3 want to replace 3 with 45 leaf tree 33 leaf leaf 12 / 32
Trees Immutable linked lists Appending lists Trees with DV (defrecord tree [value left right]) Java class with 3 immutable fields Having (tree. 3 :leaf (tree. 33 :leaf :leaf)) tree 3 want to replace 3 with 45 tree 45 leaf tree 33 leaf leaf need to create new node and point to the subtrees in general, path from root to node must be reconstructed whoever references (part of) the tree never sees any changes 12 / 32
with DV Immutable linked lists Appending lists Trees with DV dataflow variables may be unbound reader is suspended until bound dataflow variables are single-assignment reader will never see different values Considering all above we can treat DVs as immutable unless checking with IsDet operation, which is not part of declarative model 13 / 32
Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) 14 / 32
Iterative computation (1) Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) How to iterate over values with single-assignment variable? with recursion each (recursive) call creates new variables for the arguments (defn iteration [X Sum] (if (>= X 10) Sum ; stop and return result (do ; else (println X) (iteration (inc X) (+ Sum X))))) (println (iteration 0 0)) 15 / 32
Iterative computation (2) Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) On the third iteration "Result" "X" "Sum" x 1 0 s 1 0 x 2 1 s 2 0 x 3 2 s 3 1 current call new values are created on the stack in each call recursion stops at x 10 old x i and s i can safely be used by external code/threads (unlike in imperative PLs) Java final keyword for closures 16 / 32
Clojure recur Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) avoid stack grow with recur must be in tail position calls current function with tail call optimization (TCO) JVM does not allow TCO, Clojure has to generate iterative code instead of recursive calls (defn iteration [i s] (if (>= i 10) s (do ; else (println i) (recur (inc i) (+ s i)) ))) (println (iteration 0 0)) 17 / 32
Recursive computation (1) Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) Naive implementations are often wasteful stack grows because of append is not in tail position (defn append [Ls Ms] (if (= Ls ()) Ms (cons (first Ls) (append (rest Ls) Ms)))) (println (append (1 5 3) (3 2 3))) Naive definitions are often slow (defn reverse [Xs] (if (= Xs ()) () (append (reverse (rest Xs)) (list (first Xs)) ))) (println (reverse (1 2 3 4))) 18 / 32
Recursive computation (2) Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) Use accumulators and tail recursion (defn- reverse2_iter [Rs Ys] (if (= Ys ()) Rs ; return accumulated result (recur (cons (first Ys) Rs) (rest Ys)))) (defn reverse2 [Xs] (reverse2_iter () Xs)) ; empty accumulator (println (reverse2 (1 4 3 2))) Rs is an accumulator for the new list Recursive call is in tail position 19 / 32
Higher-order (1) Reverse the list [1,2,7] 1 2 7 Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) [] 1 [] 2 1 [] 7 2 1 [] {P.} {P.} {P.} accumulator values from R0=[] to R3=7 2 1 [] can be also viewed as state transform from S 0 to S 3 tansformation takes In and X and returns Out {P In X?Out} In X {P.} Out 20 / 32
Higher-order (2) Iterative computation (1) Iterative computation (2) Clojure recur Recursive computation (1) Recursive computation (2) Higher-order (1) Higher-order (2) Define the generic function proc {ForAllAcc L P In?Out} case L of nil then Out=In [] X L2 then Mid in % Mid connects In and Out {P In X Mid} {ForAllAcc L2 P Mid Out} end end % element to head, accumulated to tail proc {RCons In X?Out} Out=X In end fun {Reverse3 L} {ForAllAcc L RCons nil} end ForAllAcc is actually procedural FoldL state transform view: X may be incoming message 21 / 32
thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions 22 / 32
thread block thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions thread... end block around statement (expression) runs it in a separate thread declare A B C A=thread 4+5 end thread B=5+6 end C=thread {Abs thread 6+5 end} end threads in Oz are green threads thousands of threads are ok Mozart (Oz VM) is not able to run on multiple cores 23 / 32
Dataflow between threads thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions variable set in one thread can be read in another declare A B thread A=B+1 end B=5 thread is suspended until B is bound B is bound by the main thread Thats all! model is still declarative i.e. same results for the same inputs result completely deterministic evaluation order becomes non-deterministic 24 / 32
Evaluation order thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions control flow driven eager arguments are evaluated before a call C, Java, Oz lazy arguments are evaluated when needed Haskell data-driven Oz ByNeed operation operation is executed when data is available dataflow computers 25 / 32
Determinism of control/data flow thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions single-assignment doing its magic works indendent of evaluation order in an iteration state is evolving not changing data never changes 26 / 32
Limitation: exceptions thread block Dataflow between threads Evaluation order Determinism of control/data flow Limitation: exceptions exceptions may break declarative model observable non-determinism is introduced declare A thread A=5 end thread A=4 end A is bound to 4 or 5 depending on thread scheduling exception is thrown in unlucky thread 27 / 32
Haskell (1) Haskell (2) Haskell (3) Intel ArBB 28 / 32
Haskell (1) Haskell (1) Haskell (2) Haskell (3) Intel ArBB Parallel Haskell has 2 operations par a b sparks a and evaluates b returns b sparks adds to FIFO of planned evaluations may be evaluated on other core/processor pseq a b evaluates a and then b returns b the order is important, becuase generally Haskell compiler may not respect the order All mentioned evaluations are done up to Weak Head Normal Form (WHNF) 29 / 32
Haskell (2) Haskell (1) Haskell (2) Haskell (3) Intel ArBB cutoff = 35 fib :: Int -> Integer fib 0 = 0 fib 1 = 1 fib n = fib (n-1) + fib (n-2) fib :: Int -> Integer fib n n < cutoff = fib n otherwise = r par (l pseq l + r) where l = fib (n-1) r = fib (n-2) par sparks r pseq makes sure that main thread continues on l both return its second (right hand) argument first (left hand) argument just ensures some semantics 30 / 32
Haskell (3) Haskell (1) Haskell (2) Haskell (3) Intel ArBB Haskell can run on multiple cores There are many caveats par and pseq only evaluate up to WHNF it is enough for our fibonacci algorithm not enough for most other evaluation needs to be forced by help functions Acheiving faster programs is possible but not easy need to clearly understand Haskell execution rules 31 / 32
Intel ArBB Haskell (1) Haskell (2) Haskell (3) Intel ArBB ArBB is from another world data parallel languages its data are immutable it aims for deterministic task parallelism The lesson: immutability and determinism are the friends of anywhere. 32 / 32