Adv. Course in Programming Languages

Transcription

1 Department of Computer Science, University of Tsukuba No.3

2 How to write Code Generator for Power? In your favorite programming language!! C, C++ (with template), C# Fortran Lisp, Scheme, Clojure Java, Scala Perl, Python, Ruby, JavaScript SML, OCaml, F#, MetaOCaml Haskell (with template) Prolog etc. etc. Red ones have supports for quasiquotation or code generation.

3 How to write Code Generator for Power? In your favorite programming language!! C, C++ (with template), C# Fortran Lisp, Scheme, Clojure Java, Scala Perl, Python, Ruby, JavaScript SML, OCaml, F#, MetaOCaml Haskell (with template) Prolog etc. etc. Red ones have supports for quasiquotation or code generation.

4 Staged Power in Various Languages In the May 19th class: Oishi-kun s solution in Haskell (with template) Mano-kun s solution in Java (with string) Tanaka-kun s solution in C++ (with template) Yamaguchi-kun s solution in Terra and Lua

5 Staged Power in MetaOCaml let rec power n x = if n = 1 then x else x * (power (n-1) x)

6 Staged Power in MetaOCaml let rec power n x = if n = 1 then x else x * (power (n-1) x) let rec staged_power n x = if n = 1 then x else.<.~x *.~(staged_power (n-1).<x>.) >.

7 Staged Power in MetaOCaml let rec power n x = if n = 1 then x else x * (power (n-1) x) let rec staged_power n x = if n = 1 then x else.<.~x *.~(staged_power (n-1).<x>.) >. staged_power 3.<5>. =>.<.~<5>. *.~(staged_power 2.<5>. ) >. =>.< 5 *.~(staged_power 2.<5>. ) >.... =>.<5 * (5 * 5)>.

8 Computation in MetaOCaml Code in brackets are (usually) not evaluated:.<1 + 2>. reduces to.<1 + 2>. (not computed) But escaped expression in brackets are evaluated: let x =.<3 + 4>. in.<1 +.~x * 2>. =>.<1 +.~(.<3+4>.) * 2)>. =>.<1 + (3+4) * 2)>. And that s all!

9 Another Example of Program Generation Matrix-vector multiplication: a a 1n b 1 c =... a m1... a mn b n c m where c i = Σ 1 j n a ij b j

10 Another Example of Program Generation Matrix-vector multiplication: a a 1n b 1 c =... a m1... a mn b n c m where or c i = Σ 1 j n a ij b j c(i) = Σ 1 j n a(i, j) b(j)

11 Implementing matrix-vector multiplication let mvmul nrow ncol mat1 vec1 vec2 = for i = 0 to nrow - 1 do let tmp = ref 0.0 in for j = 0 to ncol - 1 do tmp :=!tmp +. mat1.(i * ncol + j) *. vec1.(j) done; vec2.(i) <-!tmp done OCaml specific: for i = 0 to n-1 do.. done for for-loop. let tmp = ref 0.0 for creating mutable cell.! tmp for the value in tmp. ary.(e) for the e-th element of array.

12 How to write a program generator? First of all, we need to know, which data are known statically. Situation 1: we don t know anything. Situation 2: we know the size of matrix/vector, but don t know the elements of matrix/vector. Situation 3: we know the size of matrix/vector and the matrix elements, but don t know vector elements. Situation 3 : (switch matrix/vector) Situation 4: we know everything. Program generation is effective for situations 2 and 3, where we know something and don t know something. But, sometimes, we can still use Program Generation techniques in Situation 1 (next page).

13 How to write a program generator? We know nothing about the data, but know the architecture of our machine. cache size; if we know the cache size, we can divide a very large matrix into small components which fit in the cache. target CPU; if we know the CPU has some special instruction such as vector instructions, we may generate an efficient code using them. Then we can still generate an efficient code suitable for each particular architecture.

14 Matrix-Vector Multiplication; Scenario 1 Suppose we know the size of matrix/vector, but don t know their elements. Also assume that the size is not too large. let mvmuls1 nrow ncol mat1 vec1 vec2 = gen_for 0 (nrow - 1) (fun row ->.<let tmp = ref 0.0 in begin.~(gen_for 0 (ncol - 1) (fun col ->.<tmp :=!tmp +. (.~mat1).(row * ncol + col) *. (.~vec1).(col)>.)); (.~vec2).(row) <-!tmp end>.)

15 Matrix-Vector Multiplication; Scenario 1 A helper function: let gen_for init fin loop_body =... in gen_for 1 3 (fun i ->.<foo i>.) ==>.<foo 1; foo 2; foo 3>.

16 Matrix-Vector Multiplication; Scenario 1 Generated code: mvmul1 =.<fun x_201 y_202 z_203 -> let tmp_208 = ref 0.0 in tmp_208 :=! tmp_ x_201.(0 * 6 + 0) *. y_202.(0); tmp_208 :=! tmp_ x_201.(0 * 6 + 1) *. y_202.(1); tmp_208 :=! tmp_ x_201.(0 * 6 + 2) *. y_202.(2); tmp_208 :=! tmp_ x_201.(0 * 6 + 3) *. y_202.(3); tmp_208 :=! tmp_ x_201.(0 * 6 + 4) *. y_202.(4); tmp_208 :=! tmp_ x_201.(0 * 6 + 5) *. y_202.(5); (); z_203.(0) <-! tmp_208;...>. Completely unrolling the loop.

17 Matrix-Vector Multiplication; Scenario 1 We can do better; to pre-compute the index 0*6+0 etc.... let idx = row * ncol + col in.<tmp :=!tmp +. (.~mat1).(idx) *. (.~vec1).(col)>.));... The value of idx is computed at the time of code generation, and its result is embedded into the code.

18 Matrix-Vector Multiplication; Scenario 1 Generated code: tmp_208 :=! tmp_ x_201.(0 * 6 + 0) *. y_202.(0); mvmul1 =.<fun x_201 y_202 z_203 -> mvmul2 =.<fun x_209 y_210 z_211 ->... tmp_216 :=! tmp_ x_209.(0) *. y_210.(0);...

19 Matrix-Vector Multiplication; Scenario 2 Suppose we know the size of matrix/vector and the elements of matrix, but don t know the elements of vector. Again, we assume that the size is not too large. let mvmuls3 nrow ncol mat1 vec1 vec2 =... let v = mat1.(row * ncol + col) in.<tmp :=!tmp +. v *. (.~vec1).(col)>.));... Now mat1 is static, and vec1 remains dynamic.

20 Matrix-Vector Multiplication; Scenario 2 Generated code: val mvmul3 =.<fun x_217 y_218 -> let tmp_223 = ref 0.0 in tmp_223 :=! tmp_ *. x_217.(0); tmp_223 :=! tmp_ *. x_217.(1); tmp_223 :=! tmp_ *. x_217.(2); tmp_223 :=! tmp_ *. x_217.(3); tmp_223 :=! tmp_ *. x_217.(4); tmp_223 :=! tmp_ *. x_217.(5); (); y_218.(0) <-! tmp_223;...>.

21 Matrix-Vector Multiplication; Scenario 2 If the matrix contains many occurrences of 0.0 and 1.0, then we want to optimize v * 1.0 to v, v * 0.0 to 0.0 and v to v. Generated code: val mvmul4 =.<fun mat1 x_255 y_256 -> (let tmp_261 = Pervasives.ref 0.0 in (tmp_261 := ((! tmp_261) +. (x_255.(0))); (); (); (); y_256.(0) <-! tmp_261);... ()>. These optimization can be done by the compiler, so in an ordinary program we do not have to care about them. But in program generation, we MUST avoid code explosion, that is, generating too large code.

22 Matrix-Vector Multiplication; Scenario 3 We can have other scenarios such as: size and matrix elements are known, vector elements are unknown. the matrix is too large, and we don t want to fully unroll the loop (to avoid code explosion). for a row which has less than four non-zero elements, we want to unroll it. Otherwise, we keep the for-loop as it is. Idealized Generated code: val mvmul5 =.<fun mat1 vec1 vec2 -> vec2.(0) <- 2.0 *. vec1.(2) *. vec1.(4) * vec2.(1) <- 6.0 *. vec1.(0) *. vec1.(1); let tmp = ref 0.0 in for col = 0 to 5 do tmp :=! tmp +. (mat1.(12 + col) *. vec1.(col); done; vec2.(2) <- tmp

23 Automatic vs Hand-Written Fully automatic; compilers, partial evaluators easy for programmers relatively easy to establish the correctness uniform optimization is not quite optimal for each specific domain Hand-written; staged computation hard for programmers relatively hard to establish the correctness domain-specific optimization

24 Automatic vs Hand-Written Should we write everything by hand? YES, for custom optimization. No, for automatic staging (partial evaluation).

25 Exercise Write a code-generator in your favorite programming language: C++ template Ruby (using eval function) MetaOCaml (3rd week) Scala (lightweight modular staging, 4th week) or others (any language is ok) Example application: power function, matrix multiplication, gibonacci function (x, y, x + y, x + 2y, 2x + 3y, ) Send your answer file by to me before 5th week s class, and give a presentation about it in the class.