Advances in Programming Languages: Efficiency

Transcription

1 Advances in Programming Languages: Efficiency Stephen Gilmore The University of Edinburgh March 1, 2007 Overview Computer programs should execute efficiently. The art and skill of computer programming lies in making programs which execute efficiently while also ensuring that they are reliable. Anyone can get the wrong answers fast. Efficiency and correctness are sometimes in opposition. To craft a program which executes more efficiently we might have to research or invent a better algorithm to solve the problem, or we might have to invest more time and effort in understanding our chosen compiler. However, as a general rule of thumb, more efficient algorithms are often more complex than less efficient ones, and may be more difficult to implement correctly. In addition to this there is also a welldocumented history of compilers introducing errors by including optimisations, so using a large number of combinations of switch-controlled optimisations might also introduce errors into our code. Set against this is another peril. It is becoming increasingly difficult for programmers to understand how they can optimise their programs. This is because modern compilers are much more complex than were the compilers in the frontier days of the wild west. Processors too are much more complex, with pipelined architectures which execute many instructions simultaneously. Efficiency and programming languages Choosing efficiency as an object of study, we might ask how high a price we pay for using safe programming languages, and whether more better compilers can reduce these disadvantages. We compare C, Java, C#, Cyclone and Caml using a microbenchmark, Takeuchi s function. Takeuchi s function is a three-argument recursive integer function which eventually returns the value of its first argument. As a benchmark, the important properties of this function are that its running time may be very long, without using a large amount of stack space. Quite obviously, Takeuchi s function is not a typical program which practicising developers would be commissioned to create. Benchmarks are either realistic examples which test a range of qualities of the implementation at a time, or they are microbenchmarks which test only a few qualities. Takeuchi s function tests the efficiency with which languages call functions (invoke methods) and handle simple conditional branching. These are things which even real-world programs need to do as well. One way to make this comparison is through measurement. Essentially, we just run the program and time it. Our timings will depend on (at least) three things: 1. how the program is implemented; 1

2 UG4 Advances in Programming Languages 2005/ how the program is compiled; and 3. how the measurements are obtained. We look at each of these in each case. Takeuchi s function implemented in C Takeuchi s function is implemented in C below. /* File: programs/c/takeuchi.c */ #include <stdio.h> #include <stdlib.h> int tak(int x, int y, int z) { else return tak(tak(x-1,y,z),tak(y-1,z,x),tak(z-1,x,y)); int main() { printf("%d\n", tak(16,6,1)); exit(0); Makefile for the C code We can control the behaviour of the C compiler through command-line arguments. Below, the important one is -O3, which asks for as many optimisations as possible. Thus, we produce two binaries and expect the tak.opt executable to run faster than the tak executable. # File: programs/c/makefile.timings GCC=gcc -Wall ${GCC -o tak Takeuchi.c ${GCC -O3 -o tak.opt Takeuchi.c (${TIME./tak) 2> c.timings (${TIME./tak.opt) 2>> c.timings Timings for the C code To obtain this, and all other timings, we have used the GNU time command version 1.7. This demonstrates that the optimised version runs faster than the version without optimisations turned on, as we might have expected. These, and each of the subsequent timings, were obtained on the machine slim.inf.ed.ac.uk user 0.04system 0:27.52elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (78major+14minor)pagefaults 0swaps

3 UG4 Advances in Programming Languages 2005/ user 0.02system 0:26.09elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (78major+14minor)pagefaults 0swaps Takeuchi s function implemented in C# Takeuchi s function is implemented in C# below. The requirement to have the class sealed is analogous to final in Java; this class can have no subclasses. Making the tak method both static and public gives the compiler the license to make as many optimisations as possible. /* File: programs/cs/takeuchi.cs */ using System; public sealed class Takeuchi { public static int tak(int x, int y, int z) { else return tak(tak(x-1,y,z), tak(y-1,z,x), tak(z-1,x,y)); public static void Main(string[] args) { Console.WriteLine(tak(16,6,1)); Makefile for the C# code In the version of the Mono C# compiler which I used (version ) I was unable to find the command-line option to turn on compiler optimisations. The obvious choices (-O3 and -optimise) did not work. So it could be that the code could be compiled better than I managed. # File: programs/cs/makefile.timings MCS=mcs -warn:4 ${MCS -o Takeuchi.exe Takeuchi.cs (${TIME mint Takeuchi.exe) 2> cs.timings (${TIME mono Takeuchi.exe) 2>> cs.timings Timings for the C# code Running the code in the mint interpreter runs much more slowly than the mono JIT, as might be expected user 0.38system 36:26.16elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (619major+402minor)pagefaults 0swaps 30.18user 0.01system 0:30.77elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (713major+423minor)pagefaults 0swaps

4 UG4 Advances in Programming Languages 2005/ Takeuchi s function implemented in Java The tak function is implemented as a public, static, final method. /* File: programs/java/takeuchi.java */ public final class Takeuchi { public static final int tak(int x, int y, int z) { else return tak(tak(x-1,y,z), tak(y-1,z,x), tak(z-1,x,y)); public static void main(string[] args) { System.out.println(tak(16,6,1)); Makefile for the Java code This class was compiled both with SUN s Java compiler, javac (version ) and the GNU gcj compiler (version ). # File: programs/java/makefile.timings GCJ=gcj -O3 GCJLINK=gcj -c -g -O3 javac Takeuchi.java ${GCJLINK Takeuchi.java ${GCJ --main=takeuchi -o tak Takeuchi.o (${TIME java -Xint Takeuchi) 2> java.timings (${TIME java Takeuchi) 2>> java.timings (${TIME./tak) 2>> java.timings Timings for the Java code The compiled Java versions were run in the interpreter (with the JIT turned off); with the JIT; and as a native code executable. Unsurprisingly, the interpreter runs slowest, although not as slowly as the Mono interpreter. Rather more surprisingly SUN s Java HotSpot(TM) Client VM (build b06) executes faster than the native code version compiled by GCJ, more than repaying the cost of running the JIT itself user 0.18system 5:54.33elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1264major+540minor)pagefaults 0swaps 33.19user 0.17system 0:34.36elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1266major+540minor)pagefaults 0swaps 57.25user 0.03system 0:59.49elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1360major+160minor)pagefaults 0swaps

5 UG4 Advances in Programming Languages 2005/ Takeuchi s function implemented in Cyclone The C implementation of Takeuchi s function can be used unchanged with Cyclone. /* File: programs/cyclone/takeuchi.cyc */ #include <stdio.h> #include <stdlib.h> int tak(int x, int y, int z) { else return tak(tak(x-1,y,z),tak(y-1,z,x),tak(z-1,x,y)); int main() { printf("%d\n", tak(16,6,1)); exit(0); Makefile for the Cyclone code The Makefile for the Cyclone code is very similar to the C Makefile. # File: programs/cyclone/makefile.timings CYCLONE=cyclone -Wall ${CYCLONE -o tak Takeuchi.cyc ${CYCLONE -O3 -o tak.opt Takeuchi.cyc (${TIME./tak) 2> cyclone.timings (${TIME./tak.opt) 2>> cyclone.timings Timings for the Cyclone code Perhaps surprisingly, the optimised Cyclone code outperforms the optimised GCC code. Only slightly, but it does outperform it user 0.05system 0:29.82elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (97major+41minor)pagefaults 0swaps 24.99user 0.06system 0:25.76elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (97major+41minor)pagefaults 0swaps Takeuchi s function in Objective Caml (* File: programs/caml/takeuchi.ml *) let rec tak(x,y,z) = if x<=y then y else tak(tak(x-1,y,z),tak(y-1,z,x),tak(z-1,x,y));; print_endline(string_of_int(tak(16,6,1)));;

6 UG4 Advances in Programming Languages 2005/ Makefile for the Objective Caml code The Makefile for the Objective Caml code uses the bytecode compiler ocamlc and the native code compiler ocamlopt. # File: programs/caml/makefile.timings ocamlc -o tak Takeuchi.ml ocamlopt -o tak.opt Takeuchi.ml (${TIME ocaml Takeuchi.ml) 2> caml.timings (${TIME./tak) 2>> caml.timings (${TIME./tak.opt) 2>> caml.timings Timings for the Objective Caml code The first run of the Objective Caml implementation includes compiling and running the Caml bytecode. The second runs the pre-compiled bytecode. The final run is the optimised native code, which significantly outperforms the C implementation user 0.28system 4:59.59elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (156major+487minor)pagefaults 0swaps user 0.21system 4:57.81elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (152major+120minor)pagefaults 0swaps 17.56user 0.01system 0:17.58elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (115major+26minor)pagefaults 0swaps Takeuchi s function in Ruby # File: programs/ruby/takeuchi.rb def tak(x,y,z) if (x <= y) then y else tak(tak(x-1,y,z),tak(y-1,z,x),tak(z-1,x,y)) end end puts (tak(16,6,1)) Makefile for the Ruby implementation The Makefile for the Ruby implementation simply runs the Ruby interpreter on the Ruby source code. This is compiled to bytecode and executed on the Ruby virtual machine. (/usr/bin/time ruby Takeuchi.rb) 2> ruby.timings

7 UG4 Advances in Programming Languages 2005/ Timings for the Ruby implementation The Ruby implementation is significantly slower than all of the others, with a run-time of nearly three hours (compared to, for example, 17 seconds with Objective Caml) user system 2:54:28elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+511minor)pagefaults 0swaps Results The timings from all of the runs are shown below, with the two slowest timings (for the Mono interpreter and the Ruby virtual machine) omitted to preserve the scale. Ocamlopt Cyclone O3 GCC O3 GCC Cyclone Mono C# Java GCJ Ocamlc Ocaml Java Xint Summary When one talks about software performance and predicts how optimisations can be applied pretty much only two things are certain 1. your intuitions are wrong; and 2. you need to implement and time an optimisation before you can be sure of it. The quest for efficiency should not be allowed to overtake the quest for reliability. Slow code that works is preferred over fast code that does not every time. Interpreted bytecode programs are often slower than programs compiled to native code. Programs in type-safe languages (such as Objective Caml and Cyclone) are not always slower than their equivalents in weakly-typed languages such as C. Compile-time static type-checking incurs no run-time cost.