1EC 2001 1 / 1 P1 Engineering Computation David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/1ec Hilary 2001
1EC 2001 2 / 1 Algorithms: Design, Constructs and Correctness
1EC 2001 3 / 1 Algorithm design A computation is an operation or set of operations that takes a set of input symbols I and from them produces some desired set of output symbols O. O G(I) For any high level computation to run on a machine, it must be broken down into a sequence of elementary computations which are accessible to the machine. You might write that G = E n...e 2 E 1 This breaking down is done in a number of stages. Fortunately the final stages are performed automatically by ever cleverer compilers, and often we need only be responsible for the design down to the level of a high-level programming language.
1EC 2001 4 / 1 Step 1: Specification of Inputs and Outputs Often the first step in the design of a computation G is to define it implicitly by defining its inputs I and outputs O For example At a high level: Input an ordered list of employees, their hourly pay rates, and their number of hours worked in a month; output a similarly ordered list of monthly pay. Or at a low level: Input a 32-bit integer; output its rightshift. You ll see that these statements say nothing explicitly about the nitty-gritty of how to achieve the transformation.
1EC 2001 5 / 1 Step 2: Algorithm Design The second (broad) stage is to design an Algorithm. An algorithm is simply a recipe which conforms to a particular computing genre. At the initial substages, the algorithm may include phrases in English and expressions in mathematics. At a later stage, those phrases might become comments, and be replaced by statements which embody the constructs common to all programming languages but specific to none. 1: Determine S the sum of all n of the θ i s 1: //Determine S the sum of all n of the θ i s 2: S n i=1 θ i 1: // Determine S the sum of all n of the theta i s 2: S = 0 3: for i = 1 to n step 1 do Notice that already in the example we have assumed that algorithm is following the imperative or procedural genre. In this the designer assumes control of what the machine does next, albeit in the light of data. There is a very sharp separation between control and data here, whereas in other genres the distinction is blurred.
1EC 2001 6 / 1 Step 3: Program D,B and T Only now does one enter the stage of program design. With good algorithm design, the transfer to code should be smooth, although it may be that the different data and control structures a particular level language affords make the transition more or less easy. At a particular level and usually this means the high level most programming languages share a set of common control constructs. More often the choice of language is made by looking at the way that data can be represented, and the way that data structures can be manipulated. Specify Inputs and Outputs (Natural Lang) Feedback Program Test against spec ( Algorithm Design Blackboard Lang) Program Build Program Design (Programming Lang)
1EC 2001 7 / 1 Control constructs If we think of how a imperative language runs on a machine, two constructs which are required are assignments to move data from one place to another conditional jumps so that the order of execution can be changed depending on the result of a computation. To these we add for convenience loops run the same code many times; subroutines run the same code from many places; We now spend more time looking at the four main constructs, and clarify the stylized blackboard language which we have used throughout these lectures. Loop Assignments Conditionals Subroutine
1EC 2001 8 / 1 Construct 1: Assignments We noted earlier most of the programming languages in common use are imperative languages. In these we specify what happens in terms of a sequence of changing values of program variables. An assignment consists of a variable name, followed by an equals or sign, followed by an expression. The expression is evaluated and the value given to the variable name. For example x k 355/a Assignments do not alter the flow of the program. Hence, thusfar we can only execute a sequence of statements in the order given, with no possibility of variation from that sequence.
1EC 2001 9 / 1 Construct 2: Conditionals One way of achieving variation is by testing a condition, and branching according to whether the condition is met or not. The conditional construct can take a number of forms depending on whether or not further tests are required. The simplest form is: IF if condition then statement; statement;... end if but when a block of statements is executed conditionally on the condition not being satisfied: IF with ELSE: if condition then statement; statement;... else statement; statement;... end if With further conditions, we reach the most general form: IF in most general form: if condition1 then statement1a; statement1b;... else if condition2 then statement2a; statement2b;... else if condition3 then statement3a; statement3b;... else statement4a; statement4b;... end if
1EC 2001 10 / 1 Construct 3: Loops The third key construct is the loop. Loops allow the same code to be re-run repeatedly, usually with different data, until some termination condition is satisfied. Because they are so useful, there are several flavours of loops that we will use, of which the most fundamental is the while loop. WHILE while condition do statement1; statement2;... end while Clearly, if the loop is to be entered at all, the condition must be satisfied initially. Once entered, if the loop is to terminate, one of the statements must affect the tested condition.
1EC 2001 11 / 1 Construct 3: Loops A very common condition is based on a loop counter, which is initialized and incremented, as in the example below, which computes the factorial of a variable n as f(n) = n WHILE i=0; fn=1 while i < n do i=i+1 fn = fn*i end while i=1 i Although other forms of loops are stricly unnecessary, they are useful. For example, the for loop: FOR fn=1 for i=0 to n step 1 do fn = fn*i end for
1EC 2001 12 / 1 Construct 4: Subroutines and Functions A subroutine allows a fragment of code to be run from different calling points with the flow returning to the calling point so same code from many places. Even if a piece of code is not called more than once, it might be highly desirable to write it as a subroutine in order that so that steps in an algorithm exhibit a similar level of detail. Some subroutines might be referred to a functions, because they have a single output value determined by a return statement. Others output multiple values placing them in a list of formal parameters. The inputs are always supplied in this list of parameters. As examples consider the use of a return statement: abs (x) if x < 0.0 then return x else return x end if abs (x; r) if x < 0.0 then r = x else r = x end if The general form can use a parameter list and a return: Name(input1, input2,...; output1, output2)
1EC 2001 13 / 1 Recursion in Subroutines Subroutines can call themselves, a technique called recursion. factorial(n) if n 1 then return 1 else return n factorial(n 1) end if The recursive code for the factorial will be to most far clearer than the loop code. The construct is powerful because it allows you to solve just one step of the problem. However some regard recursion as undesirable not least because in practice the way it uses the computer s stack memory can lead to unexpected failure at run time. Local storage Local storage Local storage Local storage Local storage Local storage Local storage Local storage Local storage Stack Memory Fills up
1EC 2001 14 / 1 Program Correctness Even in the shortest program it is easy to make a error. Not a syntactical error these are picked up by the compiler but a fundamental error, so that the code fails to compute what it is supposed to. For example, we might try to compute the factorial function the other way round, counting down from n to 1: This looks entirely plausible, but it isn t too difficult to values of n for which this fails. But checking an arbitrarily complicated program using specific values is really not feasible. // BAD!! fact = 1; i = n; while i > 1 do i = i - 1; fact = fact * i ; end while We now explore some techniques for performing such checking. Although the techniques are quite powerful, the examples will be rather simple so simple that some might be tempted to think what a waste of effort and why bother? You shouldn t.
1EC 2001 15 / 1 Assertions and Pre- and Post-conditions Before we can decide whether a program will do what we want it to do, we need a way of specifying what it should do. At every level, specification takes the form of some assertion about the values of the program variables at key points in the program s execution. Eg, we would like to be able to assert that after the code for factorial is run that f equals n!. English assertions are not generally precise enough, and one might wish to use a formal specification language (which we won t discuss here) or use mathematics. In computer programs, the standard way of asserting something about a program is the use of a program comment. Note that // We require n 0 f =factorial(n) // At this point f = n! the 1st comment warns us that the following makes sense only if n 0 it supplies a set of requirements or PRE-conditions. the 2nd comment asserts something about our expectation after the one line has executed. This supplies a set of POST-conditions.
1EC 2001 16 / 1 Backward Tracing from Post-conditions So far, assertions merely tell us the expectations of the programmer. What we wish to do now is to prove that the piece of code between a pre-condition and post-condition is valid. Rather than working forwards, work backward from post- to pre-conditions. There is a simple rule for generating what needs to be true before an assignment, given something which has to be true after the assignment. Namely, if we have an assignment and required post-condition as in x Expression // Post: P(x) is true here where P(x) is a logical expression (or predicate) depending on x then the necessary pre-condition is given by this rule: THE BACKWARD TRACING RULE // Pre: P(Expression) is true here x Expression // Post: P(x) is true here
1EC 2001 17 / 1 Example 1: Swapping Find statements S that satisfy the assertions // Pre: x = A AND y = B (So A,B are starting values of x,y) S 1 ; S 2 ;... // Post: x = B AND y = A The following is bad!! // Pre: x = A AND y = B x = y y = x // Post: x = B AND y = A Proof of badness: Working backwards inserting assertions A1 and A2 // Pre: x = A AND y = B // A2: y = B AND y = A x = y // A1: x = B AND x = A y = x // Post: x = B AND y = A
1EC 2001 18 / 1 Example 1: Swapping /ctd Of course, the trick to swapping is to introduce an extra variable. You are asked in 1P1K to prove that the following satisfies the assertions. // Good swapper // Pre: x = A AND y = B temp = x x = y y = temp // Post: x = B AND y = A
1EC 2001 19 / 1 Example 2: Fibonacci Series Step 1 Generate and verify a program to compute the F n from the Fibonacci series, defined by F 0 = F 1 = 1, F n = F n 2 +F n 1 for n 2 The first few terms are 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89,... We could proceed as follows: Noting that the definition of F n is split into two different cases, we might use a conditional statement to split our code // Pre: n is non-negative if n = 0 OR n = 1 then // Pre: n is 0 or 1 fn = 1 // Post: fn = F n else // Pre: n 2 C Harder!! // Post: fn = F n end if
1EC 2001 20 / 1 Fibonacci Step 2 The harder case in the definition of F n is defined by a recurrence relation F n = F n 2 + F n 1 and this suggests that a loop might be a suitable way of constructing code C, storing the values of the two preceding terms in the series to compute the current one. To speed understanding here it helps for you to see the final code without conditions F 0 = F 1 = 1, F n = F n 2 + F n 1 for n 2 So, when we go into this bit of code, m is 1, fm represents F 1 and fmm1 represents F 0. We add up fm and fp, but make sure that preserve the old value of fm to put into fmm1 for the next go round. It seems reasonable, but now let us prove it works // Pre: n 2 m = 1 fm = 1; fmm1 = 1 while m < n do temp = fm fm = fm + fmm1 fmm1 = temp m = m + 1 end while fn = fm // Post: fn = F n
1EC 2001 21 / 1 Fibonacci Step 2 m = 1 fm = 1; fmm1 = 1 while m<n do // [A2]: fmm1 is F m 1 AND fm is F m // [A6]: fm is F m AND fm+fmm1 is F m+1 temp = fm // [A5]: temp is F m AND fm+fmm1 is F m+1 fm = fm + fmm1 // [A4]: temp is F m AND fm is F m+1 fmm1 = temp // [A3]: fmm1 is F m AND fm is F m+1 m = m+1 // [A2] fmm1 is F m 1 AND fm is F m end while // [A1]: fm is F m AND m is n fn = fm // Post: fn = F n The final step is to show that [A6] and [A2] are equivalent. Work backwards
1EC 2001 22 / 1 Loop Invariants In program constructs other than loops we can always work our way through the code, working out exactly what the computer would do when executing the code. However problems arise in loop because there is no way of telling (in general) how many times the loop will execute. The way on is to use mathematical induction to deduce properties of a loop that will hold regardless of the number of times the loop body is run. // Pre: n 2 fmm1 = 1; fm = 1; m = 1 while m<n do // fmm1 is F m 1 AND fm is F m temp = fm fm = fm + fmm1 fmm1 = temp m = m+1 // fmm1 is F m 1 AND fm is F m end while // fm is F m AND m is n Recall the example of the previous section. The induction hypothesis here is that At the START of each loop iteration: fmm1 = F m 1 AND fm = F m Notice that if the hypothesis is true at the start of the loop, because no action takes place between the end of one run through the loop and the start of
1EC 2001 23 / 1 Watch out in for loops! NB While loops are obvious, but For loops hide some of the action. 1 that the m=m+1 is executed before the end of the loop; and 2 that the m=1 is executed before the loop starts. m=1 while (m<10) do // Start of loop... statements... m = m + 1 // End of loop end while for (m=1; m<10; m=m+1) do // Start of loop... statements... // End of loop? NO, m=m+1 is really here! end for Best to convert for loops into while loops beforehand.
1EC 2001 24 / 1 Example of loop invariants As another example, consider the code fragment to find the product of the first n positive integers: // n > 0 prod = 1; i = 1 // Loop Inv: prod is the prdct of first i positive integers while i<n do prod = prod*i i = i+1 end while
1EC 2001 25 / 1 Example ctd/ // n > 0 prod = 1; i = 1 // Invariant: prod is the product of the first i positive integers while i<n do // prod is i j=1 j [1] the loop inv // prod*i is i+1 j=1 j [4] prod = prod*i // prod is i+1 j=1 j [3] i = i + 1 // prod is i j=1 j [2] the loop inv end while The final step [1] [4] is to show that prod is i j=1 j prod*i is i+1 j=1 j You should contrast this with what happens if we transpose i = i+1; prod = prod * i;
1EC 2001 26 / 1 Checking that Loops Terminate It is easy to write down a loop that does not terminate, as in i = 1 while i>0 do i=i+1 end while That was easy to spot. But consider the code on the right that counts the number of 1 s in the binary representation of a positive integer number. DIV is the integer division operator. MOD is the modulo operator, ie. (evennumber MOD 2) = 0, and (oddnumber MOD 2 ) = 1 rest starts out positive // Necessary pre-condition: number 0 rest = number count = 0 // Loop Invariant: an exercise... while rest = 0 do if (rest MOD 2) == 1 then // the right-most bit of rest must be a 1 count = count + 1 rest = (rest DIV 2) end if end while // count is the total number of 1 bits in number
1EC 2001 27 / 1 Loop termination /ctd Can we be sure that this is the case? Fortunately yes. The statement rest = (rest DIV 2) is equivalent to a RightShift of the bits in rest. As examples, take the odd and even numbers Odd Number 0 0 1 1 0 1 1 1 55 Right Shift 0 0 0 1 1 0 1 1 27 Even Number 0 0 1 1 0 1 1 0 54 Right Shift 0 0 0 1 1 0 1 1 27 So each time around the loop the value of rest gets smaller, and the value of rest is always non-negative. Thus although it is not immediately clear exactly how many times the loop body will be executed, it is clear that the loop must terminate.
1EC 2001 28 / 1 Loop Variants In the previous example, the variable rest is called a LOOP VARIANT. LOOP VARIANT A loop variant is some function of the program variables that can take only a finite number of values in any particular run of a program; and that takes a different value each time around the loop. Example using a function not just // Invariant: one variable a 0 AND b 0 In the following code, each time while a>0 the loop is executed AND b>0 do (i) we can t be sure which of a or if a>b then b will get smaller, but a = a-b (ii) we do know that one will else (iii) we know that both remain b = b - a non-negative. end if Therefore a suitable simple end while expression for a Loop Variant is
1EC 2001 29 / 1 Summary We have discussed the design process in terms of input and output specification, of algorithm design and program DBT. We saw that there are four programming constructs. Aassignments and conditional are essential, and that loops and subroutines are highly desirable. Comments in programs are important for the purposes of documention, which is vital so that we can be sure both what the code is supposed to do and that it does it; furthermore, it makes it possible to alter code at a later date in a safe manner. (For code that is used in real applications, the biggest cost incurred is not when the code is originally written, but when the program is subsequently altered to meet new requirements.) Assertions about the expected state of the program variables are a vital part of code documentation. Assertions are easily added to programs in the form of comments, and assertions that are written in a mathematical style are likely to be more precise than assertions using just English. For the simplest forms of program statements (assignments and loops) we can then trace through the code, working out what we expect to happen. In particular, for assignments we can use the
1EC 2001 30 / 1 Summary /ctd For code involving loops we have a problem in that we do not know in the general case exactly how many times the loop body will be executed. However the declaration of invariants for loops gives us a straightforward method for inferring properties of programs containing loops. Invariants are normally simple for the original programmer to state and very hard for anybody else to determine! Therefore the assertion of invariants for all but trivial loops is again an important part of the documentation of a program. Although invariants tell us a lot about the state of a program when it terminates, we generally need some further argument to convince ourselves that all our loops do terminate. Informal arguments are sufficient for many cases, and for other cases the use of a variant expression will suffice.