1 Algorithms, Inputs, Outputs

Transcription

1 1 Algorithms, Inputs, Outputs In the last class we discussed Universal Turing machines and the evolution in thought from circuitry being the important thing to the table of state transitions, or computer program, being the important thing. Recall that a Universal Turing machine is one Turing machine that has wiring that can SIMULATE any other Turing machine. So we now think of a universal Turing machine, which takes as input (so what is already on the tape when it is turned on) an encoding of a state transition table describing a particular Turing machine that the universal Turing machine will simulate, and the initial state of the tape for the Turing machine to be simulated. So now instead of building special-purpose Turing machines we just build a universal one and get it to simulate any other Turing machine. It is worth mentioning that there are still situations where it is useful to build a special purpose computer. If you need to build a lot of them it is often much cheaper to build special-purpose computers (that do one task) instead of the more general-purpose computers like Universal Turing machines. For example the brains of a stoplight, are often just the wiring of a special purpose computer. It is common in the area of industrial control systems to build these special purpose computers (for cost reasons). The design of these machines involves the study of state transitions just like we have been talking about, and it is common to visualize these transitions using tools from graph theory, with vertices machine states and arcs representing transitions between states. See Wikipedia page on State Diagrams for example. It may be hard to see the connection between modern computers and (Universal) Turing machines we have been talking about. Turing machines seem so simple, all you can do is detect and manipulate patterns on the tape, and move the head around! Modern computers don t have a tape and you can do graphics and sound and stuff! It is not too far from the truth to think of a modern computer as a sophisticated Universal Turing machine in the following ways. Firstly, and importantly, modern computer memory is designed so that the head of the machine can move over the memory very quickly. In fact it can access any element on the tape in one step. You can think of the memory as a large array M and the head of the tape can access any element of the array (any cell on the tape) in one step. This is quite different than moving the head around one cell at a time and makes things much faster. This way of designing memory is called Random Access Memory, or the RAM in your computer. We saw that we could design state tables for moving binary strings around, comparing strings, telling if a string is a palindrome, adding strings representing binary numbers, and so on. Another way to conceptualize how modern computers differ from what we ve been talking about is that people have already written the state transisition tables for many of the things that people who write computer programs want to do, like work with binary strings

2 that represent letters (ASCII), compare strings, treat strings as numbers and add, multiply, etc them, represent sounds images and movies as (long!) strings of numbers,... So all this grunt work has been done and already built into modern computers. The result is modern computer languages (like python, java, C,... ) that make life much easier for the programmer. So we won t talk so much about state transitions anymore. We will instead think of the more abstract programs or ALGORITHMS, which could (in theory) be represented by a list of states and state transitions ENCODED as a STRING on the tape of a universal Turing machine, but more likely is described by some modern computer language (but again, encoded as a string!). It will also be important to think about the INPUTS to the algorithithm (just like the input to a Turing machine what is on the tape when it starts) and also the outputs of the algorithm. So to say it again, we think of programs as STRINGS that can be read by a universal Turing machine type computer and executed. Turing machines remain important as a SIMPLE but powerful MODEL of a computer. Clearly they are simple. However they are powerful because there are a number of theoretical results that suggest (roughly!) that ANYTHING that it is POSSIBLE to compute CAN be computed by a Turing machine. This belief (it is not proven in the mathematical sense) is called the Church-Turing thesis. Modern computers are obviously much more sophisticated than Turing machines, even though anything they compute can be computed by a Turing machine (though perhaps much more slowly). One big difference is in how modern computer memory is handled. The Turing machine has a memory tape which the head has to pass over one element at a time. Modern computer memory is randomly accessible (that s why it s called RAM randomly accessible memory), meaning that you can think of the entire computer memory as being numbered boxes, and you can request the contents of or change the contents of a memory element directly. IMPORTANTLY, Getting the contents of memory location 5 takes the same time as getting the contents of memory location 5000 there is no tape that needs to be passed over. Modern computer languages have some kind of array type that will allow you to store numbers at various addresses in this fashion. In addition, modern computer lanagues allow the programmer to more easily write the more sophisticated instructions and constructs we are used to from our earlier computing courses, like looping, conditionals, variable nameing, reading, and assignment. You could do these things using state tables also but it would be cumbersome. If you are used to C/C++, there are two big differences in python to be aware of. First, scoping in C/C++ is done using..., but in python it s done with indentiation. Secondly, variables in C/C++ must be declared as having a type. You don t need to do this in python. You can put anything in to a python variable. Finally the octothorpe (hash marks) mean the rest of the line is a comment.

3 2 Estimating computation times and common loop structures Consider the following python program: i = 0 s = 0 n=100 while i < n: s=s+i i=i+1 print(s) Imagine a scenario where it takes 1 unit (say 1 second) for the computer to perform ( execute ) each line. How many seconds will it take for this program to run from beginning to end? Well, let s pretend to be (simulate) the computer using pencil and paper to keep track of which instruction we are performing and the values of all the computer variables after we do each instruction. The first three lines, where we create the variables and put a number value into each takes 1 unit of time each. The way the computer does the while loop is that it just checks the condition (is i < n?) and IF SO it does each line in the loop body once, after which the computer jumps back to the start of the loop where it is again checed if the value of i is less than n or not. IF NOT then the computer jumps over the lines of the loop body and continues executing the instructions that immediately follow the loop body (so in our case the print statement.) So the line with the while condition is performed 101 values in total (once each for i = 0, 1, 2,..., 100 ), and the s = s + 1 line is performed 100 times, once each for the i values {0, 1, 2,..., 99}. The i = i + 1 line is performed 100 times also. The remaining lines are perfomed once. The following table shows how many times the work associated with each line is performed: So the total amount of time is instruction times performed i = 0 1 s = 0 1 n = while i < n: 101 s=s+i 100 i=i print( s ) = 305

4 units of time. Generalizing to any value for n, we find the total time to be (n + 1) + (n) + (n) + 1 = 3n + 5 units. Note that the time required is very close to 3n for large values of n, and the vast majority of the time is taken up by the three lines associated with the loop. In the following program, which is a different way to write a simple loop program in python, the computer will do a similar kind of thing to perform the for loop, starting i at 0, performing some kind of check that the value of variable i is less than the value of n, and if so performing the steps of the loop body and if not continuing by performing the instructions after the loop body. i = 0 s = 0 n=100 for i in range(n): s=s+i print( s ) So, if we assume that performing each line of the following program takes 1 unit (again say 1 second), and we assume that the lines of the loop body are each performed once for each value of i in {0, 1, 2, 3,..., 99} then the following table shows how many times the work associated with each line is performed: instruction times performed i = 0 1 s = 0 1 n=100 1 for i in range(n): 100 s=s+i 100 print( s ) 1 Thus, if we total the number of times any line is performed we get line executions in total, which makes for 204 line executions alltogether, and thus 204 units of time. Generalizing, we can say the above program would take n + n + 1 = 2n + 4 units of time to execute. So for large values of n the total time would be close to 2n, since the +4 would be insignificant for large n. Again, we see the nn term comes from the instructions associated with the loop, which contribute more to the total time because they are performed multiple times. Now, if you look at the above two examples, you will see that both loops do basically the same thing. Some people might look blindly at the above computations and decide that 203 is less than 305, (and more generally 2n+4 is less than 3n+5) so I ll use the second algorithm. THIS WOULD BE WRONG. The reason is because our assumptions are not correct the reality is that each line does NOT take the same amount of time (1 unit) of time to perform.

5 In reality the computational work required to perform each each line is a little bit different, depending on what the line is! However it is still extremely useful to consider how many times each line of a program is performed, because if we do assume that each line takes more or less the same amount of effort, our estimates of total time won t be far off. Furthermore we can still get useful and accurate information about how long it will take a program to run using this technique, which we will see later. Finally if we want better estimates we can incorporate more exact costs of the cost of each line into the above table and multiply that by the number of times that line gets performed for a more exact estimate of the total time required. So for now let us just continue assuming each line takes the same amount of work, and focus our attention on the number of times each line is performed. Another example, this time with a double loop. n=1000 s = 0 for i in range(n): for j in range(n): t = math.sqrt(i*j+1) s=s+t print(s) The outside loop makes i take on each of the values 0, 1, 2,..., 999, which is n different values in total. For EACH of these values of i, the inner loop using j is performed, which makes j take on each of the values {0, 1, 2,..., 999}, which is n different values. Hence the lines associated with the innermost loop, of which there are 3, are performed for each of the values (i, j) in the set {(i, j) 0 i 1000, 0 j 1000}, which is a set of size n n = n 2. So the following table shows how many times the work associated with each line is performed: instruction times performed n= s = 0 1 for i in range(n): n for j in range(n): n*n t = math.sqrt(i*j+1) n*n s=s+t n*n print(s) 1 We thus see that, again with the moderately correct assumption that each line takes 1 unit of time to perform, that the above loop takes 3n 2 + n + 3 steps to perform. Again note that as n grows larger, some lines get executed a LOT more times than others, and so will be responsible for most of the required computation time. It is VERY important that you clearly understand that when you have a program with a loop, the instructions/lines in the loop usually get executed more times than others and so

6 will contribute more to the total required time. It is therefore important for us to be aware of how many times each line gets executed (at least approximately) if we are to understand how long it takes for programs to run.