Week 1 s s Getting Started 1 3 4 5 - - Lecturer Dr Lectures Tuesday 1-13 Fulton House Lecture room Tuesday 15-16 Fulton House Lecture room Thursday 11-1 Fulton House Lecture room Friday 10-11 Glyndwr C Labs (choose one of the following two sessions) Thursday 1-13 Faraday 17 (Linux Lab) Thursday 13-14 Faraday 17 (Linux Lab) - 6 7 Course home page s Assessment s http://www.cs.swan.ac.uk/~csoliver/s01314/index.html with messages general information up-to-date versions of the slides (which serve as the script) lab sheets course works. - There will be two pieces of coursework, each worth 10%. Labs are an essential part of the course. Successful participations is worth 10%. The written examination counts 70%, it will take place in January. -
Lectures s References s The script will usually be hed out at the beginning of each week. Corrected version of the script are available on the course home-page. Usually, they will not be printed out again, except in cases where substantial changes have occurred. Left-over print-outs of the script can be found in the student s office (Faraday Building Room 06) for those who could not attend the lecture. You also have to sign the module attendance form. - There are very many textbooks in print. In this module, we will follow ( refer to as CLRS) Introduction to s (Third Edition) by Cormen, Leiserson, Rivest Stein, The MIT Press, 009. LIS Call Number: QA76.6. I58 009 (10 copies on 1 week loan, copies on 1 night loan) http://en.wikipedia.org/wiki/introduction_to_s Reading from CLRS for week 1 - Chapter 1 Chapter, Sections.1,. s ic solution s Problem: A specification in general terms of inputs outputs the desired input/output relationship. Problem Instance: An actual set of inputs for a given problem. Example 1 Problem: ing (numbers). Input: A sequence of n numbers a 1, a,..., a n. Output: A permutation (reordering) a 1, a,... a n of the input such that a 1 a... a n. Example instance: Input 10, 15, 3, 3, 70, 17, 30 - : a well-defined computational procedure that takes some value, or sequence of values, as input produces some values, or sequence of values, as output. There will always be many different for any given problem. A program is a particular implementation of some algorithm. A program is not the same as an algorithm. There are many different implementations of any given algorithm. - Output 30, 17, 15, 3, 3, 10, 70
Expressing s s We express in whatever way is the clearest most concise. English is sometimes the best way. When issues of control need to be made perfectly clear, we often use pseudocode. Pseudocode is similar to any typical imperative programming language, such as Java, C, C++, Pascal,... Pseudocode expresses to humans. Software engineering issues of data abstraction, modularity, error hling are often ignored. We sometimes embed English statements into pseudocode. Therefore, unlike for real programming language, we cannot create a compiler that translates pseudocode to machine code. - Example is a good algorithm for ing a small number of elements. It works the way you might a h of playing cards. -(A) 1 for j = to A.length key = A[j] 3 / Insert A[j] into ed sequence A[1.. j 1]. 4 i = j 1 5 while i > 0 A[i] > key 6 A[i+1] = A[i] 7 i = i 1 8 A[i+1] = key - s Rom-access machine (RAM) model s We want to predict the resources that the algorithm requires. Typically, we are interested in runtime memory number of basic operations, such as: arithmetic operations (eg, for multiplying matrices) bit operations (eg, for multiplying integers) comparisons (eg, for ing searching) In order to predict resource requirements, we need a computational model. - Instructions are executed one after another. No concurrent operations. It s too tedious to define each of the instructions their associated time costs. Instead, we recognise that we ll use instructions commonly found in real computers: Arithmetic: add, subtract, multiply, divide, remainder, floor, ceiling. Also, shift left/shift right (good for multiplying/dividing by k ). Data movement: load, store, copy. Control: conditional/unconditional branch, subroutine call return. Each of these instructions takes a constant amount of time. -
Rom-access machine (RAM) model (cont d) s How do we analyse running time? s The time taken by an algorithm depends on the input. The RAM model uses integer floating-point types We don t worry about precision, although it is crucial in certain numerical applications. There is a limit on the word size: when working with inputs of size n, assume that integers are represented by c lg n bits for some constant c 1. (lg n is a very frequently used shorth for log n.) - ing 1000 numbers takes longer than ing 3 numbers. A given ing algorithm may even take differing amounts of time on two inputs of the same size. For example, we ll see that insertion takes less time to n elements when they are already ed than when they are in reverse ed order. The input size depends on the problem studied. Usually the number of items in the input. Like the size n of the array being ed. But could be something else. If multiplying two integers, could be the total number of bits in the two integers. Could be described by more than one number. - The running time on a particular input is the number of primitive operations (steps) executed. Want to define steps to be machine-independent. Each line of pseudocode requires a constant amount of time. One line may take a different amount of time than another, but each execution of line i takes the same amount of time c i. Assumption: lines consist only of primitive operations. If the line is a subroutine call, then the actual call takes constant time, but the execution of the subroutine being called might not. If the line specifies operations other than primitive ones,then it might take more than constant time. s - Analysing - -(A) cost repetitions 1 for j = to A.length c 1 n key = A[j] c n 1 3 / Insert A[j] into ed 0 n 1 sequence A[1.. j 1]. 4 i = j 1 c 4 n 1 n 5 while i > 0 A[i] > key c 5 j= t j n 6 A[i+1] = A[i] c 6 j= (t j 1) n 7 i = i 1 c 7 j= (t j 1) 8 A[i+1] = key c 8 n 1 where: n = A.length, t j = number of times the while -loop test in line 5 is executed in the jth iteration of the for -loop. s -
Applying the exact s Remarks s Best possible situation: t j = 1 for all j, i.e., when A is already ed. Runtime: (c 1 + c + c 4 + c 5 + c 8 )n (c + c 4 + c 5 + c 8 ). Worst possible situation: t j = j for all j, i.e., when A is ed in reverse order. Runtime: ( c 5 + c 6 + c 7 ) n + (c 1 + c + c 4 + c 5 c 6 c 7 + c 8 ) n (c + c 4 + c 5 + c 8 ). Average situation: t j = j for all j, i.e., every permutation is equally likely, so expected value of t j is j. Runtime: ( c 5 4 + c 6 4 + c 7 4 )n + (c 1 + c + c 4 + c 5 4 3c 6 4 3c 7 4 + c 8)n (c + c 4 + c 5 c 6 c 7 + c 8 ). - Types of Analysis: exact (rarely achieved) best-case worst-case average-case. Worst-case versus average-case : We usually concentrate on finding the worst-case running time. Average case is often as bad as the worst case. Order of growth is another abstraction to ease focus on the important features. Will be discussed next week. - Mathematical Functions We shall assume you are comfortable with stard math functions, like exponentiation b x its inverse log b x. log b a is the number x such that b x = a. We shall usually work with binary logarithms lg x = log x. Some Useful Identities log b (xy) = log b x + log b y log b (x y ) = y log b x log b x log c x = log b c s - Two Revealing Tables Time to solve a problem instance of size n using a T (n)-time algorithm. T(n) n=10 n=0 n=50 n=100 n=500 n=1000 n lg n 33 µs 86 µs 8 µs 664 µs 4.5 ms 10 ms n 100 µs 400 µs.5 ms 10 ms 50 ms 1 s n 1 ms 1 s 36 yr 10 16 yr 10 137 yr 10 87 yr n! 4 s 10 5 yr 10 51 yr 10 144 yr 10 110 yr 10 554 yr Largest problem instance solvable in 1 minute. s - We shall often use floor x ceiling x functions: x is the largest integer x, e.g., 5.3 = 5 x is the smallest integer x, e.g., 5.3 = 6 as well as summation notation: n a i = a 1 + a + a 3 + + a n. i=1 T (n) faster machine 10 3 faster machine n lg n 3.9 10 6 7.5 10 6.7 10 9 n 7 745 10 954 44 948 n 5 6 35 n! 11 11 13
Running insertion Organise in pairs, write down the following sequence 1, 14, 5, 4, 1, 14, 5, 3, 13, 10 it by insertion-, counting the comparisons the assignments as usual in this context, only considering the real objects. s - Running insertion (cont.) Showing number of comparisons assignments for each insertion step, together with the next element to be inserted: 1, 14, 5, 4, 1, 14, 5, 3, 13, 10 1 1, 14, 5, 4, 1, 14, 5, 3, 13, 10 : (1, 1) 5, 1, 14, 4, 1, 14, 5, 3, 13, 10 : (, 4) 3 4, 5, 1, 14, 1, 14, 5, 3, 13, 10 : (3, 5) 4 1, 4, 5, 1, 14, 14, 5, 3, 13, 10 : (4, 6) 5 1, 4, 5, 1, 14, 14, 5, 3, 13, 10 : (1, 1) 6 1, 4, 5, 5, 1, 14, 14, 3, 13, 10 : (4, 5) 7 1, 3, 4, 5, 5, 1, 14, 14, 13, 10 : (7, 8) 8 1, 3, 4, 5, 5, 1, 13, 14, 14, 10 : (3, 4) s - 9 1, 3, 4, 5, 5, 10, 1, 13, 14, 14 : (5, 6) : (30, 40) Logarithms s Some binary logarithms s The powers n for 0 n 3: 1 13107 6144 4 5488 8 1048576 16 09715 3 4194304 64 8388608 18 1677716 56 3355443 51 67108864 104 1341778 048 68435456 4096 53687091 819 107374184 16384 147483648 3768 49496796 65536-1 lg(10 3 ) 10 (since 10 = 104) lg(18 10 3 ) = lg(18) + lg(10 3 ) = 7 + lg(10 3 ) 17 3 lg(10 6 ) = lg(10 3 10 3 ) = lg(10 3 ) + lg(10 3 ) 0 4 lg(7 10 6 ) = lg(7) + lg(10 6 ).8 + 0 =.8 5 lg(10 9 ) = lg(10 3 ) + lg(10 6 ) 10 + 0 = 30 6 lg(10 10 ) = lg(10) + lg(10 9 ) 3.3 + 30 = 33.3 -
Number of binary digits s Number of binary digits (cont.) s Consider a natural number n 0: Develop a formula for the number b(n) 1 of binary digits of n. - b(n) = { 1 if n = 0 lg(n) + 1 if n 1 Here are the values of b(n) for n { 0,..., 16 }: 1, 1,,, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 5 - Hint: use lg. Sums s Sums (cont.) The list of values of f (n) for n = 0,..., 10: s 0, 1, 3, 6, 10, 15, 1, 8, 36, 45, 55. Develop a formula for n f (n) := i. Start by creating a table of values. i=1 - The differences between these values are 1,, 3, 4, 5, 6, 7, 8, 9, 10. Since these differences grow, the original function is not linear (not something like a n + b). However this list of differences is linear, so the original function is quadratic (something like a n + b n + c). - Playing around yields f (n) = n i=1 i = 1 n(n + 1).
Other (homework) s Can you imagine other for ing? In what sense could they improve insertion-? How good could they be? -