Computer Algorithms. Introduction to Algorithm

Size: px

Start display at page:

Download "Computer Algorithms. Introduction to Algorithm"

Melvyn Reed
5 years ago
Views:

1 Computer Algorithms Introduction to Algorithm CISC 4080 Yanjun Li 1 What is Algorithm? An Algorithm is a sequence of well-defined computational steps that transform the input into the output. These steps are precise, unambiguous, mechanical, efficient and correct. It is a tool for solving well-specified computational problems. CISC 4080 Yanjun Li 2 1

2 List of Algorithms that You already know (1) General Recursive Algorithm Polynomial Algorithm Searching Algorithms Linear Search Binary Search Search with Hashing Sorting Algorithms Quick Sort Selection Sort Bubble Sort Insertion Sort Heap Sort Merge Sort CISC 4080 Yanjun Li 3 List of Algorithms that You already know (2) Graph Related Algorithms Bread-first Search/Traversal Depth-first Search/Traversal Single-source Shortest Path Binary Tree Related Algorithms Bread-first Traversal Depth-first Traversal In-order Pre-order Post-order CISC 4080 Yanjun Li 4 2

3 Fibonacci Numbers The Fibonacci numbers F n are generated by the simple rule: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,.. F n = F n-1 + F n-2 if n > 1; F n = 1 if n = 1; F n = 0 if n = 0; The Fibonacci numbers grow almost as fast as the powers of 2. In general, F n n. To compute the nth Fibonacci number, we need an algorithm. CISC 4080 Yanjun Li 5 A Recursive Algorithm Implementing the recursive definition of F n. We always ask the following three questions about any algorithm: Is it correct? How much time does it take, as a function of n? And can we do better? CISC 4080 Yanjun Li 6 3

4 Efficiency of the Recursive Algorithm Let T(n) be the number of computer steps needed to compute fib1(n). T(n) 2 for n 1. T(n) = T(n-1)+T(n-2)+ C for n > 1. Overall, T(n) F n. (it actually is exponential in n since Fn n.). This recursive algorithm is impractically slow except for very small values of n. The recursive algorithm is correct but hopelessly inefficient. CISC 4080 Yanjun Li 7 Can we do better? Why the recursive algorithm is so slow? Many computations are repeated. We would like to store the intermediate results: F 0, F 1,,F n-1. CISC 4080 Yanjun Li 8 4

5 A Polynomial Algorithm An array is used to store the intermediate results. The inner loop consists of a single computer step and is executed n-1 time. Therefore, the algorithm is liner in n. CISC 4080 Yanjun Li 9 Evaluation of Running Time A processor s instruction set has a variety of basic primitives branching, storing to memory, comparing numbers, simple arithmetic, and so on. When we estimate the running time of an algorithm, they are considered as one category of basic computer steps. Also the architecture-specific and hardware-specific details are ignored. CISC 4080 Yanjun Li 10 5

6 Big-O Notation Running time (number of basic computer steps) of an algorithm is expressed as a function of the size of the input n. T(n) = 5n 3 + 4n + 3 To simplify further, lower-order terms and details of coefficient in the leading term are ignored. T(n) = O(n 3 ) CISC 4080 Yanjun Li 11 Big-O Notation Let f(n) and g(n) be functions from positive integers to positive real numbers. We say f = O(g) (which means that f grows no faster than g) if there is a constant c > 0 such that f(n) c g(n) For example, 10n = O(n). When comparing two algorithm, we care when n grows, which is better. f 1 (n) =n 2, f 2 (n) =2n +20 Since f 2 (n)/f 1 (n) =(2n+20)/n 2 22 for all n, f 2 =O(f 1 ). CISC 4080 Yanjun Li 12 6

7 Common Rules Multiplicative constants can be omitted 14n 2 becomes n 2. n a dominates n b if a > b n 2 dominates n. Any exponential dominates any polynomial 3 n dominates n 5 ; 3 n dominates 2 n. Any polynomial dominates any logarithm n dominates (log n) 3 ; n 2 dominates n(log n). CISC 4080 Yanjun Li 13 Case Study: Insertion Sort Works like someone who inserts one more card at a time into a hand of cards that are already sorted. To insert 12, we need to make room for it by moving first 36 and then 24. CISC 4080 Yanjun Li 14 7

8 Problem Description Sorting Problem: Input: A sequence of n numbers (a 1, a 2,,a n ); Output: A permutation (reordering) (a 1, a 2,, a n ) of the input sequence such that a 1 a 2 a n. Pesudo-code Array Index starts from 1. CISC 4080 Yanjun Li 15 Example CISC 4080 Yanjun Li 16 8

9 Analyzing Algorithm -1 [ c + c + ( t 1)( c + c + c + c ] T ( n) = c j + c 1 ( n 1) ) t j [ 1, n 1) CISC 4080 Yanjun Li Best-case, Worst-case and Average-case Analysis Best case: the array is already sorted, t j =1: = an + b [ c + c + c + c ] T ( n) = c1( n 1) Worst case: the array is in reverse sorted order, t j = n-1. 2 T ( n) = an + bn + c We shall usually focus on the worst-case running time the upper bound. Sometime, we focus on average-case running time. CISC 4080 Yanjun Li 18 9

10 Comparison of Sorting Algorithms CISC 4080 Yanjun Li 19 10

Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types

Module 1: Asymptotic Time Complexity and Intro to Abstract Data Types Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu