Computer Algorithms CISC4080 CIS, Fordham Univ. Acknowledgement. Outline. Instructor: X. Zhang Lecture 1

Size: px

Start display at page:

Download "Computer Algorithms CISC4080 CIS, Fordham Univ. Acknowledgement. Outline. Instructor: X. Zhang Lecture 1"

Caren McKinney
5 years ago
Views:

1 Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 1 Acknowledgement The set of slides have use materials from the following resources Slides for textbook by Dr. Y. Chen from Shanghai Jiaotong Univ. Slides from Dr. M. Nicolescu from UNR Slides sets by Dr. K. Wayne from Princeton which in turn have borrowed materials from other resources 2 Outline What is algorithm: word origin, first algorithms, algorithms of today s world Pseudocode: convention and examples Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive formula for running time of recursive algorithm math help: math. induction Asymptotic notations Algorithm running time classes: P, NP 3

databases, caching,networking, compilers,... Computer graphics. Movies, video games, virtual reality, Security.

2 What are Algorithms? 4 Algorithms Etymology CS 477/677 - Lecture 1 5 Why study algorithms? Internet: web search, packet routing, distributed file sharing Biology: human genome project, protein folding, Computers: circuit layout, databases, caching,networking, compilers,... Computer graphics. Movies, video games, virtual reality, Security. Cell phones, e-commerce, voting machines, Multimedia. MP3, JPG, DivX, HDTV, face recognition, Social networks. Recommendations, news feeds, advertisements,... Physics. N-body simulation, particle collision simulation,... 6

3 Learning Goals Learn algorithms (sorting and searching, arithmetics, graph algorithms, linear programming algorithms), and practice implementing them in C++ (usage of C++ STL) Algorithms analysis: correctness, efficiency (running time and space requirement) Complexity analysis of problem itself: Lower bound analysis: comparison based sorting cannot do better than nlogn NP complete problem Learn paradigms such as divide an conquer, greedy algorithms, randomization, dynamic programming and linear programming for design algorithmic solution to new problems We will teach you how to fish for yourself 7 Greedy Algorithms Similar to dynamic programming, but simpler approach Also used for optimization problems Idea: When we have a choice to make, make the one that looks best right now Make a locally optimal choice in hope of getting a globally optimal solution Greedy algorithms don t always yield an optimal solution Applications: Activity selection, fractional knapsack, Huffman codes 8 Dynamic Programming An algorithm design technique (like divide and conquer) Richard Bellman, optimizing decision processes Applicable to problems with overlapping subproblems E.g.: Fibonacci numbers: Recurrence: F(n) = F(n-1) + F(n-2) Boundary conditions: F(1) = 0, F(2) = 1 Compute: F(5) = 3, F(3) = 1, F(4) = 2 Solution: store the solutions to subproblems in a table Applications: Assembly line scheduling, matrix chain multiplication, longest common sequence of two strings, 0-1 Knapsack problem 9

4 Decimal System & algorithms Imagine adding two Roman numerals? What is Decimal system, invented in India around AD600 Uses only 10 symbols (0,1, 9) write large number compactly easy to perform arithmetic operations 10 Oldest Algorithms Al Khwarizmi laid out basic methods for adding, multiplying and dividing numbers extracting square roots calculating digits of pi, These procedures were precise, unambiguous, mechanical, efficient, correct. i.e., they were algorithms, a term coined to honor Al Khwarizmi after decimal system was adopted in Europe many centuries later. 11 Algorithms that you ve seen Linear Search: search for an item with a matching key in an array (unsorted) Binary Search Bubble Sort, Insertion Sort, Selection Sort, Radix Sort Search in a binary search tree: algorithm + data structure Graph algorithms: traversal Group Practice: Idea (how does it work), correctness, efficiency 12

5 Example: Selection Sort Input: a list of elements, L[1 n] Output: rearrange elements in List, so that L[1]<=L[2]<=L[3]< L[n] Note that list is an ADT (could be implemented using array, linked list) Ideas (in two sentences) First, find location of smallest element in sub list L[1 n], and swap it with first element in the sublist repeat the same procedure for sublist L[2 n], L[3 n],, L[n-1 n] 13 Selection Sort (idea=>pseudocode) for i=1 to n-1 // find location of smallest element in sub list L[i n] minindex = i; for k=i+1 to n if L[k]<L[minIndex]: minindex=k //swap it with first element in the sublist if (minindex=i) swap (L[i], L[minIndex]); // Correctness: L[i] is now the i-th smallest element 14 Overview of Lab1 Implement and test three basic sorting algorithms Start from idea of the algorithms, keep refine it until it becomes code Performance measurement: how long does it take for each sorting algorithms (functions) to sort an array of a given size Basis of lab2 and lab3 15

Introduction to algorithm analysis Consider calculation of Fibonacci sequence, in particular, the n-th number in sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 16 Fibonacci Sequence 0, 1, 1, 2, 3, 5, 8,

6 Introduction to algorithm analysis Consider calculation of Fibonacci sequence, in particular, the n-th number in sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 16 Fibonacci Sequence 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, Formally, Problem: How to calculate n-th term, e.g., what is F 100, F 200? 17 A recursive algorithm Three questions: Is it correct? yes, as the code mirrors the definition How much time does it take? Can we do better? (faster?) 18

7 In pursuit of better algorithms We want to solve problems using less resource: Space: how much memory is needed? Time: how fast can we get the result? Usually, the bigger input, the more memory it takes and longer it takes it takes longer to calculate 200-th number in Fibonacci sequence than the 10th number it takes longer to sort larger array Efficient algorithms are critical for large input size/problem instance Finding F100, Searching Web Two different approaches to evaluate efficiency of algorithms: Measurement vs. analysis 19 Experimental approach Measure how much time elapses from algorithm starts to finishes needs to implement, instrument and deploy e.g., t1 = gettimeofday(&t1,null); BubbleSort (a, size); gettimeofday (&t2, NULL); double timeinseconds = (t2.tv_sec-t1.tv_sec) + (t2.tv_usec-t2.tv_usec)/ ; results are realistic, specific and random specific to language, run time system (Java VM, OS), caching effect, other processes running cannot shed light on: larger input size(not always possible to test all input size), faster CPU, Measurement is important for a production system/end product; but not informative for algorithm efficiency studies/comparison/ prediction 20 Example (Fib1: recursive) n T(n)ofFib1 F(n) 10 3e e e e e e e e e Time (in seconds) Model fitting to find out T(n)? n Running time seems to grows exponentially as n increases 21

Example (Fib2: iterative) 10 1e-06 55 11 1e-06 89 12 0 144 13 0 233 14 0 377 15 0 610 16 0 987 17 0 1597 18 0 2584 19 0 4181 20 0 6765 21 0 10946 22 0 17711 23 0 28657 24 0 46368 25 0 75025 26 0

102334155 41 1e-06 165580141 42 0 267914296 43 1e-06 433494437 44 0 701408733 1000 8e-06 Time (in seconds) n Increase very slowly as n increases Model fitting to find out T(n)?

8 Example (Fib2: iterative) 10 1e e e e e e e e e-06 Time (in seconds) n Increase very slowly as n increases Model fitting to find out T(n)? 22 Summary Review three sorting algorithms Ideas => pseudocode => implementation remember and start from ideas write comment to explain idea and why it works refine comment to become code Running time measurement: how to? record the wall-clock time before, and after time-elapsed = after-before Fibonacci number calculating: two algorithms & very different running time growth function 23 Assignment Lab1 Chapter 0 of DPV 24

Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 1

Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 1 Acknowledgement The set of slides have use materials from the following resources Slides for textbook by Dr. Y. Chen from