CSE 2010: Algorithms and Data Structures. Algorithms. Summer 2017

Size: px

Start display at page:

Download "CSE 2010: Algorithms and Data Structures. Algorithms. Summer 2017"

Barnard Doyle
5 years ago
Views:

1 CSE 010: Algorithms and Data Structures Algorithms Summer 017

Abstractions In computer science, we're interested in devising efficient methods (i.e., algorithms) to solve problems. Abstraction vs. implementation.

2 Abstractions In computer science, we're interested in devising efficient methods (i.e., algorithms) to solve problems. Abstraction vs. implementation. Algorithm 1 Compute sum of integers in array 1: procedure ArraySum(A) : sum =0 3: for each integer i in A do 4: sum = sum + i 5: end for 6: Return sum 7: end procedure Spherical chickens in the vacuum - Well, I found a solution to your problem with the chickens. But I had to consider them as spherical objects in the vacuum and with a uniform mass distribution

3 Why computer scientists need to study algorithms? To know a standard set of important algorithms To be able to tell how efficient algorithms are To be able to recognize problems that can be solved with these algorithms To help understand problems and their solutions To help develop our analytical skills

4 What is an algorithm? An algorithm is a finite sequence of unambiguous instructions for solving a problem, i.e., for obtaining the required output for any legitimate input in a finite amount of time. [Levitin 003]

5 Types of algorithms Some try to classify algorithms according the problem-solving approach they use: Iterative Algorithms Direct/Indirect Recursive Algorithms Divide-and-conquer Algorithms Dynamic-rogramming Algorithms Randomized Algorithms Brute-Force Algorithms

6 Selecting an algorithm to solve a problem Someone arriving at the airport and she needs to get to your house. What algorithm could she use? Get-a-Taxi Algorithm Call-You Algorithm Rent-a-Car Algorithm Bus Algorithm How can we compare these approaches? What do they have in common? How do they differ?

7 Fundamental of Alg. roblem Solving Algorithms are not answers to problems They are a set of specific instructions that produce the answer

Typical steps to designing algorithms DRAW 1.

Assess the capabilities of a computational

Choose between exact and approximate approach

Algorithms + Data Structures = rograms [Wirth

Implement any solution (it helps fulfill Step

8 Typical steps to designing algorithms DRAW 1. Understand the problem. Assess the capabilities of a computational device 3. Choose between exact and approximate approach 4. Decide on appropriate data structures Algorithms + Data Structures = rograms [Wirth 76] 5. Implement any solution (it helps fulfill Step 1) 6. Generate various improved solutions 7. Select the most efficient solution

9 Is the algorithm good enough? Very important factors to consider: How often are you going to need the algorithm? What is the typical problem size you are trying to solve? Less important factors that should also be considered: What language are you going use to implement your algorithm? What is the cost-benefit of an efficient algorithm?

10 ractice.1-3 Consider the searching problem: Input: Asequenceofn numbers A Dha 1 ;a ;:::;a n i and a value. Output: An index i such that D AŒi or the special value NIL if does not appear in A. Write pseudocode for linear search, whichscansthroughthesequence,looking for. Usingaloopinvariant,provethatyouralgorithmiscorrect.Makesurethat your loop invariant fulfills the three necessary properties..1-4

11 Linear/sequential search

12 insertat: Insert an element into an array

13 removeat: Remove an element from an array

14 Our first algorithm to analyze: Insertion sort ter 1: Input: Asequenceofn numbers ha 1 ;a ;:::;a n i. Figure.1 Sorting a hand of cards using insertion sort. Output: Apermutation(reordering)ha1 0 ;a0 ;:::;a0 i of the input sequence such n that a1 0 a0 a0 n. The numbers that we wish to sort are also known as the keys. Althoughconceptually we are sorting a sequence, the input comes to us in the form of an array with n elements. In this book, we shall typically describe algorithms as programs written in a pseudocode that is similar in many respects to C, C++, Java, ython, or ascal. If

15 Insertion sort (a) (b) (c) (d) (e) (f)

16 Insertion sort (a) INSERTION-SORT.A/ 1 for j D to A:length key D AŒj 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 4 i D j 1 5 while i>0and AŒi > key 6 AŒi C 1 D AŒi 7 i D i 1 8 AŒi C 1 D key (b) (c) (d) (e) Loop invariants and the correctness of insertion sort (f)

17 ractice Dh i.1- Rewrite the INSERTION-SORT procedure to sort into nonincreasing instead of nondecreasing order..1-3

18 Analysis of insertion sort (a) The sorting time depends on the size of the input sequence. (b) If two equal-size sequences are partially sorted, then sorting time will be different for each sequence. (c) In general, the longer the sequence, the longer the sorting time. (d) For running time, we can simplify the analysis by defining a cost ci to represent the time it takes to complete the instruction i. (e) (f)

19 Analysis of insertion sort INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 5 while i>0and AŒi > key c n 5 j D t j 6 AŒi C 1 D AŒi c n 6 j D.t j 1/ 7 i D i 1 c n 7 j D.t j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times (a) (b) (c) (d) (e) (f)

20 Insertion sort INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 5 while i>0and AŒi > key c n 5 j D t j 6 AŒi C 1 D AŒi c n 6 j D.t j 1/ 7 i D i 1 c n 7 j D.t j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D

21 Insertion sort INSERTION-SORT.A/ cost times Best case 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 5 while i>0and AŒi > key c n 5 j D t j 6 AŒi C 1 D AŒi c n 6 j D.t j 1/ 7 i D i 1 c n 7 j D.t j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D

22 Insertion sort INSERTION-SORT.A/ cost times Best case 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 5 while i>0and AŒi > key c n 5 j D t j 6 AŒi C 1 D AŒi c n 6 j D.t j 1/ 7 i D i 1 c n 7 j D.t j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D 5.n 1/ C C

23 Insertion sort INSERTION-SORT.A/ cost times Best case 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 5 while i>0and AŒi > key c n 5 j D t j 6 AŒi C 1 D AŒi c n 6 j D.t j 1/ 7 i D i 1 c n 7 j D.t j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D 0 5.n 1/ C C 0

24 Insertion sort Best case X T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D D Even for inputs,andthebest-caserunningtimeis of a given size, algorithm s running time may T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5.n 1/ C c 8.n 1/ D.c 1 C c C c 4 C c 5 C c 8 /n.c C c 4 C c 5 C c 8 /: We can express this running time as an C b for constants a and b that depend on the statement costs c i ;itisthusalinear function of n. If the array is in reverse sorted order that is, in decreasing order the worst

25 Insertion sort Best case X T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D 5.n 1/ C C D Even for inputs,andthebest-caserunningtimeis of a given size, algorithm s running time may T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5.n 1/ C c 8.n 1/ D.c 1 C c C c 4 C c 5 C c 8 /n.c C c 4 C c 5 C c 8 /: We can express this running time as an C b for constants a and b that depend on the statement costs c i ;itisthusalinear function of n. If the array is in reverse sorted order that is, in decreasing order the worst

26 Insertion sort Best case X T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 j D t j C c 6.t j 1/ j D C c 7.t j 1/ C c 8.n 1/ : j D 0 5.n 1/ C C D Even for inputs,andthebest-caserunningtimeis of a given size, algorithm s running time may T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5.n 1/ C c 8.n 1/ D.c 1 C c C c 4 C c 5 C c 8 /n.c C c 4 C c 5 C c 8 /: 0 We can express this running time as an C b for constants a and b that depend on the statement costs c i ;itisthusalinear function of n. If the array is in reverse sorted order that is, in decreasing order the worst

27 Insertion sort Worst case INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 The running time of the algorithm is the sum of running times t j C c 6.t j 1/ j D j D C c 7.t j 1/ C c 8.n 1/ : j D Even for inputs of a given size, an algorithm s running time may

28 Insertion sort Worst case INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 The running time of the algorithm is the sum of running times t j C c 6.t j 1/ j D j D C c 7.t j 1/ C c 8.n 1/ : j D X Even for inputs of a given size, an algorithm s running time may Useful identities: and n.n C 1/ n.n 1/ j D 1.j 1/ D j D j D

29 Insertion sort Worst case n.n C 1/ T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 1 INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times t j C c 6.t j 1/ j D j D C c 7.t j 1/ C c 8.n 1/ : j D X Even for inputs of a given size, an algorithm s running time may Useful identities: and n.n C 1/ n.n 1/ j D 1.j 1/ D j D j D

30 Insertion sort Worst case T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 n.n C 1/ 1 INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times j D t j C c 6.t j 1/ C C j D n.n 1/ C c 7.t j 1/ C c 8.n 1/ : j D c c X Even for inputs of a given size, an algorithm s running time may Useful identities: and n.n C 1/ n.n 1/ j D 1.j 1/ D j D j D

31 Insertion sort Worst case T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 n.n C 1/ 1 INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times j D t j C c 6.t j 1/ C C j D n.n 1/ C c 7.t j 1/ C c 8.n 1/ : C C j D n.n 1/ c c X Even for inputs of a given size, an algorithm s running time may c c Useful identities: and n.n C 1/ n.n 1/ j D 1.j 1/ D j D j D

32 Insertion X sort INSERTION-SORT.A/ cost times 1 for j D to A:length c 1 n key D AŒj c n 1 Worst case the worst case, the running time of I -S is T.n/ D c 1 n C c.n 1/ C c 4.n 1/ C c 5 n.n C 1/ D n.n 1/ n.n 1/ C c 6 C c 7 C c 6 C c 7 n C.c C c 4 C c 5 C c 8 /: c5 3 // Insert AŒj into the sorted sequence AŒ1 : : j 1. 0 n 1 4 i D j 1 c 4 n 1 n 5 while i>0and AŒi > key c 5 j D t j n 6 AŒi C 1 D AŒi c.t 6 j D j 1/ 7 i D i 1 c n.t 7 j D j 1/ 8 AŒi C 1 D key c 8 n 1 The running time of the algorithm is the sum of running times c 1 C c C c 4 C c 5 1 C c 8.n 1/ c 6 c 7 C c 8 n We can express this worst-case running time as an C bn C c for constants a, b, and c that again depend on the statement costs c i ;itisthusaquadratic function of n.

33 Worst case vs. Average case Average case is often as bad as the worst case. When analyzing algorithms, we will mostly focus on the worst case.

34 Rate of growth: Example functions that often appear in algorithm analysis time T(n) time T(n) Constant 1 Logarithmic log n Linear n N-Log-N n log n Quadratic n Cubic n3 Exponential n T(n) = n T(n) = n T(n) = log n T(n) = input size n uction; Analysis of

35 Rate of growth: consider only the leading term ve us more detail Running time of an C bn C c he factor of sts sts..wethusig from the leading term. We write that insert running time of.n / (pronounced theta of n-squared ) informally in this chapter, and we will define it precisely lly consider one algorithm to be more efficient than anothe order of growth. But for large enough inputs, a.n / algorithm, for example, will run more quickly in the worst case than a.n 3 / algorithm.

Analyzing Algorithms

Analyzing Algorithms Insertion Sort 5 4 4 7 10 7 5 10 (a) (d) 1 3 4 5 6 5 4 6 1 3 1 3 4 5 6 4 5 6 1 3 (b) (e) 1 3 4 5 6 5 4 6 1 3 1 3 4 5 6 1 4 5 6 3 (c) (f) 1 3 4 5 6 4 5 6 1 3 1 3 4 5 6 1 3 4 5 6 Insertion