Algorithms and Data Structures

Transcription

1 Algorithms and Data Structures Course Introduction Antonio Carzaniga Faculty of Informatics University of Lugano October 25, 2006

2 Outline General course information Program Preliminary schedule Motivating example

3 General Information On-line course information on Moodle: and on my web page:

4 General Information On-line course information on Moodle: and on my web page: Announcements you are responsible for reading the announcements page

5 General Information On-line course information on Moodle: and on my web page: Announcements you are responsible for reading the announcements page Office hours Antonio Carzaniga: by appointment Nicolas Schiper: Monday 15:30 16:30, Wed 15:30 16:30

6 Textbook 1 Introduction to Algorithms Second Edition Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein The MIT Press

7 Textbook 2 Algorithms Sanjoy Dasgupta Christos H. Papadimitriou Umesh.V. Vazirani Mc Graw-Hill

8 Evaluation +50% projects 3 5 assignments grades added together, thus resulting in a weighted average +20% midterm exam +30% final exam ±10% instructor s discretionary evaluation participation extra credits trajectory...

9 Evaluation +50% projects 3 5 assignments grades added together, thus resulting in a weighted average +20% midterm exam +30% final exam ±10% instructor s discretionary evaluation participation extra credits trajectory % plagiarism penalties

10 Plagiarism

11 Plagiarism A student should never take someone else s material and present it as his or her own. Doing so means committing plagiarism.

12 Plagiarism A student should never take someone else s material and present it as his or her own. Doing so means committing plagiarism. material means ideas, words, code, suggestions, corrections on one s work, etc. Using someone else s material may be appropriate e.g., software libraries always clearly identify the external material, and acknowledge its source. Failing to do so means committing plagiarism. the work will be evaluated based on its added value

13 Plagiarism A student should never take someone else s material and present it as his or her own. Doing so means committing plagiarism. material means ideas, words, code, suggestions, corrections on one s work, etc. Using someone else s material may be appropriate e.g., software libraries always clearly identify the external material, and acknowledge its source. Failing to do so means committing plagiarism. the work will be evaluated based on its added value Committing plagiarism on an assignment or an exam will result in failing that assignment or that exam loosing one or more points in the final note! Penalties may be escalated in accordance with the regulations of the Faculty of Informatics

14 Deadlines

15 Deadlines Deadlines are firm.

16 Deadlines Deadlines are firm. Exceptions may be granted at the instructor s discretion only for documented medical conditions or other documented emergencies

17 Deadlines Deadlines are firm. Exceptions may be granted at the instructor s discretion only for documented medical conditions or other documented emergencies Each late day will reduce the assignment s grade by one third of the total value of that assignment

18 Deadlines Deadlines are firm. Exceptions may be granted at the instructor s discretion only for documented medical conditions or other documented emergencies Each late day will reduce the assignment s grade by one third of the total value of that assignment Corollary 1. The grade of an assignment turned in more than two days late is 0

19 Deadlines Deadlines are firm. Exceptions may be granted at the instructor s discretion only for documented medical conditions or other documented emergencies Each late day will reduce the assignment s grade by one third of the total value of that assignment Corollary 1. The grade of an assignment turned in more than two days late is 0 The proof of Corollary 1 is left as an exercise

20 Now let s move on to the real interesting and fun stuff...

21 Example A sequence of numbers 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,...

22 Example A sequence of numbers 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... The well-known Fibonacci sequence Leonardo da Pisa (ca ca. 1250) son of Guglielmo Bonaccio a.k.a. Leonardo Fibonacci

23 The Fibonacci Sequence Mathematical definition of the Fibonacci sequence 0 if n = 0 F n = 1 if n = 1 F n 1 + F n 2 if n > 1

24 The Fibonacci Sequence Mathematical definition of the Fibonacci sequence 0 if n = 0 F n = 1 if n = 1 F n 1 + F n 2 if n > 1 In pseudo-code FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2)

25 Questions on Our First Algorithm FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2)

26 Questions on Our First Algorithm FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) 1. Is the algorithm correct? for every valid input, does it terminate? if so, does it do the right thing?

27 Questions on Our First Algorithm FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) 1. Is the algorithm correct? for every valid input, does it terminate? if so, does it do the right thing? 2. How much time does it take to complete?

28 Questions on Our First Algorithm FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) 1. Is the algorithm correct? for every valid input, does it terminate? if so, does it do the right thing? 2. How much time does it take to complete? 3. Can we do better?

29 Correctness

30 Correctness FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) 0 if n = 0 F n = 1 if n = 1 F n 1 + F n 2 if n > 1

31 Correctness FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) 0 if n = 0 F n = 1 if n = 1 F n 1 + F n 2 if n > 1 The algorithm is clearly correct assuming n 0

32 Performance How long does it take?

33 Performance How long does it take? Let s try it...

34 Results Time (seconds) "Python" "Scheme" "C" "C-wiz" "Java" "C-gcc" n

35 Comments

36 Comments Different implementations perform differently it is better to let the compiler do the optimization simple language tricks don t seem to pay off

37 Comments Different implementations perform differently it is better to let the compiler do the optimization simple language tricks don t seem to pay off However, the differences do not seem to be substantial all implementations, sooner or later, seem to hit a wall...

38 Comments Different implementations perform differently it is better to let the compiler do the optimization simple language tricks don t seem to pay off However, the differences do not seem to be substantial all implementations, sooner or later, seem to hit a wall... Conclusion: the problem lies with the algorithm

39 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity

40 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity Formally, but a bit superficially for now, let T(n) be the number of basic steps needed to compute FIBONACCI(n)

41 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity Formally, but a bit superficially for now, let T(n) be the number of basic steps needed to compute FIBONACCI(n) FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2)

42 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity Formally, but a bit superficially for now, let T(n) be the number of basic steps needed to compute FIBONACCI(n) FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) T(0) = 2; T(1) = 3

43 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity Formally, but a bit superficially for now, let T(n) be the number of basic steps needed to compute FIBONACCI(n) FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) T(0) = 2; T(1) = 3 T(n) = T(n 1) + T(n 2) + 3

44 Complexity of Our First Algorithm We need a mathematical characterization of the performance of the algorithm We ll call it the algorithm s computational complexity Formally, but a bit superficially for now, let T(n) be the number of basic steps needed to compute FIBONACCI(n) FIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else return FIBONACCI(n 1) + FIBONACCI(n 2) T(0) = 2; T(1) = 3 T(n) = T(n 1) + T(n 2) + 3 T(n) F n

45 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2

46 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2

47 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 )

48 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))

49 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))...

50 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))... 2 n 2

51 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))... 2 n 2 This means that T(n) ( 2) n (1.4) n

52 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))... 2 n 2 This means that T(n) ( 2) n (1.4) n T(n) grows exponentially with n

53 Complexity of Our First Algorithm (2) So, let s try to understand how F n grows with n T(n) F n = F n 1 + F n 2 Now, since F n F n 1 F n 2 F n 3... F n 2F n 2 2(2F n 4 ) 2(2(2F n 6 ))... 2 n 2 This means that T(n) ( 2) n (1.4) n T(n) grows exponentially with n Can we do better?

54 A Better Algorithm Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,...

55 A Better Algorithm Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... Idea: we can build F n from the ground up!

56 A Better Algorithm Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... Idea: we can build F n from the ground up! SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else pprev 0 6 prev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f

57 Results Time (seconds) "PythonSmart" "Scheme" "C" "C-wiz" "Java" "C-gcc" n

58 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f

59 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f T(n) =

60 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f T(n) = 6 + 6(n 1)

61 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f T(n) = 6 + 6(n 1) = 6n

62 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f T(n) = 6 + 6(n 1) = 6n The complexity of SMARTFIBONACCI(n) is linear in n

63 Complexity of SmartFibonacci SMARTFIBONACCI(n) 1 if n = 0 2 then return 0 3 elseif n = 1 4 then return 1 5 else prev 0 6 pprev 1 7 for i 2 to n 8 do f prev + pprev 9 pprev prev 10 prev f 11 return f T(n) = 6 + 6(n 1) = 6n The complexity of SMARTFIBONACCI(n) is linear in n... almost...