Lecture 2: Sorting and the Big O CS204/209 : Algorithms (and Scientific Computing) Niall Madden Wednesday, 16 September 2009 CS204/209 Lecture 2: Sorting and the Big O 1/18
In today s lecture 1 Recall... Algorithms 2 Sorting Bubble Sort Insertion Sort 3 The correctness of algorithms Loop Invariants 4 Comparing Algorithms 5 The Big-O Most of the content is based on Chap. 2 of Cormen et al s Introduction to Algorithms, and Sections 2.1 and 2.2 of Rosen s Discrete Mathematics (511 ROS). CS204/209 Lecture 2: Sorting and the Big O 2/18
Last week s problem Problem 1 6 busybodies each have a piece of news. How many phone calls are required so that all of them know all the news? Any ideas? CS204/209 Lecture 2: Sorting and the Big O 3/18
Recall... Algorithms Definition An Algorithm is a finite set of precise instructions for performing a computation or for solving a problem. CS204/209 Lecture 2: Sorting and the Big O 4/18
Sorting The basic problem is Given a list of integers {a 1, a 2,..., a n } rearrange them into a list {b 1, b 2,..., b n } such that b 1 b 2 b n. Methods that we will look at include the Bubble, Insertion, Shaker, Merge, Quick and Tournament sort algorithms. CS204/209 Lecture 2: Sorting and the Big O 5/18
Sorting Bubble Sort Bubble sort is one of the simplest and slowest algorithms for sorting. The basic idea is go through the list {a 1, a 2,..., a n 1, a n } from left to right. If an adjacent pair of element are out of order, swap them. At the end of the 1st pass, the largest element should be at the end. Now repeat the process for the list {a 1, a 2,..., a n 1 }. and then for the list {a 1, a 2,..., a n 2 }. Continue until you have sorted the list {a 1, a 2 }. CS204/209 Lecture 2: Sorting and the Big O 6/18
Sorting Bubble Sort Here is a pseudo-code description of the algorithm: BubbleSort(a 1, a 2,..., a n ) FOR i 1, 2,..., n 1 FOR j 1, 2,..., n i IF a j > a j+1 swap(a j, a j+1 ) END END END Bubble Sort CS204/209 Lecture 2: Sorting and the Big O 7/18
Sorting Bubble Sort Example Use Bubble Sort to arrange the following integers in ascending order { 9, 12, 3, 2, 6, 8 } CS204/209 Lecture 2: Sorting and the Big O 8/18
Sorting Insertion Sort Insertion Sort This can be thought of as being like how one might sort a hand of cards. Start with the cards face down on the table. 1 Pick the first card: it forms the trivially sorted list {a 1 }. 2 Pick the next card from the table, and insert it into the correct place (i.e., before or after a 1 ), giving the sorted list {a 1, a 2 }. 3 Pick a third cards and insert it into the correct place in the list. Now we have sorted {a 1, a 2, a 3 }. 4 In general, at step j, insert a j into the correct location of {a 1, a 2,..., a j 1 }. CS204/209 Lecture 2: Sorting and the Big O 9/18
Sorting Insertion Sort Pseudocode for Insertion Sort 1: for j = 2 to n do 2: key a j 3: i j 1 4: while i > 0 and a i > key do 5: a i+1 a i 6: i i 1 7: end while 8: a i+1 key 9: end for CS204/209 Lecture 2: Sorting and the Big O 10/18
Sorting Insertion Sort Example Use Insertion Sort to arrange the following integers in ascending order { 9, 12, 3, 2, 6, 8 } CS204/209 Lecture 2: Sorting and the Big O 11/18
The correctness of algorithms The algorithm described above seems to work: after it has run we end up with a sorted list. But how can we prove this. The idea is to use a Loop Invariant this notion will repeat through-out the course. At the end of each pass j through the algorithm given above, the entries in the sublist {a 1, a 2,..., a j } are fist j entries of the original list, but in sorted order. But we need to state this a bit more formally... CS204/209 Lecture 2: Sorting and the Big O 12/18
The correctness of algorithms Loop Invariants To prove an algorithm works using a loop invariant, we use Initialisation: before the first iteration of the loop, the statement is true. Maintenance: if it is true at the start of an iteration of the loop, it is true at the end of it. Termination: conclude that when the loop finishes, the statement is true. Now lets use this idea to show that Insertion Sort works: CS204/209 Lecture 2: Sorting and the Big O 13/18
Comparing Algorithms We now have two algorithms for sorting a list. In general, how do we know which one to choose? One of the most important determinants is the amount of effort required for the method. That requires us to have some way of expressing the expense associated with a given procedure for solving a particular size. Exercise Suppose we have two algorithms for sorting a list of length n. One takes 100n 2 operations and the other takes n 3 + n. Which should we use? CS204/209 Lecture 2: Sorting and the Big O 14/18
The Big-O The above exercise is interesting, but very simplified/idealised. The time taken to run the same algorithm on the same problem is very likely to vary from one computer to another. But this difference probably won t depend on n the size of the problem. So we use the Big-O notation. CS204/209 Lecture 2: Sorting and the Big O 15/18
The Big-O Definition (Big-O) Given functions f : N R and g : N R, we say that f (x) is O ( g(x) ) if there are constants C and k such that for all x > k. f (x) C g(x) In English, we read this as f is big-oh of g. The Big-O notation was introduced by Paul Bachmann (Germany) in 1892. The symbol is sometimes called the Landau symbol in honour of Edmund Landau. It was introduced into computer science by (the legendary) Donald Knuth. CS204/209 Lecture 2: Sorting and the Big O 16/18
The Big-O Example Show that f (x) = x 2 + 2x + 1 is O(x 2 ). Note that x 2 is also O ( x 2 + 2x + 1 ). CS204/209 Lecture 2: Sorting and the Big O 17/18
The Big-O Example Show that f (x) = 7x 2 is O(x 3 ). Is it the case that x 3 is O(7x 2 )? CS204/209 Lecture 2: Sorting and the Big O 18/18