Course Introducton Course Topcs Exams, abs, Proects A quc loo at a few algorthms 1 Advanced Data Structures and Algorthms Descrpton: We are gong to dscuss algorthm complexty analyss, algorthm desgn technques (greedy methods, dvde and conquer, dynamc programmng), and examples of effcent algorthms for varous problems. We wll also dscuss advanced data structure such as Hash Tables, Bnary Search trees, and ed-blac trees. We wll also dscuss the noton of NP-completeness and NP-complete problems. Prerequstes: Computer Scence II (COSC 220) and Dscrete Mathematcs (MATH 210), both completed wth a grade of C or better. Prmary eference: Introducton to Algorthms 3 rd edton by Thomas H. Cormen, Charles E. eserson, onald. vest and Clfford Sten, MIT Press 2 Advanced Data Structures and Algorthms Topcs nclude: Mathematcal Foundaton for Analyss of Algorthms Set Notaton, Asymptotc Notatons (Bg-, Bg-O and Bg-, lttle-o, lttle- notatons) Best-Case, Worst-Case, Average- Case unnng Tmes Sortng Algorthms and Analyss Inserton Sort, Selecton Sort, Bubble Sort, Shell Sort, Merge Sort, Heap Sort, Quc Sort, near tme Sortng Algorthms: adx Sort, Bucet Sort, Countng Sort and analyss of these algorthms. Selecton Problems Maxmum, Mnmum, MnMax, General Selecton Problems Advanced Data Structures and Algorthms Topcs nclude: Advanced Data Structures Bnary Search Trees, varous Balanced Bnary Search Trees, ed-blac trees, Heaps and Prorty Queues, Dsont Set Data Structures, Hash Tables and algorthms used for these Data Structures. Graph Algorthms Graph representatons, Breadth-Frst Search, Depth-Frst Search, Mnmum Spannng Tree, Shortest Path, Maxmum Flow, and analyses of these algorthms. NP-Completeness Polynomal tme, Polynomal tme verfcaton, NPcompleteness and reducblty 3 4 Advanced Data Structures and Algorthms Exams: There wll be three exams -- two mdterms and a fnal. Once the dates are set, they wll be posted on the course web page. There wll be no mae-ups or reschedulng of exams (except n emergency cases wth evdence). abs & Proects: There wll be a seres of labs and/or proects that consst of mplementng varous algorthms and data structures or learnng new language features. Each student s requred to attend a lab sesson each wee. It s mportant to arrve on tme for each lab sesson. Each student needs to read any assgned pre-lab readngs before startng each lab sesson. Some of the labs can be completed n the lab sesson, but others wll be more dffcult and tme consumng and wll need to be completed outsde of class. Algorthm What s an algorthm? An algorthm s any well-defned computatonal procedure that taes some value, or set of values, as nput and produces some value, or set of values, as output. Input Algorthm Output Fall 2017 5 6 1
Algorthm Algorthm An algorthm s a sequence of computatonal steps that transform the nput nto the output. An algorthm s a tool for solvng a well-specfed computatonal problem. Ex) sortng problem A sequence of n numbers <a 1, a 2, a 3, a n > Sortng Algorthm Permutaton <a 1, a 2, a 3, a n > of nput sequence such that a 1 a 2 a 3 a n What s a correct algorthm for solvng a problem? An algorthm s sad to be correct f t halts wth the correct output for every possble nput. What crtera are used to determne whch algorthm s better? Measure of effcency s speed How long an algorthm taes to produce output. How do we evaluate whch algorthm s better? By analyzng algorthms mathematcally 7 8 Greedy Method follows the problem solvng heurstc of mang the locally optmal choce at each stage n the hope of fndng a global optmum. Dvde and Conquer wors by recursvely breang down a problem nto two or more sub-problems of the same (or related) type, untl these sub-problems become smple enough to be solved drectly. Dynamc Programmng solves complex problems by breang them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes of overlappng sub-problems whch are only slghtly smaller (Greedy: Con Change) Problem Mae change for a gven amount usng the smallest possble number of cons. The avalable cons are: dollars (100 cents) quarters (25 cents) dmes (10 cents) ncels (5 cents) pennes (1 cent) 9 10 (Greedy: Con Change) (Greedy: Con Change) Greedy strategy Sum = 0 epeatedly choose the largest con such that Sum + Total change amount Greedy methods do not wor n all cases!! For example: Con set = 8, 5, 1; Change = 10. Greedy Soluton = 8, 1, 1 Optmal Soluton (.e., fewest # of cons) = 5, 5 11 12 2
(Dynamc Programmng: Fbonacc Sequence) (Dynamc Programmng: Fbonacc Sequence) nt fb(nt n) f n == 0 or n == 1 return 1; else return fb(n-1) + fb(n-2); Notce that f we call, say, fb(5), we produce a call tree that calls the functon on the same value many dfferent tmes. In partcular, fb(2) s calculated three tmes from scratch. In larger examples, even more values of fb,.e., subproblems, are recalculated, resultng n an exponental tme algorthm. 1. fb(5) 2. fb(4) + fb(3) 3. (fb(3) + fb(2)) + (fb(2) + fb(1)) 4. ((fb(2) + fb(1)) + (fb(1) + fb(0))) + ((fb(1) + fb(0)) + fb(1)) 5. (((fb(1) + fb(0)) + fb(1)) + (fb(1) + fb(0))) + ((fb(1) + fb(0)) + fb(1)) We can mprove recurrence verson of Fbonacc sequence by usng dynamc programmng dea. Now, suppose we have a smple map obect, m, whch maps each value of fb that has already been calculated to ts result. The resultng functon requres only O(n) tme nstead of exponental tme (but requres O(n) space): map M (ey, value) M(0) = 1; M(1) = 1; nt fb(nt n) f map M does not contan ey n else return M(n)= fb(n-1)+ fb(n-2); return M(n) (Dvde & Conquer: Merge Sort) Merge Sort The merge sort algorthm s representatve of the dvde-and-conquer paradgm. Dvde: Dvde the n-element sequence to be sorted nto two subsequences of n/2 elements each. Conquer: Sort each subsequence recursvely usng merge sort. Combne: Merge the two sorted subsequences to produce the sorted result. (Dvde & Conquer: Merge Sort) f (p < r) Frst element ndex of array A ast element ndex of array A 15 16 (Dvde & Conquer: Merge Sort) (Dvde & Conquer: Merge Sort) MergeSort(A, 0, 7) A MergeSort(A, 0, 3) MergeSort(A, 4, 7) 17 18 3
(Dvde & Conquer: Merge Sort) (Dvde & Conquer: Merge Sort) MergeSort(A, 0, 1) MergeSort(A, 2, 3) MergeSort(A, 4, 5) MergeSort(A, 6, 7) 19 20 (Dvde & Conquer: Merge Sort) (Dvde & Conquer: Merge Sort) 0 1 2 3 3 4 5 6 2 3 4 6 0 1 3 5 Merge Merge Merge 4 6 2 3 0 5 1 3 Merge Merge Merge Merge Merge(A, p, q, r) n1 = q p + 1; n2 = r q; for = 1 to n1 [] = A[p + 1]; // s a local array for = 1 to n2 [] = A[q + ]; // s a local array [n1 + 1] = ; [n2 + 1] = ; = = 1; for = p to r f [] [] then A[] = []; = + 1; else A[] = []; = +1; // end for-loop // end of Merge functon 21 22 p q q+1 r = p 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 23 24 4
0 2 2 3 5 0 1 3 4 6 0 0 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 25 26 0 0 1 3 5 0 1 3 4 6 0 0 1 2 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 27 28 0 0 1 2 2 0 1 3 4 6 0 0 1 2 2 3 1 3 4 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 29 30 5
0 0 1 2 2 3 3 3 4 6 0 0 1 2 2 3 3 4 4 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 31 32 = r 0 0 1 2 2 3 3 4 5 6 0 0 1 2 2 3 3 4 5 6 0 2 2 3 5 0 1 3 4 6 0 2 2 3 5 0 1 3 4 6 33 34 Merge Sort Performance Snce Merge sort dvdes a lst nto two balanced lsts, the runnng tme s the same all of the tme (.e., Best case = Worst case). (1), T ( n) 2T ( n ) ( n), 2 n 1 n 1 35 6