Outline CS38 Introdution to Algorithms Leture 1 April 1, 2014 administrative stuff motivation and overview of the ourse stale mathings example graphs, representing graphs graph traversals (BFS, DFS) onnetivity, topologial sort, strong onnetivity April 1, 2014 CS38 Leture 1 2 Administrative Stuff Text: Introdution to Algorithms (3 rd Edition) y Cormen, Leiserson, Rivest, Stein CLRS reommended ut not required letures self-ontained slides posted online Administrative Stuff weekly homework ollaoration in groups of 2-3 enouraged separate write-ups (larity ounts) midterm and final indistinguishale from homework exept umulative, no ollaoration allowed April 1, 2014 CS38 Leture 1 3 April 1, 2014 CS38 Leture 1 4 Administrative Stuff no programming in this ourse things I assume you are familiar with: programming and asi data strutures: arrays, lists, staks, queues asymptoti notation ig-oh sets, graphs proofs, espeially indution proofs exposure to NP-ompleteness Computaility and Complexity Algorithms Systems and Software Design and Implementation Theory April 1, 2014 CS38 Leture 1 5 April 1, 2014 CS38 Leture 1 6 1
at the heart of programs lie algorithms in this ourse algorithms means: astrating prolems from aross appliation domains worst ase analysis asymptoti analysis ( ig-oh ) main figure of merit rigorous proofs paradigm (vs. heuristis ) algorithms as a key tehnology think aout: mapping/navigation Google searh Shazam word proessing (spelling orretion, layout ) ontent delivery and streaming video games (graphis, rendering ) ig data (querying, learning ) April 1, 2014 CS38 Leture 1 7 April 1, 2014 CS38 Leture 1 8 In a perfet world for eah prolem we would have an algorithm the algorithm would e the fastest possile What would CS look like in this world? Our world (fortunately) is not so perfet: for many prolems we know emarrassingly little aout what the fastest algorithm is multiplying two integers or two matries fatoring an integer into primes determining shortest tour of given n ities for many prolems we suspet fast algorithms are impossile (NP-omplete prolems) for some prolems we have unexpeted and lever algorithms (we will see many of these) April 1, 2014 CS38 Leture 1 9 April 1, 2014 CS38 Leture 1 10 Two main themes: algorithm design paradigms algorithms for fundamental prolems (data strutures as needed) definitions and onventions NP-ompleteness and introdution to approximation algorithms April 1, 2014 CS38 Leture 1 11 April 1, 2014 CS38 Leture 1 12 2
What is a prolem? Some examples: given n integers, produe a sorted list given a graph and nodes s and t, find a shortest path from s to t given an integer, find its prime fators prolem assoiates eah input to an output a prolem is a funtion: f:σ * Σ * What is an algorithm? a prolem is a funtion: f:σ * Σ * formally: an algorithm is a Turing Mahine that omputes funtion f more informal: a preisely speified sequene of asi instrutions omputing f level of detail is a judgment all high-level desription, detailed pseudo-ode April 1, 2014 CS38 Leture 1 13 April 1, 2014 CS38 Leture 1 14 Underlying model Offiially, Random Aess Mahine (RAM) essentially, low level programming language like assemly ode Will not ome up in this ourse We all an distinguish etween, e.g. x à i-th element of array A (single step) x à minimum element of array A (not a single step) Worst-ase analysis Figure of merit: resoure usage running time (primary for this ourse) storage spae others Always measure resoure usage via: funtion of the input size value of the fn. is the maximum quantity of resoure used over all inputs of given size alled worst-ase analysis April 1, 2014 CS38 Leture 1 15 April 1, 2014 CS38 Leture 1 16 Asymptoti notation Measure time/spae omplexity using asymptoti notation ( ig-oh notation ) disregard lower-order terms in running time disregard oeffiient on highest order term example: f(n) = 6n 3 + 2n 2 + 100n + 102781 f(n) is order n 3 write f(n) = O(n 3 ) Asymtoti notation aptures ehavior for large n 3n 2 +100 2 n /30 f(n) n! April 1, 2014 CS38 Leture 1 17 April 1, 2014 CS38 Leture 1 18 3
Asymptoti notation Definition: given funtions f,g:n R +, we say f(n) = O(g(n)) if there exist positive integers, n 0 suh that for all n n 0 f(n) g(n). meaning: f(n) is (asymptotially) less than or equal to g(n) if g > 0 an assume n 0 = 0, y setting = max 0 n n0 {, f(n)/g(n)} Asymptoti notation fats logarithmi : O(log n) log n = (log 2 n)/(log 2 ) so log n = O(log 2 n) for any onstant ; therefore suppress ase when write it polynomial : O(n ) = n O(1) also: O(log n) = O(n ) = n O(1) exponential : O(2 nδ ) for δ > 0 eah ound asymptotially less than next April 1, 2014 CS38 Leture 1 19 April 1, 2014 CS38 Leture 1 20 Why worst ase, asymptoti? Why worst-ase? well-suited to rigorous analysis, simple stringent requirement etter Why asymptoti? not produtive to fous on fine distintions are aout ehavior on large inputs general-purpose alg. should e salale exposes genuine arriers/motivates new ideas example April 1, 2014 CS38 Leture 1 21 April 1, 2014 CS38 Leture 1 22 Motivation: n medial students and n hospitals eah student has ranking of hospitals eah hospital has ranking of students Goal: math eah student to a hospital Goal: make the mathing stale Definition: (student x, hospital y) pair unstale if x prefers y to its math and y prefers x to its math Captures many settings, e.g., employee/employer students/dorms men/women Usually desried via men/women: ranked list of n women for eah of n men ranked list of n men for eah of n women f produe a stale mathing (no unstale pairs) April 1, 2014 CS38 Leture 1 23 April 1, 2014 CS38 Leture 1 24 4
Does a stale mathing always exist? Is there an effiient algorithm to find one? We have a well-defined prolem a proposed algorithm Now we need to prove orretness ound running time, possily requiring filling in implementation details April 1, 2014 CS38 Leture 1 25 April 1, 2014 CS38 Leture 1 26 mathed, and with no unstale pair Proof: terminates? April 1, 2014 CS38 Leture 1 27 mathed, and with no unstale pair Proof: all mathed? April 1, 2014 CS38 Leture 1 28 mathed, and with no unstale pair Proof: unstale pair (m, w)? April 1, 2014 CS38 Leture 1 29 mathed, and S ontaining no unstale pair Proof: terminates: only n 2 possile proposals, 1 per iteration all mathed: suppose not. Then some m unmathed and some w unmathed. So w never proposed to. But m proposed to everyone if ends unmathed. April 1, 2014 CS38 Leture 1 30 5
mathed, and S ontaining no unstale pair Proof: pair (m, w) not in S ase 1: m never proposed to w, ) m prefers his urrent partner ase 2: m proposed to w ) w rejeted m (in line 6 or line 5) in oth ases (m, w) is not an unstale pair. April 1, 2014 CS38 Leture 1 31 Lemma: an implement with running time O(n 2 ) Proof: reate two arrays wife, husand wife[m] = w if (m,w) in S, 0 if unmathed (same for husand) April 1, 2014 CS38 Leture 1 32 implementing step 5? for eah preferene list pref an reate inv-pref via: for i = 1 to n do inv-pref[pref[i]] = i w prefers m to m iff inv-pref[m] < inv-pref[m ] O(n 2 ) preproessing; O(1) time for eah iteration of loop We proved: Theorem (Gale-Shapley 62): there is an O(n 2 ) time algorithm that is given n rankings of women y eah of n men n rankings of men y eah of n women and outputs a stale mathing of men to women. April 1, 2014 CS38 Leture 1 34 Graphs Basi graph algorithms Graph G = (V, E) direted or undireted notation: n = V, m = E (note: m n 2 ) adjaeny list or adjaeny matrix a a a 0 1 1 0 0 0 0 1 0 a April 1, 2014 CS38 Leture 1 35 April 1, 2014 CS38 Leture 1 36 6
Graphs Graphs model many things physial networks (e.g. roads) ommuniation networks (e.g. internet) information networks (e.g. the we) soial networks (e.g. friends) dependeny networks (e.g. topis in this ourse) so many fundamental algorithms operate on graphs Graphs Graph terminology: an undireted graph is onneted if there is a path etween eah pair of verties a tree is a onneted, undireted graph with no yles; a forest is a olletion of disjoint trees a direted graph is strongly onneted if there is a path from x to y and from y to x, 8 x,y2v a DAG is a Direted Ayli Graph April 1, 2014 CS38 Leture 1 37 April 1, 2014 CS38 Leture 1 38 Graph traversals Graph traversal algorithm: visit some or all of the nodes in a graph, laeling them with useful information readth-first: useful for undireted, yields onnetivity and shortest-paths information depth-first: useful for direted, yields numering used for topologial sort strongly-onneted omponent deomposition Breadth first searh BFS(undireted graph G, starting vertex s) 1. for eah vertex v, v.olor = white, v.dist = 1, v.pred = nil 2. s.olor = grey, s.dist = 0, s.pred = nil 3. Q = ;; ENQUEUE(Q, s) 4. WHILE Q is not empty u = DEQUEUE(Q) 5. for eah v adjaent to u 6. IF v.olor = white THEN 7. v.olor = grey, v.dist = u.dist + 1, v.pred = u 8. ENQUEUE(Q, v) 9. u.olor = lak Lemma: BFS runs in time O(m + n), when G is represented y an adjaeny list. Proof? April 1, 2014 CS38 Leture 1 39 Breadth first searh BFS(undireted graph G, starting vertex s) 1. for eah vertex v, v.olor = white, v.dist = 1, v.pred = nil 2. s.olor = grey, s.dist = 0, s.pred = nil 3. Q = ;; ENQUEUE(Q, s) 4. WHILE Q is not empty u = DEQUEUE(Q) 5. for eah v adjaent to u 6. IF v.olor = white THEN 7. v.olor = grey, v.dist = u.dist + 1, v.pred = u 8. ENQUEUE(Q, v) 9. u.olor = lak BFS example from CLRS Lemma: BFS runs in time O(m + n), when G is represented y an adjaeny list. Proof: eah vertex enqueued at most 1 time; its adj. list sanned one; O(1) work for eah neighor 7