1.5 Case Study. dynamic connectivity quick find quick union improvements applications

Transcription

1 . Case Study dynamic connectivity quick find quick union imrovements alications Subtext of today s lecture (and this course) Stes to develoing a usable algorithm. Model the roblem. Find an algorithm to solve it. Fast enough? Fits in memory? If not, figure out why. Find a way to address the roblem. Iterate until satisfied. The scientific method. Mathematical analysis. Algorithms in Java, th Edition Robert Sedgewick and Kevin Wayne Coyright 9 January, :: AM Dynamic connectivity Given a set of objects Union: connect two objects. Find: is there a ath connecting the two objects? more difficult roblem: find the ath union(, ) dynamic connectivity quick find quick union imrovements alications union(, ) union(, ) union(, 6) find(, ) find(, ) union(, ) union(, ) union(, 6) no yes 6 union(, ) find(, ) yes find(, ) yes

2 Network connectivity: larger examle Modeling the objects Q. Is there a ath from to q? Dynamic connectivity alications involve maniulating objects of all tyes. Variable name aliases. Pixels in a digital hoto. Comuters in a network. Web ages on the Internet. Transistors in a comuter chi. Metallic sites in a comosite system. q When rogramming, convenient to name objects to N-. Use integers as array index. Suress details not relevant to union-find. can use symbol table to translate from object names to integers (stay tuned) A. Yes. but finding the ath is more difficult: stay tuned (Chater ) 6 Modeling the connections Imlementing the oerations Transitivity. If is connected to q and q is connected to r, then is connected to r. Connected comonents. Maximal set of objects that are mutually connected. Find query. Check if two objects are in the same set. Union command. Relace sets containing two objects with their union. 6 6 union(, ) connected comonents connected comonents

3 Union-find data tye (API) Goal. Design efficient data structure for union-find. Number of objects N can be huge. Number of oerations M can be huge. Find queries and union commands may be intermixed. ublic class UnionFind UnionFind(int N) create union-find data structure with N objects and no connections boolean find(int, int q) are and q in the same set? void union(int, int q) relace sets containing and q with their union dynamic connectivity quick find quick union imrovements alications 9 Quick-find [eager aroach] Quick-find [eager aroach] Data structure. Integer array id[] of size N. Interretation: and q are connected if they have the same id. Data structure. Integer array id[] of size N. Interretation: and q are connected if they have the same id. i 6 9 id[i] and 6 are connected,,, and 9 are connected i 6 9 id[i] and 6 are connected,,, and 9 are connected Find. Check if and q have the same id. id[] = 9; id[6] = 6 and 6 not connected 6 9

4 Quick-find [eager aroach] Quick-find examle Data structure. Integer array id[] of size N. Interretation: and q are connected if they have the same id. i 6 9 id[i] and 6 are connected,,, and 9 are connected Find. Check if and q have the same id. id[] = 9; id[6] = 6 and 6 not connected Union. To merge sets containing and q, change all entries with id[] to id[q] i 6 9 id[i] union of and 6,,,, 6, and 9 are connected 6- roblem: many values can change roblem: many values can change Quick-find: Java imlementation Quick-find is too slow ublic class QuickFind rivate int[] id; ublic QuickFind(int N) id = new int[n]; for (int i = ; i < N; i++) id[i] = i; set id of each object to itself (N array accesses) Quick-find defect. Union too exensive (N array accesses). Trees are flat, but too exensive to kee them flat. algorithm union find quick-find N rivate int find(int ) return id[]; ublic boolean find(int, int q) return find() == find(q); ublic void union(int, int q) int id = find(); for (int i = ; i < id.length; i++) if (id[i] == id) id[i] = id[q]; return comonent id for ( array access) check whether and q are in the same comonent change all entries with id[] to id[q] (N array accesses) Ex. Takes N array accesses to rocess sequence of N union commands on N objects. 6

5 Quadratic algorithms do not scale Rough standard (for now). 9 oerations er second. 9 words of main memory. Touch all words in aroximately second. a truism (roughly) since 9! Ex. Huge roblem for quick-find. 9 union commands on 9 objects. Quick-find takes more than oerations. + years of comuter time! Paradoxically, quadratic algorithms get worse with newer equiment. New comuter may be x as fast. But, has x as much memory so roblem may be x bigger. With quadratic algorithm, takes x as long! dynamic connectivity quick find quick union imrovements alications Quick-union [lazy aroach] Quick-union [lazy aroach] Data structure. Integer array id[] of size N. Interretation: id[i] is arent of i. Root of i is id[id[id[...id[i]...]]]. kee going until it doesn t change Data structure. Integer array id[] of size N. Interretation: id[i] is arent of i. Root of i is id[id[id[...id[i]...]]]. kee going until it doesn t change i 6 9 id[i] q i 6 9 id[i] q 's root is 9; 's root is 6 Find. Check if and q have the same root. 's root is 9; 's root is 6 and are not connected 9

6 Quick-union [lazy aroach] Quick-union examle Data structure. Integer array id[] of size N. Interretation: id[i] is arent of i. Root of i is id[id[id[...id[i]...]]]. kee going until it doesn t change i 6 9 id[i] q Find. Check if and q have the same root. Union. To merge sets containing and q, set the id of 's root to the id of q's root. i 6 9 id[i] 's root is 9; 's root is 6 and are not connected 6 q roblem: trees can get tall only one value changes Quick-union: Java imlementation Quick-union is also too slow ublic class QuickUnion rivate int[] id; ublic QuickUnion(int N) id = new int[n]; for (int i = ; i < N; i++) id[i] = i; rivate int find(int i) while (i!= id[i]) i = id[i]; return i; set id of each object to itself (N array accesses) chase arent ointers until reach root (deth of i array accesses) Quick-find defect. Union too exensive (N array accesses). Trees are flat, but too exensive to kee them flat. Quick-union defect. Trees can get tall. Find too exensive (could be N array accesses). ublic boolean find(int, int q) return find() == find(q); check if and q have same root (deth of and q array accesses) algorithm union find quick-find N ublic void union(int, int q) int i = root(), j = root(q); id[i] = j; change root of to oint to root of q (deth of and q array accesses) quick-union N N includes cost of finding root worst case

7 Imrovement : weighting Weighted quick-union. Modify quick-union to avoid tall trees. Kee track of size of each set. Balance by linking small tree below large one. dynamic connectivity quick find quick union imrovements alications Ex. Union of and. Quick union: link 9 to 6. Weighted quick union: link 6 to 9. size 9 6 q 6 Weighted quick-union examle Weighted quick-union: Java imlementation Data structure. Same as quick-union, but maintain extra array sz[i] to count number of objects in the tree rooted at i. Find. Identical to quick-union. return find() == find(q); no roblem: trees stay flat Union. Modify quick-union to: Merge smaller tree into larger tree. Udate the sz[] array. int i = find(); int j = find(q); if (sz[i] < sz[j]) id[i] = j; sz[j] += sz[i]; else id[j] = i; sz[i] += sz[j];

8 Weighted quick-union analysis Weighted quick-union analysis Analysis. Find: takes time roortional to deth of and q. Union: takes constant time, given roots. Proosition. Deth of any node x is at most lg N. Analysis. Find: takes time roortional to deth of and q. Union: takes constant time, given roots. Proosition. Deth of any node x is at most lg N. Pf. When does deth of x increase? Increases by when tree T containing x is merged into another tree T. The size of the tree containing x at least doubles since T! T. Size of tree containing x can double at most lg N times. Why? 6 9 T T N = deth(x) =! lg N x x 9 Weighted quick-union analysis Imrovement : ath comression Analysis. Find: takes time roortional to deth of and q. Union: takes constant time, given roots. Quick union with ath comression. Just after comuting the root of, set the id of each examined node to oint to that root. Proosition. Deth of any node x is at most lg N. algorithm union find 9 6 quick-find N quick-union N N weighted QU lg N lg N 6 includes cost of finding root 9 root(9) Q. Sto at guaranteed accetable erformance? A. No, easy to imrove further.

9 Path comression: Java imlementation Weighted quick-union with ath comression examle Standard imlementation: add second loo to find() to set the id[] of each examined node to the root. Simler one-ass variant: halve the ath length by making every other node in ath oint to its grandarent ublic int find(int i) while (i!= id[i]) id[i] = id[id[i]]; i = id[i]; return i; only one extra line of code! no roblem: trees stay VERY flat In ractice. No reason not to! Kees tree almost comletely flat. 6- WQUPC erformance Summary Proosition. [Tarjan 9] Starting from an emty data structure, any sequence of M union and find os on N objects takes O(N + M lg* N) time. Proof is very difficult. But the algorithm is still simle! actually O(N + M α(m, N)) see COS Bottom line. WQUPC makes it ossible to solve roblems that could not otherwise be addressed. algorithm worst-case time quick-find M N Linear algorithm? Cost within constant factor of reading in the data. In theory, WQUPC is not quite linear. In ractice, WQUPC is linear. because lg* N is a constant in this universe Amazing fact. No linear-time algorithm exists. N lg* N lg* function number of times needed to take the lg of a number until reaching quick-union weighted QU QU + ath comression weighted QU + ath comression M union-find oerations on a set of N objects Ex. [ 9 unions and finds with 9 objects] WQUPC reduces time from years to 6 seconds. Suercomuter won't hel much; good algorithm enables solution. M N N + M log N N + M log N N + M lg* N 6

10 Union-find alications dynamic connectivity quick find quick union imrovements alications Percolation. Games (Go, Hex). Network connectivity. Least common ancestor. Equivalence of finite state automata. Hoshen-Koelman algorithm in hysics. Hinley-Milner olymorhic tye inference. Kruskal's minimum sanning tree algorithm. Comiling equivalence statements in Fortran. Morhological attribute oenings and closings. Matlab's bwlabel() function in image rocessing. Percolation Percolation A model for many hysical systems: N-by-N grid of sites. Each site is oen with robability (or blocked with robability -). System ercolates if to and bottom are connected by oen sites. A model for many hysical systems: N-by-N grid of sites. Each site is oen with robability (or blocked with robability -). System ercolates if to and bottom are connected by oen sites. ercolates blocked site does not ercolate model system vacant site occuied site ercolates N = emty oen site does not ercolate full oen site site connected to to no oen site connected to to electricity material conductor insulated conducts fluid flow material emty blocked orous social interaction oulation erson emty communicates 9

11 Likelihood of ercolation Percolation hase transition Deends on site vacancy robability. When N is large, theory guarantees a shar threshold *. > *: almost certainly ercolates. < *: almost certainly does not ercolate. Q. What is the value of *? low does not ercolate medium ercolates? high ercolates ercolation robability N =.9 * site vacancy robability N = Monte Carlo simulation UF solution to find ercolation threshold Initialize N-by-N whole grid to be blocked. Declare random sites oen until to connected to bottom. Vacancy ercentage estimates *. How to check whether system ercolates? Create an object for each site. Sites are in same set if connected by oen sites. Percolates if any site in to row is in same set as any site in bottom row. brute force algorithm needs to check N airs full oen site (connected to to) emty oen site (not connected to to) blocked site full oen site (connected to to) emty oen site (not connected to to) blocked site N =

12 UF solution to find ercolation threshold UF solution to find ercolation threshold Q. How to declare a new site oen? Q. How to declare a new site oen? A. Take union of new site and all adjacent oen sites. oen this site oen this site full oen site (connected to to) 6 9 full oen site (connected to to) emty oen site (not connected to to) blocked site emty oen site (not connected to to) blocked site N = N = 6 UF solution: a critical otimization UF solution: a critical otimization Q. How to avoid checking all airs of to and bottom sites? Q. How to avoid checking all airs of to and bottom sites? A. Create a virtual to and bottom objects; system ercolates when virtual to and bottom objects are in same set. virtual to row full oen site (connected to to) 6 9 full oen site (connected to to) emty oen site (not connected to to) blocked site emty oen site (not connected to to) blocked site virtual bottom row N = N =

13 Percolation threshold Subtext of today s lecture (and this course) Q. What is ercolation threshold *? A. About.96 for large square lattices. ercolation constant known only via simulation ercolation robability *.9 site vacancy robability Stes to develoing a usable algorithm. Model the roblem. Find an algorithm to solve it. Fast enough? Fits in memory? If not, figure out why. Find a way to address the roblem. Iterate until satisfied. The scientific method. Mathematical analysis. Fast algorithm enables accurate answer to scientific question. 9