Electrical & Computer Engineering University of Waterloo Canada March 8, 2007

Size: px

Start display at page:

Download "Electrical & Computer Engineering University of Waterloo Canada March 8, 2007"

Milo Miles
5 years ago
Views:

R). Some common axioms relations may satisfy: 1.

1 Electrical & Computer Engineering University of Waterloo Canada March 8, 2007 Binary Relations I Recall that a binary relation on a set X is a set R X 2. We may interpret a binary relation as a directed graph G = (X, R). Some common axioms relations may satisfy: 1. Transitive (T ): x, y, z. (R(x, y) R(y, z) R(x, z)) If there is a path from x to z, there is an edge from x to z.

2 Binary Relations II 2. Reflexive: x. R(x, x) Every vertex has an edge to itself. 3. Symmetric (S) x, y. R(x, y) R(y, x) If there is an edge from x to y, there is an edge from y to x. Usually one draws the graph without arrows: and it is called simply a graph rather than a directed graph. Binary Relations III 4. Antisymmetric (A) x, y. R(x, y) R(y, x) (x = y) When the relation is reflexive, transitive and also antisymmetric, it is a partial order. A rough classification of : Binary Relation/Directed Graph Graph (S) Preorder/Quasiorder (T,R) Equivalence (T,R,S) Partial order/poset (T,R,A) Tree order Total order

3 Binary Relations IV Good algorithms for managing the common classes of are known. If you can identify the abstract relation(s) underlying a problem, this may lead you directly to efficient algorithms. Part I Equivalence Relations

4 Equivalence relations and partitions I An equivalence relation is a binary relation that is reflexive, transitive, and symmetric. (The most familiar example: equality, = ). Pictured as a graph, an equivalence relation is a collection of cliques: a b e f c d g For an equivalence X 2, we write [a] = {b X : a b} for the equivalence class of a; Equivalence relations and partitions II X / for the set of equivalence classes induced by : X / = {[a] : a X } X / is a partition. (Recall that a partition of a set X is a collection of subsets Y 1,..., Y k of X that are pairwise disjoint and satisfy Y i = X.) Example: In the above figure, the equivalence classes are {{a, b, c, d}, {e, f, g}}. Example: take N with a b (a mod 5 = b mod 5). The equivalence classes N/ are {{0, 5, 10,...}, {1, 6, 11,...},..., {4, 9, 14,...}}. Common algorithmic problems we encounter with equivalence classes: Answering queries of the form Is a b?

5 Equivalence relations and partitions III Maintaining an equivalence relation as we progressively decide objects are equivalent. (This results from an inductively defined equivalence relation.) Example: the Nelson-Oppen method for equational reasoning [7]. Maintaining an equivalence relation as we progressively decide objects are not equivalent. (This results from a co-inductive definition of equivalence [6].) Example: minimizing states of a DFA [4], maintaining bisimulations, congruence closure [3]. A system of representatives is the primary means for efficient manipulation of equivalence relations. A system of representatives for is a function s : (X / ) X choosing a single element from each block of the partition, such that a b if and only if s(a) = s(b) Equivalence relations and partitions IV Example: to reason about equivalence of integers modulo 5, we could choose the representatives 0, 1, 2, 3, and 4. The integer 1 represents the equivalence class [1] = {1, 6, 11, 16,...}. With a means to quickly compute representatives, we can test whether a b by computing the representatives of the equivalence classes [a] and [b], then using equality. If the equivalence relation is static, one can precompute a system of representatives as e.g., a table. If the equivalence relation is discovered dynamically, more sophisticated methods are needed.

6 Disjoint Set Union I Disjoint Set Union is algorithms-speak for maintaining an inductively-defined equivalence relation: Initially we have a set of objects, none of which are known to be equivalent. We gradually discover that objects are equivalent, and we wish to maintain a representation of the equivalence relation that lets us quickly answer queries of the form Is a b? Interface: union(a, b): include a b in the equivalence relation find(a): returns an equivalence class representative (ECR) for a. There is wonderfully elegant data structure due to Tarjan [8] that performs these operations in O(nα(n)) time, where α(n) 3 for n less than (cosmologists best estimate of) the number of particles in the universe. Disjoint Set Union II Tarjan s data structure maintains the equivalence relation on the set X as a forest a collection of trees. Each node in a tree is an element of the set X, each tree is an equivalence class, and each root is an equivalence class representative. a b c d e f g A forest representation of the equivalence classes {{a, b, c, d}, {e, f, g}}. Each element has a pointer to its parent; to determine the equivalence class representative, we follow the parent pointers to the root of the tree.

7 Disjoint Set Union III The efficiency of the representation depends on how deep the trees are. To keep the trees shallow, two techniques are employed: (i) path compression; and (ii) union by rank. Record representation: for each element x X, we track parent(x): a pointer to the parent of x, or a pointer to itself if it is the root (alternately, a null pointer can be used.) rank(x): indicates how deep trees are (but, not depth per se). Pseudocode for find(a): find (a) if parent(a) a then parent(a) find(parent(a)) return parent(a) Disjoint Set Union IV This recursively follows the parent pointers up to the root, then rewrites all the parent pointers so they point directly at the root, called path compression : d f e d f c e c Left: tree. Right: after calling find(c). A simple way to implement union(a,b): just make the root of a s tree have b as a parent. union(a,b) parent( find (a)) b

8 Disjoint Set Union V However, this can lead to poorly balanced trees. For better asymptotic efficiency, one can track how deep the trees are and always make the deeper tree the parent of the shallower tree: called union by rank. union(a,b) pa find(a) pb find(b) if pa=pb then return if rank(pa) > rank(pb) then parent(pb) pa else parent(pa) pb if (rank(pa) = rank(pb)) rank(pb) rank(pa) + 1 Disjoint Set Union VI Tarjan proved that using both path compression and union by rank, a sequence of n calls to union and find requires O(nα(n)) time, where α(n) 3 for > = n >; i.e., a tower of powers-of-two. The function α(n) is the inverse of the Ackermann function; see CLR [2] or [8] for details. For any practical purpose, the time required by Tarjan s algorithm is indistinguishable from O(n) for a sequence of n operations; or O(1) per operation amortized time (to come.)

9 Part II Graphs Representation of Graphs I Here are four common methods of representing graphs. If the graph is large (e.g., infinite), the structure is not known beforehand, etc., we may choose an implicit representation for the graph, where vertices and edges are computed on-the-fly as needed. For example, the graph G = (N, E) where (x, y) E if and only if y divides x, is an infinite graph where the edges can be computed on the fly by factorization. An explicit representation is one where we directly encode the structure of the graph in a data structure. Some common methods for this:

10 Representation of Graphs II Adjacency matrix: an n n matrix A of 0 s and 1 s, with A ij = 1 if and only if v i, v j E. Row i indicates the out edges for vertex i, and column i indicates the in edges. A = b d a c Representation of Graphs III Adjacency lists: each vertex maintains a set of vertices to/from which there is an edge e.g. out(a) = {b, c} out(b) = {d} out(c) = {d} out(d) = If the graph structure is static (i.e., not changing as the algorithm runs), it is common to represent lists of inand out- edges as vectors, for efficiency. For more elaborate algorithms on e.g. weighted graphs, a representation of this sort is commonly used:

11 Representation of Graphs IV public class Edge { Vertex x, y; double weight; } public class Vertex { Set<Edge> out; Set<Edge> in; } Depth-First Search I One of the commonest operations on a graph is to visit the vertices of the graph one by one in some desired order. This is commonly called a search. In a depth-first search, we explore along a single path into the graph as far as we can until no new vertices can be reached; then we return to some earlier point where new vertices are still reachable and continue. (Think of exploring a maze.) Example of a depth-first search (yellow) starting at the center vertex of this graph:

12 Depth-First Search II As we visit each new vertex, we perform some action there. The choice of action depends on what we hope to accomplish; for now we will just call it visiting the vertex, but later we will see examples of specific useful actions. We might choose to visit the vertex the first time we see it (preorder), or the last time we see it (postorder) Here is a recursive implementation of depth-first search. It uses a set Seen to track which vertices have been visited. One can also include a flag field as part of the vertex data structure that can be marked to indicate the vertex has been seen. Depth-First Search III dfs(x) dfs(x, ) dfs(x, Seen) if x Seen Seen Seen {x} preordervisit (x) // Do something For each edge (x, y), dfs(y,seen) postordervisit (x) // Do something This search is easily implemented in a nonrecursive version, using a stack data structure to keep track of the current path into the graph:

13 Depth-First Search IV dfs(x) Seen = Stack S push(s,x) while S is not empty, y pop(s) if y Seen then Seen Seen { y } preordervisit (y) for each edge (y, z), push(s,z) Topological Sort I A Directed Acyclic Graph (DAG) is a graph in which there are no cycles (i.e., paths from a vertex to itself.) The reflexive, transitive closure of a DAG is a partial order. (If you add to a DAG an edge (x, y) whenever there is a path from x to y, plus self-loops (x, x), the resulting edge relation is a partial order: reflexive, transitive, and anti-symmetric.) Every finite partial order can be extended to a total order: i.e., if is a partial order on a finite set, there is a total order such that (x y) (x y); or, more obtusely,. (Axiom of choice implies this for infinite sets also.)

14 Topological Sort II Example: let V = N 2 (pairs of natural numbers), and for all i, j, put edges (i, j) (i + 1, j) and (i, j) (i, j + 1):... Then the transitive reflexive closure of this graph is a partial order where (i, j) (i, j ) if and only if i i and j j : (2, 0) (1, 1) (0, 2) (1, 0) (0, 1) (0, 0) Topological Sort III One way to extend to a total order is:. An example of what computer scientists call dovetailing. Topological sort is a method for obtaining a total-order extension of a partial order.

15 Topological Sort IV Example: Suppose we want to evaluate a digital circuit: a b c Build a graph where signals are vertices, and an edge indicates that one signal depends upon another (a dependence graph ): d e e d c a b Topological Sort V The transitive, reflexive closure of this graph yields an order, where e.g., e d means signal e can be evaluated only after signal d. Extending to a total order gives us a valid order in which to evaluate the signals, e.g., e d c b a If we evaluate signals in the order a, b, c, d, e we respect the dependencies. Other examples: Ordering the presentation of topics in a course or paper. Solving equations Makefiles Planning (keeping track of task dependencies) Spreadsheets and dataflow languages [5] Ordering static initializers in programming languages Dynamization of static algorithms e.g. [1]

16 Topological Sort VI Here is an algorithm for topological sort based on depth-first search. Note that there are many ways in which a partial order can be extended to a total order; this is just one method. TopologicalSort (V,E) Set<Node> visited; List <Node> order; for x V dfs(x, visited, order) dfs(x, visited, order) if x visited visited.add(x) for each out edge (x,y) dfs(y, visited, order) order. insertback(x) Topological Sort VII We search the dependence graph depth-first, visiting vertices postorder at which time we insert them at the back of the list. Example: for the circuit example, a depth-first search might visit the vertices in the order a, b, d, c, e.

17 Connected components of undirected graph I Defn: A set of vertices Y V is connected if for every a, b Y there is a path from a to b. Y is a maximal connected component if it cannot be enlarged, i.e., for any connected set of vertices Y with Y Y, Y = Y. Note that the connected components of a graph form a partition of the vertices: a b q g d c e The connected components are {{a, b, g, q}, {c, d, e}}. Using Tarjan s disjoint set union, there is a very simple algorithm for connected components: Connected components of undirected graph II 1. Have a parent pointer and rank associated with each vertex (e.g., by creating a separate record for each vertex, or by storing these fields directly in the vertex data structure.) 2. For each edge (a, b), call union(a, b). No searching is necessary! The complexity is O( E + V α( E + V )), practically linear in the number of vertices and edges.

18 I [1] Umut A. Acar, Guy E. Blelloch, Robert Harper, Jorge L. Vittes, and Shan Leung Maverick Woo. Dynamizing static algorithms, with applications to dynamic trees and history independence. In SODA 04: Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms, pages , Philadelphia, PA, USA, Society for Industrial and Applied Mathematics. bib pdf [2] Thomas H. Cormen, Charles E. Leiserson, and Ronald R. Rivest. Intoduction to algorithms. McGraw Hill, bib II [3] Peter J. Downey, Ravi Sethi, and Robert Endre Tarjan. Variations on the common subexpression problem. Journal of the ACM (JACM), 27(4): , bib pdf [4] J. E. Hopcroft. An n log n algorithm for minimizing the states in a finite-automaton. In Z. Kohavi, editor, Theory of Machines and Computations, pages Academic Press, bib [5] Wesley M. Johnston, J. R. Paul Hanna, and Richard J. Millar. Advances in dataflow programming languages. ACM Comput. Surv., 36(1):1 34, bib pdf

19 III [6] Y. N. Moschovakis. Elementary Induction on Abstract Structures. North-Holland, Amsterdam, bib [7] Greg Nelson and Derek C. Oppen. Fast decision procedures based on congruence closure. Journal of the ACM (JACM), 27(2): , bib pdf [8] R. E. Tarjan. Efficiency of a good but not linear disjoint set union algorithm. Journal of the ACM (JACM), 22: , bib pdf

Outline Purpose Disjoint Sets Maze Generation. Disjoint Sets. Seth Long. March 24, 2010

Outline Purpose Disjoint Sets Maze Generation. Disjoint Sets. Seth Long. March 24, 2010 March 24, 2010 Equivalence Classes Some Applications Forming Equivalence Classes Equivalence Classes Some Applications Forming Equivalence Classes Equivalence Classes The class of all things equivelant