The block triangular form and bipartite matchings

Transcription

1 Outline The block triangular form and bipartite matchings Jean-Yves L Excellent and Bora Uçar GRAAL, LIP, ENS Lyon, France CR-07: Sparse Matrix Computations, November /48 CR09

2 Outline Outline 1 The block triangular form The BTF, computation and properties Bipartite matching 2 2/48 CR09

3 Definitions: Reducibility The BTF, computation and properties Bipartite matching Reducible matrix: An n n square matrix is reducible if there exists an n n permutation matrix P such that ( ) PAP T A11 A = 12, O A 22 where A 11 is an r r submatrix, A 22 is an (n r) (n r) submatrix, where 1 r < n. Irreducible matrix: There is no such permutation matrix. Theorem: An n n square matrix is irreducible iff its directed graph is strongly connected. 3/48 CR09

4 The BTF, computation and properties Bipartite matching Definitions: Fully indecomposability Fully indecomposable matrix: There is no permutation matrices P and Q such that «A11 A 12 PAQ =, O A 22 with the same condition on the blocks and their sizes as above. Decomposable matrix: There are permutation matrices P and Q such that the permuted matrix PAQ is of the form above. In general we are interested in fully indecomposable A 11 and A 22. That is, the matrices in the block triangular form 0 1 A 11 A 12 A 1K O A 22 A 2K PAQ = C.. A, O O A KK where each A kk is fully indecomposable. 4/48 CR09

5 Solving Ax = b in BTF The BTF, computation and properties Bipartite matching Find Ax = b using PAQy = Pb, with 0 1 A 11 A 12 A 1K O A 22 A 2K PAQ = C.. A, O O A KK where A kk contains indices r(k)-tor(k + 1) 1. A PAQ permute A in into BTF x Pb permute b for k = K downto 1 do I 1 : r(k) 1 indices above the kth block J r(k) : r(k + 1) 1 indices of the kth block x(j) A(J, J)\x(J) x(i ) x(i ) A(I, J) x(j) x Qx undo the permutation 5/48 CR09

6 BTF: Basic definitions The BTF, computation and properties Bipartite matching Bipartite graph representation is used for rectangular matrices and also unsymmetric matrices. The bipartite graph H of a matrix A has two sets of vertices (R and C) corresponding to respectively the row and columns indices of A. a r,c 0 (r, c) H(A) A bipartite graph with m rows and n columns has the Hall property if every set of k column vertices is adjacent to at least k row vertices for all 0 k n. Hall theorem A bipartite graph has a column matching iff it has the Hall property. 6/48 CR09

7 Strong Hall property The BTF, computation and properties Bipartite matching A bipartite graph with m rows and n columns has the strong Hall property if every set of k column vertices is adjacent to at least k + 1 row vertices for all 0 k < n. Any matrix that is not strong Hall can be permuted in block triangular form. Decomposable matrix: There are permutation matrices P and Q such that A 11 A 12 A 1K O A 22 A 2K PAQ =......, O O A KK with each diagonal block being square and fully indecomposable. 7/48 CR09

8 Finding the BTF The BTF, computation and properties Bipartite matching The block triangular form is based on a canonical decomposition of bipartite graphs known as the Dulmage-Mendelsohn decomposition (late 50s, early 60s; Pothen 84, Pothen and Fan 90, Duff and U. 09). A two (or three) step algorithm (for square, full rank sparse matrices) 1 Find a perfect matching in the bipartite graph of the matrix, 2 Permute the matrix to have a zero free diagonal, 3 Find the strong components of the directed graph of the permuted matrix. 8/48 CR09

9 Permutation matrices The BTF, computation and properties Bipartite matching A permutation matrix is a square (0, 1)-matrix where each row and column has a single 1. If P is a permutation matrix, PP T = I, i.e., it is an orthogonal matrix. Let, A = 2 3 and suppose we want to permute columns as [2, 1, 3]. Define p 2,1 = 1, p 1,2 = 1, p 3,3 = 1, and B = AP (if column j to be at position i, set p ji = 1) B = 2 = /48 CR09

10 The BTF, computation and properties Bipartite matching Matching in bipartite graphs and permutations A matching in a graph is a set of edges no two of which share a common vertex. We will be mostly dealing with matchings in bipartite graphs. In matrix terms, a matching in the bipartite graph of a matrix corresponds to a set of nonzero entries no two of which are in the same row or column. A vertex is said to be matched if there is an edge in the matching incident on the vertex, and to be unmatched otherwise. In a perfect matching, all vertices are matched. The cardinality of a matching is the number of edges in it. A maximum cardinality matching or a maximum matching is a matching of maximum cardinality. 10/48 CR09

11 The BTF, computation and properties Bipartite matching Matching in bipartite graphs and permutations Given a square matrix whose bipartite graph has a perfect matching, such a matching can be used to permute the matrix such that the matching entries are along the main diagonal. r 1 r 2 c 1 c 2 r 3 c = /48 CR09

12 The BTF The block triangular form The BTF, computation and properties Bipartite matching A two (or three) step algorithm 1 Find a perfect matching in the bipartite graph of the matrix, 2 Permute the matrix to have a zero free diagonal, 3 Find the strong components of the directed graph of the permuted matrix. The strong components found in the last step are independent of the matching found in the first step. 12/48 CR09

13 The BTF The block triangular form The BTF, computation and properties Bipartite matching The strong components found in the last step are independent of the!"#$%&'()**'+$,-.')#/'0%1/-/%1/%"'$2',)"+30%& matching found in the first step (figure from J. Gilbert).! # $ % " & '! # $ % " & '! # & $ " ' %! # & $ " ' %!! ## $$ %% && '' "" $! % # " ' &! # $ % " & '! # & $ " ' %! # & $ " ' % 13/48 CR09 $!!# %$ #% '& &' ""

14 Matching: Definitions The BTF, computation and properties Bipartite matching A matching M is a set of edges if no two edges in M are incident on the same vertex. A vertex is said to be matched (with respect to a given matching) if there is an edge in the matching incident on the vertex, and to be unmatched otherwise. 14/48 CR09

15 Matching: Definitions The BTF, computation and properties Bipartite matching M-alternating walk is a sequence of edges where every second edge is in M. M-alternating tour is an alternating walk that starts and ends at the same vertex. M-alternating path is a path whose edges are alternately in M and not in M. u M v denotes that vertex u reaches vertex v with an M-alternating path (we assume that u and v are different, in other words an alternating path has at least one edge). M-alternating cycle is an alternating walk without repeated vertices other than the start and end vertices. 15/48 CR09

16 The BTF, computation and properties Bipartite matching Finding the BTF (general sparse matrix case) A two (or three) step algorithm 1 Find a maximum cardinality matching in the bipartite graph of the matrix, 2 Decompose into strong Hall blocks H C S C V C H R A H S R O A S. (1) V R O O A V The block A H, formed by the rows in the set H R and the columns in the set H C, is underdetermined; The block A S, formed by the rows in the set S R and the columns in the set S C, is square and can itself have a btf with the blocks on the diagonal being all square (fine decomposition); The block A V, formed by the rows in the set V R and the columns in the set V C, is overdetermined. 16/48 CR09

17 The BTF, computation and properties Bipartite matching Finding the BTF (general sparse matrix case) V R = {rows reachable via alt. path from some unmatched row} V C = {cols reachable via alt. path from some unmatched row} H R = { rows reachable via alt. path from some unmatched col} H C = { cols reachable via alt. path from some unmatched col } S R = R \ V R \ H R S C = C \ V C \ H C 17/48 CR09

18 The BTF, computation and properties Bipartite matching Finding the BTF (general sparse matrix case)!"#$%&'()'*+'#,-.*/+'0-$1-,232-*!! # $ % " & ' ( )!*!!!#! # $ % " & ' ( )!*!!!" #" $"! # $ % " & ' ( )!* # $ % " & ' ( )!*!!!% #% $%!!!# Figure from J. R. Gilbert. 18/48 CR09

19 The BTF, computation and properties Bipartite matching Dulmage-Mendelsohn decomposition: Properties The rows in H R are completely matched to the columns in H C. The columns in S C are completely matched to the rows in S R and vice versa. The columns in V C are completely matched to the rows in V R. The block triangular form is unique. In other words, any maximum matching yields the same sets H R, H C, S R, S C, V R, and V C. The previous two properties also imply that all entries of a maximum matching should reside in the blocks on the diagonal of the btf. In the block triangular form of a structurally rank deficient, square matrix, the horizontal and vertical blocks both should be present. The square block may be missing. 19/48 CR09

20 The BTF, computation and properties Bipartite matching Dulmage-Mendelsohn decomposition: Properties No edge joins V R and S C, or V R and H C, or S R and H C. The subgraph induced by V R and V C has the strong Hall property. The transpose of the subgraph induced by H R and H C has the strong!"#$%&'()'*+'#,-.*/+'0-$1-,232-* Hall property.!! # $ % " & ' ( )!*!!!#! # $ % " & ' ( )!*!!!" #" $"! # $ % " & ' ( )!* # $ % " & ' ( )!*!!!% #% $%!!!# 20/48 CR09

21 The BTF, computation and properties Bipartite matching Dulmage-Mendelsohn decomposition: Fine decomposition In the fine decomposition, the horizontal and vertical blocks have block diagonal structure where the individual blocks on the diagonal are horizontal and vertical, respectively. Find the connected components of A H and A V ; these will be the diagonal blocks in those submatrices with the same shape as the encompassing blocks. The fine decomposition of the square block A S is obtained by identifying fully indecomposable blocks. Include all rows and columns corresponding to the vertices in an alternating tour in a single block. 21/48 CR09

22 The BTF, computation and properties Bipartite matching Fine decomposition of the square block Two columns are equivalent if they lie on an alternating tour. This is an equivalence relation. Let C 1, C 2,..., C K be the equivalence classes, and let R k be the set of rows matched to C k.!"#$%&'#$()*+,-#./#'0#*12"'$0#32+421&5&2' # " # $ # %! # $ % " & '! # $ % " & '! "! $! % &'()*+,-! # $ % " & '! #! # $ % " & ' 6*)07(08,1)'/2, 94-5.*/')(*(0,1)'/2,34-5 $ % " Figure from J. R. Gilbert. & $ 22/48 CR09

23 The BTF, computation and properties Bipartite matching Fine decomposition of the square block Let C 1, C 2,..., C K be the equivalence classes, and let R k be the set of rows matched to C k. The row subsets R k and column subsets C k can be renumbered such that if C i has a nonzero in the row set R j, then j i (hence giving the block upper triangular form). The subgraph induced by R k and C k has strong Hall property. If non-matching edges are directed from rows to columns and the matching ones are shrunk into single vertices the resulting directed graph (the directed graph of the permuted matrix) has strongly connected components C 1, C 2,..., C K. A bipartite graph has the strong Hall property iff every pair of edges is on some alternating tour iff the graph is connected and every edge is in some perfect matching. 23/48 CR09

24 Augmenting paths An augmenting path with respect to a matching M is an alternating path that starts and ends at an unmatched vertex. Theorem [Berge, 57] A matching M is of size maximum cardinality iff there is no augmenting path with respect to M. Proof: Assume M is maximum and P an M-augmenting path. Then M = M \ E(P) E(P) \ M is also a matching with M = M + 1 contradiction. 24/48 CR09

25 Augmenting paths An augmenting path with respect to a matching M is an alternating path that starts and ends at an unmatched vertex. Theorem [Berge, 57] A matching M is of size maximum cardinality iff there is no augmenting path with respect to M. Proof (cont ): Assume no M-augmenting path exists, but there is a matching M such that M > M. Consider D = M \ M M \ M. Since both M and M are matchings, each vertex has at most two edges of D incident on it. Case 0: nothing to do. Case 2 (cycles): equal number of M and M edges (skip). Case 1 (paths): Since M > M, there should be at least one path who has one more edge in M than in M : an augmenting path wrt M contradiction. 25/48 CR09

26 Augmenting paths Theorem [Berge, 57] A matching M is of size maximum cardinality iff there is no augmenting path with respect to M. a 1 b 1 a 1 b 1 a 1 b 1 a 2 b 2 a 2 b 2 a 2 b 2 a 3 b 3 a 3 b 3 a 3 b 3 a 4 b 4 a 4 b 4 a 4 b 4 26/48 CR09

27 Bipartite matching algorithms The essence of bipartite cardinality matching algorithms is to start with an empty matching and then to augment it until no further augmentations are possible. The existing algorithms mostly differ in the way the augmenting paths are found and the way the augmentations are performed. 27/48 CR09

28 Components of a maximum matching algorithm Suppose that matching of size k has been found (i.e., there exist permutations to place entries on the first k diagonal positions). augmenting paths: try to find a matching of size k + 1 using a matching of size k, back-tracking with depth first search X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6

29 Illustration of the algorithm 1 X X 2 X X X X 0 X X X X X X r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6 29/48 CR09

30 Illustration of the algorithm 1 2 X X 0 X X X X 0 X X X X X X r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6 30/48 CR09

31 Illustration of the algorithm X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 c1 c2 c3 c4 c5 r6 c X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c /48 CR09

32 Illustration of the algorithm X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 c1 c2 c3 c4 c5 r6 c X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c /48 CR09

33 Illustration of the algorithm X X X X 0 0 X 0 X X 0 0 X X X X X X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 r6 r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6 c1 c2 c3 c4 c5 c /48 CR09

34 Illustration of the algorithm X X X X 0 0 X 0 X X 0 0 X X X X X X X X X 0 0 X 0 X X 0 0 X X X X X r1 r2 r3 r4 r5 r6 r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6 c1 c2 c3 c4 c5 c6 34/48 CR09

35 Illustration of the algorithm X X X X 0 0 X 0 X X 0 0 X X X X X X X X X 0 X X 0 X 0 X 0 X X X X r1 r2 r3 r4 r5 r6 r1 r2 r3 r4 r5 r6 c1 c2 c3 c4 c5 c6 c1 c2 c3 c4 c5 c6 35/48 CR09

36 upper may triangular be found form in the of A,. book by The Duff blocket triangular al. [6]. form We implemented an algorithm similar to the one Rows in [5], but found it to be Columns slow on the denser problems we considered. This motivated the design of the variant of the O(m) maximum matching algorithm described below. We found that our implementation of this algorithm was competitive with Duff s 0 (m-) algorithm on sparser problems, while it was faster than the latter on denser problems. A description of our O(m) matching algorithm follows. The basic algorithm Algorithm 1. Maximum Matching Step 0. [Initialize] Set the matching M and the set of unmatched columns U to be empty. Step 1. [cheap matching] for each column vertex c E C do match c to the first unmatched row vertex r E c&(c), if there is such a row; if c cannot be matched, add c to U; end for Step 2. [augment u,., := 0 repeat matching] ACM Transactions on Mathematical Software, Vol. 16, No. 4, December (perform one pass of the augmenting procedure) for each column vertex c E U do search for an augmenting path from c, visiting only row vertices that have not been visited previously in this pass; mark all row vertices reached as visited, if an augmenting end for u:= u,,,; u,., := PI; until no augmenting path is found in a pass. path is found, augment M; else include c in U,,,; Inasmuch as an augmenting path has an unmatched column vertex at one end and an unmatched row vertex at the other, it is no loss of generality to search for augmenting paths from unmatched columns only. Since we are searching by alternating paths, the depth-first searches (dfs) have a simple structure. From 36/48 CR09

37 The basic algorithm: Analysis Each pass costs O(τ)-time, where τ = nnz(a). There are a total of n passes (at most), hence the runtime is O(nτ). Theorem: The cheap matching phase finds at least as many vertices as half the maximum cardinality. There are different proofs; one is based on the following theorem. König-Frobenius theorem: Let A be an m n matrix, where m n, and let k n be a nonnegative integer. A necessary and sufficient condition that the size of a maximum matching of A is n k is that it contains an s t submatrix with s + t = m + k. Suppose cheap matching obtained a cardinality of l; reorder rows and columns of A such that «A11 A 12 A =, O A 21 then by the above theorem (m l) + (n l) m + k, and hence l (n k)/2. 37/48 CR09

38 Definitions: Fully indecomposability (returning to Lecture on Graphs) Theorem: An n n square matrix A is fully indecomposable iff for some permutation matrix P, the matrix PA is irreducible and has a zero-free main diagonal. Proof: We will come later in the semester to the if part. Now we complete the proof. Use the König-Frobenius theorem. 38/48 CR09

39 Proof. /s x//-l. IMI The block triangular form IPI-<_ 2Ls /J/(s Us /J)/ <_ 2Lw/J + 1. A faster Thus, for algorithm each < r, IPil is one(at of theleast L/J / positive in theory) odd integers less than or equal to 2L/3 / 1. Also I/1, "", IPI contribute at most s- r distinct integers, and the total number of distinct integers is less than or equal to Lx/J / + Fx/ < 2Lv/J + 2, and the proof is complete. In view of Corollaries 3 and 4 and Theorem 3, the computation of the sequence {M} breaks into at most 2Lx/-] + 2 phases, within each of which all the augmenting paths found are vertex-disjoint and of the same length. Since these paths are vertex-disjoint, they are all augmenting paths relative to the matching with By Hopcroft which the phase and iskarp, 73. begun. This gives us an alternative way of describing the computation of a maximum matching. ALGORITHM A (Maximum matching algorithm). StepO. M. Step 1. Let l(m) be the length of a shortest augmenting path relative to M. Find a maximal 2 set of paths {Q, Q2,"", Q} with the properties that (a) for each i, Qi is an augmenting path relative to M and 1(21 -/(M); (b) the Q are vertex-disjoint. Halt if no such paths exist. - Step 2. M @ Qt; go to 1. COROLLARY 5. If the cardinality ofa maximum matching is s, then Algorithm A constructs a maximum matching within 2Lx/ + 2 executions of Step 1. This way of describing the construction of a maximum matching suggests that we should not regard successive augmentation steps as independent computations, but should concentrate instead on the efficient implementation of an entire phase (i.e., the execution of Step in Algorithm A). The next section shows the advantage of this approach in the case where G is a bipartite graph. A set is maximal with a given property if it has the property and is not properly contained in any set that has the property. 39/48 CR09

40 A faster algorithm: HK Remark: Let M 1 be a matching of size r and M 2 be a matching of size s, where s > r. Then, there exists s r many vertex disjoint M 1-augmenting paths. Remark: Let M be a matching. Suppose M = r, and s > r is the maximum cardinality of a matching. Then, there is an M-augmenting path of length 2 r/(s r) + 1 (all together r edges from M). Remark: If P is a shortest M-augmenting path and P is an (M P)-augmenting path, then P P + P P. Remark: Let M 0 = and consider the sequence M 0, M 1,... of matchings where P i is a shortest augmenting path relative to M i and M i+1 = M i P i. Then P i P i+1. If P i = P j, then P i and P j are vertex disjoint. Remark: Let s be the maximum cardinality of a matching. The number of distinct integers in the sequence P 0, P 1,..., P s is less than or equal to 2 s /48 CR09

41 IPI-<_ 2Ls /J/(s Us /J)/ <_ 2Lw/J + 1. The block triangular form Thus, for each < r, IPil is one of the L/J / positive odd integers less than or The run equal totime 2L/3 / of 1. Also HKI/1, "", IPI contribute at most s- r Lx/J / + Fx/ < 2Lv/J + 2, and the proof is complete. distinct integers, and the total number of distinct integers is less than or equal to In view of Corollaries 3 and 4 and Theorem 3, the computation of the sequence {M} breaks into at most 2Lx/-] + The HK algorithm finishes in 2 2 phases, within each of which all the augmenting paths found are vertex-disjoint and of the same length. Since these paths are vertex-disjoint, they are all augmenting s + paths 2 executions relative to the of matching Step 1. with which the phase is begun. This gives us an alternative way of describing the computation of a maximum matching. ALGORITHM A (Maximum matching algorithm). StepO. M. Step 1. Let l(m) be the length of a shortest augmenting path relative to M. Find a maximal 2 set of paths {Q, Q2,"", Q} with the properties that (a) for each i, Qi is an augmenting path relative to M and 1(21 -/(M); (b) the Q are vertex-disjoint. Halt if no such paths exist. - Step 2. M @ Qt; go to 1. COROLLARY 5. If the cardinality ofa maximum matching is s, then Algorithm A constructs a maximum matching within 2Lx/ + 2 executions of Step 1. This way of describing the construction of a maximum matching suggests The cost of Step 1 is O(m + n + τ) for a bipartite graph on m + n that we should not regard successive augmentation steps as independent vertices computations, and τ edges. but should concentrate instead on the efficient implementation of an entire phase (i.e., the execution of Step in Algorithm A). The next section shows the advantage of this approach in the case where G is a bipartite graph. A set is maximal with a given property if it has the property and is not properly contained in any set that has the property. 41/48 CR09

42 Then ( The block triangular form The run time of HK (P,/), where Lo O L, U U Li._ U (gi, {free girls}), E 0 U E U U El,_ 2 U {(u, 1))Iv e Li,_ and u e {free girls}}. An example of a graph ( is given in Fig. 3. The following properties of ( are immediate. The cost of(i) Step The vertices 1 is at O(m even + levels n + (Lo, τ) L2, for aare bipartite boys, and those graph at odd onlevels m + n are vertices and τ edges. girls. (ii) If (u, v) e/, then for some i, u e L + and v e Li. Build an auxiliary (iii) ( is acyclic. graph: (B) L o E o L E L E z L E L E L (G) (B) (G) (B) (G) FIG. 3. The graph used by Algorithm B. Dotted arrows indicate edges in current matching. Do a DFS-based search from unmatched vertices. Each discovered edge either becomes part of an augmenting path or there is no augmenting path using that edge. No need to revisit that edge. 42/48 CR09

43 The run time of HK The cost of Step 1 is O(m + n + τ) for a bipartite graph on m + n vertices and τ edges. There are at most O( n) executions of Step 1, hence the run time is O( n τ). 43/48 CR09

44 Summary Hall property: Let G = (U, V, E) be a bipartite graph. It is possible to match every vertex of U with a vertex of V if and only if for all U U, U N(U ). If Hall property holds then U can be matched into V. Dulmage-Mendelsohn decomposition of a matrix: 1 find a maximum cardinality matching in the bipartite graph of the matrix, 2 find the horizontal block by following alternating paths from unmatched columns, 3 find the vertical block by following alternating paths from unmatched rows, 4 the remaining row and cols form the square block, 5 find the connected components of the vertical and horizontal blocks, 6 find the strongly connected components of the square block (permuted to have a zero-free diagonal). 44/48 CR09

45 Summary Maximum cardinality matching algorithms: start with an empty matching, augment until there is no augmenting paths. 45/48 CR09

46 Illustration of maximum transversal ordering (matrix orani678) Original matrix Permuted matrix nz = nz = Better numerical stability Easier to factorize for solvers that work on A + A T 46/48 CR09

47 Question Prove that a k regular bipartite graph G = (U, V, E) has a perfect matching. k U = k V U = V 47/48 CR09

48 Question Prove that a k regular bipartite graph G = (U, V, E) has a perfect matching. k U = k V U = V Let U U. Let E U be the set of edges incident with U and let E N(U ) be the set of edges incident with the neighbors of the set U. We have E U E N(U ). Therefore, k U = E U E N(U ) = k N(U ) ; hence Hall s property holds. 48/48 CR09