AA220/CS238 Parallel Methods in Numerical Analysis. Introduction to Sparse Direct Solver (Symmetric Positive Definite Systems)

Transcription

1 AA0/CS8 Parallel ethods in Numerical Analysis Introduction to Sparse Direct Solver (Symmetric Positive Definite Systems) Kincho H. Law Professor of Civil and Environmental Engineering Stanford University ay, 00

2 A Typical Finite Element Program Nonlinear and/or Adaptive Solver User Interface (UI) (esh Gen., B.C., etc..) Element Library atrix, RHS Formation and Assembly Linear Solver (Direct, Iterative) Element Characteristics Post Processing Direct Solution Scheme... Ku = T LDL u Ly = T DL u p = p = y p K: Symmetric +ve Definite L: Lower Triangular atrix D: Diagonal atrix

3 otivation Example: Bandwidth inimization Reverse Cuthill ckee Algorithm

4 otivation Example: Sparsity within Profile.... Reverse Cuthill ckee Algorithm

5 otivation Example: Sparse atrix Ordering inimum Degree Ordering

6 Introductory Example x x x = y y y Stiffness atrix x y x = y x y Eq. /8 x Eq. Eq. /8 x Eq x 0.00 x 7.7 x y = y y Eq. (-0./7.) x Eq.

7 Symbolic Representation of Symmetric atrices and Gaussian Elimination x x x x y x = y x y y = y y Graph G of A is constructed as follows:. Deleting node and its incident edges. Adding edges to the graph so that the adjacent nodes of are now pairwise adjacent in graph G Graph G of Graph G of A0 A [ ]

8 Symbolic Representation of Gaussian Elimination x x x y = y y x y x = y x y Graph G of A0 Graph G of A x 0.00 x 7.7 x y = y y 7.00 ( 0.00)( 0.07) Graph G of A

9 Summary of Results Given a symmetric system of equations Ax = y x x x y = y y Numerical factorization of A into LU Graph of A Graph of L+U x 0.00 x 7.7 x y = y y For symmetric system, A can be factored as LDL T x 0.07 x.000 x y = y y

10 = y y y y y y x x x x x x Another Example Ax = y A = LDL T Stiffness atrix Not all entries within profile (band) are zero!

11 = y y y y y y x x x x x x Graph Representation of Gaussian Elimination Graph of A Graph after elimination of node

12 Graph after elimination of node Graph after elimination of node Graph after elimination of node Graph after elimination of node

13 atrix A Nonzero Structure of atrix A atrix Factor L Nonzero Structure of atrix L+L T.0000 Graph of atrix A Graph of atrix L+L T

14 atrix Numbering as Graph Ordering Purpose: inimize the number of fill-in nonzero entries in the matrix factor L inimum Degree Ordering Scheme (A Greedy Strategy). Select a node with a minimum degree from the graph and label the node. Eliminate the node and perform graph transformation by adding fill-in edges if necessary. Repeat steps and until all nodes are labelled

15 e a b d f g c g f e d c b a a b d f g c b d f g c d f g c d g c d c c 7 7 Original Graph and atrix Structure 7 Reordered Graph and atrix Structure

16 Summary of Numerical Factorization for Sparse Symmetric atrices Given a symmetric matrix A, compute the matrix factor L and D such that A = LDL T. Order the matrix A such that it has a desirable structure. Symbolic Factorization to determine the structure of L. Numerically factorize the matrix A into L and D utilizing the nonzero structure of L T. Forward and backward solutions: Lz = y; DL x = z Focus of Discussion: Graph and Tree Representation of the Sparse Factor L

17 = s u u A T + = = = d w D w L D w w D L L D L w L d D w L s u u A T T T T T T T Direct (Cholesky) Factorization T L D L ] ][ ][ [ ] [ = [ ] L = [ ] D = Suppose The problem is to compute w and d

18 {} {} {} u = [ L ][ D ] w w = [ D ] [ L ] {} u {} w Structure of {w} is related to forward solve / / = / = 0.08 / / s = T T { w} [ D ]{ w} + d d = s { w} [ D ]{ w} d T = =

19 Definition for a Tree Structure of atrix Factor Define PARENT(j) = min {i L(i,j) 0} Note: The list array PARENT represents the row subscript of the first nonzero entry in each column of the lower triangular matrix factor L Lemma: If A(i,j) (or L(i,j)) 0, then for each k = PARENT( PARENT. (PARENT(j) ) L(i,k) 0, where k < i. That is, given the tree T(A) and the nonzero entries of A, we can obtain the nonzero entries per each row of the matrix factor L by tracing the path along the tree from the nonzero offdiagonal column subscript of A to the row number of interest atrix Structure of L+L T Tree of atrix Factor L (Law and Fenves 8, 8; Liu 8, 88; Schreiber 8)

20 ALGORITH: ROW_STRUCTURE /* Determine the data structure for row i of matrix factor L */ BEGIN Sort the column subscripts of the nonzero entries in ascending order and store them in a linked list array LIST; j = HEAD of LIST; WHILE j 0 DO BEGIN IF LIST(j) 0 THEN next = LIST(j) ELSE next = i; r = j; WHILE 0 < r < next DO BEGIN add subscript r to row i of L; r = PARENT(r); ENDWHILE; IF r 0 and r < i THEN sort r to LIST; j = LIST(j); ENDWHILE; END.

21 ALGORITH: ROW_STRUCTURE /* Determine data structure for row i of matrix factor L */ BEGIN Sort the column subscripts of the nonzero entries in ascending order and store them in a linked list array LIST; j = HEAD of LIST; WHILE j 0 DO BEGIN IF LIST(j) 0 THEN next = LIST(j) ELSE next = i; r = j; WHILE 0 < r < next DO BEGIN add subscript r to row i of L; r = PARENT(r); ENDWHILE; IF r 0 and r < i THEN sort r to LIST; j = LIST(j); ENDWHILE; END. atrix Structure Tree of atrix Factor L Example: Row of atrix K HEAD = ; LIST:<0,, 0, 0,, 0> j = next = LIST() = ; r = ; add entry to row of L (i.e. L(,) 0); r = PARENT() = ; sort to list; (i.e. LIST:<0,,, 0,, 0>); (j = LIST() = ;)

22 ALGORITH: ROW_STRUCTURE /* Determine data structure for row i of matrix factor L */ BEGIN Sort the column subscripts of the nonzero entries in ascending order and store them in a linked list array LIST; j = HEAD of LIST; WHILE j 0 DO BEGIN IF LIST(j) 0 THEN next = LIST(j) ELSE next = i; r = j; WHILE 0 < r < next DO BEGIN add subscript r to row i of L; r = PARENT(r); ENDWHILE; IF r 0 and r < i THEN sort r to LIST; j = LIST(j); ENDWHILE; END. Nonzero entries denoted in the linked list LIST atrix Structure Tree of atrix Factor L Example: Row of atrix K (cont d) (HEAD = ; LIST:<0,,, 0,, 0>;) (j = LIST() = ;) next = LIST() = ; r = ; add entry to row of L (i.e. L(,) 0); r = PARENT() = ; j = LIST() = (LIST()=0) NET= r = ; add entry to row of L (i.e. L(,) 0); r = PARENT() = ; j = LIST() = 0.

23 ALGORITH: TREE-STRUCTURE /*Given the nonzero entries of A, determine the tree structure of matrix A */ BEGIN Initialize array PARENT to 0; FOR each row i = TO n, DO FOR each nonzero entry A(i,j) of row i, DO r = j; WHILE ((PARENT(r) 0) AND (PARENT(r) i)) DO r = PARENT(r); ENDWHILE. IF (PARENT(r) = 0) THEN PARENT(r) = i; ENDFOR ENDFOR. END. atrix A atrix of L+L T

24 ALGORITH: TREE-STRUCTURE (with tree compression) (Ref: Liu 8, 88) /*Given the nonzero entries of A, determine the tree structure of matrix A */ BEGIN Initialize arrays PARENT and ANCESTOR to 0; FOR each row i = TO n, DO FOR each nonzero entry A(i,j) of row i, DO r = j; WHILE ((ANCESTOR(r) 0) AND (ANCESTOR(r) < i)) DO r = ANCESTOR(r); ANCESTOR(r) = i; r = t; ENDWHILE. IF (ANCESTOR(r) = 0) THEN DO ANCESTOR(r) = i; PARENT(r) = i; ENDIF. ENDFOR ENDFOR. END.

25 Example model ordered using RC (bandwidth minimization) algorithm F F Note: Zero entries within the band!

26 Restructuring of Ordered Elimination Tree : ALGORITH: Binary-Tree Representation /*Given the PARENT array */ BEGIN FOR each node i, DO IF (PARENT(I) 0) THEN DO BEGIN r = PARENT(r); IF (CHILD(r) 0) THEN DO BEGIN SIBLING(r)=CHILD(r); CHILD(r) = i; ENDIF. ELSE CHILD(r)=I; ENDIF ENDFOR. END. PARENT CHILD SIBLING

27 Restructuring of Ordered Elimination Tree : Post-Order Traversal ALGORITH: POST-ORDER(r,number) /*Given the Binary Tree Representation */ /* Initially set r=n (root of T(K)) and number = */ BEGIN t=r; IF (t 0) THEN DO BEGIN POST-ORDER(CHILD(t),number); label node t = number; number = number + ; POST-ORDER(SIBLING(t),number); ENDIF END.

28 Post-ordering of the elimination tree F 0 9 F Preserve number of fill-in entries 9 Reveal matrix partitioning Allow block data structure : principal submatrix, row segments 0 0 9

29 Summary of Numerical Factorization for Sparse Symmetric atrices Given a symmetric matrix A, compute the matrix factor L and D such that A = LDL T. Order the matrix A such that it has a desirable structure. Symbolic Factorization to determine the structure of L. Given structure of A, determine the tree structure T(A) of A. Given T(A) and structure of A, determine the structure of L. Numerically factor the matrix A into L and D utilizing the nonzero structure of L

30 A Typical Finite Element Program Nonlinear and/or Adaptive Solver User Interface (UI) (esh Gen., B.C., etc..) Element Library atrix, RHS Formation and Assembly Linear Solver (Direct, Iterative) Element Characteristics Direct Solution Scheme Ordering the equations Data structure for system matrix Profile solver Sparse solver Post Processing

31 8 Performance of Sparse Linear Solver Sparse Solvers Traditional Solver Neq ultind ind Profile Square Humboldt Humboldt Plate 00x0..7. Square Performance (second) 0 8 Profile inimum Sol Degr 0 ultilevel Nested Diss Number of Equations (Ref: Jun Peng 00)

32 Comparison of Different Linear Solvers The odels:. Brickx8x0;. Humboldt;. Humboldt;. Square00x00 0 SymSparse::ultiND SymSparse::inD SymSparse::GenND Profile SuperLU UmfPack Performance (Second) Tested odels (Ref: Jun Peng 00)

33 General Remarks References: Law and Fenves, A Node Addition odel for Symbolic Factorization, AC TOS, ():7-0, 98. ackay, Law and Raefsky, An Implementation of A Generalized Sparse/Profile Finite Element Solution ethod, Computer and Structure, ():7-77, 99. George and Liu, Computer Solution of Large Sparse Positive Definite Systems, Prentice Hall, 98. Duff, Erisman and Reid, Direct ethods for Sparse atrices, Oxford Science Publications, 98. Software Packages: Sparspak, YSP, UFPACK, SuperLU, etc Next Lecture: Parallel Implementation of a Sparse Direct Solver