Pipeline Givens sequences for computing the QR decomposition on a EREW PRAM q

Transcription

1 Parallel Computing 32 (2006) Pipeline Givens sequences for computing the QR decomposition on a EREW PRAM q Marc Hofmann a, *, Erricos John Kontoghiorghes b,c a Institut d informatique, Université de Neuchâtel, Emile-Argand 11, Case Postale 2, CH-2007 Neuchâtel, Switzerland b Department of Public and Business Administration, University of Cyprus, Cyprus c School of Computer Science and Information Systems, Birkbeck College, University of London, UK Received 4 March 2005; received in revised form 25 September 2005; accepted 19 November 2005 Available online 18 January 2006 Abstract Parallel Givens sequences for computing the QR decomposition of an m n (m > n) matrix are considered. The Givens rotations operate on adjacent planes. A pipeline strategy for updating the pair of elements in the affected rows of the matrix is employed. This allows a Givens rotation to use rows that have been partially updated by previous rotations. Two new Givens schemes, based on this pipeline approach, and requiring respectively n 2 /2 and n processors, are developed. Within this context a performance analysis on an exclusive-read, exclusive-write (EREW) parallel random access machine (PRAM) computational model establishes that the proposed schemes are twice as efficient as existing Givens sequences. Ó 2005 Elsevier B.V. All rights reserved. Keywords: Givens rotation; QR decomposition; Parallel algorithms; PRAM 1. Introduction Consider the QR decomposition of the full column rank matrix A 2 R mn : Q T A ¼ R n 0 m n ; ð1þ where Q 2 R mm and R is upper triangular of order n. The triangular matrix R in (1) is derived iteratively from Q T i ea i ¼ ea iþ1, where ea 0 ¼ A and Q i is orthogonal: the triangularization process terminates when ea v (v >0)is upper triangular. Q = Q 0 Q 1 Q v is not computed explicitly. R can be derived by employing a sequence of Givens rotations. The Givens rotation (GR) that annihilates the element A i,j when applied from the left of A has the form q This work is in part supported by the Swiss National Foundation Grants * Corresponding author. Fax: addresses: marc.hofmann@unine.ch (M. Hofmann), erricos@ucy.ac.cy (E.J. Kontoghiorghes) /$ - see front matter Ó 2005 Elsevier B.V. All rights reserved. doi: /j.parco

2 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) G i;j ¼ diag ði i 2 ; eg i;j ; I m i Þ with eg i;j ¼ c s ; s c where c = A i 1,j /t, s = A i,j /t and t 2 ¼ A 2 i 1;j þ A2 i;jð6¼ 0Þ. (a) SK (b) msk Fig. 1. The SK and modified SK Givens sequences to compute the QR decomposition, where m = 16 and n =6. Fig. 2. Partial annihilation of a matrix by the PipSK scheme when n =6.

3 224 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) A GR affects only the ith and (i 1)th rows of A. Thus, bm/2c rotations can be applied simultaneously. A compound disjoint Givens rotation (CDGR) comprises rotations that can be applied simultaneously. Parallel algorithms for computing the QR decomposition based on CDGRs have been developed [2,3,8 10,12 14]. The Greedy sequence in [4,11] was found to be optimal; that is, it requires less CDGRs than any other Givens strategy. However, the computation of the indices of the rotations which do not involve adjacent planes is non-trivial. The employment of rotations between adjacent planes facilitates the development of efficient factorization strategies for structured matrices [6]. An EREW (exclusive-read, exclusive-write) PRAM (parallel random access machine) computational model is considered [6]. It is assumed that there are p processors which can perform simultaneously p GRs. A single time unit is defined to be the time required to execute the operation of applying a Givens rotation to two oneelement vectors. Thus, the elapsed time necessary to perform a rotation depends on the length of the vectors involved. Computing c and s requires 6 flops. Rotating two elements requires another six flops. Hence, annihilation of an element and performing the necessary updating of an m n matrix requires 6n flops. Notice that the GR is not applied to the first pair of elements the components of which are set to t and zero, respectively. To simplify the complexity analysis of the proposed algorithms it is assumed that m and n are even. Complexities are given in time units. (a) PGS-1 (b) PGS-2 (c) PGS-2 cycles Fig. 3. Parallel Givens sequences for computing the QR decomposition, where m = 16 and n = 6.

4 New parallel Givens sequences to compute the QR decomposition are proposed. Throughout the execution of a Givens sequence the annihilated elements are preserved. In the next section a pipelined version of the parallel Sameh and Kuck scheme is presented. Two new pipeline parallel strategies are discussed in Section 3. A theoretical analysis of complexities of the various schemes is presented. Section 4 summarizes what has been achieved. 2. Pipeline parallel SK sequence M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) The parallel Sameh and Kuck (SK) scheme in [14] computes (1) by applying up to n GRs simultaneously. Each GR is performed by one processor. Specifically, the elements of the ith column of A begin to be annihilated by the (2i 1)th CDGR as illustrated in Fig. 1(a). The numeral i and the symbol denote an element annihilated by the ith CDGR and a non-zero element, respectively. The number of CDGRs and time units required by the SK scheme are given, respectively, by C SK ðm; nþ ¼m þ n 2 Fig. 4. Triangularization of a 16 6 matrix by the PipPGS-1 algorithm.

5 226 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) and T SK ðm; nþ ¼ðm 1Þn þ Xn j¼2 ðn j þ 1Þ ¼nð2m þ n 3Þ=2. Here it is assumed that p = n. Let g ðkþ i;j (k =1,...,n j + 1) denote the application of G i,j to the kth pair of elements in positions (i,j + k 1) and (i 1,j + k 1). That is, the rotation G i,j operating on the (n j + 1) element subvectors of rows i and i 1 is now expressed as the application of a sequence of elementary rotations g ð1þ i;j ;...; g ðn jþ1þ i;j to the pairs of elements {(i,j),(i 1,j)},...,{(i,n),(i 1,n)}, respectively. The rotation G i+1,j+1 can begin before the application of G i,j has been completed. Specifically, g ð1þ iþ1;jþ1 can be applied once gð2þ i;j has been executed. Thus, several GRs can operate concurrently on different elements of a row. The Pipelined SK (PipSK) scheme employs this strategy. The first steps of the PipSK scheme are illustrated in Fig. 2. As in the case of the SK scheme, PipSK initiates a CDGR every second time unit and its overall execution time is given by T PipSK ðm; nþ ¼2C SK ðm; nþ 1 ¼ 2m þ 2n 5. The number of CDGRs being performed in parallel is n/2 requiring 2i processors each (i = 1,..., n/2). The PipSK scheme thus requires p ¼ P n=2 i¼1 2i n2 =4 processors. Its annihilation pattern a modified SK scheme is shown in Fig. 1(b). Fig. 5. Annihilation cycle of the PipPGS-1 algorithm when n = 6.

6 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) Fig. 6. Annihilation cycle of the PGS-2 scheme when n =6. Fig. 7. Annihilation cycle of the PipPGS-2 algorithm when n = 6.

7 228 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) New pipeline parallel Givens sequences Alternative parallel Givens (PG) sequences to compute the QR decomposition and which are more suitable for pipelining are shown in Fig. 3. The number of CDGRs applied by the first PG sequence (PGS-1 [1]) is given by C PGS-1ðm; nþ ¼m þ 2n 3. The Pipelined PGS-1 (PipPGS-1) is illustrated in Fig. 4. It requires p n 2 /2 processors. It initiates a CDGR every time unit and its time complexity is given by the total number of CDGRs applied. That is, T PipPGS-1ðm; nþ ¼C PGS 1 ðm; nþ ¼m þ 2n 3. The PipPGS-1 operates in cycles of n + 1 time units. This is shown in Fig. 5. In every time unit up to n processors initiate a GR, while the other processors are updating previously initiated rotations. Each processor executes two GRs in one cycle with complexities T 1 and T 2 respectively, such that T 1 + T 2 = n + 1 time units. The PipPGS-1 performs better than the SK scheme utilizing approximately n/2 times the number of processors. That is, T SK (m,n)/t PipPGS-1 (m,n) n. Hence, the efficiency of the PipPGS-1 is twice that of the SK scheme. The PGS-2 illustrated in Fig. 3(b) employs p = n processors. A cycle involves one CDGR and n consecutive GRs and annihilates up to 2n elements in n + 1 steps. This is illustrated in Fig. 6. The PGS-2 scheme requires more steps than the SK scheme. When m and n are even, the sequence consists of (m + n 2)/2 cycles which Fig. 8. Initial annihilation steps of a 16 6 matrix by the PipPGS-2 algorithm.

8 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) Fig. 9. Final annihilation steps of a 16 6 matrix by the PipPGS-2 algorithm. can be partitioned in three sets comprising cycles {1,...,n 1}, {n,...,(m 2)/2} and {m/2,...,(m + n 2)/ 2}. The sets, cycles and constituent steps are detailed in Fig. 3. The ith cycle in the first, second and third set applies i +1,n + 1 and 2(n/2 i + 1) steps, respectively. The total number of steps applied by the PGS-2 is given by C PGS-2ðm; nþ ¼ Xn 1 ði þ 1Þþðm 2nÞðn þ 1Þ=2 þ Xn=2 2i ð2mn þ 2m n 2 Þ=4. i¼1 The Pipelined PGS-2 (PipPGS-2) applies 2n GRs in a pipeline manner. Fig. 7 illustrates the annihilation cycle of the PipPGS-2 which annihilates 2n elements in n + 1 time units. This is equivalent to two CDGRs of the SK scheme for which 2n time units are required. Figs. 8 and 9 illustrate, respectively, the initial and the final phase of the annihilation process of a 16 6 matrix by the PipPGS-2. An asterisk denotes the start of a new cycle. In every time unit a processor initiates a GR. When m and n are even, the first (m 2)/2 cycles are executed in n + 1 time units each. The ith cycle in the third set requires 2(n/2 i + 1) + 1 time units (i =1,...,n/2). The overall execution time of the PipPGS-2 algorithm is given by T PipPGS-2 ðm; nþ ¼ðm 2Þðn þ 1Þ=2 þ Xn=2 ð2i þ 1Þ ð2mn þ 2m þ n 2 Þ=4. Furthermore, if m n and for the same number of processors, T SK ðm; nþ T PipPGS-2 ðm; nþ 2. That is, the proposed scheme is twice as fast as the SK scheme. i¼1 i¼1

9 230 M. Hofmann, E.J. Kontoghiorghes / Parallel Computing 32 (2006) Table 1 Summary of the complexities of the SK and pipelined schemes Scheme Processors Complexity SK n n(2m + n 3)/2 PipSK n 2 /4 2m +2n 5 PipPGS-1 n 2 /2 m +2n 3 PipPGS-2 n (2mn +2m + n 2 )/4 4. Conclusions A new pipeline parallel strategy has been proposed for computing the QR decomposition. Its computational complexity is compared to the SK scheme of Sameh and Kuck in [14]. The complexity analysis is not based on the unrealistic assumption that all CDGRs, or cycles, have the same execution time. Instead, the number of operations performed by a single Givens rotation is given by the size of the pair of vectors used in the rotation. It was found that for an equal number of processors the pipelined scheme solves the m n QR decomposition twice as fast as the SK scheme when m n. The complexities are summarized in Table 1. Block versions of the SK scheme have previously been designed to compute the orthogonal factorizations of structured matrices which arise in econometric estimation problems [5,15]. Within this context the Givens rotations are replaced by orthogonal factorizations which employ Householder reflections. Thus, it might be fruitful to investigate the effectiveness of incorporating the pipeline strategy in the design of block algorithms [7,16]. Acknowledgements The authors are grateful to Maurice Clint, Ahmed Sameh and the anonymous referee for their valuable comments and suggestions. References [1] A. Bojanczyk, R. Brent, H. Kung, Numerically stable solution of dense systems of linear equations using mesh-connected processors, SIAM J. Sci. Statist. Comput. 5 (1984) [2] M. Cosnard, M. Daoudi, Optimal algorithms for parallel Givens factorization on a coarse-grained PRAM, J. ACM 41 (2) (1994) [3] M. Cosnard, J.-M. Muller, Y. Robert, Parallel QR decomposition of a rectangular matrix, Numerische Mathematik 48 (1986) [4] M. Cosnard, Y. Robert, Complexité de la factorisation QR en parallèle, C. R. Acad. Sci. Paris, ser. I 297 (1983) [5] E.J. Kontoghiorghes, Parallel Algorithms for Linear Models: Numerical Methods and Estimation Problems, Advances in Computational Economics, volume 15, Kluwer Academic Publishers, Boston, MA, [6] E.J. Kontoghiorghes, Parallel Givens sequences for solving the general linear model on a EREW PRAM, Parallel Algorithms Appl. 15 (1 2) (2000) [7] E.J. Kontoghiorghes, Parallel strategies for rank-k updating of the QR decomposition, SIAM J. Matrix Anal. Appl. 22 (3) (2000) [8] E.J. Kontoghiorghes, Greedy Givens algorithms for computing the rank-k updating of the QR decomposition, Parallel Comput. 28 (9) (2002) [9] F.T. Luk, A rotation method for computing the QR decomposition, SIAM J. Sci. Statist. Comput. 7 (2) (1986) [10] J.J. Modi, Parallel Algorithms and Matrix Computation, Oxford Applied Mathematics and Computing Science series, Oxford University Press, [11] J.J. Modi, M.R.B. Clarke, An alternative Givens ordering, Numerische Mathematik 43 (1984) [12] A.H. Sameh, Solving the linear least squares problem on a linear array of processors, in: Algorithmically Specialized Parallel Computers, Academic Press, Inc., 1985, pp [13] A.H. Sameh, R.P. Brent, On Jacobi and Jacobi like algorithms for a parallel computer, Math. Comput. 25 (115) (1971) [14] A.H. Sameh, D.J. Kuck, On stable parallel linear system solvers, J. ACM 25 (1) (1978) [15] P. Yanev, P. Foschi, E.J. Kontoghiorghes, Algorithms for computing the QR decomposition of a set of matrices with common columns, Algorithmica 39 (2004) [16] P. Yanev, E.J. Kontoghiorghes, Efficient algorithms for block downdating of least squares solutions, Appl. Numer. Math. 49 (2004) 3 15.