AMS subject classifications. 65F05 Direct methods for linear systems and matrix inversion

Transcription

1 EFFICIENT SOLUTION OF A x (k) = b (k) USING A 1 ADI DITKOWSKI, GADI FIBICH, AND NIR GAVISH Abstract. I this work we cosider the problem of solvig Ax (k) = b (k), k = 1,..., K where b (k+1) = f(x (k) ). We show that whe A is a full matrix ad K c, where c 1 depeds o the specific software ad hardware setup, it is faster to solve Ax (k) = b (k) for k = 1,..., K by explicitly evaatig the iverse matrix A 1 rather tha through the LU decompositio of A. We also show that the forward error is comparable i both methods, regardless of the coditio umber of A. AMS subject classificatios. 65F05 Direct methods for liear systems ad matrix iversio 1. Itroductio. Oe of the first thigs we lear i a basic umerical liear algebra course is that i order to solve the liear system Ax = b, we should ot calculate the iverse matrix A 1. For example, we quote from Matlab s user guide: I practice, it is seldom ecessary to form the explicit iverse of a matrix. A frequet misuse of iverse arises whe solvig the system of liear equatios. Oe way to solve this is with x = iv(a) b. A better way, from both a executio time ad umerical accuracy stadpoit, is to use the matrix divisio operator x = A\b. This produces the sotio usig Gaussia elimiatio, without formig the iverse. Similar statemets appear i the classical umerical aalysis textbooks. For example, Gob ad va Loa [7, page 121] say:... As a fial example we show how to avoid the pitfall of explicit iverse computatio... whe a matrix iverse i ecoutered i a formula, we must thik i terms of solvig equatios rather tha i terms of explicit iverse formatio. Similarly, Cote ad de Boor [8, page 166] say:... give this simple prescriptio for calculatig the iverse of a matrix, we haste to poit out that there is usually o good reaso for ever calculatig the iverse.... wheever A 1 is eeded merely to calculate a vector A 1 b (as i solvig Ax = b) or a matrix product A 1 B, A 1 should ever be calculated explicitly. Higham [9, page 262] also states: Not oly is the iversio approach three times more expesive, but it is much less stable.... we see that matrix iversio is likely to give a much larger residual tha Gaussia elimiatio with partial pivotig... Thus, the reasos for preferrig Gaussia elimiatio over explicit calculatio of A 1 to solve Ax = b are performace, ad accuracy ad stability. I this study we cosider the case where we wat to solve the equatios Ax (k) = b (k), k = 1,..., K, (1.1) whe 1. A is a full matrix. 2. Equatios (1.1) have to be solved sequetially, e.g. whe b (k+1) = f(x (k) ). The stadard, LU-based approach ivolves a preprocessig stage i which the LU decompositio of the matrix A is calculated: [L, U] = (A). (1.2) The, for each right-had-side the liear system is solved usig a forward substitutio ad a backward substitutio: y (k) = L\b (k) ; x (k) = U\y (k), k = 1,..., K. (1.3) A secod, A 1 based approach for solvig problem (1.1) is to calculate the iverse of A i a preprocessig stage: iva = iv(a). (1.4) The, for each right-had-side the liear system is solved by a sigle matrix-vector multiplicatio: x (k) = iva b (k), k = 1,..., K. (1.5) 1

2 2 I this study we show that it is better to solve equatios (1.1) usig A 1 rather tha through LU decompositio. I Sectio 2 we cosider the performace issue. We first show that the two methods require the same umber of arithmetic operatios. However, the actual performace has a lot to do with the way that optimized computer codes, such as Itel s MKL BLAS (Basic Liear Algebra Support) package, hadle computer architecture issues, such as cachig ad memory allocatio. Ideed, because the data structure of a square matrix (A 1 ) is iheretly simpler tha that of two triagular matrices (L ad U), i our umerical tests we observe that it faster to solve these equatios usig A 1 tha through the LU decompositio of A. I Sectio 2.4 we show that there are more efficiet implemetatios of the LU algorithm the the oe give i equatios (1.2,1.3). Eve with these improvemets, however, the A 1 algorithm is still more efficiet. As oted, the secod argumet agaist usig A 1 is accuracy ad stability. This argumet is based o aalysis which showed that the backward error i the A 1 based method is larger tha i the LU-based method, ad that the ratio betwee the two icreases with the coditio umber of A. While there has bee o such aalytical results for the forward error, it was implicitly assumed that it would behave similarly. However, i Sectio 3, our simulatios with radomly-chose matrices, as well as with three families of ill-coditioed matrices, suggest that this implicit assumptio is wrog. Ideed, we observe that the forward error is of the same magitude for both the A 1 based method ad the LU-based method, regardless of the coditio umber of A. I Sectio 4 we list various applicatios that are expected to ru faster whe implemeted with the A 1 based method (quasi-newto iteratios, iverse power method, ad spectral differetiatio methods). We cocde by showig that the iverse power method for fidig the smallest eigevae of a matrix ca be 11 times faster with the A 1 based method tha with the LU-based method, ad just as accurate. Our cocsio that usig A 1 is superior to usig LU decompositio applies to problems i which: 1. A is full, sice whe A is sparse the L ad U are sparse but A 1 is full. 2. Equatios (1.1) have to be solved sequetially, hece oly level 2 BLAS optimizatio (matrix-vector operatios) ca be used. Ideed, whe all the b (k) are kow i advace, all the right-had sides ca be simultaeously solved with a block LU algorithm, i.e., replacig (1.3) with Y = L\B; X = U\Y, where B = [b (1) b (K) ], X = [x (1) x (K) ], Y = [y (1) y (K) ]. I this case, the block LU algorithm is more efficiet tha usig X = A 1 B, sice oe ca exploit level 3 BLAS optimizatio (matrix-matrix operatios). 2. Performace Arithmetic operatios cout. We begi with the stadard cout of arithmetic operatios: Lemma 2.1. The overall umber of arithmetic operatios eeded to solve the system (1.1) i the LUbased method is ad i the A 1 based method N = O( 2 ) + 2K 2 + O(K), }{{}}{{} eq. eq. (1.3) (1.2) N A 1 = O( 2 ) + 2K 2 + O(K). }{{}}{{} eq. eq. (1.5) (1.4) Proof. For the LU-based method the preprocessig stage ivolves fidig the LU decompositio of A, which requires O ( 2) arithmetic operatios, see [2, page 152]. The, solvig each liear system ivolves a backward ad forward substitutio. Each substitutio requires 2 + O() arithmetic operatios. Therefore, a overall of 2K 2 + O(K) arithmetic operatios are eeded to solve (1.3).

3 Similarly, for the A 1 based method the preprocessig stage ivolves fidig A 1, which require O ( 2) arithmetic operatios, see [2, page 161]. The, solvig each liear system requires multiplyig b (k) by A 1, which requires O() arithmetic operatios. Therefore, a overall of 2K 2 + O(K) arithmetic operatios are eeded to solve (1.5). Observatio 2.1 shows that the umber of arithmetic operatios for the LU-based method is smaller tha for the A 1 based method for K, ad is the same for K. More precisely, N 1 4 N A 1 K N = 2+6λ 8+6λ N A 1 K = λ, λ = O(1) N N A 1 K Numerical tests. The stadard approach to estimate ru-time of algorithms is to cout the umber of arithmetic operatios. Hece, Lemma 2.1 ad equatio (2.1) seem to suggest that the LU-based method is faster tha the A 1 based method. We ow show that this is ot always the case. To demostrate this, we solved liear systems of the form (1.1) usig the LU-based ad A 1 based methods. For each method ad for each = 2 j, j = 5,..., 12, we radomly choose 3 differet matrices, for each matrix we solve the liear system Ax = b (k) 1000 times, sequetially, ad calculate the average CPU time per RHS. This measuremet of CPU time does ot icde the preprocessig time of LU decompositio or of fidig the iverse. The tests were coducted o a Petium 4 with 1Mb cache ad 2GB RAM usig Matlab 7.0 R14 for Liux. I the first test we used Itel s MKL BLAS (Basic Liear Algebra Support) package. The average CPU time per RHS with the A 1 based method is smaller by a factor of compared with the LU-based method (see Figure 2.1), for matrix sizes varyig from to (2.1) average CPU time Fig Average CPU time for solvig Ax = b (k) usig the backward ad forward substitutio (eq. (1.3), ) ad through multiplicatio by A 1 (eq. (1.5), ). Next, we repeated the same test, but chaged the software package that Matlab uses for the liear algebra subrouties from Itel s MKL BLAS to ATLAS BLAS. I this case, the average CPU time for a forward ad a backward substitutio did ot chage, but the average CPU time for multiplicatio by A 1 icreased by a factor of , hece the A 1 based method was oly 8-16 times faster tha the LU-based method, see Figure 2.2. Fially, we ra a bechmark test i which we implemeted the multiplicatio by A 1 ad the backward ad forward substitutio usig a aive straightforward o-vectorized code of a double loop. I that case, the average CPU time is about the same for forward ad backward substitutio ad for multiplicatio by A 1, see Figure 2.3. As expected, the aive implemetatio was slower tha the MKL BLAS implemetatio. The matrix-vector multiplicatio by A 1 was times faster (usig MKL BLAS) tha with the aive

4 4 average CPU time Fig Same data as i Figure 2.1. Also plotted, is the average CPU time usig ATLAS BLAS for solvig Ax = b (k) usig the forward ad backward substitutio (eq. (1.3), ) ad through multiplicatio by A 1 (eq. (1.5, ). implemetatio. O the other side, forward ad backward substitutio usig MKL BLAS was oly times faster tha forward ad backward substitutio with the aive implemetatio. average CPU time Fig Same data as i Figure 2.1. I additio average CPU time per RHS usig a aive implemetatio for the LU-based method (solid) ad the A 1 based method (dashes). We also coducted the same tests o a differet Petium 4 machie (0.5Mb cache ad 0.5GB RAM usig Matlab 7.0 R14 for Widows XP). Uder this cofiguratio, all qualitative results remaied the same, but the performace ratios chaged. For example, the A 1 based method was oly times faster tha the LU-based method usig Itel s MKL BLAS, ad 8-12 faster uder ATLAS BLAS Data aalysis. The simulatio of Sectio 2.2 shows that, for sequetially solvig the system ad eglectig the preprocessig costs, the A 1 based method is sigificatly faster tha the LU-based method (per RHS). Sice the umber of arithmetic operatios per RHS is the same for both methods (see Sectio 2.1), we cocde that the A 1 based method is faster sice the implemetatio of matrix multiplicatio i MKL BLAS or i ATLAS BLAS is faster tha the implemetatio of a forward ad backward substitutio. Ideed, whe we used a aive implemetatio, there was o performace differece betwee the two methods. I retrospect, the fact that the A 1 based method is faster is ot surprisig sice the memory structure ad idexig of a full matrix is iheretly simpler tha that of a triagular matrix. Therefore, matrix

5 5 multiplicatio ca be more easily accelerated through software ad hardware optimizatio. I other words, the performace differece betwee the two methods is due to the implemetatio of the umerical liear algebra software package. This cocsio is further supported by the observatio that chagig the BLAS package leads to a chage i the performace ratio from to 8-16, see Figure 2.2. I the umerical experimets show so far, we eglected the preprocessig cost ad cosidered oly the cost of the solvig the liear systems (1.3) ad (1.5). This correspods to the case where K, sice the the preprocessig costs are egligible. We ow take ito accout the preprocessig cost ad look at the overall cost of solvig (1.1) i both methods. For example, settig K=10 ad comparig the overall cost (preprocessig+solvig for 10 RHS) of the two methods, we obtai that the A 1 based method is faster that the LU-based method for matrices of size < th ad slower for > th, where the threshold is th (K = 10) = 700, see Figure 2.4. Equivaletly, for a give we ca fid a threshold K th = K th (), such the overall cost for solvig (1.1) is faster for the A 1 method whe K > K th ad slower whe K < K th. There is a liear relatio betwee K th ad. Ideed, the overall cost of the LU-based method is T preprocessig + K T RHS = α preprocessig 3 + K β RHS 2 + O(K, 2 ), (2.2) where T preprocessig is the preprocessig cost of (1.2) ad T RHS is the cost of the solvig the liear systems (1.3) by forward ad backward substitutio. The overall cost of the A 1 based method is where T preprocessig iverse Hece, T preprocessig iverse K th = T preprocessig iverse + K Tiverse RHS = α preprocessig iverse 3 + K β RHS iverse 2 + O(K, 2 ), (2.3) is the preprocessig cost (1.4) ad T RHS iverse is the cost of multiplicatio by A 1, see (1.5). T RHS = (αpreprocessig iverse (β RHS T preprocessig Tiverse RHS α preprocessig ) 3 + O( 2 ) βiverse RHS )2 + O() αpreprocessig iverse β RHS α preprocessig β RHS iverse + O(1). (2.4) I Figure 2.5 we plot the vae of K th as a fuctio of. The results show that K th , i agreemet with (2.4). As oted, Observatio 2.1 has bee traditioally iterpreted to imply that the LU-based method is faster tha the A 1 based method for K = O() ad that the ru time of the two methods is the same for K. I cotrast, i Figure 2.5 we see that the A 1 based method is faster for K > c, where c The reaso for this disagreemet is the implicit assumptio that the umber of arithmetic operatios is a good measure for compariso of ru-times. This assumptio fails, however, to capture the large differece betwee βiverse RHS ad βrhs that results from the differece i implemetatio of matrix-vector multiplicatio ad of forward ad backward substitutios. K=10 overall CPU time th Fig Overall cpu time of the LU-based method (dots) ad the A 1 based method (solid) with K = 10.

6 6 K th Fig K th as a fuctio of th (solid). Dotted lie is the fitted liear curve K th = Improvig the LU solver. I our umerical tests so far we used the most straightforward implemetatio of the LU algorithm for solvig (1.1). We first calculated the LU decompositio (1.2). The, for each right-had-side we solved the liear system usig a forward substitutio ad a backward substitutio, see equatio (1.3), which are implemeted by a backslash ( \ ) or equivaletly by the mldivide Matlab commad. As stated by the Matlab s help, the commad [L, U] = (A); returs a upper triagular matrix i U, ad a psychologically lower triagular matrix (i.e., a potetially permuted lower triagular matrix) i L. The psychologically lower triagular form makes it harder to efficietly implemet forward substitutio. A faster way is, therefore, to eforce L to be a strict lower triagular matrix by usig the commad [L, U, P ] = (A), (2.5) where P is the permutatio matrix. The, the sotio to Ax = b ca be obtaied by y = L\(P b); (2.6) x = U\y; The additioal calculatios i this modificatio (i.e., the calculatio of P b) are O(), hece are egligible compared with the time saved by forcig L to be triagular. Ideed, Figure 2.6 shows that usig the strict lower triagular matrix is about 2.5 times faster tha the stadard method. The backslash commad, e.g. A\b, ivolves a preprocessig stage i which the properties of the matrix A are checked i order to pick the most appropriate solver for the problem. Thus, i the case of equatio (2.6), the backslash commad idetifies that L ad U are triagular matrices ad uses backward- or forwardsubstitutio to obtai the sotio. Sice it is kow that L ad U i (2.6) are lower- ad upper-triagular matrices, it is possible to bypass the preprocessig stage by usig the lisolve commad istead of the backslash commad. Therefore, we use (2.5) to obtai a strict lower triagular matrix L, but replace (2.6) with optlt.lt = true; % lower triagular property (2.7) optu T.U T = true; % upper triagular property y = lisolve(l, P b, optlt ); x = lisolve(u, y, optut ); Figure 2.6 shows that usig (2.5) ad (2.7) is 4.6 times faster tha the stadard method, i.e. equatios (1.2) ad (1.3), ad 1.3 times faster tha usig a strict lower triagular matrix without lisolve, i.e, (2.5) ad (2.6). Eve after all these improvemets, however, the A 1 method is still cosiderably faster.

7 average CPU time N Fig Average CPU time for solvig Ax = b (k) usig stadard LU method (solid), LU with explicit pivot matrix (dash-dotted), LU ad lisolve (dotted) ad A 1 method (dashed). 3. Accuracy ad stability. The secod commo argumet i the Literature agaist solvig Ax = b with A 1 is its umerical accuracy ad stability (see citatios i the Itroductio). As oted, this argumet is based o aalysis which showed that the backward error i the A 1 based method is larger tha i the LU-based method, ad that the ratio betwee the two icreases with the coditio umber of A. While there has bee o such aalytical results for the forward error, it was implicitly assumed that it would behave similarly. Our simulatios however suggest that this implicit assumptio is wrog. Ideed, we observe that the forward error is of the same magitude for both the A 1 based method ad the LU-based method, regardless of the coditio umber of A Forward error. I the A 1 based method, x x cod(a) b b. I the LU-based method x is determied by solvig the liear systems (1.3), hece, x x cod(l)cod(u) b b. Sice cod(a) cod(l)cod(u), the worst case relative (forward) error x x is likely to be larger for the LU-based method tha for the A 1 based method. O the other had, the roudoff error ivolved i matrix vector multiplicatio is greater tha the roudoff error of backward ad forward substitutio [9, page 262]. We ow show that, i practice, the relative error of the A 1 based method is oly slightly larger tha the relative error of the LU-based method. To do so, we first solved liear systems of the from (1.1) usig the LU-based ad A 1 based methods. For each = 100, 200,, 4000, we radomly choose 25 differet matrices ad a sotio vector x. For each matrix A, we first geerate the vector b = Ax, the solve with each method Aˆx = b, calculate the relative forward error ˆx x x ad the average over all 25 matrices. Results i Figure 3.1A show that the forward error of the LU-based is smaller by a factor of 2-4 tha for the A 1 based method. This 2-4 factor does ot chage sigificatly as chages from 100 to 4000, ad cod(a) varies betwee 10 3 to I order to see whether larger differeces would appear for ill-coditioed matrices, we perform the same test with Matlab s matrix gallery of ivotory ill-coditioed matrices (Figure 3.2A). I this case, the relative forward error for the LU-based method is slightly below the error for the A 1 based method, differig by a factor of 1 1.5, as cod(a) varies betwee Repeatig this test for tridiagoal ill-coditioed matrices ad asymmetric ill-coditioed matrices gave early idetical results, with factors of 1-2 ad , respectively.

8 Relative forward error A Relative backward error B Fig A: Relative forward error ˆx x x same data. for ˆx = A 1 b (solid) ad ˆx = A\b (dots). B: Relative backward error for the Relative forward error A Relative backward error B cod(a) cod(a) Fig A: Forward error x ˆx for ˆx = A x 1 b (solid) ad for ˆx = A\b (dots). The two lies are almost idistiguishable. B: Backward error Aˆx b for the same data. b 3.2. Backward error. The backward error Aˆx b is kow to be sigificatly larger i the A 1 based method tha i the LU-based method whe A is ill-coditioed [9, page 262]. Ideed, Figure 3.1B shows that the backward error of the LU-based method is smaller by a factor of tha for the A 1 based method, for the same matrices used to geerate Figure 3.1A. Similarly, Figure 3.2B shows that the backward error of the LU-based method becomes expoetially smaller tha for the A 1 based method, for the same ill-coditioed ivotory matrices used to geerate Figure 3.2A. These results, thus, cofirm that the LU-based method is superior to the A 1 based method i terms of backward error. However, compariso betwee Figures 3.1A ad 3.1B ad betwee Figures 3.2A ad 3.2B shows, that the backward error is a poor predictio to the forward error. Remark: If the matrix A ca be diagoalized, the worst case sceario is that b is the eigevector of the maximal eigevae (i absote vae) ad b is proportioal to the eigevector of the miimal eigevae. However, i most applicatios arisig from PDE, this is ot the case. The mai mass is cocetrated i the lower modes, i.e. i the eigevector correlated to the lower eigevaes, while the oise is related to the high frequecy modes. Therefore i such applicatios the worst case sceario is ulikely to happeed. 4. Potetial Applicatios. Ay applicatio that ivolves solvig may liear systems with a costat full matrix A ca beefit from adoptig the A 1 based approach over the LU-based method. We ow list various applicatios which are potetially suitable for the A 1 based method:

9 9 1. Spectral differetiatio methods for PDEs: While fiite differece methods require solvig liear systems with a sparse baded matrix, spectral methods, i may cases, require solvig of liear systems with full matrices [3]. For example, whe solvig a parabolic PDE i a implicit method, each time step ivolves solvig a liear system of the form Dx (t+ t) = F (x (t) ), where D is a full matrix which does ot vary with time t. I this case, if the umber of time steps K > K th (), it is faster to adopt the A 1 based approach, i.e., we ivert D at the begiig of the simulatio, ad the at each step calculate x (t+ t) = D 1 F (x (t) ). 2. Modified Newto methods: Newto methods are used to fid the roots of oliear equatios. They ivolve the iteratios ( x (k+1) = x (k) [J (k) ] 1 F x (k)), (4.1) where J (k) = J ( x (k)) is the Jacobia of the fuctio F at the poit x (k). Hece, x (k+1) is the sotio of the liear system ( J (k) x (k+1) = b (k), b (k) = J (k) x (k) F x (k)). (4.2) The calculatio of the Jacobia at every iteratio is typically very expesive. To reduce the cost of the Jacobia calculatio, i the modified Newto method [6], the Jacobia is calculated oly oce every several (K) iteratios. The commo approach is to avoid the explicit calculatio of J 1, ad fid x (k+1) from (4.2). Our aalysis shows that it is faster to use the A 1 based approach ad calculate x (k+1) from (4.1) for small matrices of size < th (K). A similar algorithm which may beefit from applyig the A 1 based method is the Update Skippig BFGS algorithm used i optimizatio problems [1]. I this case, the iteratios of the BFGS method are ( B k x (k+1) = B k x (k) λj x (k)), (4.3) where B k is a approximatio to the Hessia of F at the poit x (k) [5]. To reduce the cost of the Hessia calculatio, i the Update Skippig BFGS algorithm, B k is updated oly oce every K steps. I this case, Our aalysis shows that it is faster to use the A 1 based approach ad calculate x (k+1) from x (k+1) = x (k) λ k B 1 k J (x (k)), (4.4) for matrices of size < th (K). 3. Iverse iteratio: The smallest eigevae of a full matrix A (ad the correspodig eigevector) ca be foud from the sequece v(k+1), where [2], v (k) Av (k+1) = v (k). (4.5) The commo approach is to calculate v (k+1) usig the LU-based method, i.e., Ly = v (k), Uv (k+1) = y. (4.6) However, i this case, calculatig v (k+1) usig the A 1 based approach is faster if the umber of iteratios K > K th (). v (k+1) = A 1 v (k) (4.7)

10 10 Method Equatios CPU time (secods) [L, U] (1.2), (1.3) [L, U, P ] (2.5), (2.6) 7.6 [L, U, P ] + lisolve (2.5), (2.7) 4.62 A 1 (1.4), (1.5) 1.88 Table 4.1 Compariso of CPU ru times for the iverse iteratio method. As a fial example, we implemet the iverse power method (4.5) usig both methods as follows. We radomly choose 250 radom matrices. For each matrix we measure the overall cpu-time (preprocessig ad iteratios) eeded to calculate the smallest eigevae usig the iverse power method implemeted with the A 1 based method (4.7) ad the LU-based method (4.6). We stop the iteratios whe the relative error is below (with respect to the smallest eigevae foud by Matlab s eig fuctio). I our simulatios we observed that the A 1 based method was about 11 times faster tha the LUbased method (see Table 4.1). The average umber of iteratios eeded for the iteratios to coverge was 269 i A 1 based method ad 264 i the LU-based method. This 2% differece is probably due to the larger roudoff errors i the A 1 based method. Therefore, i this case, the A 1 based method is sigificatly faster tha the LU-based method, ad just as accurate. Usig the accelerated LU algorithms (see Sectio 2.4) lead to a cosiderable reductio i the overall CPU time, but was still 2.5 times slower tha usig A 1. Ackowledgmets. We thak Eli Turkel ad Siva Toledo for useful discussios. REFERENCES [1] J.E. Deis ad Jr. Schabel ad B. Robert, Numerical Methods for Ucostraied Optimizatio ad Noliear Equatios, Pretice-Hall, New York, [2] N.L. Trefethe ad D. Bua, Numerical Liear Algebra, Siam, Philadelphia, [3] N.L. Trefethe, Spectral methods i MATLAB, Siam, Philadelphia, [4] C.T. Kelley, Solvig Noliear Equatios with Newto s Method, Siam, Philadelphia, [5] K.G. Tamara ad D. P. O Leary ad L. Nazareth, BFGS with update skippig ad varyig memory,siam J. Optim.,Vol. 8, [6] A. Quarteroi ad R. Sacco ad F. Saleri, Numerical Mathematics, Spriger, New-York, 2000 [7] G. Gob ad C. va Loa, Matrix computatios, 2rd ed., The Johs Hopkis Uiversity Press, Lodo, [8] S. D. Cote ad C. de Boor, Elemetary Numerical Aalysis, 3rd ed., McGraw-Hill book compay, [9] N. J. Higham, Accuracy ad Stability of Numerical Algorithms, Siam, Philadelphia, 1996.