Towards a Fast and Reliable Software Implementation of SLI-FLP Hybrid Computer Arithmetic

Transcription

1 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) Towards a Fast and Reliale Software Implementation of SLI-FLP Hyrid Computer Arithmetic UNANG SHEN AND PETER R. TURNER Department of Mathematics and Computer Science Clarkson University Potsdam, N 3699 U.S.A. pturner@clarkson.edu Astract: - In this paper we descrie a C++ implementation of a hyrid system comining SLI (symmetric level-index) arithmetic and FLP (floating-point) arithmetic. The principal motivation for the work to e presented is to promote the use of SLI arithmetic as a practical framework for scientific computing. This hyrid arithmetic is essentially overflow and underflow free, and its implementation has shown the suitaility to e used in real life computations. Key-Words: - symmetric level-index, computer arithmetic, overflow, underflow, hyrid, numer representation. Introduction This paper is concerned with the implementation, which could e used in practical computations, of an over/underflow free arithmetic ased on symmetric level-index (SLI) system. The implementation currently can e run in a C++ environment of either WINDOWS or LINU, and is availale for download at There are other software implementations of SLI arithmetic, typically in MATLAB, FORTRAN [6], or PASCAL [9]. The purposes of these implementations are to verify the algorithms, and to show the effectiveness of a potential hardware implementation y some numerical experiments. Besides those motivations, the new C++ implementation emphasizes more its practical use as software, with some recent improvements of the algorithms and the design of a hyrid numer representation scheme. The choice of programming language is ased on the fact that C++ is efficient, flexile, versatile, and popular in scientific computation. We claim the implementation is appropriate for use in some real life computational prolems, where over/underflow causes troule. A systematic testing process was executed to show its reliaility and efficiency in section 4. An application is also descried in [8] with some performance records of the program. Ease of use is also an important concern for the implementation. Little knowledge aout SLI arithmetic is required, and no additional analysis to the computational prolem is needed to use the C++ lirary. In other words, it is not necessary to predict which variale has the risk of overflow or underflow in floating-point (FLP) arithmetic, or to estimate how large or how small the result could e, if the FLP representale range is to e exceeded. When the desired operations in a program are supported y this C++ implementation, all a programmer needs to do is to simply replace the data type doule with type SLI for all real numer variales. The maority of this paper will discuss the design of the hyrid numer representation scheme, and the algorithms to perform arithmetic operations, as well as a few implementational details of them in C++. A little overview of SLI arithmetic is provided meanwhile. 2 Hyrid Numer Representation The level-index (LI) numer representation is to represent a positive real numer y x, where x, 0 x<, = φ ( x) = () φ ( x ) e, x, and φ is called the generalized exponential function. It follows that x = l + f, where the index f [0,) is given y f = ln(ln(...( ln ))), (2) the natural logarithm eing taken l times. So the level l must e a nonnegative integer. The symmetric level-index system presents a numer in (0, ) y the LI image of its reciprocal. Thus a real numer can e represented y ± r =± φ ( x) = s φ( l + f ), (3) and four pieces of information are needed to store a SLI numer -- numer sign s, level l, index f, and reciprocal sign r.

2 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) SLI numer representation has an essentially infinite representale numer range. It is sufficient to allocate ust 3 its to the level when working to any conceivale implemented precision, and the four asic arithmetic operations would e closed in the finite numer set of SLI representation. The details of LI and SLI numer representation and their algorithms for arithmetic operations are descried in [2], [4], [3], [], and [7]. It is not surprising that to perform arithmetic operations would e more complicated in the SLI system than in FLP arithmetic. Accordingly a hyrid system is proposed in [8] to take the advantages of oth SLI arithmetic and FLP arithmetic. If the magnitude of a numer is within a certain range, [2-5, 2 5 ] for example, the numer is represented in the IEEE standard form [5]. SLI numer representation is used otherwise. In this hyrid system, a doule precision numer can e packed into a 64-it word. See [8] for a possile representation scheme in a potential hardware implementation. However, it is not efficient to program this scheme in C++. We implemented the hyrid numer representation using a structure of total length of 70 its, as illustrated in Fig.. Note a SLI numer in this hyrid system is at 5 least level 4, since φ ( 4.57) 2. yte 8 ytes it field of length 3: 0: indicates an FLP numer; 4~7: the level l of an SLI numer. it field: numer sign s for SLI. it field: reciprocal sign r for SLI. not used doule: the value of an FLP numer; or the index f of an SLI numer. Fig. : SLI-FLP hyrid numer representation implemented in C++ This way no it manipulations are required, and the software implementation can run faster, with the tradeoff of a little extra space complexity. 3 Algorithms for Arithmetic Operations The algorithms implemented are ased on the modified algorithms for SLI arithmetic operations descried in []. Those modifications allow more parallel implementations in a potential hardware design. With a small change of initial calculations which will e descried later, the modified algorithms also enefit a software implementation. There are a few maor modifications to the algorithms in [], including the treatments of hyrid numer representation, and an approximation method ased on Taylor expansions for the algorithm of addition/sutraction. 3. Algorithm for Addition/Sutraction The addition/sutraction prolem is to solve the equation rz r r Z = sz φ ( z) = s φ( x) ± sφ( = ± (4) for s Z, r Z, z given s, s, r, r, x, y. We may assume r r φ( x ) φ( (or ) without losing generality. In the hyrid system, different paths of the algorithm will e executed for operands in different ranges. The figure elow displays which path would e chosen according to the larger magnitude operand in the addition/sutraction algorithm. pure FLP addition or sutraction pure SLI or mixed SLI-FLP, using Taylor approximation, or full algorithm φ (.0) φ(4.57) φ(5.84) E+2 trivial addition or sutraction Fig. 2: Domain decomposition of the algorithm for addition/sutraction in the hyrid system, when the larger magnitude operand satisfies > The domain decomposition is almost symmetric aout φ (.0) noticing φ (.0) = = φ(.0). So it is only necessary to show the numer line for >. Both SLI and FLP numers are used to denote magnitudes in the numer line. Notice a doule precision FLP numer can only reach 2 024, which is aout φ (4.632). The algorithm path in the second sudomain is most complex, and can not e shown in a single illustration. Figures of finer domain decomposition can e found in [7] and [8]. Trivial arithmetic operations are defined as the cases where the operation result equals to the larger magnitude operand. In doule precision arithmetic, addition/sutractions are all trivial except for φ( x) φ( x) if φ( x ) φ( ), so Z = and the algorithm is complete. The most eneficial modification in the algorithm is to adopt FLP arithmetic operations. A pure FLP addition or sutraction will e performed, if oth the

3 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) operands are within the reduced FLP range [2-5, 2 5 ]. It will e much faster than a pure SLI operation in any conceivale software implementations. If the larger magnitude operand is a FLP numer and the other operand is a SLI numer, it is safe to convert the SLI operand into FLP form and do a pure FLP operation. But for the other SLI-FLP mixed case, the SLI algorithm needs to e performed. The modified full algorithms are ased on computing sequences defined y a = / φ( x ); c = / φ( y ); = φ( z ) / φ( x ). (5) Assuming s = s = in equation (4) for simplicity, we give the full algorithm for SLI addition or sutraction, which is modified from [] as follows. Algorithm : Modified full SLI addition/sutraction of positive arguments for nontrivial cases Input (r, x), (r, r where r φ( x ) φ( and l, l 2. large= ( r == r == ); Initialize Booleans mixed = ( r! = r small = ( r r Z = r. Compute = exp( f ), a l = al = exp( / a ) ( = l = exp( f = exp( / ) ( = l a exp large a, a+ d = exp, mixed a a exp, small a c= ± d c= + r a ln c < l ), == r,...,2); );,...,2); == ). while and c a = +, c = + a ln c if c< a then z = + c / a, go to Output = l, h= f + ln c while h = +, h= ln h z = + h Output (r Z, z) Operator ± is denoted to compute the initial value of the c-sequence. It means that + is used for addition and - is used for sutraction. The computation of sequences in this algorithm is mathematically ut not numerically equivalent to what is defined in []. In order to avoid possile overflow/underflow prolems when generating the c-sequence with FLP internal operations, the initial value of the c-sequence is calculated in a different way. The new definition of the c-sequence is also more efficient to compute in a software implementation, y saving one exponential function evaluation. The tradeoff is that it requires oth operands to e at least level 2. But this will not e a prolem in the hyrid system, since then all SLI numers start from level 4. Another saving is that the flipover cases are all avoided. A flipover addition is defined as the case where the reciprocal sign of the larger addend is changed after a SLI addition. But any two SLI numers with different reciprocal signs are sufficiently distant in this hyrid system to avoid flipover additions, considering the limited precision of a 64-it numer representation. Algorithm is the full algorithm for pure SLI addition or sutractions. As shown in Fig. 2, the algorithm will e considered when the larger magnitude operand satisfies = s r x φ( x) and 2 5 < φ ( x ) < φ(5.84). There can e two simple variations for the algorithm, depending on the magnitude of the other operand. When is represented as a FLP numer in this hyrid system, it is not necessary to compute the -sequence. Notice only, the last value in the sequence, is useful in the later computations. It is possile to skip generating the -sequence in the SLI-FLP mixed case, y redefining d = exp( / a) (6) and performing a large addition in the rest of the algorithm. This variation can e easily verified according to the sequence definitions in (5). When has a magnitude sufficiently smaller than, the Taylor approximation scheme (see [7] for details) is applicale and the computation of the c-sequence can e avoided. In this particular implementation, a linear interpolation is performed for each addition or sutraction to decide if the first order Taylor approximation method is useful. If it is, calculate

4 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) d, large or mixed ε = a a... 2 al (7) d /( d+ ), small and output ( r, x± ε ). The approximation of ε is ased on a different expression than in [7], ecause the original full algorithm is used in [7], while the modified algorithm is adapted in this paper. However, the formula deduction of ε is very similar. 3.2 Error Bounds in Implementation Since all the internal operations are performed in standard doule precision FLP arithmetic, the error of the implemented SLI arithmetic operations is larger than what is predicted similarly as in [4]. When performing an SLI addition, we assume that the sequence {a } is stored with asolute precision γ while all the other quantities are stored to asolute precisionγ. Let δz represent the error of z caused y inherent errors δ x in x and δ y in y. As stated in [4], δ z in a large addition satisfies δz c + γ l Z (8) (2ρ+ 2) γ + (/ e lnγ ) ρλγ where ρ= + / φ' () + / φ'(2) (9) and e λ = ( 4+ e ) e (0) On replacing ρ, λ and e y their numerical values, the result reduces to δz 6.784γ + ( lnγ ) γ () Due to the restriction of FLP arithmetic, the internal operations in this software implementation have the precision of Sustituting othγ andγ with 2-52 in (), we computed the error δ z in a large addition is ounded y approximately , and similar error ounds can e otained for mixed and small cases. The largest 4 among the error ounds is , which would e acceptale for most computations. Although the aove error analysis is ased on the full algorithm for pure SLI addition, the error ounds also work for the variations of the algorithm. The SLI-FLP mixed case is expected to have a etter precision since the roundoff error in generating the -sequence is avoided. The Taylor approximation is deemed to e applicale only when its result perfectly matches the result of the full algorithm at the expected precision level, so the approximation method does not introduce any further error to the implementation. Sutractions in this implementation have similar errors, unless cancellations occur. The correctness testing in section 4 supports the error analysis in this section. 3.3 Algorithm for Multiplication/Division The domain decomposition of multiplication/division is similar to the one for addition/sutraction, except that trivial cases start from a different numer. The threshold is aout φ( 6.839) for multiplication/ division in doule precision SLI arithmetic. The full algorithm for multiplication or division is also modified from []. But note here φ( x) φ( r instead of r φ( x ) φ( is required for a pure SLI multiplication/division. Algorithm 2: Modified full SLI multiplication/ division of positive arguments for nontrivial cases Input (r, x), (r, where φ( x) φ( and l, l 3. Initialize Booleans large, mixed, small, r Z = r. Compute = exp( f ), a l al = exp( / a = exp( f ), = exp( / ) ( = l a2 ± exp a22 c = a2 m exp a22 c= ± a ln c 2 ) ( = l 2 2,,,...,3);,...,3); large or small mixed = 2 while < l and c a = +, c = + a ln c if c< a then if = 2 and c < exp( / a2 ) then z = + c / exp( / a2 ), go to Output else z = + c / a, go to Output = l, h= f + ln c while h = +, h= ln h z = + h Output (r Z, z)

5 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) When a ± operation appears in this algorithm, + should e used for multiplication and - would e used for division. This algorithm can also have variations to make it run faster for some special cases. However, the Taylor approximation method is not very helpful for multiplication/division, since its applicale domain is very limited. The SLI-FLP mixed case can still simplify the prolem. We first swap with if the leading operand is an FLP numer. Then the value of c can e computed y ± exp( / a2 ) ln, largeor mixed c = (2) m exp( / a2 ) ln, small Again the -sequence is not needed. The multiplication/division algorithm is expected to have similar error as the addition algorithm, unless cancellations happen. 3.4 Algorithm for Powering Operation It is possile to derive a similar algorithm for powering operation, ut there would e many special cases to consider. So the powering operation was implemented y calling the suroutine of multiplication. For example, if φ ( φ ( z) = φ( x), (3) taking logarithms of oth sides of the equation, we will get φ( z ) = φ( x ) φ(. (4) So the large case of powering prolem ecomes a multiplication prolem at different levels. However, a separate powering algorithm could e marginally faster ecause of the saving of extra function calls. 4 Testing One important purpose of this software implementation is to promote the use of SLI arithmetic in practical scientific computation, so the correctness and the performance of the program are oth critical. A systematic testing procedure was designed and executed, so that we can ensure the usefulness of the implementation y providing the testing results. 4. Correctness Testing We are not ale to directly test the correctness of this SLI-FLP hyrid system. It is easy to verify the cases where FLP arithmetic operations are performed, ut it will e hard to tell if the results are correct for numers that exceed the doule precision FLP representale range, especially for cases using the Taylor approximation method. So our testing egan from the pure SLI system using only the full original algorithm introduced in [4]. The original algorithm is identical to the modified algorithm in [] except for the definitions of the -sequence. But there are fewer restrictions that are needed for a pure SLI system using FLP internal operations. We may compare the results from pure SLI arithmetic with the ones from FLP arithmetic if the magnitudes of testing data are ounded, so that the operations will not cause overflow or underflow in FLP arithmetic. A large numer of such tests are ale to cover all ranches of the algorithm. If the implementation gives correct results for all tests in the FLP range, we may reason that the algorithm remains correct for numers that exceed this range, ecause all cases in the pure SLI system are essentially the same computation at different levels. Next we can test the hyrid arithmetic ased on the pure SLI system. The agreement of their results in arithmetic operations is enough to conclude the correctness of the implementation of the hyrid system. To test the pure SLI arithmetic using only the full original algorithm, the testing cases were chosen oth y hand and y a random numer generator. 0,000 pairs of random operands with a log-normal distriution in magnitudes, as shown in Fig. 3, were generated for the testing of the four asic arithmetic operations. The log-normal distriution is the est choice for the testing cases among the well known distriutions, ecause the testing needs to cover a range of doule precision FLP numers as large as possile, ut at the same time, sufficient testing cases need to e picked around, in order to test all the three flipover cases which appear infrequently. Fig. 3: Cumulative distriution of the correctness testing data for asic arithmetic operations in the pure SLI system

6 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) The random operands for testing the powering operation could e generated similarly. But FLP overflow/underflow happens more frequently in powering than in the asic arithmetic operations, so the numer range of the testale cases is very limited, and a normal distriution is used instead. A testing tool in LINU called GCOV is used to verify the coverage of all ranches of the algorithm. When some cases with a true zero in operations are manually added, every line in the arithmetic operation suroutines is covered in testing. We claim a test successful if the result in SLI arithmetic and the result in FLP arithmetic differ y no more than a small tolerance. According to the error ounds predicted in Section 3.2, a tolerance of is used in the index. The FLP result of an operation is converted into an SLI numer, and its index is compared with the index of the SLI result, in the situation that oth results have the same level in SLI forms. 4 Using the tolerance , all tests of addition/sutraction were successful. A few tests 4 would fail if a smaller tolerance such as 2 0 is used. The results verified the error ounds predicted in section 2. Almost all the tests of multiplication/division and powering operation succeeded with the same error tolerance. A few exceptions were oserved, since cancellations occur more often in multiplications and divisions, with the log-normal distriuted testing data. Fig. 4: Cumulative distriution of the correctness testing data for arithmetic operations in the SLI-FLP hyrid system Next we tested the implementation of the hyrid system. The random numer generator can now work with SLI numers to get a proper distriution of testing data, and the uniform distriution, as shown in Fig. 4, works well for the testing of all arithmetic operations, including powering. Note the signs of the horizontal coordinates denote for the reciprocal signs of SLI numers instead of numer signs. The reason for having a horizontal line segment in the center is that φ (.0) = / φ(.0), and the length of this line segment is zero in SLI arithmetic. For the hyrid system, we have tested 0,000 pairs of random operands following a uniform distriution in magnitude, plus a few special cases manually added. All ranches of the algorithm were tested, and the results matched the ones from FLP arithmetic or from the well tested pure SLI system. Hence we conclude that to perform arithmetic operations in this SLI-FLP hyrid system is reliale. 4.2 Performance Testing In order to evaluate the performance, the time costs of performing arithmetic operations in the SLI implementation will e compared to the costs in FLP arithmetic. However, the SLI arithmetic was programmed in C++ as a class, and has a lot of overhead in running time, while the FLP arithmetic was uilt in chips. The results of a direct comparison will not reflect the efficiency difference etween their algorithms. An FLP class was then programmed to make the comparison fairer. In this performance comparison we still run the functions defined y ANSI C++ to do FLP arithmetic operations, ut all the operations are now encapsulated into a class, in order to add similar overhead to FLP computations as the ones SLI arithmetic has in the software implementation. Note the comparison is not yet fair for SLI arithmetic, since the logical structure of the algorithm cannot e efficiently programmed in a high level language. The performance testing was done for four types of arithmetic operations -- addition/sutraction, multiplication, division and powering. 0,000 random numers were generated as operands, and each operation etween any two numers was repeated thousands of times for a more accurate comparison. Because the hyrid system uses different algorithms to perform arithmetic operations for operands in different numer ranges, we made three tests with different sets of testing data. All three sets of data follow a uniform distriution in SLI representation, ut with various numer ranges as shown in Tale. The numer range specified in the tale is only for those greater or equal to. If a numer is etween 0 and, it must e within the reciprocal of the corresponding range. For example, must e within 022 (2,) in data set. There are negative numers

7 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) Tale : Properties of data sets used in performance testing Data set Numer range (for those. 0 ) Operands in FLP over/underflow Results in FLP over/underflow (,2 ) 0% 3.3% 2 ( φ (4.6322), φ(7.0)) 00% 00% 3 (, φ (7.0)) 39.09% 50.05% as well, and their asolute values fall in the same numer ranges. Data set contains only numers in the doule precision FLP representale range. However, it is possile that the results of arithmetic operations overflow or underflow in FLP arithmetic, especially in powering operations. The performance of the implementation in this numer range is our maor concern, since most practical computations are still in this range. The time costs of other operations in different systems are shown in the following tale, providing that an FLP addition/sutraction costs one time unit. The experiments were performed using a desktop with a 3GHz Pentium 4 CPU. Tale 2: Time comparison for operands in doule precision FLP range, normalized with respect to FLP addition Operation FLP SLI hyrid +(-) * / The pure SLI system used in comparison is implemented ased on the original algorithm without the Taylor approximation method. The results in Tale 2 clearly show that the hyrid system enefits a lot from using FLP arithmetic in an appropriate numer range. The hyrid system costs less than five times the FLP arithmetic does. The cost ratio will e even etter in practice, since a real life prolem usually spends a large portion of time on operations other than real numer computations. See [8] for an example. Tale 3: Time comparison for operands eyond FLP range, normalized with respect to FLP addition Operation FLP SLI hyrid +(-) * / Data set 2 contains numers eyond doule precision FLP range. In this numer range we may tell how the Taylor approximation scheme and the other modifications to the algorithm enefit the performance of SLI arithmetic operations. The time comparisons are displayed in Tale 3. Note that the FLP results are all exceptions (0, Inf or NaN), so the times indicated do not reflect meaningful results. Data set 3 is used to evaluate the overall performance of the hyrid system, so oth FLP numers and SLI numers are included. Tale 4: Time comparison for operands uniformly distriuted in the hyrid numer representale range, normalized with respect to FLP addition Operation FLP SLI hyrid +(-) * / In Tale 4, the hyrid system clearly shows its advantage over the pure SLI system without the Taylor approximation scheme. The overall performance is almost douled for most operations. Although the FLP arithmetic is still much faster, it does not always give desirale results. As shown in Tale, 39% of numers and 50% of operations overflow or underflow in FLP arithmetic for this data set. The performance testing shows that the hyrid system works much faster than a pure SLI system, and its performance is acceptale even compared with a pure FLP system, when a comparison is appropriate. 5 Conclusion The software implementation of SLI-FLP hyrid arithmetic descried in this paper is roust and ready to e used in practical computations. The numer representation scheme and the algorithms for arithmetic operations were all modified, with the goal of eing efficiently implemented using a high level computer language. In order to ensure the reliaility and efficiency of the implementation, oth correctness testing and performance testing were carefully designed and executed. With acceptale efficiency, this C++ lirary can expediently help mathematicians, engineers and

8 Proceedings of the 6th WSEAS Int. Conf. on Systems Theory & Scientific Computation, Elounda, Greece, August 2-23, 2006 (pp0-08) programmers to control overflow and underflow prolems in computing. We hope the implementation also help to motivate further research on SLI arithmetic, y ringing it to the interest of more people who need its advantages. References: [] M.A. Anuta, D.W. Lozier, N. Schaanel & P.R. Turner, Basic Linear Algera Operations in SLI Arithmetic, Computing and Applied Mathematics Laoratory (Applied and Computational Mathematics Division), NIST Report, NISTIR58, 996. [2] C.W. Clenshaw & F.W.J. Olver, Level-index Arithmetic Operations, SIAM J. Numer. Anal. 24, 987, [3] C.W. Clenshaw, F.W.J. Olver & P. R. Turner, Level-Index Arithmetic: An Introductory Survey, Numerical Analysis and Parallel Processing (P.R. Turner, ed.), Springer-Verlag, Berlin, 989, [4] C.W. Clenshaw & P.R. Turner, The Symmetric Level-Index System, IMA J. Numer. Anal. 8, 988, [5] IEEE 754/854, IEEE Standard for Floating Point Arithmetic, ANSI/IEEE Std. Vols 754/854, IEEE, New ork. [6] D.W. Lozier & P.R. Turner, Symmetric Level Index Arithmetic in Simulation and Modeling, J. Res. Natl. Inst. Stand. Technol. 97, 992, [7]. Shen & P.R. Turner, Taylor Approximation for Symmetric Level-Index Arithmetic Processing, IMA J. Numer. Anal.26, 2006, [8]. Shen & P.R. Turner, A Hyrid Numer Representation Scheme Based on Symmetric Level-Index Arithmetic, CSC'06 (WORLDCOMP'06), CSREA Press, Las Vegas, 2006, [9] P.R. Turner, A Software Implementation of SLI Arithmetic, Proc. ARITH9, IEEE Computer Society, Washington DC, 989, 8-24.