Wavefront Cache-friendly Algorithm for Compact Numerical Schemes

Size: px

Start display at page:

Download "Wavefront Cache-friendly Algorithm for Compact Numerical Schemes"

Lydia Webb
5 years ago
Views:

1 NASA/CR ICASE Report No Waveront Cache-riendly Alorithm or Compact Numerical Schemes Alex Povitsky ICASE, Hampton, Virinia Institute or Computer Applications in Science and Enineerin NASA Lanley Research Center Hampton, VA Operated by Universities Space Research Association National Aeronautics and Space Administration Lanley Research Center Hampton, Virinia Prepared or Lanley Research Center under Contract NAS October 1999

2 WAVEFRONT CACHE-FRIENDLY ALGORITHM FOR COMPACT NUMERICAL SCHEMES ALEX POVITSKY æ Abstract. Compact numerical schemes provide hih-order solution o PDEs with low dissipation and dispersion. Computer implementation o these schemes requires numerous passes o data throuh cache memory that considerably reduces perormance o these schemes. To reduce this diæculty, a novel alorithm is proposed here. This alorithm is based on a waveront approach and sweeps throuh cache only twice. Key words. cache locality, compact scheme, waveront alorithm, banded linear systems Subject classiæcation. Computer Science, Applied and Numerical Mathematics 1. Introduction. Compact numerical schemes are widely used or challenin problems o computational physics ë1ë. Compact ænite diæerence ormulas are deæned as expressions where derivatives at diæerent mesh points appear simultaneously. These schemes mimic spectral schemes with low dissipation and dispersion. In spite o the act that the number o arithmetic operations per rid node is approximately equal or explicit and compact ormulations o the same approximation order ë2ë, the computational time is considerably larer or the compact schemes. Tam and Webb èë3ë, p. 278è reported about the order o manitude computational time diæerence between explicit and compact ormulations. The poor perormance o compact schemes is explained by architectural eatures o modern computers. The ap between ormal CPU perormance and actual perormance is likely to increase because the CPU speed tends to increase much aster than the speed o memory access. To increase the computational eæciency o modern computers, the interace between a processor and its memory includes a number o cache memories that are placed èloicallyè between the processor and the physical main memory, ivin the processor ast access to data stored in the cache ë4ë. Access to main memory typically requires dozens or hundreds o æops, and reduction o the main memory-cache exchane represents a challene or scientiæc computin. Compact schemes require solution o banded linear systems, which consider each spatial partial derivative separately. Solution o banded linear systems by Gaussian elimination requires orward and backward sweeps throuh data. These sweeps are repeated in all three spatial directions ollowed by a Rune-Kutta temporal update èrkè and computin the riht-hand sides o compact ormulations. Thus, compact 3-D solvers pass data throuh the memory cache eiht times. On the contrary, an explicit central-diæerence alorithm may be easily written in such away that data passes throuh the cache only once. This study proposes a new ormulation o a compact-scheme based numerical alorithm which sweeps throuh data only twice per stae o RK. The alorithm is based on a waveront approach, where a current ront nodes are computed usin only the values at previous ronts. Then this alorithm is expanded to any number o levels o cache memory. Numerical solution at each time step is exactly the same as or a standard compact alorithm. æ Staæ Scientist, ICASE, NASA Lanley Research Center, Hampton, VA è aeralpo@icase.eduè. This research was supported by the National Aeronautics and Space Administration under NASA Contract No. NAS while the author was in residence at the Institute or Computer Applications in Science and Enineerin èicaseè, NASA Lanley Research Center, Hampton, VA

3 è1è 2. Hih-order Numerical Methods. Consider a multi-dimensional partial diæerential equation èpdeè: du dt = where t is the time, k =1; 2; 3 are spatial coordinates. The riht-hand side terms are approximated usin compact ænite diæerence schemes ë1ë: è2è æu 0 i,2 + æu 0 i,1 + U 0 i + æu 0 i+1 + æu 0 i+2 = a 2æx èu i+1, U b i,1è+ 2æx èu i+2, U i,2è; where æx is the rid step and primes denote derivatives with respect to x: Expansion to systems with second spatial derivatives ènavier-stokes typeè is straiht-orward as the compact ormulation or derivatives and the method o their computation are similar to those or the ærst derivatives. Equation è1è is discretized in time with an explicit Rune-Kutta scheme. The solution is advanced rom time level n to time level n + 1 in several sub-staes ë6ë è3è H M = S + M U M+1 = U M + b M+1 æth M ; + M + a M H M,1 ; where M is the particular stae number; and the coeæcients a M and b M depend upon the order o the RK scheme. To compute spatial derivatives, we solve the sets o independent linear banded systems o equations where each system corresponds to one line o the numerical rid. For example, a system correspondin to a line in the x direction has a scalar tridiaonal matrix N x æ N x : è4è a k;l Z k,1;l + b k;l Z k;l + c k;l Z k+1;l = k;l ; where k = 1; :::; N x ; l = 1; :::; N y æ N z ; a k;l ;b k;l and c k;l are the coeæcients, Z k;l are the unknown variables, and N x ;N y and N z are the number o rid nodes in the x; y and z directions, respectively. The ærst step o the Thomas alorithm is LU actorization è5è d 1;l = b 1;l ; c k,1;l d k;l = b k;l, a k;l ; k =2; :::; N x ; d k,1;l and orward substitution èfsè è6è 1;l = 1;l d 1;l ; k;l =,a k;l k,1;l + k;l d k;l ; k =2; :::; N x : The second step o the Thomas alorithm is backward substitution èbsè è7è Z Nx;l = Nx;l; Z k;l = k;l, Z k+1;l c k;l d k;l ; k = N x, 1; :::; 1: The coeæcients a k ;b k and c k are constant or compact schemes; thereore, LU actorization is perormed only once and the ærst step computations include only orward substitution è6è. The standard alorithm or compact numerical solution o the system è1è is perormed as ollows: Alorithm A Step 1. Compute the riht-hand side o equation è2è usin values o the overnin variable U rom the previous time step. Step 2. Compute the spatial derivatives solvin tridiaonal systems in all spatial directions. 2

4 Step 3. Compute the riht-hand side o equation è1è usin the spatial derivatives computed in Step 2 and update overnin variables by Rune-Kutta scheme. Step 4. Repeat computational Steps 1-3 or all Q staes o Rune-Kutta scheme. Step 5. Repeat computational Steps 1-4 or all time steps. This alorithm passes data throuh the cache to perorm Step 1, then it passes data throuh the cache twice per direction to compute the spatial derivatives èstep 2è, and ænally the alorithm touches each rid point to compute the temporal update èstep 3è. Thereore, the data pass throuh the cache 2 + 2D times, where D is the number o directions. For explicit schemes, coeæcients æ and æ are equal to zero; thereore, Step 2 in the above alorithm is reduced to local computations o spatial derivatives. Hence, explicit alorithms can be easily written in such away that the data passes throuh the cache once. 3. Proposed Cache-riendly Alorithm. In this section we develop a cache-aware compact numerical alorithm where the data passes throuh the cache only twice. Let us consider ærst the two-dimensional case. The waveronts are deæned as subsets o rid nodes èi;jè with I + J = const within a waveront. I a rid node èi;jè belons to the ront W, its neihbors èi, 1;Jè and èi;j, 1è belon to the previous ront WM and two other neihbors èi;j + 1è and èi +1;Jè belon to the next ront WP: The ollowin Alorithm B sweeps throuh rid nodes only twice and perorms exactly the same computations as Alorithm A èsee the previous sectionè. Alorithm B or iw=if,...,il or in=1,...,igèiw è Compute the riht-hand side o equation è2è Compute the orward step o the Thomas alorithm è6è in the x and y directions. or iw=il,...,if or in=1,...,igèiw è Perorm the backward step o the Thomas alorithm è7è in the x and y directions. Compute the Rune-Kutta temporal update. Here, IF and IL are the ærst and the last waveronts; indexed variable IG deænes the number o ridnodes within a waveront iw: The orward-step computations è6è in both spatial directions use already computed orward-step coeæcients in rid nodes èi, 1; Jè and èi; J, 1è; whereas the backward-step computations è7è use already computed values in rid nodes èi +1;Jè and èi;j + 1è èsee Fiure 1è. This alorithm exploits the data-independence o the solution o banded linear systems in diæerent spatial directions, i.e., the systems in the x and y directions are solved simultaneously. Additionally, the riht-hand sides o compact 3

5 Table 1 Number o data passes throuh cache or basic compact scheme èalorithm Aè, waveront compact scheme èalorithm Bè and explicit scheme. Number o data passes throuh cache Dimension Alorithm A Explicit Alorithm B 2-D D ormulations are computed simultaneously with the orward-step computations and the RK computations are perormed immediately ater completion o the backward-step computations or a rid node. This alorithm can be easily expanded to penta-diaonal matrices where two neihborin ronts rom either side are used in computations. This alorithm is expanded to the three-dimensional case where waveronts represent planes o rid nodes èi;j;kè with I + J + K = const: Similar to the 2-D case, three previous neihbors belon to the plane WM èsee Fiure 2è and the next three neihbors belon to the plane WP: Still, the alorithm sweeps twice throuh the 3-D array èsee Table 1è. 4. Extension to Multi-level Cache. Let us ærst consider two cache levels, primary level L 1 and secondary level L 2 : We cover the computational domain with boxes èsquares in the 2-D caseè that æt the cache size L 1 ; and renumber the boxes as super-nodes èi b ;J b ;K b è: Then, we deæne box waveronts as subsets o boxes where I b + J b + K b = const: Each box is considered as a computational domain where the orward and the backward steps o Alorithm B are applied. A computational domain covered with nine cache boxes is shown in Fiure 3. These boxes orm æve waveronts in the orward and backward directions. The ærst box waveront includes the box è1; 1è; the second box waveront includes boxes è2; 1è and è1; 2è and so on. The computed waveronts within each box are shown or the ærst two box waveront levels in the orward and backward directions. Let us deæne the two-level alorithm as ollows: Alorithm C or ib=ibf,...,ibl or iw=ifèib è,...,ilèib è Perorm orward-step computations o alorithm B or ib=ibl,...,ibf or iw=ilèib è,...,ifèib è Perorm backward-step computations o alorithm B Alorithm C is consistent, i.e., the previous waveront is completed by the time orward- or backwardstep computations bein or the current waveront. Alorithm C requires a diæerent way o storae o 4

6 overnin array U than the traditional way o column-by-column placement o array in memory. Instead, here the array should be stored box-by-box. This alorithm is easily expanded to cases with any number o cache levels. A cache box is covered with smaller boxes o the size o the next èsmallerè level o cache. In this case the number o nested loops is equal to the number o cache levels. The inner loop sweeps throuh rid nodes that belon to a waveront, whereas outer loops sweep throuh box waveronts correspondin to diæerent levels o cache. 5. Conclusion. The cache-aware compact numerical alorithm has been developed. The data pass throuh cache only twice or 2-D and 3-D cases. The alorithm is expanded to any number o cache levels. Interaction o the proposed alorithm with compilers will be studied in our uture research. REFERENCES ë1ë S. K. Lele, Compact Finite Diæerence Schemes with Spectral Like Resolution, Journal o Computational Physics, 103 è1992è, pp ë2ë T. Colonius, Lectures on Computational Aeroacoustics, presented at the lecture series on Aeroacoustics and Active Noise Control, von Karman Institute o Fluid Dynamics, 1997, ë3ë C. K. W. Tam and J. C. Webb, Dispersion-relation-preservin Finite Diæerence Schemes or Computational Acoustics, Journal o Computational Physics, 107 è1993è, pp ë4ë R. Y. Kain, Advanced Computer Architecture èa System Desin Approachè, Prentice-Hall, Inc., ë5ë Tien-Pao Shih, Goal-directed Perormance Tunin or Scientiæc Applications, Ph.D. Thesis, University o Michian, ë6ë R. V. Wilson, A. O. Demuren and M. Carpenter, Hih-order Compact Schemes or Numerical Simulation o Incompressible Flows, ICASE Report No ,

7 (I, J+1) (I-1,J) (I+1,J) (I,J) (I,J-1) WM W WP Fi. 1. Waveront alorithm in a 2-D case, where WM;W and WP are three consequent waveronts. Solid arrows represent orward-step computations and dashed arrows represent backward-step computations (I-1,J,K) (I,J,K-1) (I,J,K) (I,J-1,K) y Fi. 2. Waveront alorithm in a 3-D case. Shaded plane is the previous waveront. 6

8 (1,3) (2,3) (3,3) (1,2) (2,2) (3,2) (1,1) (2,1) (3,1) Fi. 3. Domain partitionin or two-level cache. 7

Parallelization of the Pipelined Thomas Algorithm

NASA/CR-1998-208736 ICASE Report No. 98-48 Parallelization of the Pipelined Thomas Algorithm A. Povitsky ICASE, Hampton, Virginia Institute for Computer Applications in Science and Engineering NASA Langley