Minimizing Memory Access By Improving Register Usage Through High-level Transformations

Transcription

1 Minimizing Memory Access By Improving Register Usage Troug Hig-level Transformations San Li Scool of Computer Engineering anyang Tecnological University anyang Avenue, SIGAPORE Moammed Javed Absar STMicroelectronics Asia Pacific Pte. Ltd. 20 Science Park Road, SIGAPORE Abstract Multimedia signal processing software typically ave to process large amounts of data. Te algoritms often involve te andling of data arrays in te form of nested loops. Experiments sow tat for tis kind of applications data transfer (memory access) operations consume muc more power tan data-pat operations. Our objective is to reduce memory access related power consumption by reducing te number of data transfers between processor and memory, or between a iger (closer to processor) level of memory and a memory at a lower level using source program transformation. In tis paper, we propose a source-to-source transformation metod to improve register usage for multi-dimensional arrays in nested loops. Significant improvement is demonstrated troug some bencmark programs. I. ITRODUCTIO Power consumption as become an increasingly important cost factor in embedded system designs. One of te reasons is te proliferation of battery powered portable and-eld devices. Moreover, ig power dissipation also means costly packaging and cooling requirements and lower reliability. Consequently, power-efficiency is a critical concern wen designing an embedded system. Our specific target of power reduction is multimedia applications. Tis kind of applications is data-intensive and access large amounts of data. Power consumption of tese applications is largely contributed by data transfer and memory access operations [8], [9], [14], [10], [11]. In contrast, data-pat operations consume muc less power. Hence, reducing data memory access will lead to a reduction of power consumed by te application. Generally, in a program data are stored as arrays and are often manipulated inside (nested) loops. Witout any optimization to te source code, redundant access to te arrays often exist and sould terefore be minimized. Our approac to tis problem is to perform ig-level source-to-source transformations so tat temporally reused data can be stored in registers instead. However, temporal reuse can appen in various forms. In order to select and perform suitable ig-level transformations for different kinds of temporal reuse automatically, a systematic metod is required. Tis paper proposes suc a metod tat improves te register allocation of subscripted array variables under normal compilation conditions, wit compilers wic use coloring-based register allocators. Te major contribution of our work is tat multiple-index subscripted variables are explored in our metod. In contrast, te previous works [7], [6], [12], [13], [5] restrict array references to be single-index subscripts. Tis limits te maximum exploitation of temporal reuse. II. MULTIPLE-IDEX SUBSCRIPTED VARIABLE Subscripted variable is array reference inside a loop nest (e.g. x[i][j]). Multiple-index subscripted variable means tat an array reference can ave more tan one loop index variables in eac of its dimensions (e.g. x[i + j][j] and x[2i j]). A general form of for (I 1 = b 1; I 1 < (b 1 +1); I 1 += t 1) for (I 2 = b 2; I 2 < (b 2 +1); I 2 += t 2) for (I M = b M; I M < (b M +1); I M += t M) x[ ] H c [ I1 I2 IM] M1 or [ f ( f ( ] [ I 1 11 Fig. 1. loop M M1 c M [ c1 c2 M I M M2 c 2 c ] I M M c ] General form of multiple-index subscripted variable x in a nest Multiple-index subscripted variable is illustrated in Fig. 1. Te array reference x[ ] as dimensions inside a loop nest wit a dept of M. In eac dimension, a subscript expression f n ( (1 n ) can ave maximal M loop index variables I m (1 m M). Tat is to say, te subscript value of te nt dimension of te array is determined by more tan one loop index variables. Wen it is te case, te array reference x is said to be a multiple-index subscripted variable. A multiple-index subscripted variable generates temporal reuse if it accesses te same datum in different iterations; tis kind of variables is exploitable in improving register allocation. Tose tat fail to generate temporal reuses will not be considered during register allocation. Weter multiple-index subscripted variable can generate temporal reuse is determined by bot its subscript expressions and te boundary values of te loop nest. Tis is explained in more detail in te section III-A. Temporal reuse generated by a multiple-index subscripted variable is referred to self-temporal reuse. However, tis is different from te self-temporal reuse described in [15]. In [15] te self-temporal reuse of an array reference is generated by invariant loop index variable only, wereas, te self-temporal reuse ere is generated by multiple loop indices. Fig. 2 gives an example of te two cases. Bot reference x[i] in Fig. 2(a) and reference x[i + j] in Fig. 2(b) generate temporal reuses. x[i] is invariant to loop index j. Wit te same value of i, x[i] accesses te same datum trougout te entire loop j. In tis example, x[i] as 10 instances from x[0] to x[9]. Every instance is accessed 10 times wic is te boundary value of loop j. In contrast, x[i + j] does not ave any invariant loop. Its subscript is a function of bot loop indices i and j. Te temporal reuse generated by x[i + j] results from bot loop indices i and j. For example, x[i + j] accesses te instance x[1] twice for (i = 1 j = 0) and

2 Fig. 2. for (i = 0; i < 10; i++) for (j = 0; j < 10; j++) = x[i] (a) for (i = 0; i < 10; i++) for (j = 0; j < 10; j++) = x[i+j] (b) Examples of two kinds of self-temporal reuse (i = 0 j = 1). Furtermore, x[i + j] accesses eac instance wit different number of times. x[0] is accessed once for (i = j = 0), wereas, x[9] is accessed 10 times for different value combinations of i and j. Tis example sows tat multiple-index subscripted variable generates self-temporal reuse in a different way from loop invariant reference. Terefore, te self-temporal reuse described in tis paper complements te original concept in [15]. In a more general case like Fig. 1, te situation will be more complicated. Tis requires a systematic approac to detect and exploit multiple-index subscripted variable. III. METHOD Conceptually, te exploitation metod is simple. It tries to minimize memory access by replacing te frequently used multiple-index subscripted variables to temporary scalar variables. Tis data transformation is called scalar replacement [5]. To enable tis transformation, an systematic approac is developed. Te flow of te overall approac is sown in Fig. 3. Te steps in te approac are discussed in te following sub-sections. detection loop intercange unroll-and-jam scalar replacement Fig. 3. variable Te flow of te approac exploiting Multiple-index subscripted A. Detection Process Given a program, te compiler sould find out not only te multiple-index subscripted variables but also teir ability of generating temporal reuses. Tis process starts from te subscript expressions of an array reference. Te subscript expressions can be written in te form of a matrix called access matrix, H (defined in Fig. 1). For a loop invariant reference, te access matrix, H, contains one or more rows of zeros, meaning a loop index is not in any subscript expression of te reference. Tis evidence of temporal reuses is obvious to detect. In contrast, te situation for a more general form of array reference including multiple-index subscripted variable is not tat direct. For a general form of array reference, its ability of generating temporal reuses can be found by comparing te rank of its access matrix wit te dept of te loop nest it resides. For a multipleindex subscripted variable, te rank of its access matrix is computed by applying Ecelon reduction [3]. Tis rank is used to compare wit te loop nest s dept less te number of invariant loops, since a multiple-index subscripted variable can also be a loop invariant reference. Assuming te number of loops invariant to a multipleindex subscripted variable is p, te results of te comparison are sown below. Wen rank(h) = M p, tere is no temporal reuse is generated by te multiple indices. Wen rank(h) > M p, tis situation sould not exist in a correctly written program. Tere is no reuse in tis case. Wen rank(h) < M p, te reference migt generates temporal reuses wit its loop indices. It sould be noticed tat, tese temporal reuses are exploitable only wen tey occur witin te loop bounds b and b. Wen it is found tat a reference as rank(h) < M p, te distance vector sould be computed using Equation 1. Distance vector measures te distance between multiple accesses to a datum in terms of loop iterations. Its principle is tat self-temporal reuse occurs wen any two instances, x[ i)] and x[ j)], of an array reference x[ ] access te same datum at different iterations i and j ( i j). i) = j) and i j ih + c = jh + c ( i j)h = 0 d = i j dh = 0 (1) Te distance vectors for x[i] in Fig. 2(a) and x[i+j] in Fig. 2(b) are [0, 1] and [1, 1] respectively. From te distance vector of x[i + j], it can be seen tat tere is a coupling effect between te values of te two loop indices. Tat is to say, wenever loop index i increases by 1, loop index j sould decrease by 1. Tis coupling effect of te indices generates temporal reuses for exploitation. B. Loop Intercange And Related Tests Loop intercange is a loop transformation tat permutes te loops inside a loop nest [2], [3]. It is performs before unroll-and-jam in te approac. Loop intercange itself is not elpful to reveal reuses from multiple loop indices. Instead, it is performed for loop invariant references. Loop invariant reference is exploitable only wen its invariant loop is te innermost loop. Altoug tis paper is to exploit multiple-index subscripted variable, te occurrence of loop invariant reference makes loop intercange necessary. To determine weter loop intercange is necessary, te metod does te followings. It finds out all te invariant loops, wic are legal to sift to te innermost loop, in a loop nest. Ten it selects one of te invariant loops. By assuming tat loop as te innermost loop, it computes te amount of exploitable reuses. Te invariant loop tat generates maximum number of reuses will be cosen to be te innermost loop if it is not yet te innermost. If invariant loop does not exist in te loop nest, no loop intercange and related tests will be performed. C. Unroll-And-Jam Unroll-and-jam is a kind of loop transformation tecnique tat unrolls a loop body several times [6], [13]. It can be used in conjunction wit scalar replacement to improve register allocation [5], since it reveals te reuses carried by te loop indices. As mention in section III-A, te distance vector of reuse generated by multiplexindex subscripted variables reveals coupling effects between loop indices. Tus, wen applying unroll-and-jam, multiple loops sould be unrolled to expose tis kind of reuses. Altoug unrolling multiple loops does not cause any legality problem, jamming te unrolled loop body into te innermost loop is not always legal. For an outer loop, unroll-and-jam is allowed wen te outer loop can be intercange to te innermost loop. Te most important problem in unroll-and-jam is to determine te unrolling vector of te loop nest. For loop nest wit dept M, te

3 For every loop carrying legal temporal reuses, its unrolling factor v is initialized to te corresponding t m. Te minimal ratio R min is initialized to 1. Te optimum unrolling factor v opt is initialized to tm. Assuming tere are L loops carrying legal temporal reuses, form=1, to L for v t to max endfor m endfor m v m compute te number of reuses, e. compute te number of registers, r. if (r < te number of registers available) break; endif. compute te total number of memory accesses,a. compute te number of memory access ratio R. if (R < R opt ((R==R opt opt opt opt opt ) && (( v1 v2 vm vl) ( v1 v2 vm vl ) ))) R opt = R. endif Optimal unrolling vector is assigned wit te current unrolling vector. for (i = 0; i < 9; i += 3) for (j = 0; j < 9; j += 3) = x[i + j] ; = x[i + j + 1] ; = x[i + j + 1] ; = x[i + j + 3] ; = x[i + j + 3] ; = x[i + j + 4] ; = x[i + 9] ; = x[i + 10] ; = x[i + 11] ; for(j = 0; j < 10; j++) = x[9+j] ; (a) Unroll-and-jam for (i = 0; i < 9; i += 3) x3 = x[i]; x4 = x[i + 1]; for (j = 0; j < 9; j += 3) x0 = x3; x1 = x4; x2 = x[i + j + 2]; x3 = x[i + j + 3]; x4 = x[i + j + 4]; = x0 ; = x1 ; = x1 ; = x3 ; = x3 ; = x4 ; = x[i + 9] ; = x[i + 10] ; = x[i + 11] ; for(j = 0; j < 10; j++) = x[9+j] ; (b)scalar replacement Fig. 4. Iterative algoritm of searcing optimal unrolling vector Fig. 5. Scalar replacement in round robin fasion after unroll-and-jam unrolling vector is defined as v = [v 1 v 2... v m... v M ]. Wit te understandings of te multiple-index subscripted variables, we develop a parameter estimation model to determine te unrolling vector for loop nests containing multiple-index subscripted variables. Tis estimation model as taken into account te distance vectors of te reuses, te boundaries and te incremental step size of te loop nest, and te number of available registers. In our model te extended code size from unroll-and-jam is not considered, since tis is not purpose of tis paper. However, furter consideration on extended code size sould be incorporable into our metod witout muc modification. 1) Parameters To Estimate: Te optimal unrolling vector not only reveals a significant amount of reduction in memory access but also avoids register spilling. To find suc an optimal unrolling vector for a given loop nest, measurements are needed. Te measurements take bot te effect of scalar replacement and te effect of unroll-and-jam into account. Te details are listed below. Te total number of reuses exposed by unroll-and-jam, e. Reuse is te read after te first access. Te number of scalar variables required by scalar replacement, r. Te number of registers required cannot be greater tan te number of available registers. Tis is a condition to prevent register spilling. Te total number of memory accesses after te optimization, a. Tis computes te total number of memory accesses after scalar replacement in round robin fasion. Te ratio of te memory accesses after and before te optimization, R. Tis parameter reflects te effect of optimization. Tis is te objective to minimize. 2) Unrolling Vector: Te algoritm for selecting an optimized unrolling vector based on te parameter estimations is presented in Fig. 4. In Fig. 4, te first step is to do some initialization work for te iterative searcing of te optimal unrolling vector. It initializes bot te temporary unrolling vector (for searcing) and te default optimal unrolling vector to te original incremental step size vector t. During te searc, it performs te parameters estimation for every possible unrolling vector. Te unrolling vector leading to minimal memory accesses under te register restriction will be cosen. If tere is more tan one suc unrolling vector, te one wit smaller code size will win. If no unrolling vector satisfies te register restriction, te original incremental step size is te optimal unrolling vector. D. Scalar Replacement Scalar replacement replaces subscripted variables (array references) by temporary scalar variables to effect reuse [5]. Scalar replacement is te last transformation in te metod, wic is performed after unroll-and-jam. Tis transformation increases te register usage for most normal compiler wit coloring-based register allocators and is te most effective way of reducing memory operands [14]. Register operand also as sorter running times due to elimination of potential stalls and cace misses. Wen applying scalar replacement, tere are few tings to consider. Firstly, legality sould always be taken into account. Scalar replacement is performed on variables aving true dependence (readafter-write) or input dependence (read-after-read). Wen te reuse is true dependence, it sould be ensure tat tere is no data update between te write and te read of te datum. Wen te reuse is input dependence, scalar replacement will not cause any legality problem. For multiple-index subscripted variables, te reuse generated by multiple loop indices is self-temporal reuse, wic is always input dependence. Hence, tere is no legality problem during scalar replacement. Secondly, wen performing scalar replacement after unroll-and-jam, te temporary scalar variables are used in round robin fasion. For example, te code in Fig. 2(b) is optimized and sown in Fig. 5. In Fig. 5(a) unroll-and-jam is performed to expose te temporal reuses generated by te multiple-index subscripted array reference x[i+j]. Bot loop i and loop j are unrolled by an unrolling factor of 3. Te temporal reuses are te references aving te same subscripts, wic are ten reduced by scalar replacement as sown in Fig. 5(b). Scalar replacement is applied in round robin fasion by introducing temporary variables x0, x1, x3 and x4. Te assignments of x3 to x0 and x4 to x1 clear te data in x3 and x4, tus x3 and x4 can be loaded wit new values. To enable te round robin fasion, x3 and x4 are initialized in te outer loop. In te outer loop, tere is no data passing between x3 and x4, since te distance of two array references carried by loop i is 1, wic is less tan te incremental step size 3. Hence, te round robin fasion is not applied in te outer loop. Tis example sows tat scalar replacement in te round robin fasion greatly reduces te number of array references. Tis is because te data wit lifetime across iterations are passed troug scalar variables.

4 TABLE I BECHMARK FROM HLSYTH95/MEMORY Bencmark Compression Laplace LowPass SOR Wavelet E. Complexity Description An image compression sceme A Laplace algoritm to perform edge enancement A low-pass filter to an image A Successive Over-Relaxation (SOR) algoritm Te Daubecies 4-Coefficient Wavelet filter Te algoritm as to first detect te distance vectors in a loop nest. Specifically, te process of searcing multiple-index subscripted variable is found in O(n) time, n is te number of subscripted array references. After te detection process, te algoritm performs te test for loop intercange. Tis test is usually fast and requires O(M) time, M is te dept of te loop nest. Following loop intercange, te algoritm searces te loop index space for optimal unrolling vector based on te parameter estimations. Tis process requires time O(MK), M is te dept of a loop nest, and K is te number of iterations in a loop. In practice, te algoritm runs surprisingly fast. Tis migt be due to te acceptable complexity of practical applications. IV. PERFORMACE EVALUATIO Our proposed metod is applied to five bencmark programs taken from te Hig-level Syntesis Design Repository ((HLSynt95/memory) [1] listed in Table I. All te programs involve array accesses inside nested loops. Performance of our metod is measured in terms of 1) te accuracy of te measurements. Tis ensures tat te optimal unrolling vector is found correctly. 2) te percentage of temporal reuses exploited. A. Accuracy Accuracy refers to te reliability of te measurements for an unrolling vector. Measurements provide an estimation of te practical situation. Accurate measurements sould be close enoug to te practical situation during unroll-and-jam. To see ow close our measurements are to te practical situation, we compare te estimated access ratio R wit te practical access ratio R from te execution of te transformed program in a simulator [4]. Te estimation is restricted by te number of available registers. For te sake of evaluation, te number of registers is arbitrarily set to 10. However, te principle can be applied to oter values depending on te number of registers in a particular processor. For eac program, we follow te procedure in section III to perform loop intercange (if any), unroll-and-jam and scalar replacement. For te unroll-andjam, te program is transformed for eac possible unrolling vector. By executing te transformed program in te simulator, we ave te practical access ratio. Experiments sow tat estimated access ratio matces te practical access ratio wit a ig accuracy. For example, Table II sows te experimental results of te bencmark Compression. Tis proves tat te measurements in section III-C accurately estimate te effect of te transforms. B. Exploitation Rate Te effectiveness and flexibility of te procedure are now examined. Tis is done by computing te percentage of temporal reuse TABLE II COMPARISO OF ACCESS RATIOS FOR COMPRESSIO UDER 10-REGISTER RESTRICTIO [v1 v2] Estimated R Practical R [2 2] [2 3] [3 2] TABLE III EXPLOITATIO RATE UDER 10-REGISTER RESTRICTIO Prog. Opt. v Reg. Used R ideal R P rac Expl. Rate (%) Comp. [2 3] Lapl. [1 3] LowP. [1 3] SOR [2 3] Wave exploited in a program using (2). ExploitationRate = 1 Rprac 1 R ideal 100% (2) were R prac is te practical memory access ratio after and before te optimization. R ideal is te ideal memory access ratio. Te ideal case appens wen tere are sufficient registers to store all te data aving temporal reuses. Te ideal access ratio is inerent to computations and does not depend on te way a program written. Evaluation is performed on te bencmarks under te 10-register restriction. Te results are sown in Table III. Te second column in te table is te optimal unrolling vector selected by te procedure. Te tird column is te number of registers actually used during te optimization. Te results in te last column sow tat te exploitation rate varies from 49% to 87% for different bencmarks. Tis is because te exploitation rate is determined by te data dependences in a particular program and te number of available registers. Low exploitation rate can be due to data dependences preventing transformations and/or limited number of registers. Te last two columns in Table III sow tat significant improvements to data intensive applications ave been acieved by our metod. V. COCLUSIO An automated metod for reducing te number of memory access by transforming te source code of a data intensive program is proposed in tis paper. Tis metod as te advantage tat te exploration space is broadened compared wit previous works by considering temporal reuse generated by multi-index subscripts. Second, tis approac performs a sequence of ig-level compiler tecniques to exploit maximum amount of temporal reuses under register restriction. Tird, te measurements designed in our approac give an accurate estimation of te practical effect. Experimental results on a number of bencmarks confirm tat significant improvements can be acieved for data intensive programs using our metod. REFERECES [1] 1995 ig-level syntesis design repository. ttp://ftp.ics.uci.edu/pub/lsynt/hlsynt95/, [2] J. R. Allen and K. Kennedy. Automatic loop intercange. In Proceedings of te ACM SIGPLA 84 Symposium on Compiler Construction, pages ACM, June [3] U. Banerjee. Loop Transformations for Restructuring Compilers: Te Foundations. Kluwer Academic Publisers, Boston/Dordrect/London, 1993.

5 [4] D. Burger and T. M. Austin. Te simplescalar tool set, version 2.0. June [5] D. Callaan, S. Carr, and K. Kennedy. Improving register allocation for subscripted variables. In Proceedings of te ACM SIGPLA 1990 conference on Programming language design and implementation, pages ACM, June [6] S. Carr and Y. Guan. Unroll-and-jam using uniformly generated sets. In Proceedings of te 30t annual ACM/IEEE international symposium on Microarcitecture, pages , December [7] S. Carr and K. Kennedy. Improving te ratio of memory operations to floating-point operations in loops. ACM Transactions on Programming Languages and Systems, 16(6): , ovember [8] F. Cattoor, K. Danckaert, S. Wuytack, and. Dutt. Code transformations for data transfer and storage exploration preprocessing in multimedia processors. IEEE Journal on Design and Test of Computers, 18(3):70 82, May-June [9] F. Cattoor, S. Wuytack, E. D. Greef, F. Balasa, L. actergaele, and A. Vandecappelle. Custom Memory Management Metodology: Exploration of Memory Organization for Embedded Multimedia System Design. Kluwer Academic Publisers, Boston/Dordrect/London, [10] R. Gonzales and M. Horowitz. Energy dissipation in general-purpose microprocessors. IEEE Journal of Solid-state Circuit, SC-31(9): , September [11] M. Kandemir,. Vijaykrisnan, M. J. Irwin, and W. Ye. Influence of compiler optimizations on system power. In Proceedings of te 37t Design Automation Conference, pages ACM, June [12] A. Koseki, H. Komastu, and Y. Fukazawa. A metod for estimating optimal unrolling times for nested loops. Information Processing Society of Japan Journal, 37(06): , [13] V. Sarkar. Optimized unrolling of nested loops. In Proceedings of te 14t international conference on Supercomputing, pages ACM, June [14] V. Tiwari, S. Malik, and A. Wolfe. Power analysis of embedded software: a first step towards software power minimization. In Proceedings of te IEEE Conference on Computer Aided Design, pages IEEE, ovember [15] M. E. Wolf and M. S. Lam. A data locality optimizing algoritm. In Proceedings of te ACM SIGPLA 91 Conference on Programming Language Design and Implementation, pages 30 44, June 1991.