System-Level Parallelism and Throughput Optimization in Designing Reconfigurable Computing Applications

Transcription

1 System-Level Parallelism and hroughput Optimization in Designing Reonfigurable Computing Appliations Esam El-Araby 1, Mohamed aher 1, Kris Gaj 2, arek El-Ghazawi 1, David Caliga 3, and Nikitas Alexandridis 1 1 he George Washington University, 2 George Mason University, 3 SRC Computers {esam, mtaher}@gwu.edu, kgaj@gmu.edu,{tarek, alexan}@gwu.edu, aliga@sromp.om Abstrat Reonfigurable Computers (RCs) an leverage the synergism between onventional proessors and FPGAs to provide low-level hardware funtionality at the same level of programmability as general-purpose omputers. In a large lass of appliations, the total I/O time is omparable or even greater than the omputations time. As a result, the rate of the DMA transfer between the miroproessor memory and the on-board memory of the FPGA-based proessor beomes the performane bottlenek. In this paper, we perform a theoretial and experimental study of this speifi performane limitation. he mathematial formulation of the problem has been experimentally verified on the state-of-the art reonfigurable platform, SRC-6E. We demonstrate and quantify the possible solution to this problem that exploits the system-level parallelism within reonfigurable mahines. 1. Introdution Reonfigurable Computers ombine the flexibility of traditional miroproessors with the power of Field Programmable Gate Arrays (FPGAs). he programming model is aimed at separating programmers from the details of the hardware desription, and allowing them to fous on an implemented funtion. his approah allows the use of software programmers and mathematiians in the development of the ode, and substantially dereases the time to the solution. he SRC-6E Reonfigurable Computer is one example of this ategory of hybrid omputers [1]. In this paper we will disuss the existing limitations on the performane of reonfigurable omputers, and propose an optimization tehnique that improves this performane. Our experimental results onfirm the effiieny of the proposed solution. 2. SRC-6E Reonfigurable Computer 2.1. Hardware Arhiteture SRC-6E platform onsists of two general-purpose miroproessor boards and one MAP reonfigurable proessor board. Eah miroproessor board is based on two 1 GHz Pentium 3 miroproessors. he SRC MAP board onsists of two MAP reonfigurable proessors. Overall, the SRC-6E system provides a 1:1 miroproessor to FPGA ratio. Miroproessor boards are onneted to the MAP board through the SNAP interonnet. SNAP ard plugs into the DIMM slot on the miroproessor motherboard [1]. Hardware arhiteture of the SRC MAP proessor is shown in Fig. 1. his proessor onsists of two programmable User FPGAs, six 4 MB banks of the onboard memory (OBM), and a single Control FPGA. In the typial mode of operation, input data is first transferred through the Control FPGA from the miroproessor memory to OBM. his transfer is followed by omputations performed by the User FPGA, whih fethes input data from OBM and transfers results bak to OBM. Finally, the results are transmitted bak from OBM to miroproessor memory. Figure 1. Hardware Arhiteture of SRC-6E 2.2. Programming Model he SRC-6E has a similar ompilation proess as a onventional miroproessor-based omputing system, but needs to support additional tasks in order to produe

2 logi for the MAP reonfigurable proessor, as shown in Fig. 2. here are two types of appliation soure files to be ompiled. Soure files of the first type are ompiled targeting exeution on the Intel platform. Soure files of the seond type are ompiled targeting exeution on the MAP reonfigurable proessor. A file that ontains a program to be exeuted on the Intel proessor is ompiled using the miroproessor ompiler. All files ontaining funtions that all hardware maros and thus exeute on the MAP are ompiled by the MAP ompiler. MAP soure files ontain MAP funtions omposed of maro alls. Here, maro is defined as a piee of hardware logi designed to implement a ertain funtion. Sine users often wish to extend the built-in set of operators, the ompiler allows users to integrate their own VHDL/Verilog maros. Objet files Appliation soures μp Compiler Linker MAP Compiler.o files.o files Appliation exeutable.v files Maro soures. or.f files.vhd or.v files HDL soures Netlists Logi synthesis.ngo files Plae & Re.bin files Configuration bitstreams Figure 2. SRC Compilation Proess 3. Current Performane Limitations he total exeution time for any appliation on a reonfigurable mahine onsists of the omputations time and the total I/O time as shown in Fig. 3. In a large lass of appliations, the total I/O time is omparable or even greater than the omputations time. As a result, the rate of the DMA transfer between the miroproessor memory and the on-board memory beomes the performane bottlenek. One possible solution is the redesign of the system hardware in suh a way that it supports a higher data transfer rate. aking into aount the ost of the hardware system upgrade, this solution may not be pratial. Additionally, even with the higher data transfer rate, there might be still appliations in whih the DMA time is omparable or even longer than the omputations time. herefore, our goal has been to find a general solution to speed-up a large lass of appliations running on a reonfigurable omputer with any hanges to the system hardware. Our solution exploits the system-level parallelism within the SRC mahine, and requires only small hanges in the appliation ode. Figure 3. Exeution time with overlapping 4. he Proposed Optimization ehnique 4.1. Model Formulation he objetive of our optimization tehnique is to overlap omputations with the data transfer whih substantially redues the total exeution time. his tehnique is onstrained by both the mahine and the nature of the appliation. he mahine onstraints an be in terms of the I/O bandwidth, total number of onurrent DMA hannels, the apability of overlapping the input with the put DMA hannels, and the asymmetry between the input and put DMA hannels bandwidths. In our model, we assume a generi hypothetial mahine that has all the above mentioned onstraints (see Fig. 4). In other words, we assume asymmetri I/O transfers, non-equal number of onurrent input and put DMA hannels, and varying overlapping ability among the DMA hannels. On the other hand, the appliation fores some onstraints, depending on its nature, whih makes it diffiult to model all the possible variations. wo essential variations in this ontext are the nature of data aeptane and data proessing by the appliation. For the data aeptane, our model assumes that the appliation is periodi, i.e. data are fed into the appliation sequentially in fixed-size bloks. Periodiity, in general, aommodates for the speial nature of pipelined appliations as a subset of the range of appliations it overs. For the nature of proessing, we assume onurrent proessing of multiple bloks of data as well as linear dependeny between the omputations time and the amount of data being proessed. hese assumptions are met by a large lass of appliations, inluding enryption [2, 3, 4], ompression, and seleted image and data proessing algorithms [5, 6, 7]. he details of the presented tehnique are illustrated in Fig. 5. Both the DMA-IN and DMA-OU transfers are divided into a sequene of n data transfers eah. Eah of these transfer parels is further divided into a number of onurrent transfer parels equal to the number of the DMA hannels available in eah diretion. he omputation period has been divided into a number of partial omputation periods spanning the time interval between the end of the first DMA-IN transfer and the beginning of the last DMA-OU transfer. he first and the last data parels are speial, as no omputations an be performed in parallel with these data transfers.

3 Figure 4. Model arhiteture a) Non-overlapped DMA hannels (V=0) b) Overlapped DMA hannels (V=1) Figure 5. Overlapping Computations with Data ransfers 4.2. Analysis he following notation will be used in our mathematial model: n DMA-IN is the number of input data parels n COMP is the number of partial omputations n DMA-OU is the number of put data parels K DMA-IN is the input transfer onurreny (multipliity) fator, i.e. the number of onurrent input hannels K DMA-OU is the put transfer onurreny (multipliity) fator, i.e. the number of onurrent put hannels K DMA is the total DMA onurreny (multipliity) fator K DMA = K DMA-IN K DMA-OU (1) K C is the omputations onurreny (multipliity) fator, i.e. the number of onurrent proessing units, it is also the number of independent data hannels between the OBM and the omputations on the FPGA in either diretion (e.g. number of OBM memory banks) B DMA-IN is the bandwidth for the input data transfer from the miroproessor memory to the OBM per single DMA hannel B COMP-IN is the bandwidth for the input data transfer between the OBM and a single omputational unit B COMP-OU is the bandwidth for the put data transfer from a single omputational unit to the OBM B DMA-OU is the bandwidth for the put data transfer from the OBM to the miroproessor memory per single DMA hannel D BLOCK-IN is the data blok size for eah of the onurrent input parels D BLOCK-COMP is the data blok size for eah of the onurrent omputations D BLOCK-OU is the data blok size for eah of the onurrent put parels D DMA-IN is the total data size for the input transfer D DMA IN = ndma IN D BLOCK IN K (2) DMA IN

4 D COMP-IN is the total input data size for the omputations and it is equal to the total data transferred in by the DMA DCOMP IN = ncomp D BLOCK COMP K (3) C DCOMP IN = D (4) DMA IN D COMP-OU is the total put data size from the omputations and it is equal to the total data to be transferred by the DMA DCOMP OU = β D (5) COMP IN DCOMP OU = D (6) DMA OU β is the data prodution-onsumption fator; i.e. β>1 for data-produing appliations, and β<1 for data-onsuming appliations D DMA-OU is the total data size for the put transfer DDMA OU = ndma OU DBLOCK OU K (7) DMA OU DMA-IN is the single-hannel DMA transfer time from the miroproessor memory to the on-board memory DDMA IN DMA IN = (8) B DMA IN DMA-OU is the single-hannel DMA transfer time from the on-board memory to the miroproessor memory DDMA OU DMA OU = (9) B DMA OU DMA is the single-hannel total DMA transfer time DMA = DMA-IN DMA-OU (10) COMP is the total omputations time for the ase of no-overlapping NoOverlap is the total exeution time for the ase of no-overlapping Overlap is the total exeution time for the ase of overlapping V is the DMA hannel-overlapping fator; i.e. V=0 for no overlapping between input and put DMA transfers, V=1 for maximum overlapping between input and put DMA transfers (see Fig. 5) We also introdue the following notation for the ratios of respetive times. DMA IN in =, DMA OU =, COMP = (11) DMA DMA DMA Equations (1) to (9) show the soures of asymmetry in DMA transfer times. Asymmetry an be aused by differene in the number of hannels, K DMA-IN, K DMA-OU, and in bandwidths, B DMA-IN, B DMA-OU, between the transfers, whih are mahine onstraints. It an also be aused by transferring different data sizes in eah diretion depending on whether the appliation being either data-produing (D DMA-OU >D DMA-IN ; i.e. β>1) or data-onsuming (D DMA-OU <D DMA-IN ; i.e. β<1). In addition to these fators, asymmetry an be aused by differene in the number of input parels, n DMA-IN, and the number of put parels, n DMA-OU. o limit the asymmetry onditions to those fators whih are only fored by the mahine and/or appliation onstraints, not by the proposed tehnique itself, we an deliberately selet the number of transfer parels in both diretions to be equal; i.e. n DMA-IN =n DMA-OU = n. In general, asymmetry an be aused by either one or all of the above three onstraints. he resultant effets of all these asymmetry onstraints are olletively modeled in equations (2), (7), (8), (9), (10), and (11). As a oneptual representation [8] of the model, Fig. 5 suggests some bounds on the number of input hannels, K DMA-IN, and the number of put hannels, K DMA-OU, when related to the number of onurrent proessing units, K C. he existene of OBM, whih is very ommon in almost all types of RCs, serves as a buffering mehanism whih relaxes any bounding limits on the relation between K DMA-OU and K C. In other words, these two fators an boundlessly be hanged independently. On the other hand, there an be a lower bound for the relation between K DMA-IN and K C. his lower bound is fored by the fat that the first blok of transferred data should be at least the minimum amount of data neessary to start the proessing; i.e. (K DMA-IN * D BLOCK-IN ) (K C *D BLOCK-COMP ). Equations derived to assess the effetiveness of the proposed overlapping tehnique are grouped together in able 1. Based on Figs. 3 and 5, equations (12) and (13) have been derived to desribe the total exeution time for both ases of no-overlapping and overlapping. o evaluate the effetiveness of the tehnique, the speedup in the total exeution time, S, is defined by equation (14). Based on equations (12), (13), and (14), equation (15) gives a simplified formula for S for the different values of the ratio. he upper limit on the speedup, and the asymptoti behavior of this limit, for both ases of nonoverlapped DMA hannels, V=0, and maximally overlapped DMA hannels, V=1, are given in equation (16) under the onditions of symmetri DMA transfers. In Fig. 6, the asymptoti dependene between the speedup, S, and, is plotted for different values of the system parameters, K DMA-IN, K DMA-OU, K C, and V. Based on equation (15) and Fig. 6, our tehnique, for a given K C, gives the best results for the ase of in =, K DMA-IN =

5 K DMA-OU, i.e., symmetri data transfer where data transfer-in and data transfer- take the same amount of time. If this is not the ase, the speedup, S, shifts downward, shaded areas in Fig. 6, from the peak values when varies between Cmin and Cmax, where Cmin and Cmax are defined in able 1, equation (15). In other words, when the DMA transfer-in time differs from the DMA transfer- time, the maximum performane degrades from the peak value, i.e. the DMA asymmetry introdues some speedup losses. An asymmetry between the DMA-IN throughput and DMA-OU throughput exists in the urrent version of the SRC-6E system. he ase of min = in =0.4 and max = =0.6, shown in Fig. 6, orresponds to the experimentally measured differene between the DMA- IN and DMA-OU times for the SRC system. From Fig. 6 and equation (16), it an be seen that the hange in the asymptoti maximum in speedup, S max, is in diret proportion to the hange in the number of hannels, K DMA, while the relative hange in its loation shifts left to less (faster omputations), i.e. the omputation for whih this maximum is ahieved, is in inverse proportion to the hange in the number of hannels. It an also be seen that as the level of DMA hannel-overlapping, V, inreases, the effet of the number of hannels on the speedup inreases, and the speedup loss, the shaded areas in Fig. 6, inreases. In addition to this, the hange in the asymptoti maximum in able 1. Equations desribing the performane of the proposed tehnique = = (1 ) NoOverlap DMA IN COMP DMA OU DMA Overlap [ n 1 V ( n 2)] Max(, ) Max(, ) V ( n 2) Max(,, ) nk K[2( n 1) V( n 2)] K[2( n 1) V( n 2)] nkdma OU nkdma IN K[2( n 1) V( n 2)] nk DMA IN nk DMA IN DMA OU DMA IN DMA IN DMA OU DMA IN DMA OU DMA OU = nk DMA OU (12) (13) S = NoOverlap Overlap (14) S n(1 ) n [ n V ( n 2)] kmin kmax K...0 kmax kmin n(1 ) =... min nn [ 1 V( n 2)] n kmin kmax K[2( n 1) V( n 2)] n(1 )... max n ( ) S max max min where 1 K KDMA... V = 0 2 = lim S = n = max K KDMA... V = 1 K[2( n 1) V( n 2)] K[2( n 1) V( n 2)] in min = k = Min(, n min n KDMA IN KDMA ) OU K[2( n 1) V( n 2)] K[2( n 1) V( n 2)] in max = k = Max(, n max n KDMA IN KDMA ) OU and max min 2K... V = 0 KDMA = = K... V = 1 KDMA (15) (16) where 1 K DMA IN = KDMA OU = KDMA 2 1 min = Min( in, ) = max = Max( in, ) = 2 Symmetri DMA ransfers Figure 6. heoretial asymptoti speedup (n )

6 Figure 7. heoretial asymptoti speedup for different K (SRC-6E ase) speedup, S max, is in diret proportion to the hange in the number of onurrent proessing units, K C, and the shift in its loation, right shift to larger (slower omputations), is also diretly proportional to the hange in K C SRC-6E Case Study Model Parameters SRC-6E reonfigurable omputer has been used as our testbed to verify our model. o apply our model to SRC-6E system we set the model parameters with some experimentally measured values, and others from the mahine speifiations. he mahine parameters are set to the following values: in =0.4, =0.6 (asymmetri DMA) K DMA-IN = K DMA-OU =1 (single-hannel DMA) V=0 (non-overlapped DMA hannels) Fig. 7 shows a plot of the asymptoti speedup against C for different K C. his figure shows the effet on speedup of the DMA asymmetry, as well as the effet of omputations onurreny, i.e. the number of onurrent proessing units and/or the number of independent data hannels between the OBM and the FPGA (see Fig. 4). he peak speedup, S max, an be alulated from the following equation: 1 1 S = K = K = K 0.83 (17) max 2 2 max In the experimental verifiation of this model we investigated only appliations with the parameter K C set to he Design Problem he design problem an be stated as follows: given the mahine onstraints and the appliation onstraints, what is the minimum number of transfer parels that ahieves a speedup as lose as possible to the asymptoti maximum for that appliation? In other words, given in,, K DMA- IN, K DMA-OU,V, and K C, we are trying to find the minimum n, n min, that gives speedup S very lose to S with an a) b) Figure 8. he design regions for different n (SRC-6E ase)

7 able 2. Design values for the minimum number of transfer parels, n min effiieny E near to 1, where S is the asymptoti value of S for this speifi appliation, and E is the ratio between S and S. From Fig. 8 the design problem an be broken down into two ases, namely when 2 min > max and when 2 min < max. able 2 serves as a guideline to finding the required n min. 5. Experimental Results he experimental work has been performed, as mentioned earlier, on the SRC-6E. In our experiments, we seleted a ertain value of, and then repeated the experiment multiple number of times with the different number of transfer parels, n. We started with n=1 (no overlap, speedup = 1), then n = 2, 4, 8, 16, 32. hen, we repeated the experiments for different values of. he results of experiments are summarized in Fig. 9. All urves, for any value of, start with the unity speedup when n = 1 (no overlap ase), then as n inreases the speedup inreases. After the speifi number of stages the speedup starts to saturate. In our experiments, we have obtained the maximum speedup when was equal to one and n was equal to 16. he speedup obtained for these parameters was equal to 1.78, and was onsistent with our theoretial preditions as in equation (17) for the ase of min = 0.4, max = 0.6 and K C = 1. We also onfirmed experimentally that the maximum performane ould be aomplished when was lose to one. When was larger or lower than one, the speedup deteriorated. Figue 9. Experimental values of speedup

8 For the ase of larger than one, the only gain in speedup is to hide the DMA time within the omputations time. When the DMA transfer is very short relative to the omputations time, the gain will also be very small. Similarly, when is smaller than one, the gain is to hide the omputations time within the DMA time. So, the idea is always to hide the shorter time within the longer time, and when both times are lose to eah other, we an obtain a speedup lose to Conlusions In this paper, a tehnique for optimizing the performane of a reonfigurable omputer is introdued. A mathematial model for this tehnique has been derived for a generi reonfigurable mahine, taking into aount the onstraints imposed by both the system and the appliation. his tehnique depends on overlapping the omputations on the User FPGAs with the I/O transfer. his overlapping requires dividing data transfers into multiple transfer parels that an be overlapped with partial omputations. he presented tehnique has been implemented and experimentally verified on the SRC-6E reonfigurable omputer. Both theoretial analysis and experimental results proved that this tehnique is effiient in speeding up the exeution time. he maximum theoretial speedup was shown to be 2 for an appliation with one proessing unit and a system with a single DMA hannel perfetly balaned for DMA-IN and DMA-OU transfers. For the urrent generation of the SRC system, the theoretial maximum speedup was shown to be 1.83, and the orresponding experimental maximum speed-up was Referenes [1] SRC-6E C-Programming Environment Guide, SRC Computers, In [2] Fidani O. D., Diab H., El-Ghazawi., Gaj K., and Alexandridis N., Implementation rade-offs of riple DES in the SRC-6e Reonfigurable Computing Environment, Pro. MAPLD [3] Fidani O.D., Poznanovi D., Gaj K., El-Ghazawi., and Alexandridis N., Performane and Overhead in a Hybrid Reonfigurable Computer, Reonfigurable Arhitetures Workshop, RAW 2002, Pro. International Parallel and Distributed Proessing Symposium (IPDPS) Workshops 2003, Nie, Frane, April 22-26, 2003, pp [4] Mihalski A., Gaj K., El-Ghazawi., An Implementation Comparison of an IDEA Enryption Cryptosystem on wo General-Purpose Reonfigurable Computers, Pro. FPL 2003, Lisbon, Sept. 2003, pp [5] Parhi K.K., VLSI Digital Signal Proessing Systems: Design and Implementation, John Wiley & Sons, NY, [6] El-Ghazawi. and Le Moigne J., Multiresolution Wavelet Deomposition on the MasPar Massively Parallel System, International Journal of Computers and heir Appliations, September [7] Mallat S.G., A heory for Multiresolution Signal Deomposition: he Wavelet Representation, IEEE ransations on Pattern Analysis and Mahine Intelligene, Vol. 11, No. 7, July [8] Hwang K. and u Z., Salable Parallel Computing: ehnology, Arhiteture, Programming, MGrawHill, 1998.