u Delay Delay x p Data, u x p1 Encoder 1 Puncturer Interleaver p2 p2 Encoder 2

Transcription

1 Implementaton of Near Shannon Lmt Error-Correctng Codes Usng Recongurable Hardware Benjamn Levne, R. Reed Taylor, and Herman Schmt Abstract Error correctng codes (ECCs) are wdely used n dgtal communcatons. Recently, new types of ECCs have been proposed whch permt error-free data transmsson over nosy channels at rates whch approach the Shannon capacty. For wreless communcaton, these new codes allow more data to be carred n the same spectrum, lower transmsson power, and hgher data securty and compresson. One new type of ECC, referred to as \Turbo Codes," has receved a lot of attenton, but s computatonally expensve to decode and dcult to realze n hardware. Low Densty Party Check Codes (LDPCs), another ECC, also provde near Shannon lmt error correcton ablty. However, LDPCs use a decodng scheme whch s much more amenable to hardware mplementaton. Ths paper wll rst present an overvew of these codng schemes, then dscuss the ssues nvolved n buldng an LDPC decoder usng recongurable hardware. We present a hypothetcal LDPC mplementaton usng a commercal FPGA, whch wll gve an dea of future research ssues and performance gans. Keywords turbo codes, error correctng codes, lowdensty party check codes I. Introducton Error correctng codes (ECCs) are wdely used n dgtal communcaton systems. These codes allow eectvely error-free communcatons to occur over nosy channels by encodng the data to be transmtted and usng decodng algorthms to detect and correct errors n the receved data. The encodng process adds redundant nformaton to the data beng transmtted. It s ths redundant nformaton that allows for the detecton and correcton of errors. Some amount of computaton s requred to perform ths error detecton and correcton. As the sgnal to nose rato (SNR) of the transmsson channel decreases, error correcton becomes more dcult and may requre the use of derent types of ECCs. The ablty to transmt and receve data over channels wth low SNRs has many techncal and economc advantages. If sgnal strengths can be kept low, more wreless devces can share the same frequency spectrum wth lmted nter-devce nterference. Lower sgnal strengths also reduce power requrements, an mportant consderaton for wreless devces, whch are often powered by batteres. Beng able to acheve low error rates over nosy channels allows the transmsson of compressed and encrypted data, whch havealow tolerance for errors, as even a sngle error can render the data unusable and requre the The authors are wth the Department of Electrcal and Computer Engneerng at Carnege Mellon Unversty, 5 Forbes Avenue, Pttsburgh, PA, E-mal addresses are fblevne, rt2, hermang@ece.cmu.edu re-transmsson of the aected data. Better ECCs allow transmttng data at hgher data rates as well as lower error rates. Hgher data rates allow the delvery of multmeda content and other dense forms of data to portable devces. Codng schemes that are currently n wde used do not haveasmuch error correcton performance as would be desrable. The mplementaton of new ECCs wth better propertes wll provde many benets and wll enable the development of new wreless devces and applcatons. Shannon [1] showed that there s a lmt to the rate at whch data can be sent through a channel wth a gven SNR. Ths rate s called the channel capacty. By usng a sucently sophstcated codng scheme, t s theoretcally possble to transmt nformaton at any rate less than or equal to the channel capacty wth an arbtrarly small probablty that the receved data wll be decoded ncorrectly. Communcaton systems desgners would lke codes that allow data transmsson at rates as close as possble to the channel capacty whle keepng the error probablty as low as necessary. ECCs that allow transmsson of data at rates near the channel capacty wth low probablty of error are often referred to as \near Shannon lmt codes." Untl recently, there have been no known near Shannon lmt ECCs wth practcal decodng algorthms. In the past several years, two general classes of practcal near Shannon lmt ECCs have been been developed. These are turbo codes and low-densty party check codes. These codes requre computatonally ntensve decodng algorthms, but they allow transmsson of data at low SNRs and hgh data rates, wth error rates much lower than those aorded by other ECCs currently n use. Whle the ECCs used n wreless devces are usually mplemented n VLSI, and the ecent mplementaton of near Shannon lmt codes n VLSI s an ongong research topc (see [2], for example), there are several reasons to explore the mplementaton of these codes usng recon- gurable hardware. These nclude algorthm desgn and testng, system prototypng and development, compatblty wth dverse and evolvng communcatons standards, and adaptve codng to match changng data and channel characterstcs. There s stll substantal current research nto determnng the most eectve ECCs. Much of ths researchnvolves emprcal testng of varous algorthm parameters and must be done by smulaton. Smulatng ECCs can be very tmeconsumng n software, even wth fast workstatons. Recon- gurable hardware could provde a way to speed up these

2 smulatons and would also allow the testng of these algorthms wth real data at system data rates. Once tested, the algorthms could be mplemented usng the same recon- gurable hardware for prototypes and low-volume systems. Communcatons standards for wreless devces are stll evolvng, and near Shannon lmt ECCs are not yet n wde use n these standards. By usng recongurable hardware, wreless devces would have the exblty to comply wth new standards as they are developed and to comply wth standards n use n derent geographc locatons. Even when the use of these codes s well understood, and standards are well establshed, there wll stll be potental applcatons for recongurable hardware. One these s mplementng adaptve error correcton. A wreless devce could be desgned to adaptvely swtch between derent ECCs dependng on the sgnal envronment, the data beng transmtted, power requrements, and other changng parameters. For nstance, when transmttng and recevng voce data to and from another portable devce nearby, a smple code wth low computaton requrements would be sucent, but f t was then necessary to transmt an encrypted le to a more dstant devce, a more complex code wth much lower error rates could be used. The purpose of ths paper s threefold. Frst, we wll ntroduce near Shannon lmt ECCs n the context of ther mplementaton n hardware. Turbo codes wll be dscussed n Secton II and LDPCs wll be dscussed n Secton III. Secondly, n Secton III.C, we wll show reasons why LD- PCs may be more amenable to hardware mplementaton than turbo codes. Thrdly, n Secton IV we wll ntroduce a possble mplementaton of an LDPC decoder as a startng place for further exploraton. Secton V presents conclusons and future work. II. Turbo Codes Turbo codes were ntroduced n 1993[3]. The term \turbo code" refers to ECCs that perform teratve, probablstc decodng of data encoded usng multple, concatenated encoders. A. Encodng Turbo codes encode data by combnng two or more encoders, ether n parallel or seral, as well as a number of data nterleavers. A typcal turbo encoder s shown n Fgure 2. Ths parallel concatenated turbo encoder uses two encoders, an nterleaver, and a puncturer. The encoders are typcally recursve systematc convolutonal (RSC) encoders. A smple four state RSC encoder s shown n Fgure 1. RSC encoders wth more states are usually used n real systems, wth 16 states beng common. These encoders are smple to mplement, as they consst of only a relatvely small number of regsters and modulo-2 adders. The two encoders used n a turbo encoder of ths type are usually, but not necessarly, dentcal. The nterleaver takes the nput data and reorders t n a xed, repeatable, pseudo-random manner, such that adjacent bt pars n the orgnal data are as dstant as possble n the nterleaved data. Assumng RSC encoders, each en- Fg. 1. u Delay x p Delay Recursve systematc convolutonal encoder. coder produces one output bt for each nput bt. Ths means that the turbo encoder shown would have a code rate r =1=3, producng 3 bts of encoded data for every bt of nput data. Whle codes wth low rates are sutable for many applcatons, hgher rate codes are desrable n others. When hgher rate codes are needed, a puncturer can be used, as shown. The puncturer reduces the amount of data to be transmtted by throwng out some of the party bts produced by the encoders. The decoder usually assumes that these dscarded bts were all zeros (or all ones). Snce less redundant nformaton s transmtted, more computaton s necessary to decode punctured turbo codes. If our nput data u = (u 1 ;:::;u k ) s a bnary vector wth length k, the nterleaved data wll be another bnary vector ^u, also of length k, contanng the same data, but n a derent order. u s encoded by encoder 1, producng a bnary vector of party bts, x p1, and ^u s encoded by encoder 2, producng another bnary vector of party bts, x p2. The subsets of each party bt vector chosen by the puncturer wll be desgnated x p1 and x p2, respectvely. One of the strengths of turbo encodng s that rather than usng one encoder to generate one set of party bts, multple encoders are used to generate multple sets of party bts. The nterleaver s desgned to ncrease the dstance between bts that are adjacent n the orgnal data stream. Thus the data fed nto the second encoder generates a set of party bts very derent from the party bts produced by the rst encoder. By usng party bts from both encoders, small amounts of redundancy provde a great deal of error correctng capablty. Data, u Interleaver B. Decodng Fg. 2. u Encoder 1 Encoder 2 x p1 x Puncturer x p2 p2 Parallel concatenated turbo encoder. A farly complex decoder s needed to decode the turbo encoded transmtted data. The orgnal data (also called x p1 2

3 the systematc data) and the party bts from the encoder are sent over the communcatons channel and receved corrupted by nose. The receved verson of the systematc data, u, s desgnated y s, and the receved versons of the punctured party bts are desgnated y p1 and y p2. The receved party vectors are usually padded wth zeros n those places where party bts were thrown away by the puncturer, to produce vectors that are agan k values long. The padded versons of the receved party data wll be desgnated y p1 and y p2, respectvely. A turbo decoder for the encoder n Fgure 2 s shown n Fgure 3. It conssts of two decoders, one correspondng to each of the orgnal encoders, two nterleavers, and a denterleaver. The de-nterleaver smply rearranges the data n the nverse manner to the nterleaver, such that data passed through the nterleaver and then the de-nterleaver (or vce-versa) would have ts orgnal order restored. Each decoder uses the receved systematc data, party data, and output of the other decoder to calculate probabltes about the contents of the orgnal data. So ntally decoder 1 uses y p1 and y s to determne a set of probabltes for the values of the orgnal data. Then decoder 2 uses y p2, y s, and the output of decoder 1 to rene these probabltes. These rened probabltes are passed back to decoder 1, whch further renes them. The probabltes passed from one decoder to the next are referred to as extrnsc probabltes and are ndcated as L e1 and L e2 n Fgure 3. Ths teratve process repeats untl ether some stoppng crtera are met, or more usually, after a xed number of teratons. y y s p2 y p1 MAP Decoder 1 Fg. 3. L e1 Interleaver Interleaver De-nterleaver MAP Decoder 2 Parallel concatenated turbo decoder. The decoders used can be of any type that can decode the codes produced by the correspondng encoders. Usually a moded form of the BCJR algorthm [5] s used to decode the RSC encoders typcally employed. Ths algorthm nds, the bt value gvng the maxmum a posteror probablty for each bt; that s, the probabltes gven the observaton of the receved data. Decoders usng ths algorthm are often referred to as MAP decoders. Note that MAP decoders do not have to be used n turbo decoders; other types of decodng algorthms, such as the soft-output Vterb algorthm (SOVA) [9] can also be used. However, MAP s a provably deal decoder and thus gves the best performance. The basc dea of the MAP decoder s to determne the probabltes of values of bts n the orgnal data, based on the observed values of the receved L e2 data and to choose the value wth the maxmum probablty. The probablty that the th bt n the orgnal data, u was a zero, based on the observed data, s wrtten as P = P (u =jy s ;y p1 ;y p2 ) and the probablty that ths bt was a one s wrtten as P 1 = P (u =1jy s ;y p1 ;y p2 ). Note that the probabltes for each bt are calculated usng all of the receved data, not just the data correspondng to the partcular bt. Determnaton of the decoded bt s straghtforward; f P 1 P the th decoded bt s set to be one, otherwse P 1 <P and the th decoded bt s set to be zero. Rather than keep track of the two probabltes, a log lkelhood rato of the probabltes, L(u ) s usually used, where L(u ) s dened as follows: P 1 L(u ) = log (1) P The decson functon s then to set the th decoded bt to one f L(u ) and to set the th decoded bt to zero f L(u ) <. The extrnsc values L e1 and L e2 are log lkelhood ratos. The notaton L e1 refers to the extrnsc value for L e (u ) from decoder 1, and smlarly for L e1. The MAP algorthm speces how these posteror probabltes are to be determned, whch s the core of the problem. A gven RSC encoder has an assocated trells dagram, whch detals the derent states that the encoder can be n, possble transtons between states, and the nputs and outputs assocated wth any gven transton between states. The trells dagram(s) for the encoders used must be known n order to determne these probabltes. Fgure 4 shows a trells dagram correspondng to the RSC encoder shown n Fgure 1. The state that the encoder starts n s desgnated s 1, the next state s desgnated s 2, and so on through the last state, s k+1. The encoder starts n State, so s 1 =. If the rst encoder nput, u 1 =, the encoder stays n State, meanng s 2 =, and outputs a party bt x p 1 =, as ndcated n the trells dagram. If u 1 =1,then encoder changes to State 1, so s 2 = 1, and x p 1 = 1. In ths way the trells dagram encapsulates the behavor of the RSC for all possble nputs. State State 1 State 2 State 3 s 1 s s s s s u = u = Fg. 4. Trells dagram for four state RSC encoder. (After Fg. 6.7 n [4]). It can be shown (see [5],[6]) that the maxmum a posteror probabltes for ths decodng problem can be found from (2) and (3). The dervatons of these equatons can 3

4 be found n the cted references and wll not be dscussed here. P = X S (s )(s ;s +1 )(s +1 ) (2) P 1 = X S 1 (s )(s ;s +1 )(s +1 ) (3) The summaton n (2) s over all encoder state transtons caused by an nput data bt wth value and smlarly the summaton n (2) s over all encoder state transtons caused by an nput data bt wth value 1. Each transton conssts of ntal state s and a nal state s +1. From (1), (2), and (3), we can compute the log lkelhood rato for the a posteror data bt values as: P P (s )(s ;s +1 )(s +1 ) S L(u ) = log 1 C A (4) (s )(s ;s +1 )(s +1 ) S. The (s ;s +1 ) term s the probablty of a state transton from state s to state s +1, gven that the encoder was n state s when bt u was encoded and gven the receved data bt y s and the correspondng receved party bt y p (Ths last s y p1 for decoder 1 and y p2 for decoder 2). (s ;s +1 ) s referred to as the branch metrc. The branch metrc s used n (2) and (3), and s also needed for calculatng the forward state metrcs (s ) and the reverse state metrcs (s +1 ), as wll be descrbed below. The branch metrc can be computed as: (s ;s +1 )=P (s +1 js )P (y s js ;s +1 ) (5). There are two components to ths calculaton, P (s +1 js ) and P (y s js ;s +1 ). The rst s the probablty that we end up n state s +1 gven that we are n state s. Snce we know the encoder trells, ths probablty s ether P or P 1, dependng on whether a data bt u wth value zero or one produces the state transton n queston. These are the values we are tryng to determne, so for the purposes of the the branch metrc calculaton, we use the extrnsc probablty, L e (u ), from the other decoder. Snce ths s a log lkelhood rato, we must convert to the desred probablty wth (6). We know u n ths case, snce t s the data bt value necessary to create the state transton we are evaluatng. A s a constant that wll appear n the numerator and denomnator of (4) and wll therefore cancel, so we wll gnore t. (2u, 1)L e (u ) P (s +1 js )=A exp (6) 2 The second component tothe calculaton s the probablty that we receve a bt y s, gven that the encoder made a transton from state s to state s +1. Ths depends on the encoder trells and also on the channel propertes and modulaton. If we assume the channel has addtve whte Gaussan nose wth varance 2, transmtted values of,1 1 for a bt and +1 for a 1 bt, and code rate r, the probablty can be calculated as shown n (7). Note that n addton to knowng the value u for the current transton, we also know what the correspondng party bt x p would be from our knowledge of the encoder trells. The B term s another constant that, lke A, wll cancel n (4). y s P (y s js ;s +1 )=B exp (2u, 1) + y p xp 2 We can combne (5), (6), and (7) to gve (8), where C = A B. (2u, 1)L e (u ) (s ;s +1 )=C exp 2 y s (2u, 1) + y p exp xp 2 The (s ) term n (4) represents the probablty that at the tme that data bt u was encoded, the encoder state was s, gven the subset of receved data values startng at y s 1 and endng at y s,1. (s ) can be computed recursvely from (9), where the summaton s over s,1 2 A, meanng all prevous encoder states s,1 that are connected n the trells dagram to the current state, s. (s )= X s,12a (7) (8) (s,1 )(s,1 ;s ) (9) The (s +1 ) term n (4) represents the probablty that at the tme that data bt u was encoded, the encoder state was s +1, gven the subset of receved data values startng at y s and endng at k ys. (s +1 ) can be computed recursvely n a backwards fashon (startng from the end of the receved data and progressng towards the begnnng of the receved data) from (1). The summaton s over s +1 2 B, meanng all future encoder states s +1 that are connected n the trells dagram to the current state, s. (s )= X s +12B (s +1 )(s ;s +1 ) (1) In order to decode a block of data, k branch metrcs must be computed from (8). Then the k forward state metrcs must be found from (9), workng teratvely through the data from begnnng to end. The k reverse state metrcs must also be calculated, usng (1) and workng teratvely through the data n the reverse drecton. It s mportant to note that the state metrcs for each must be computed serally, as each s dependent on the prevous value. Ths poses a problem for hardware mplementaton, as the amount of explotable parallelsm s lmted by ths aspect of the algorthm. The branch and state metrc values can then be used to calculate the posteror probabltes usng (4). These extrnsc probabltes are then passed to the other decoder, whch must then do the same calculatons. Ths consttutes one teraton of the algorthm, and there may be ten or twenty teratons before the decodng process s complete. It s clear from nspecton of the relevant 4

5 equatons that ths s a very computatonally complex algorthm. Numerous multplcatons and exponentals must be computed for each bt and the encoder trells structure must be referenced to ensure that the correct terms are used n each calculaton. There are some modcatons that can be made to ths algorthm that reduce the complexty somewhat. The so-called log-map varant takes the log of all the metrcs so that the multplcatons n (4) can be replaced by addtons. The log values of the metrcs can be determned drectly wth some manpulaton of the relevant equatons. Ths log-map algorthm s the usual choce for hardware mplementatons [2], [1], [11]. III. Low-Densty Party-Check Codes Another, related type of ECC that can exhbt near Shannon lmt performance s the low-densty party-check (LDPC) code. Lke turbo codes, they use an teratve decodng method whchnvolves the calculaton of probablstc nformaton that s passed from one teraton to another. The encodng schemes are qute derent, however. Turbo codes uses concatenated convolutonal encoders and nterleavers, whereas LDPCs use a partycheck matrx for block encodng. As wll be descrbed, the encodng process for LDPCs s a smple multplcaton of a sparse matrx and a vector. There are two man varatons of LDPCs, Gallager codes and the more recent MN codes, whch are a varaton of Gallager codes, although they were developed ndependently. We wll dscuss Gallager codes exclusvely n ths paper. The derences are mnor and are dscussed n depth n [8]. A. Encodng Gven a bnary data vector, u, havng length k, we can select a transmtted vector length n, gvng a rate k=n code. Ths means we are ntroducng m = n, k party check bts. The transmtted vector, t, s created by multplyng the source vector by a generator matrx G T, such that t = G T u mod 2. Ths generator matrx s derved from the party check matrx, A, whch dstngushes one specc LDPC from another. The party check matrx A can be created by randomly constructng an m n matrx wth exactly weght w per column (that s, there should be exactly w ones n each column) and weght w(n=m) per row. Ths type of party check matrx results n what s called a regular Gallager code; f the weght per row s not exactly w(n=m), the resultng code s called an rregular Gallager code. Irregular Gallager codes have better error correctng propertes [8] but t can be more dcult to mplement decoders for rregular codes n hardware, as wll be dscussed below. There are other propertes of party matrces that aect ther performance, but they do not aect the decodng process and wll not be addressed here. We can use Gaussan elmnaton and reorderng of the columns of A to produce an equvalent party check matrx H, of the form H =[PjI m ], where P s an m k matrx contanng the actual party checks and I m s the m m dentty matrx. From ths form of the partycheck matrx, we can create the generator matrx as: G T Ik = P where I k s the k k dentty matrx. The complete LDPC codng scheme s shown n Fgure 5. The source vector u s fed nto the encoder to produce the encoded vector t. The encoded vector s then sent through the channel and corrupted by nose. The corrupted verson of ths vector, as receved by the decoder, s desgnated r. The decoder uses r and knowledge of the encoder to decode the data. The operaton of the decoder wll be dscussed next. u B. Decodng T G Encoder t Fg. 5. CHANNEL r Bt Nodes j r Check Nodes j q Decoder LDPC encoder and decoder. The algorthm used to decode LDPCs s the messagepassng algorthm, also known as the sum-product algorthm. It appears qute derent from the MAP algorthm used for turbo codng, but t s theoretcally related [7]. Smlar to the algorthms used n the turbo decoder, the message-passng algorthm determnes the a posteror probabltes for bt values based on a pror nformaton, mprovng the accuracy of these calculatons wth each teraton. The decodng algorthm for LDPCs can be thought ofas a bpartte graph, where two sets of nodes perform computatons n parallel, then communcate wth each other over connectons descrbed by the edges of the graph. The messages communcated between nodes consst of estmates of probabltes. The nature of the nodes n the graph and the structure of the graph's nterconnectons are completely descrbed by the number and locaton of ones n the party check matrx A. There are two knds of nodes, check nodes and bt nodes. The check nodes determne the probablty that a party check s satsed f one partcular data bt s set to be a one (or zero) and the other data bts havevalues wth a probablty dstrbuton correspondng to the known a pror probabltes. The bt nodes determne the probablty that a data bt has the value one (or zero), gven the nformaton from all of the other checks. Only bts and checks that are related by havng a one at a specc correspondng locaton n the party check matrx need to be consdered n these calculatons. As an example, take the matrx and graph shown n Fgure 6. The square nodes along the top row of the graph represent the check nodes, of whch there s one for each e 5

6 r Fg. 6. A= Party Check Matrx Message Passng Graph A1 A2 A3 A4 A5 A6 X1 X2 X3 X4 X5 X6 X7 X8 The message-passng structure of a 6x9 LDPC. row of the matrx A. Each row represents a sngle party check. Smlarly, the round nodes at the bottom of the graph represent the bt nodes, of whch there s one for each column n A, and thus one for each transmtted bt. The locaton of the ones and zeroes n A determne whch nodes are connected n the message passng graph. Havng a one at row j, column smply ndcates that check node j s connected to bt node. Lookng at the rst row of A n Fgure 6, one can see ones n the rst, fourth, and seventh columns; ths s reected n the graph as connectons between check node A1 (represented by the rst row) and bt nodes X1, X4, and X7 (each represented by ther respectve columns). The number of ones n a row determnes the number of ports that the correspondng check node wll have, and the number of ones n a column wll determne the number of ports that the correspondng bt node wll have. In the case of regular party check matrces, the total number of ones n each row wll be equal to all other rows, and lkewse for columns. The check nodes generate r j values, where r j s the probablty that check j s satsed f t s assumed that data bt t = and where r 1 j s the probablty that check j s satsed f t s assumed that data bt t =1. These probabltes are computed as shown n (11) and (12). The notaton 2 row[j]nfg smply means the ndces (1 n) of all bts n row j (1 j m) whch have value 1, not ncludng the current bt ndex,. r j = 1 2 [1 + r 1 j = 1 2 [1, 2row[j]nfg 2row[j]nfg q X9 (q j, q1 j)] (11) (q j, q1 j)] (12) The bt nodes generate the q j values, where q j s the probablty that bt t =,gven the values of all checks other than j and q 1 j s the probablty that bt t = 1, gven the values of all checks other than j. These probabltes are computed as shown n (13) and (14). The notaton j 2 col[]nfjg means the ndces j (1 j m) of all checks n col (1 n) whch have value 1, not ncludng the current check ndex, j. j s a normalzng value chosen so that q + j q1 =1. j p and p1 represent the current estmate of the posteror probabltes for each bt. These are the extrnsc values, as dscussed below, for all teratons after the rst. For the rst teraton, they are ntalzed to values determned by the data receved from the channel. For nstance, f the channel demodulator determnes that the sgnal receved s close to that expected for a one, t would assgn a hgh p 1. It s acceptable f the channel supples bnary values for p and p 1. q j = j p q 1 j = j p 1 j 2col[]nfjg j 2col[]nfjg r j (13) r 1 j (14) In addton, the bt nodes also calculate the extrnsc probabltes, e, whch are the computed posteror probabltes of bt t havng a gven value;.e., e s the computed probablty that bt t =. These extrnsc probabltes are used to determne what the decoded values are for each bt and are used n the bt nodes equatons (13) and (14). The accuracy of these probabltes mproves wth each teraton of the algorthm. The extrnsc probablty calculatons are performed as shown n (15) and (16). The notaton j 2 col[] means the ndces j (1 j m) of all checks n col (1 n) whch have value 1. j s another normalzng value chosen so that e + e 1 =1. C. Complexty e = e 1 = j 2col[] j 2col[] r j (15) r 1 j (16) A comparson of the complexty of MAP decodng, log- MAP decodng, and LDPC decodng s shown n Table I. These numbers assume a rate r = 1=3 LDPC code and a turbo code wth wth 16 encoder states. The numbers shown are per teraton and per data bt; for nstance, f there are 4 data bts and 1 teratons are performed, then the total number of computatons requred s 4, tmes the numbers shown n the table. There are many derent ways to mplement these algorthms and derng complexty numbers wll results from each, but these values are sucent for general comparson of the type and number of operatons requred for each decoder. The message passng algorthm for decodng LDPCs s somewhat less complex n terms of operaton count than the MAP algorthm for decodng turbo codes, as shown n Table I. The check and bt node equatons are relatvely smple, and there are no logs or exponentals as there are for MAP decodng. LDPCs have other propertes that make 6

7 TABLE I Decoder complexty n operatons per bt for each teraton. Decoder Add/Sub. Mult. Bt Log Exp Dv MAP log-map LDPC them even more advantageous for mplementaton n hardware, however. The prmary advantage s that there s much less data dependency n the LDPC algorthm. The calculatons n each bt node are ndependent of all other bt nodes, and the calculatons for each check node are ndependent of all other check nodes. Thus all of the check nodes could potentally be mplemented so as to operate n parallel, as could all of the bt nodes. If one vews the graph n Fgure 6 as a data dependency graph for the LDPC decoder, t can be seen that t s qute wde and relatvely shallow, meanng that there s a great deal of potentally explotable parallelsm. The MAP algorthm requres the seral calculaton of all of the state metrcs. The only parallelsm s across the encoder states, the number of whch s relatvely small (e.g., 4 n our example encoder), compared to k, the number of seral calculatons, snce k s typcally n the hundreds or thousands. The trells dagram n 4 can be seen as an approxmaton of a porton of the data dependency graph for the MAP algorthm. If the trells dagram s extended to length k and rotated 9 degrees, the resultant data dependency graph s narrow, and very deep, meanng that there s relatvely lttle explotable parallelsm n the MAP algorthm. The LDPC algorthm has an addtonal advantage, n that the encoder propertes are contaned by the graph structure and thus no encoder nformaton has to be stored explctly, as the trells nformaton must be for MAP decoders. D. Nodes The message-passng algorthm s mplemented by constructng a network of bt nodes and check nodes. The functonalty of the nodes themselves depends only on the number of ports (or, the number of ones n the approprate row or column) and not on the locaton of the nonzero bts. Ths means that a regular party check matrx wll dene a system wth only two unque knds of nodes n t: bt nodes wth some xed number of ports, and check nodes wth some other number of ports. Ths characterstc makes ths algorthm attractve for mplementaton n hardware, snce t can be mplemented by a replcatng a number of dentcal sub-unts. Buldng a complete network of nodes s qute smple: merely replcate the approprate nodes to match the numberofrows and columns, then connect them accordng to the locaton of the ones n A. Irregular party check matrces requre sub-unts wth derng numbers of ports and thus t s somewhat more dcult to construct hardware decoders for rregular codes. Approprate use of recongurable hardware can reduce the sgncance of ths problem. Graphcal representatons of the nternal structure of the bt and check nodes are shown n Fgures 7 and 8. 1 r, r,1 r,2 q, j q 1, j q 2, j Fg. 7. Fg. 8. p Norm p 1 Norm Norm Norm Bt node (wth 3 ports) Check node j (wth 3 ports) q, q,1 q,2 e r, j r 1, j r 2, j The bt nodes requre normalzers for the values of q j and e to ensure that q j +q1 = 1 and j e +e1 =1. Each of these pars of values should always sum to 1 due to the fact that they represent a complete set of probabltes, but the calculated values wll not usually have ths property. The normalzer nds the scale factor j for (13) and (14) and the scale factor for (15) and (16). The scale factor s smply the recprocal of the sum of the probabltes; for nstance, j =1=(q + j q1 j ), where q j s the value calculated from (13) before normalzaton. The dvson requred for normalzaton can be dcult to mplement ecently n hardware. One ecent mplementaton for the normalzer s shown n Fgure 9. Ths crcut generates a correctly scaled dvdend n the module labeled Dvgen so that the quotent computed by the 8 Dvcell modules has 8 bts of 1 Note that the bt nodes depcted have three ports rather than two, meanng that they are not dentcal to the bt nodes depcted n Fgure 6. Ths was done for llustratve purposes, because bt nodes wth two or fewer ports are somewhat deprecated n that they do not requre two levels of multplcaton. 7

8 precson. The Varshft module rescales the result after the probablty smultpled by the scale factor. Only the normalzed verson of one probablty needs to be produced, snce the sum of both s known to be equal to one after normalzaton. The dvson s performed usng a dgtrecurrence algorthm [12] and the mplementaton of each Dvcell s shown n Fgure 1. Each Dvcell s dentcal. The value for W suppled to the rst Dvcell s the dvdend, and the last Q value s the quotent. R s used nternally only. Ths normalzer desgn can be easly ppelned for best performance. p p 1 + <<1 dvsor Dvgen dvdend Dvcell Dvcell Dvcell 8 x dvcells operand multplers, and 17 normalzers! Fortunately, even the very large party check matrces currently used n the eld are so sparse that they typcally have 6 or fewer ones per column. On the other hand, the hardware needed to mplement a party check matrx A scales lnearly wth the sze of A, gven a xed number of ones n every row and column. In other words, addng addtonal columns and rows to the matrx smply adds addtonal nodes to the graph; t does not ncrease the complexty of the nodes. Ths means that as long as an mplementaton of an LDPC decodng scheme ts on a devce, the desgn can be \grown" by smply addng more nodes and connectng them to ther neghbors n the graph. Bt nodes, and to a lesser extent, check nodes are qute large. On bg recongurable devces, t should be easy to t at least one of each; however, mplementng very large numbers of nodes, or nodes wth large numbers of ports, could be dcult. Fgure 11 shows how many LUTs are consumed by check nodes (of varyng bt-wdths and numbers of ports) mplemented on a Xlnx Vrtex chp Ports 4 Ports 5 Ports 6 Ports Varshft quotent LUTs normalzed p 1 Fg. 9. Normalzer Crcut Bts per Port W R Q Fg. 11. LUTs consumed by acheck node wth varous bt-wdths and port counts Dvsor Sgn Bt + <<1 1 W+1 R+1 Q+1 Fg. 1. Dvder Cell. IV. Implementaton of LDPCs The bt and check nodes do not scale gracefully as ther number of ports ncreases, prmarly as a result of the multplers and normalzers n the nodes. For nstance, a 16- port bt node would requre operand multplers, 32 2 There are many possble approaches to buldng a complete LDPC decoder on a recongurable devce whch can only t a few bt and check nodes. We wll take a close look at one partcular smple desgn, takng note of some of the ssues nvolved n ts mplementaton, and makng some estmates as to ts performance. We wll also dscuss a few other optons for desgnng such a system. A. A Two-Node System In order to better understand the nature of the ssues nvolved n mplementng an LDPC decoder, consder a nave two-node mplementaton. In ths system, we wll mplement one bt node and one check node (lke those shown n n Fgures 7 and 8) on the recongurable devce smultaneously, and then sequence the r and q values through them. The values wll be stored n memory between teratons. A system of ths knd s depcted n Fgure 12. 8

9 TABLE II Store Unt Resources consumed by LDPC Decoder Components q Memory Fetch Unt Bt Node Fetch Unt p Memory e Memory Component Resources Percentage Normalzers 1644 LUTs 54.1% Bt Node 848 LUTs 27.9% Check Node 512 LUTs 16.8% Controller 35 LUTs 1.2% Data Memory 96 Bytes 64.% Address ROM 54 Bytes 36.% Check Node Store Unt r Memory Fg. 12. Block dagram of a complete system for performng LDPC Decodng Ths mplementaton functons as follows: The check node reads groups of values out of the q memory, processes them sequentally, and wrtes the resultng values nto the r memory. Then, the bt node takes over, readng r and p values, and producng q and e values. The values themselves are represented n a xed-pont format wth the radx postoned to represent probabltes rangng from to 1. A savngs n memory sze can be acheved by only storng the r, p, q, and e values; the r 1, p 1, q 1, and e 1 quanttes can be recovered by subtractng from 1. Accessng the values n memory s somewhat complcated: n ths case there are 4 ports nto the check node, and 3nto the bt node, so each cycle the memory must be able to provde the approprate number ofvalues. For our smple mplementaton, a sngle-ported memory was used to lmt the complexty of the memory. The port was clocked repeatedly to fetch and store the three or four values one at a tme. Because the order and groupngs of the memory accesses were known beforehand, the addresses could be loaded nto a smple ROM. An nstance of the system shown n Fgure 12, desgned to run a rate 1/4 code wth a block sze of 12 on a Xlnx Vrtex chp took 339 LUTs and a total of 96 bytes of RAM, whch s small enough to t on many devces avalable today. Ths gure was acheved wthout the use of any specal multplers or memory structures. The multplers were smply those generated by the synthess tools, and the memory used was the on-chp RAM. The synthess tool and the chp speccatons ndcate that ths mplementaton would be able to complete full bt and check node calculatons at a frequency of 5 MHz. If the block sze of the desred code s 12 bts, and decodng s run for 1 full teratons before returnng, ths Vrtex mplementaton wll fully decode bts at a rate of 4 megabts per second. Snce the code rate s 1/4, user data wll be processed at 1 Mb/s. If a large devce s avalable, or f ochp RAM s used, smply doublng the amount of memory used, as descrbed above, would double ths performance. Though t s functonally correct, ths smple mplementaton has several drawbacks, one of whch s qute obvous: the bt and check nodes cannot operate smultaneously or they may overwrte data whch has not been processed by the other node. (Remember that the accesses to memory are scattered randomly through the address space.) Ths could be remeded by doublng the memory capacty, creatng two banks each for the q, r, p, and e memores. Then, the nodes could operate on two datasets smultaneously, swappng between banks so as not to nterfere wth one another. Ths smple mprovement would prevent half of the hardware from remanng dle durng executon, doublng the performance of the algorthm. There are many other mprovements whch may not be as straghtforward to acheve but whch could certanly brng gans n speed of executon. It seems that most of the problems wth the presented mplementaton are related to the way n whch values are stored n memory: the explotable parallelsm n ths desgn was completely lmted by the memory sze. Addtonal performance gans could have been acheved by ncreasng the number of memory ports so as to allowmultple bt and check nodes to operate n parallel; however, the sze and complexty of the memores was a lmtng factor. Addtonally, the ROM used to step through the addresses s qute szeable. It would be nce to smplfy the addressng technque so as to elmnate the need for the ROM. B. Related Work It s not easy to compare the performance gures gven above to those reported n other research. LDPCs are new enough that there are no publshed results avalable for LDPC mplementatons on hardware at ths tme. Even the nformaton on turbo code mplementatons s sparse; addtonally, t s not generally clear how to farly compare performance results for LDPCs wth those for turbo codes. Reported performance gures for turbo decoder mplementatons, especally commercal products, often do not nclude the code characterstcs, number of teratons, or actual supported data rates, makng vercaton and comparson of performance even more dcult. In order to accurately compare ECC mplementatons, values for the 9

10 actual error rate and user data rate, across dentcal channels and wth dentcal channel modulaton would need to be compared. Nonetheless, a survey of turbo decoder mplementaton performance results s ncluded here, whch generally fall nto a range comparable wth the results gven above. Masera et al desgned a CMOS mplementaton of a log-map decoder n.5 mcron CMOS technology [2]. Ther smulatons ndcate that ths desgn should support a 2 Mbt/s data rate. The desgn s ntended for deep space applcatons. Hong et al desgned a comparable decoder n.6 mcron CMOS [13]. Petrobon developed a prototype mult-board system usng FPGAs to mplement the log- MAP algorthm wth a 356 kbt/s data rate [11]. Several commercals turbo decoder mplementatons are now avalable. Advanced Hardware Archtectures oers an ASIC that they clam can perform decodng usng a propretary algorthm at rates of 36 Mbt/s for 2 teratons only [14]. They are usng a somewhat derent type of codng than descrbed heren, but clam smlar error correcton performance to turbo codng. Comatlas oers a turbo decoder ASIC wth clamed 4 Mbt/s decodng rate [15]; no detals as to the algorthm used or number of teratons were avalable. Small World Communcatons oers turbo decoder cores for Xlnx XC4XV FPGAs and clam 31 Mbt/s decoder rates [16]. The supported data rates wll be lower, proportonal to the number of teratons. of lnear codes for mnmzng symbol error rates," IEEE Trans. Informaton Theory, Mar. 1974, p [6] W. E. Ryan, \A turbo code tutoral," Unpublshed, avalable at code/overvew.html [7] R. McElece, D. MacKay, and J. Cheng, \Turbo-decodng as an nstance of Pearl's belef propagaton algorthm," IEEE J. Selected Areas n Comm., Vol.16, Feb. 1998, [8] D.J.C. MacKay, \Good error-correctng codes based on very sparse matrces," IEEE Trans. on Informaton Theory, vol.45, no.2, Mar. 1999, p [9] J. Hagenauer and P. Hoeher, \A Vterb algorthm wth softdecson outputs and and ts applcatons," Proc. GlobeCom 1989, pp [1] S. Halter, M. Oberg, P.M. Chau, and P.H. Segel, \Recongurable sgnal processor for channel codng and decodng n low SNR wreless communcatons," 1998 IEEE Workshop on Sgnal Processng Systems, pp [11] S.S. Petrobon, \Implementaton and performance of a turbo/map decoder," Internatonal J. of Satellte Communcatons, vol.16, no.1, Jan.-Feb. 1998, p [12] M.D. Ercegovac and T. Lang, Dvson and Square Root: Dgt- Recurrence Algorthms and Implementatons, Kluwer, [13] S. Hong, J., and W. Stark, \VLSI desgn and mplementaton of low-complexty adaptve turbo-code encoder and decoder for wreless moble communcatons applcatons" 1998 IEEE Workshop on Sgnal Processng Systems, p [14] Advanced Hardware Archtectures, Pullman, WA, USA, \AHA451 Astro: 36 Mbt/sec turbo product code encoder/decoder," datasheet, avalable at [15] Comatlas, Cesson-Sevgne, France, \CAS 593: Turbo-code codec," datasheet, avalable at [16] Small World Communcatons, Adelade, Australa, \MAP4T Very Hgh Speed MAP Decoder," datasheet, avalable at V. Conclusons Recently, several new famles of error correctng codes have emerged whch enable error-free communcatons at lower SNRs wth hgher bandwdth. In ths paper, we examned two of these codes: turbo codes and low densty party check codes. They were evaluated n terms of ther amenablty to mplementaton n hardware. We concluded that LDPCs are more sutable because ther decoders exhbt more explotable parallelsm, the computatons requre operators that are more easly mplementable n hardware, and they are more regular and tleable n nature than turbo decoders. We presented an llustratve desgn whch s mplementable on commercal FPGAs, and whch should dsplay performance comparable to current VLSI turbo code mplementatons. Acknowledgments The authors would lke to acknowledge the support of DARPA grant xxxxx. References [1] C.E. Shannon, \A mathematcal theory of communcatons," Bell Syst. Tech. J., vol.27, 1948, pp , [2] G. Masera, G. Pccnn, M. Roch, and M. Zambon, \VLSI archtectures for turbo codes," IEEE Tran. VLSI Sys., vol.7, no.3, Sep. 1999, p [3] C. Berrou, A. Glaveux, and P. Thtmajshma, \Near Shannon lmt error-correctng codng and cecodng: Turbo codes," Proc Int. Conf. Comm., Geneva, Swtzerland, May 1993, pp [4] B. Sklar, Dgtal Communcatons: Fundamentals and Applcatons, Prentce Hall, [5] L. Bahl, J. Cocke, F. Jelnek, and J. Ravv, \Optmal decodng 1