LP Decoding. Martin J. Wainwright. Electrical Engineering and Computer Science UC Berkeley, CA,

Transcription

1 Jon Feldman LP Decodng Industral Engneerng and Operatons Research Columba Unversty, New York, NY, Martn J. Wanwrght Electrcal Engneerng and Computer Scence UC Berkeley, CA, Davd R. Karger Laboratory for Computer Scence MIT, Cambrdge, MA, Abstract. Lnear programmng (LP) relaxaton s a common technque used to fnd good solutons to complex optmzaton problems. We present the method of LP decodng : applyng LP relaxaton to the problem of maxmum-lkelhood (ML) decodng. An arbtrary bnary-nput memoryless channel s consdered. Ths treatment of the LP decodng method places our prevous work on turbo codes [6] and low-densty party-check (LDPC) codes [8] nto a generc framework. We defne the noton of a proper relaxaton, and show that any LP decoder that uses a proper relaxaton exhbts many useful propertes. We descrbe the noton of pseudocodewords under LP decodng, unfyng many known characterzatons for specfc codes and channels. The fractonal dstance of an LP decoder s defned, and t s shown that LP decoders correct a number of errors equal to half the fractonal dstance. We also dscuss the applcaton of LP decodng to bnary lnear codes. We defne the noton of a relaxaton beng symmetrc for a bnary lnear code. We show that f a relaxaton s symmetrc, one may assume that the all-zeros codeword s transmtted. 1 Introducton The problem of maxmum-lkelhood (ML) decodng s to fnd the codeword most lkely to have been transmtted, gven a corrupted codeword from a nosy channel. Lnear programmng s the problem of fndng an optmal soluton to a system of lnear nequaltes under a lnear objectve functon [2]. In ths paper, we consder lnear programmng (LP) formulatons of the ML decodng problem on bnary codes. We use LP varables to represent code bts, and the LP objectve functon s defned by the channel lkelhood ratos. Prevous work on LP decodng [6, 4, 7, 8, 5] has focused on two specfc cases: turbo codes [1] and low-densty party-check codes [11]. These two famles of codes have receved a lot of attenton recently due to ther excellent performance. Performance bounds for LP decodng n these cases are for specfc LP formulatons, code constructons, and/or channel models. In ths paper we consder LP decoders for arbtrary bnary codes, under an arbtrary bnarynput memoryless channel. We provde a framework for desgnng LP decoders, and general technques for analyzng them. Central to every LP decoder s ts assocated polytope: the set of ponts that satsfy the constrants of the LP. A decodng polytope should contan every codeword, and should also exclude every bnary word that s not a codeword. We defne such polytopes as proper. We show that LP decoders that use proper polytopes have the ML certfcate property: whenever they output a codeword, t s guaranteed to be the ML codeword. In general, for any sub-optmal decoder, the pseudocodewords correspond to the set of possble results of the decoder. Ths set ncludes the codewords, but also some non-codewords Research supported by an NSF Mathematcal Scences Postdoctoral Research Fellowshp. Research supported by NSF contract CCR and a Davd and Luclle Packard Foundaton Fellowshp.

2 that can fool the algorthm. In ths paper, we provde a general characterzaton of the pseudocodewords for any LP decoder, under any bnary-nput memoryless channel. Ths characterzaton allows us to gve an exact expresson for the word error rate (WER) of the decoder: the probablty that the transmtted codeword s not the most lkely pseudocodeword. When appled to specfc codes, polytopes and channels, LP pseudocodewords are equvalent to other sets of pseudocodewords that have been studed n prevous work. For example, n the bnary erasure channel (BEC), when the polytope from [8] s used, the set of LP pseudocodewords s equvalent to the stoppng sets of the code, as defned by D et. al [3]. For tal-btng trellses, when the polytope from [5] s used, the set of LP pseudocodewords s equvalent to the pseudocodewords defned by Forney et. al [9]. The fractonal dstance of an LP decoder s defned as an analog to the (classcal) dstance. It s shown that under the bnary symmetrc channel (BSC), LP decoders correct a number of errors equal to half the fractonal dstance of the code. We also dscuss the applcaton of LP decodng to bnary lnear codes. We defne the noton of a polytope beng symmetrc for a bnary lnear code. We show that f a polytope s symmetrc, one may assume that the all-zeros codeword s transmtted, whch greatly smplfes analyss for ths rch class of codes. 1.1 Channel Model. In ths paper we assume an arbtrary bnary-nput memoryless channel;.e., the data s transmtted as dscrete symbols from {0, 1}, and each transmtted symbol s affected by the nose n the channel ndependently. Let y C denote the transmtted codeword. We wll use ỹ = (ỹ 1,..., ỹ n ) to denote the receved (corrupted) word. Each ỹ s a symbol from some space Σ that depends on the channel model. For example, n the bnary symmetrc channel (BSC), we have Σ = {0, 1}; n the AWGN channel, we have Σ = R. In our analyss of bnary lnear codes n Secton 3, we also assume a symmetrc channel;.e., the nose affects 0s and 1s n the same way. Formally, symmetry tells us that Σ can be parttoned nto pars (a, a ) such that Pr[ ỹ = a y = 0 ] = Pr[ ỹ = a y = 1 ], and (1) Pr[ ỹ = a y = 1 ] = Pr[ ỹ = a y = 0 ]. (2) 2 The Method of LP Decodng 2.1 Lnear Programmng Relaxaton. A lnear program (LP) conssts of a set of lnear nequaltes (constrants) and a lnear objectve functon over a set of varables. Solvng the lnear program means fndng a settng of the varables that satsfes the nequaltes, and optmzes the objectve functon. Lnear programs can be solved effcently usng the smplex algorthm [19], whch runs effcently n practce, or the ellpsod algorthm [12], whch has worst-case run-tme guarantees. Although many mportant problems can be solved as LPs, not all problems are drectly amenable to ths treatment. One ssue s that LP solutons can be real-valued, whereas the varables (n certan problems) may only be meanngful as ntegers (e.g., number of seats n an arplane). If we add the restrcton that all varables must be ntegers, we obtan an nteger lnear programmng (ILP) problem, whch (unfortunately) s NP-hard n general. A natural strategy for fndng an approxmate soluton to an ILP, then, s to remove the nteger constrants, solve the resultng LP, and then transform the soluton nto a meanngful one. (For example, roundng technques, often randomzed, are one method of transformng an LP soluton nto a decent soluton to the ILP of nterest.) Ths generc technque s referred to as lnear programmng relaxaton, and many successful approxmaton algorthms to NP-hard optmzaton problems are based on t [14]. 2.2 An LP Relaxaton of ML Decodng. Suppose we wsh to decode a bnary code C {0, 1} n under some bnary-nput memoryless channel. Let y C denote the transmtted code-

3 word, and let ỹ denote the receved codeword. Let γ be the log-lkelhood rato of the th code bt: ( ) Pr[ỹ y = 0] γ = ln. (3) Pr[ỹ y = 1] The sgn of the log-lkelhood rato γ determnes whether transmtted bt y s more lkely to be a 0 or a 1. (In partcular, f y s more lkely to be a 1, then γ wll be negatve, whereas f y s more lkely to be a 0, then γ wll be postve.) We wll refer to γ as the cost of code bt y, where γ represents the cost ncurred by settng a partcular bt y to 1, and to the sum γ y as the cost of a partcular codeword y. Wth these defntons, the ML codeword s exactly the codeword of mnmum cost [5]. Our LP relaxatons for decodng wll have LP varables f for each code bt, where {1,..., n}. Suppose we were able to solve the followng problem: n mnmze γ f s.t. f C. =1 Any optmal soluton f to ths system s an ML codeword. However, optmzng over C s too complex n general. Therefore, we optmze nstead over a less complex polytope P [0, 1] n, defned by a set of lnear constrants on the varables f. The partcular nature of the constrants wll depend on the underlyng code. In prevous work [6, 8], we have defned polytopes for turbo codes, LDPC codes, and arbtrary bnary lnear codes. In each of these cases, our polytopes contan a lnear (n n) number of constrants, and are therefore solvable effcently. Snce we are lookng for codewords, t should be the case that our polytope ncludes all the codewords, and does not nclude any non-codewords. Defnton 1. A polytope P s proper for code C f the ntegral ponts n P are exactly the codewords of C;.e., P s proper f P {0, 1} n = C. Gven a proper polytope P, our LP decoder solves the followng lnear program: n mnmze γ f s.t. f P (4) =1 Defne the cost of a pont f P as n =1 γ f. The LP n equaton (4) wll fnd the pont n P wth mnmum cost. If the LP soluton s ntegral (.e., all f are ether 0 or 1), then the LP decoder outputs the codeword f. In contrast, f the LP soluton s fractonal (.e., some f s non-ntegral), then the decoder outputs error. Theorem 2. An LP decoder usng a proper polytope has the ML certfcate property: f the decoder outputs a codeword, t s guaranteed to be an ML codeword. Proof. If the LP decoder outputs a codeword f C, then the cost of the pont f P s at most the cost of any pont n P. Snce P s proper, we have P C, and so f has cost at most the cost of any codeword y C. We conclude that f s the ML codeword. Example. Suppose we have the lnear code C = {0000, 1101, 1011, 0110}. Ths code can be characterzed by the party check equatons (y 1 y 2 y 3 ) = 0 and (y 2 y 3 y 4 ) = 0. We defne a polytope R on four varables {f 1, f 2, f 3, f 4 } as the set of ponts that satsfy the followng lnear nequaltes: (A) (B) (C) f 1 f 2 + f 3 f 2 f 3 + f 4 0 f 1 1 f 2 f 1 + f 3 f 3 f 2 + f 4 0 f 2 1 f 3 f 1 + f 2 f 4 f 2 + f 3 0 f 3 1 f 1 + f 2 + f 3 2 f 2 + f 3 + f f 4 1

4 The (C) constrants ensure that all f take on values between zero and one. The (A) and (B) constrants ensure that the polytope R s proper;.e., the set of bnary words of length four that satsfy the above constrants are exactly the set of codewords of C. To see ths, consder the (A) constrants; the bnary words that satsfy these constrants are exactly the words that satsfy the party check equaton (y 1 y 2 y 3 ) = 0. Smlarly, the (B) constrants correspond to the party check equaton (y 2 y 3 y 4 ) = 0. Ths polytope s a specal case of a general-purpose polytope for bnary lnear codes and LDPC codes [8, 5]. 2.3 Success Condtons for LP Decodng. Overall, the LP decoder succeeds f the transmtted codeword s the unque optmal soluton to the LP. The decoder fals f the transmtted codeword s not an optmal soluton to the LP. In the case of multple LP optma (whch for many nose models has zero probablty), we wll be conservatve and assume that the LP decoder fals. Therefore, we have the followng theorem. Theorem 3. For any bnary-nput memoryless channel, an LP decoder usng polytope P wll fal f and only f there s some pont n P other than the transmtted codeword y wth cost less than or equal to the cost of y. We use WER y to denote the word error rate (WER) of the LP decoder, gven a partcular transmtted codeword y. By Theorem 3, we have: [ WER y = Pr f P, f y : γ f ] γ y (5) 2.4 Vertces, Codewords and Pseudocodewords. An extreme pont, or equvalently a vertex of a polytope s a pont that cannot be expressed as the convex combnaton of other ponts n the polytope. Let V(P) be the set of vertces of the polytope P. A fundamental fact of lnear programmng s that the optmal soluton to an LP can always be found at a vertex of the polytope assocated wth the LP [19]. Therefore, the LP decoder wll always fnd the lowest cost vertex of the polytope P. Theorem 4. For any polytope P [0, 1] n that s proper for C, every codeword y C s a vertex of P. Proof. In the unt hypercube [0, 1] n, bnary words of length n cannot be expressed as the convex combnaton of other ponts n the hypercube. Snce P s contaned wthn the hypercube [0, 1] n, we have that all ponts n P {0, 1} n are vertces of P. Snce P s proper, we have P {0, 1} n = C, and the theorem follows. It s mportant to note that the converse statement (.e., every polytope vertex s a codeword) may not hold, however, snce the polytope could have fractonal (non-ntegral) vertces. So, n general we have C V(P) P [0, 1] n. In LP decodng, vertces take on the role of pseudocodewords: the set of possble results that a sub-optmal decoder may produce. Pseudocodewords are a superset of the codewords, and may contan false codewords that fool the algorthm. Whle the set of codewords s a functon of the code tself, the set of pseudocodewords s a functon of the sub-optmal decodng algorthm beng used. Understandng the pseudocodewords of a sub-optmal algorthm allows a thorough analyss of ts WER. For example, D et. al [3] explot the structure of stoppng sets to analyze the word error rate of teratve decodng on the bnary erasure channel. Even and Halab [4] derve combnatoral theorems about promenades (the pseudocodewords of an LP relaxaton for

5 rate-1/2 repeat-accumulate codes [6]), and use them to show tght upper and lower bounds on the WER of LP decodng. Example. Consder the polytope R desgned earler for the code C = {0000, 1101, 1011, 0110}. The vertces of ths polytope nclude the codewords, as well as the fractonal vertces (1, 1 2, 1 2, 0) and (0, 1 2, 1 2, 1). Note that nether of the fractonal vertces can be expressed as convex combnatons of codewords. We have V(R) = { (0, 0, 0, 0), (1, 1, 0, 1), (1, 0, 1, 1), (1, 1, 1, 1), (1, 1 2, 1 2, 0), (0, 1 2, 1 2, 1)}. Ths s the set of pseudocodewords for the LP decoder usng R on ths code. Remarks: Pseudocodewords have been studed by Wberg [21], Forney et. al [9] and Frey et. al [10] as codewords of an teratve decoder computaton tree, under mn-sum decodng. They have also been studed for tal-btng trellses [9], LDPC codes n the bnary erasure channel [3], and Tanner graph covers [15]. In many of these specfc cases, LP pseudocodewords are equvalent to (or very related to) prevously studed pseudocodeword sets (see [8, 5] for detals). It would be nterestng to see how the known technques for analyzng the weghts of pseudocodewords could be used to analyze the weghts of LP pseudocodewords for other codes and channels. y (3) PSfrag replacements y (2) a b c d f y (4) y (5) y (1) Fgure 1. A decodng polytope P (dotted lne) and the convex hull P ML (sold lne) of the codewords y (1) through y (4). Also shown are the four possble cases (a d) for the objectve functon, and the normal cones to both P and P ML. 2.5 Geometrc Perspectve. Fgure 1 provdes a geometrc perspectve of LP decodng, and ts relaton to exact ML decodng. The nner sold lne encloses the convex hull of the codewords (.e., the set of ponts that are convex combnatons of codewords), denoted by P ML. The dotted lne n the fgure represents the relaxed LP decodng polytope P, and the crcles represent pseudocodewords (vertces of P). The black crcles n the fgure represent codewords (also members of P ML ), whereas the gray crcles represent fractonal vertces that are not codewords. The objectve functon s the only element of the LP that depends on channel nose. An LP objectve functon can be seen as a drecton nsde the polytope; solvng the LP amounts to fndng the pont n the polytope that s furthest n that drecton. If there s no nose n the channel, then the transmtted codeword wll be the ML codeword, and thus the lowest cost codeword. The gray arrow n Fgure 1 represents the objectve functon wthout nose, and t ponts drectly toward the transmtted codeword y (1). Nose n the channel appears n the LP as a perturbaton of the objectve functon away from the no nose drecton. If the perturbaton s small, then y (1) wll reman the optmal pont of the LP. If the perturbaton s large (.e., hgh channel nose), then y (1) wll no longer be optmal. Both exact ML and relaxed LP decodng can be seen as mnmzng the LP objectve spec-

6 fed by the channel, but over dfferent constrant sets. In exact ML decodng, the constrant set s the convex hull P ML of codewords, whereas relaxed LP decodng uses the larger polytope P. As a concrete llustraton, consder agan the set-up of Fgure 1, n whch codeword y (1) was transmtted. The four arrows labeled (a) (d) correspond to dfferent nosy versons of the LP objectve functon. (a) If there s very lttle nose, then both ML decodng and LP decodng succeed, snce both have the transmtted codeword y (1) as the optmal pont. (b) If more nose s ntroduced, then ML decodng succeeds, but LP decodng fals, snce the fractonal vertex f s optmal for the relaxaton. (c) Wth stll more nose, ML decodng fals, snce y (4) s now optmal; LP decodng stll has a fractonal optmum f, so ths error s detected. (d) Fnally, wth a lot of nose, both ML decodng and LP decodng have y (4) as the optmum, and so both methods fal and the error s undetected. Note that n the last two cases (c,d), when ML decodng fals, the falure of the LP decoder s n some sense the fault of the code tself, as opposed to the decoder. 2.6 Normal Cones. The behavor of relaxed LP decodng and exact ML decodng can be dstngushed n terms of the normal cones [13] assocated wth the LP and ML polytopes at a gven codeword y C. The (negatve) normal cones are defned as follows: N y (P) = { γ R n : N y (P ML ) = { γ R n : γ (f y ) 0 for all f P }, γ (f y ) 0 for all f P ML }. Note that N y (P) corresponds to the set of cost vectors γ such that y s an optmal soluton to the LP defned by polytope P, and the objectve functon γ f. The set N y (P ML ) has a smlar nterpretaton as the set of cost vectors γ for whch y s an ML codeword. Snce P ML P, t s mmedate from the defnton that N y (P ML ) N y (P) for all y C. For example, n Fgure 1, cost vector (a) belongs to both N y (1)(P ML ) and N y (1)(P). In contrast, the vector (b) belongs to N y (1)(P ML ), but not to N y (1)(P). If codeword y s transmtted, the success probablty of an LP decoder s equal to the total probablty mass of N y (P), under the dstrbuton on cost vectors defned by the channel. The success probablty of ML decodng s smlarly related to the probablty mass n the normal cone N y (P ML ). Thus, the dscrepancy between the normal cones of P and P ML s a measure of the gap between exact ML and relaxed LP decodng. The cone N y (P) can be seen as a sgnal-space characterzaton of the LP pseudocodewords. Such characterzatons have been gven by Frey et. al [10] n the case of teratve decodng and by Koetter and Vontobel [15] usng the noton of graph covers. In partcular, the fundamental cone studed by Koetter and Vontobel [15] for graph covers s polar [13] to the normal cone N 0 n(q) assocated wth the polytope Q defned n [8] for LDPC codes. 2.7 The Fractonal Dstance. We motvate the noton of fractonal dstance by provdng an alternatve defnton for the (classcal) dstance n terms of a proper polytope P. Recall that any proper polytope P s characterzed by a one-to-one correspondence between codewords and ntegral vertces of P;.e., C = P {0, 1} n. The Hammng dstance between two ponts n the dscrete space {0, 1} n s equvalent to the l 1 dstance between the ponts n the space [0, 1] n. Therefore, gven a proper polytope P, we may defne the dstance of a code as the mnmum l 1 dstance between two ntegral vertces,.e., d = mn y,y (V(P) {0,1} n ) y y n y y. =1

7 The LP polytope P may have addtonal non-ntegral vertces, as llustrated n Fgure 1. Accordngly, we defne the fractonal dstance d frac of a polytope P as the mnmum l 1 dstance between an ntegral vertex (codeword) and any any other vertex (pseudocodeword) of P;.e., d frac = mn y C f V(P) f y n y f. =1 Note that ths fractonal dstance s always a lower bound on the classcal dstance of the code, snce every codeword s a polytope vertex (n the set V(P)). Moreover, the performance of LP decodng s ted to ths fractonal dstance. The proof of the followng theorem s essentally the same as the proof n [8], and so t s omtted. We refer the reader to the thess [5] for detals. Theorem 5. Let C be a bnary code and P a proper polytope n an LP relaxaton for C. If the fractonal dstance of P s d frac, then the LP decoder usng P s successful f at most d frac /2 1 bts are flpped by the bnary symmetrc channel. 3 Symmetrc Polytopes for Bnary Lnear Codes Bnary lnear codes have some specal algebrac structure that can be exploted n the analyss of decodng algorthms. For example, for most message-passng decoders, one may assume wthout loss of generalty that the all-zeros codeword 0 n, whch s always a codeword of a bnary lnear code, was transmtted. Ths all-zeros assumpton greatly smplfes analyss as well as notaton. Furthermore, the dstance of a bnary lnear code s equal to the lowest weght of any non-zero codeword, where the weght of a codeword y s defned as y. In ths secton we dscuss the applcaton of LP decodng to bnary lnear codes. We frst defne the noton of a polytope beng C-symmetrc for a partcular bnary lnear code C. We then prove that f a decoder uses a C-symmetrc polytope, then the all-zeros assumpton s vald, and the fractonal dstance s equal to the lowest weght of any non-zero polytope vertex. Ths result not only smplfes analyss, but t also allows us to compute effcently the fractonal dstance of a polytope. 3.1 C-symmetry of a Polytope. For a pont f [0, 1] n, we defne ts relatve pont f [y] [0, 1] n wth respect to codeword y as follows: for all {1,..., n}, let f [y] = f y. Note that ths operaton s ts own nverse;.e., we have (f [y] ) [y] = f for all f [0, 1] n. Intutvely, the pont f [y] s the pont that has the same spatal relaton to the pont 0 n as f has to the codeword y (and vce-versa). Defnton 6. A proper polytope P for the bnary code C s C-symmetrc f, for all ponts f n the polytope P and codewords y n the code C, the relatve pont f [y] s also contaned n P. 3.2 All-Zeros Assumpton. The valdty of the all-zeros assumpton s not mmedately clear n the context of LP decodng. In ths secton, we prove that one can make the all-zeros assumpton when analyzng LP decoders, as long as the polytope used n the decoder s C-symmetrc. Theorem 7. For any LP decoder usng a C-symmetrc polytope to decode C under a bnarynput memoryless symmetrc channel, the probablty that the LP decoder fals s ndependent of the codeword that s transmtted. Proof. For an arbtrary transmtted word y, we need to show that WER y = WER 0 n. Defne BAD(y) Σ n to be the set of receved words ỹ that cause decodng falure, assumng y s

8 transmtted: { BAD(y) = ỹ Σ n : f P, f y, where γ f γ y }, where the cost vector γ = γ(ỹ) s a functon of the receved word ỹ. (Note that ths defnton s conservatve, n that t ncludes the case of multple LP optma as decodng falure.) Rewrtng equaton (5), we have that for all codewords y, WER y = Pr[ ỹ y ]. (6) ỹ BAD(y) As a partcular case, for the codeword 0 n, we have WER 0 n = ỹ BAD(0 n ) Pr[ ỹ 0n ]. We now show that the space Σ n of possble receved vectors can be parttoned nto pars (ỹ, ỹ 0 ) such that Pr[ ỹ y ] = Pr[ ỹ 0 0 n ], and ỹ BAD(y) f and only f ỹ 0 BAD(0 n ). Ths partton, along wth equaton (6), gves WER y = WER 0 n. The partton s performed accordng to the symmetry of the channel. Fx some receved vector ỹ. Defne ỹ 0 as follows: let ỹ 0 = ỹ f y = 0, and ỹ 0 = ỹ f y = 1, where ỹ s the symbol symmetrc to ỹ n the channel. (See Secton 1.1 for detals on symmetry.) Note that ths operaton s ts own nverse and therefore gves a vald partton of Σ n nto pars. Frst,we show that Pr[ ỹ y ] = Pr[ ỹ 0 0 n ]. From the channel beng memoryless, we have n Pr[ ỹ y ] = Pr[ỹ y ] = Pr[ỹ 0 0] Pr[ỹ 1] (7a) =1 :y =0 = :y =0 = :y =0 = Pr[ ỹ 0 0 n ] :y =1 Pr[ỹ 0 0] :y =1 Pr[ỹ 0 0] :y =1 Pr[ỹ 0] Pr[ỹ 0 0] Equatons (7a) and (7c) follow from the defnton of ỹ 0, whereas equaton (7b) follows from the symmetry of the channel (equatons (1) and (2)). Now t remans to show that ỹ BAD(y) f and only f ỹ 0 BAD(0 n ). Let γ be the cost vector when ỹ s receved, and let γ 0 be the cost vector when ỹ 0 s receved, as defned n equaton (3). Suppose y = 0. Then, ỹ = ỹ 0, and so γ = γ 0. Now suppose y = 1; then ỹ 0 = ỹ, and so γ 0 = log ( ) Pr[ỹ y = 0] Pr[ỹ y = log = 1] ( ) Pr[ỹ y = 1] = γ. Pr[ỹ y = 0] Ths follows from the symmetry of the channel (equatons (1) and (2)). We conclude that (7b) (7c) γ = γ 0 f y = 0, and γ = γ 0 f y = 1. (8) Fx some pont f P and consder the relatve pont f [y]. We clam that the dfference n cost between f and y s the same as the dfference n cost between f [y] and 0 n. In partcular, we reason as follows: γ f γ y = γ f [y] γ f [y] (9a) :y =0 :y =1 = = :y =0 γ 0 f [y] γ 0 f [y] + :y =1 γ 0 0. γ 0 f [y] (9b) (9c)

9 Equaton (9a) follows from the defnton of f [y], and equaton (9b) follows from equaton (8). Now suppose ỹ BAD(y), and so by the defnton of BAD there s some f P, where f y, such that γ f γ y 0. By equaton (9c), we have that γ0 f [y] γ0 0 n 0. Because P s C-symmetrc, f [y] P, and by the fact that f y, we have that f [y] 0 n. Therefore ỹ 0 BAD(0 n ). A symmetrc argument shows that f ỹ 0 BAD(0 n ) then ỹ BAD(y). Snce the all-zeros codeword has zero cost, the all-zeros assumpton gves the followng corollary to Theorem 3: Corollary 8. Under the all-zeros assumpton, for any bnary lnear code C over any bnarynput memoryless symmetrc channel, the LP decoder usng the C-symmetrc polytope P wll fal f and only f there s some non-zero pont n P wth cost less than or equal to zero. 3.3 Fractonal Dstance of Symmetrc Polytopes. The (classcal) dstance of a bnary lnear code s equal to the mnmum weght of a non-zero codeword. Ths fact s very mportant when analyzng the dstance of lnear codes. It turns out that we can make a smlar smplfyng assumpton when we analyze fractonal dstance. We refer the reader to [5] for a proof of the followng: Theorem 9. The fractonal dstance of a C-symmetrc polytope P for a bnary lnear code C s equal to the mnmum weght of a non-zero vertex of P. In contrast to the classcal dstance, the fractonal dstance of a C-symmetrc polytope P for a bnary lnear code C can be computed effcently. Ths can be used to bound the worst-case performance of LP decodng for a partcular code and polytope. Snce the fractonal dstance s a lower bound on the real dstance, we thus have an effcent algorthm to gve a non-trval lower bound on the dstance of a bnary lnear code. We refer the reader to [8, 5] for the detals of ths algorthm. 4 Concluson In ths paper we outlned the basc technque of LP decodng. We derved general success condtons for an LP decoder, and showed that any decoder usng a proper polytope has the ML certfcate property. The fractonal dstance of a polytope was defned n ths general settng, and t was shown that LP decoders correct a number of errors up to half the fractonal dstance. Furthermore, for bnary lnear codes, we establshed symmetry condtons for the polytope that allow for the all-zeros assumpton, and regardng fractonal dstance as fractonal weght. It s our hope that ths paper wll be a good startng pont for the desgn and analyss of LP decoders. There are many open questons n the area of LP decodng (for a full dscusson, see [5]). Most mportantly, we would lke to see code constructons that take advantage of the smple characterzaton of pseudocodewords offered by the LP decoder. Analytc performance bounds usng LP decoders have been proved for rate-1/2 repeat-accumulate codes [6, 4] and LDPC codes [8]. It s mportant to prove bounds for more complex turbo codes, as well as mprove the bounds for LDPC codes (whch are not yet close to the observed performance [5] of LP decoders n ths case). Gven a specfc proper polytope for an LP decoder, we can employ a number of dfferent known technques to tghten the polytope [18, 20, 16, 17], thereby obtanng an mproved decoder (at the expense of addtonal computaton). We dscussed ths dea somewhat n [5], but have yet to use t to strengthen our analytc performance bounds.

10 Fnally, t would be nterestng to look at codes over non-bnary alphabets, and over nonmemoryless channels. We could model a non-bnary code n an LP by usng several 0 1 varables as ndcators of a symbol takng on a partcular value. Alternatvely, we could map the code to a bnary code, and use an LP relaxaton for the bnary code. It would be nterestng to see f anythng s ganed by by representng the larger alphabet explctly. References [1] C. Berrou, A. Glaveux, and P. Thtmajshma. Near Shannon lmt error-correctng codng and decodng: turbo-codes. Proc. IEEE Internatonal Conference on Communcaton (ICC), Geneva, Swtzerland, pages , May [2] D. Bertsmas and J. Tstskls. Introducton to lnear optmzaton. Athena Scentfc, Belmont, MA, [3] C. D, D. Proett, T. Rchardson, E. Telatar, and R. Urbanke. Fnte length analyss of low-densty party check codes. IEEE Transactons on Informaton Theory, 48(6), [4] G. Even and N. Halab. Improved bounds on the word error probablty of RA(2) codes wth lnear programmng based decodng. In Proc. 41st Annual Allerton Conference on Communcaton, Control, and Computng, October [5] J. Feldman. Decodng Error-Correctng Codes va Lnear Programmng. PhD thess, Massachusetts Insttute of Technology, [6] J. Feldman and D. R. Karger. Decodng turbo-lke codes va lnear programmng. Proc. 43rd annual IEEE Symposum on Foundatons of Computer Scence (FOCS), November To appear n Journal of Computer and System Scences. [7] J. Feldman, D. R. Karger, and M. J. Wanwrght. Lnear programmng-based decodng of turbolke codes and ts relaton to teratve approaches. In Proc. 40th Annual Allerton Conference on Communcaton, Control, and Computng, October [8] J. Feldman, M. J. Wanwrght, and D. R. Karger. Usng lnear programmng to decode lnear codes. 37th annual Conference on Informaton Scences and Systems (CISS 03), March Submtted to IEEE Transactons on Informaton Theory, May, [9] G. D. Forney, R. Koetter, F. R. Kschschang, and A. Reznk. On the effectve weghts of pseudocodewords for codes defned on graphs wth cycles. In Codes, systems and graphcal models, pages Sprnger, [10] B. Frey, R. Koetter, and A. Vardy. Sgnal-space characterzaton of teratve decodng. IEEE Transactons on Informaton Theory, 47(2): , [11] R. Gallager. Low-densty party-check codes. IRE Trans. Inform. Theory, IT-8:21 28, Jan [12] M. Grotschel, L. Lovász, and A. Schrjver. The ellpsod method and ts consequences n combnatoral optmzaton. Combnatorca, 1(2): , [13] J. Hrart-Urruty and C. Lemaréchal. Convex analyss and mnmzaton algorthms, volume 1. Sprnger-Verlag, New York, [14] D. Hochbaum, edtor. Approxmaton Algorthms for NP-hard Problems. PWS Publshng, [15] R. Koetter and P. O. Vontobel. Graph-covers and teratve decodng of fnte length codes. In Proc. 3rd Internatonal Symposum on Turbo Codes, September [16] L. B. Lasserre. An explct exact SDP relaxaton for nonlnear 0 1 programs. K. Aardal and A.M.H. Gerards, eds., Lecture Notes n Computer Scence, 2081: , [17] M. Laurent. A comparson of the Sheral-Adams, Lovász-Schrjver and Lasserre relaxatons for 0 1 programmng. Techncal Report PNA R0108, Centrum voor Wskunde en Informatca, CWI, Amsterdam, The Netherlands, [18] L. Lovász and A. Schrjver. Cones of matrces and set-functons and 0 1 optmzaton. SIAM Journal on Optmzaton, 1(2): , [19] A. Schrjver. Theory of Lnear and Integer Programmng. John Wley, [20] H. D. Sheral and W. P. Adams. A herarchy of relaxatons between the contnuous and convex hull representatons for zero-one programmng problems. SIAM Journal on Optmzaton, 3: , [21] N. Wberg. Codes and Decodng on General Graphs. PhD thess, Lnkopng Unversty, Sweden, 1996.