LP Decoding. LP Decoding: - LP relaxation for the Maximum-Likelihood (ML) decoding problem. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.

Transcription

1 LP Decoding Allerton, 2003 Jon Feldman Columbia University David Karger MIT Martin Wainwright UC Berkeley J. Feldman, D. Karger, M. Wainwright, LP Decoding p.1/18

2 LP Decoding Linear Programming (LP): - Finding a solution to a set of linear inequalities that optimizes a linear objective function. Integer Linear Programming (ILP): - LP where variables constrained to be integers. LP Relaxation: - Using an LP to find a good (approximate) solution to an ILP. LP Decoding: - LP relaxation for the Maximum-Likelihood (ML) decoding problem. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.2/18

3 LP Decoding Previous work on specific code families/constructions: - Turbo codes [FK, FOCS 02] [EH, A 03] [F 03]. - LDPC codes [FKW, CISS 03] [F 03]. - New iterative algs. [FKW, Allerton 02] [F 03]. This paper: general treatment of LP decoding, for any binary code, memoryless channel (BSC, AWGN). - Proper polytope (ML certificate). - LP pseudocodeword. - Fractional Distance. - Symmetric polytope (linear codes). J. Feldman, D. Karger, M. Wainwright, LP Decoding p.3/18

4 Maximum-Likelihood (ML) Decoding Log-likelihood ratio (LLR) of as a cost function: - - more likely more likely For any binary-input memoryless channel: ML DECODING: Given LLRs, find such that is minimized. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.4/18

5 Maximum-Likelihood (ML) Decoding min rtex ise ise Convex hull( ) CH( ) = convex hull of codewords; CH( ) ML Decoding: Minimize s.t. CH( ). Problem: CH( ) is too complex (not poly-size).. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.5/18

6 LP Decoding min Polytope ise ise pseudocodeword Convex hull( ) Proper relaxation polytope : Alg: Solve LP. If ML certificate property integral, output, else error.. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.6/18

7 LP Decoder Example ts Define polytope on variables : Is proper (does )? J. Feldman, D. Karger, M. Wainwright, LP Decoding p.7/18

8 LP Decoder Example ts Polytope: Vertices: J. Feldman, D. Karger, M. Wainwright, LP Decoding p.8/18

9 LP Decoding Success Conditions e e Objective function cases (e) (a) (b) (c) (d) pseudocodeword some other (a) No noise (b) Both succeed (c) ML succeed, LP fail (d) Both fail, detected (e) Both fail, undetected ) transmitted J. Feldman, D. Karger, M. Wainwright, LP Decoding p.9/18

10 LP Pseudocodewords In general, pseudocodewords are the set of possible results of a sub-optimal decoder : - PCWs codewords; - Algorithm finds min-cost PCW; - WER = Pr[ transmitted = min-cost PCW ]. Example: It. decoding in the BEC [Di et. al, 02]. - PCWs = stopping sets codewords; - Iterative decoding finds min-cost stopping set. LP Decoding: - PCWs = polytope vertices codewords - LP Decoder find min-cost polytope vertex. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.10/18

11 ts Unifying Other Known PCWs = trellis flow polytope [FK 02] Vertices(polytope ) = LDPC code polytope [FKW 03] Tail-biting trellis PCWs [FKMT 01] Rate-1/2 RA code promenades [EH 03] BEC stopping sets [DPRTU 02] PCWs of graph covers [KV 03] J. Feldman, D. Karger, M. Wainwright, LP Decoding p.11/18

12 Using PCWs for Performance Bounds Turbo code polytope [FK 02, F 03]: Theorem: if In BSC, AWGN, then WER, for any. - Bounds improved by [EH, Allerton 03]., LDPC code polytope [FKW, CISS 03]: For any graph with girth, left-degree : Theorem: LP decoding corrects errors (adversarial). - With log-girth, can correect errors. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.12/18

13 Fractional Distance Another way to define (classical) distance : - = min dist. between two integral vertices of. Fractional distance: - = min distance between an integral vertex and any other vertex of. - Lower bound on classical distance:. Theorem: In the binary symmetric channel, LP decoders can correct up to errors. Linear codes: Given facets of can be computed efficiently., fractional distance J. Feldman, D. Karger, M. Wainwright, LP Decoding p.13/18

14 Symmetric Polytopes for Linear Codes ML decoding: - If is linear, may assume - Simplifies analysis, notation. - Min-distance = min-weight. is transmitted. Same assumption can be made for iterative algorithms, since pseudocodewords obey symmetry. LP Decoding: Definition: Polytope is -symmetric if, for all and, we have (where Theorem: If polytope is proper and -symmetric, then WER of LP decoder using is independent of the transmitted codeword. ). J. Feldman, D. Karger, M. Wainwright, LP Decoding p.14/18

15 noise noise ll( ) Tightening the Relaxation If constraints are added to the polytope, the decoder can only improve. Original polytope Tightened polytope Generic tightening techniques [LS 91] [SA 90]. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.15/18

16 Using Lift-And-Project 10 0 WER Comparison: Random Rate-1/4 (3,4) LDPC Code LP Decoder Lift-and-project Decoder ML Decoder 10-1 Word Error Rate AWGN Signal-to-noise ratio (E b / N 0 in db) Length 36, left degree, right degree. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.16/18

17 Future Work New PCW-based performance bounds for turbo/ldpc polytopes? - Better turbo codes (rate-1/3 RA); - Other LDPC codes. New (better?) polytopes for turbo/ldpc codes? Using lifting procedures (generic, specialized) to tighten relaxation? Deeper connections to sum-product (belief-prop)? Improved running time over simplex/ellipsoid algorithm? LP decoding of new code families, channel models? J. Feldman, D. Karger, M. Wainwright, LP Decoding p.17/18

18 Performance Comparison WER Comparison: Random Rate-1/4 (3,4) LDPC Code Min-Sum Decoder LP Decoder Sum-Product Decoder ML Decoder Word Error Rate BSC Crossover Probability Length 60, left degree, right degree. J. Feldman, D. Karger, M. Wainwright, LP Decoding p.18/18