Code Design in the Short Block Length Regime

Transcription

1 October 8, 2014 Code Design in the Short Block Length Regime Gianluigi Liva, Institute for Communications and Navigation German Aerospace Center, DLR

2 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

3 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

4 Outline 1 Introduction Motivations Preliminaries 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

5 Page 1/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Motivations: Why Short Codes? 1. Update on the race towards the best performance, in the short block length regime, with some hints on the code design 2. While for long block lengths well-established tools exist to design capacity-approaching codes (mostly, iteratively-decodable), short codes (dimension 100 < k < 1000) lack of universal design guidelines. 3. Considerable gap from theoretical bounds for moderate-length and short codes... Challenge!

6 Page 1/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Motivations: Why Short Codes? 1. Update on the race towards the best performance, in the short block length regime, with some hints on the code design 2. While for long block lengths well-established tools exist to design capacity-approaching codes (mostly, iteratively-decodable), short codes (dimension 100 < k < 1000) lack of universal design guidelines. 3. Considerable gap from theoretical bounds for moderate-length and short codes... Challenge!

7 Page 1/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Motivations: Why Short Codes? 1. Update on the race towards the best performance, in the short block length regime, with some hints on the code design 2. While for long block lengths well-established tools exist to design capacity-approaching codes (mostly, iteratively-decodable), short codes (dimension 100 < k < 1000) lack of universal design guidelines. 3. Considerable gap from theoretical bounds for moderate-length and short codes... Challenge!

8 Page 2/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Usual Paradigm for Iterative Code Design Step 1: Identify a code ensemble with near Shannon-limit performance via density evolution 1 / extrinsic information transfer 2 analysis VN EXIT function CN EXIT function ĪE,V, ĪA,C Eb/N0 =0.56 [db] ĪA,V, ĪE,C 1 T. Richardson, A. Shokrollahi, and R. Urbanke. Design of Capacity-Approaching Irregular Low-Density Parity-Check Codes. In: IEEE Trans. Inf. Theory (2001). 2 S. ten Brink. Convergence Behavior of Iteratively Decoded Parallel Concatenated Codes. In: IEEE Trans. Commun. (2001).

9 Page 3/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Usual Paradigm for Iterative Code Design Step 1b: Identify a code ensemble with near Shannon-limit performance via density evolution (DE) 3 / extrinsic information transfer (EXIT) 4 analysis under given constraints on ensemble distance properties VN EXIT function CN EXIT function 3 x ĪE,V, ĪA,C G(ω) Eb/N0 =0.56 [db] ĪA,V, ĪE,C ω x T. Richardson, A. Shokrollahi, and R. Urbanke. Design of Capacity-Approaching Irregular Low-Density Parity-Check Codes. In: IEEE Trans. Inf. Theory (2001). 4 S. ten Brink. Convergence Behavior of Iteratively Decoded Parallel Concatenated Codes. In: IEEE Trans. Commun. (2001).

10 Page 4/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Usual Paradigm for Iterative Code Design Step 2: For the block length of interest, perform the finite-length code design according to techniques that allow picking good ensemble members (e.g., in terms of minimum distance). For turbo codes, interleaver design 5 For low-density parity-check codes, girth optimization techniques 67 5 S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC X.-Y. Hu, E. Eleftheriou, and D.M. Arnold. Regular and irregular progressive edge-growth Tanner graphs. In: IEEE Trans. Inf. Theory (2005). 7 T. Tian et al. Selective Avoidance of Cycles in Irregular LDPC Code Construction. In: IEEE Trans. Commun. (2004).

11 Page 5/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Usual Paradigm for Iterative Code Design Step 1 (ensemble design) typically relies on asymptotic results DE/EXIT predict the ensemble iterative decoding threshold in the limit of infinite block length The ensemble distance properties are often captured by the growth rate of the weight distribution 8 1 G(ω) = lim log E(N( ωn )) n n where E(N( ωn )) is the expected number of codewords of weight ωn and we define the typical minimum distance ω = inf{ω > 0 : G(ω) 0} Step 1 may fail to capture the ensemble performance behavior when the block length is fixed to a few hundred bits... 8 R. Gallager. Low-density parity-check codes

12 Page 5/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Usual Paradigm for Iterative Code Design Step 1 (ensemble design) typically relies on asymptotic results DE/EXIT predict the ensemble iterative decoding threshold in the limit of infinite block length The ensemble distance properties are often captured by the growth rate of the weight distribution 8 1 G(ω) = lim log E(N( ωn )) n n where E(N( ωn )) is the expected number of codewords of weight ωn and we define the typical minimum distance ω = inf{ω > 0 : G(ω) 0} Step 1 may fail to capture the ensemble performance behavior when the block length is fixed to a few hundred bits... 8 R. Gallager. Low-density parity-check codes

13 Sphere-Packing Bound Random Coding Bound Page 6/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Multi-Edge-Type LDPC Code from J. Thorpe. Low-Density Parity-Check (LDPC) Codes Constructed from Protographs. IPN Progress Report Block Error Rate Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db Eb/N0 [db]

14 Page 6/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Multi-Edge-Type LDPC Code from J. Thorpe. Low-Density Parity-Check (LDPC) Codes Constructed from Protographs. IPN Progress Report Bad typical minimum distance, excellent threshold Block Error Rate Sphere-Packing Bound Random Coding Bound 10 5 Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db Eb/N0 [db]

15 Page 7/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Spatially-Coupled Protograph-based LDPC code from M. Lentmaier et al. Approaching Capacity with Asymptotically Regular LDPC Codes. In: Information Theory and Applications Workshop (ITA) Block Error Rate Sphere-Packing Bound Random Coding Bound Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db SC-LDPC (4,8), dmin = O(n), (Eb/N0) =0.35 db Eb/N0 [db]

16 Page 7/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Spatially-Coupled Protograph-based LDPC code from M. Lentmaier et al. Approaching Capacity with Asymptotically Regular LDPC Codes. In: Information Theory and Applications Workshop (ITA) Block Error Rate Good typical minimum distance, excellent threshold Sphere-Packing Bound Random Coding Bound Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db SC-LDPC (4,8), dmin = O(n), (Eb/N0) =0.35 db Eb/N0 [db]

17 Sphere-Packing Bound Page 8/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Non-Binary LDPC code from C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, d c)-ldpc codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008) Block Error Rate Random Coding Bound Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db SC-LDPC (4,8), dmin = O(n), (Eb/N0) =0.35 db LDPC Code, GF(256), dmin = O(logn), (Eb/N0) =0.48 db Eb/N0 [db]

18 Page 8/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Example: (2048, 1024) Codes Non-Binary LDPC code from 10 0 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, d c)-ldpc codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008) Bad typical minimum distance, good threshold Block Error Rate Sphere-Packing Bound Random Coding Bound Multi-Edge-Type LDPC, dmin = O(logn), (Eb/N0) =0.25 db SC-LDPC (4,8), dmin = O(n), (Eb/N0) =0.35 db 10 6 LDPC Code, GF(256), dmin = O(logn), (Eb/N0) =0.48 db Eb/N0 [db]

19 Page 9/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Code Design is Challenging While for long codes asymptotic design tools work reliably, at short block lengths we miss a universal approach to the code design problem As a results, the short block length iterative code design is often the domain of Ad-hoc code constructions, driven by intuition Experimental evidence Lack of deep understanding

20 Page 9/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Code Design is Challenging While for long codes asymptotic design tools work reliably, at short block lengths we miss a universal approach to the code design problem As a results, the short block length iterative code design is often the domain of Ad-hoc code constructions, driven by intuition Experimental evidence Lack of deep understanding

21 Page 9/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Code Design is Challenging While for long codes asymptotic design tools work reliably, at short block lengths we miss a universal approach to the code design problem As a results, the short block length iterative code design is often the domain of Ad-hoc code constructions, driven by intuition Experimental evidence Lack of deep understanding

22 Page 9/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Code Design is Challenging While for long codes asymptotic design tools work reliably, at short block lengths we miss a universal approach to the code design problem As a results, the short block length iterative code design is often the domain of Ad-hoc code constructions, driven by intuition Experimental evidence Lack of deep understanding

23 Page 10/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Race to the Best Performance For Long Blocks

24 Page 11/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Race to the Best Performance for Short Blocks

25 Page 12/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Codes? Short blocks are of interest for a number of emerging applications 1. Messaging 2. Machine-to-Machine 3. Wireless Sensor Networks 4. Data traffic originated by signaling in cellular networks In this presentation, we will focus on short codes under iterative decoding due to their technology readiness We shall however define what is short is for us...

26 Page 12/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Codes? Short blocks are of interest for a number of emerging applications 1. Messaging 2. Machine-to-Machine 3. Wireless Sensor Networks 4. Data traffic originated by signaling in cellular networks In this presentation, we will focus on short codes under iterative decoding due to their technology readiness We shall however define what is short is for us...

27 Page 12/158 G. Liva Short Codes Design Introduction October 8, 2014 Introduction Why Short Codes? Short blocks are of interest for a number of emerging applications 1. Messaging 2. Machine-to-Machine 3. Wireless Sensor Networks 4. Data traffic originated by signaling in cellular networks In this presentation, we will focus on short codes under iterative decoding due to their technology readiness We shall however define what is short is for us...

28 Page 8 13/158 G. Liva Short Codes Design Introduction October 8, 2014 Eb, [db] N PB =10 4 R =1/ Sphere Packing Bound 1 Non-Achievable Region k [bits]

29 Page 8 14/158 G. Liva Short Codes Design Introduction October 8, 2014 Eb, [db] Very Short Codes N PB =10 4 R =1/ Sphere Packing Bound 1 Non-Achievable Region k [bits]

30 Page 8 15/158 G. Liva Short Codes Design Introduction October 8, 2014 Eb, [db] Very Short Codes Short Codes N PB =10 4 R =1/ Sphere Packing Bound 1 Non-Achievable Region k [bits]

31 Page 8 16/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N PB =10 4 R =1/ Sphere Packing Bound 1 Non-Achievable Region k [bits]

32 Page 8 17/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code QR Code PB =10 4 R =1/2 4 ebch Code 3 2 Sphere Packing Bound 1 Non-Achievable Region k [bits]

33 Page 8 18/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 3 ebch Code 2 Sphere Packing Bound 1 Non-Achievable Region k [bits]

34 Page 8 19/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 3 ebch Code 2 Sphere Packing Bound CCSDS Turbo Codes 1 Non-Achievable Region k [bits]

35 Page 8 20/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 3 ebch Code 2 Sphere Packing Bound CCSDS Turbo/LDPC Codes CCSDS Turbo Codes 1 Non-Achievable Region k [bits]

36 Page 8 21/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 CCSDS LDPC Codes (Up-Link) ebch Code 3 2 Sphere Packing Bound CCSDS Turbo/LDPC Codes CCSDS Turbo Codes 1 Non-Achievable Region k [bits]

37 Page 8 22/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 CCSDS LDPC Codes (Up-Link) ebch Code 3 DVB-RCS2 Turbo Code 2 Sphere Packing Bound CCSDS Turbo/LDPC Codes CCSDS Turbo Codes 1 Non-Achievable Region k [bits]

38 Page 8 23/158 G. Liva Short Codes Design Introduction October 8, 2014 Very Short Codes Short Codes Long Codes Eb, [db] N Golay Code (7, 1/2) Conv. Codes QR Code PB =10 4 R =1/2 4 CCSDS LDPC Codes (Up-Link) ebch Code 3 DVB-RCS2 F256 Turbo/LDPC Codes 2 Sphere Packing Bound CCSDS Turbo/LDPC Codes CCSDS Turbo Codes 1 Non-Achievable Region k [bits]

39 Outline 1 Introduction Motivations Preliminaries 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

40 Page 24/158 G. Liva Short Codes Design Introduction October 8, 2014 Channel Model Memory-less binary-input additive white Gaussian noise (biawgn) channel Block length n (bits), code dimension k (bits), R = k/n X 2 X = { 1, +1} Y 2 R E s = 1 N N N 0, 2 E b N 0 = 1 2R 2

41 Page 25/158 G. Liva Short Codes Design Introduction October 8, 2014 Maximum-Likelihood Decoding The block-wise maximum-likelihood (ML) decoding criterion decides for the codeword maximizing the likelihood function p(y x), i.e., ˆx = arg max p(y x) x C Over the biawgn channel, it reduces to finding the codeword a minimum Euclidean distance from the observation Voronoi regions in R n

42 Page 26/158 G. Liva Short Codes Design Introduction October 8, 2014 Complete vs. Incomplete Decoding The ML decoding rule leads to complete decoding algorithms For short codes, it is often preferable to have incomplete decoding algorithms to reduce the probability of undetected errors 9 (here, the use of error detection codes is not an option due to overhead) Though, we shall accept a loss w.r.t. ML decoding... Voronoi regions in R n Decision regions of the iterative decoder 9 S. Dolinar et al. Bounds on error probability of block codes with bounded-angle maximum-likelihood incomplete decoding. In: International Symposium on Information Theory and Its Applications

43 Page 27/158 G. Liva Short Codes Design Introduction October 8, 2014 Binary Linear Block Codes A (n, k) binary linear block code C can be defined by its k n binary generator matrix G, whose k rows span the k-dimensional code space Alternatively, it may be defined through its (n k) n binary parity-check matrix H via the equation xh T = 0 where 0 is the n-elements all-zero vector The parity-check matrix rows span the subspace orthogonal to C. We denote such subspace as C. Thus, we may write x v x C, v C. The vectors in C form a linear block code of dimension n k, referred to as dual code of C

44 Page 28/158 G. Liva Short Codes Design Introduction October 8, 2014 Turbo Codes Parallel Concatenation Ensembles For given component codes and for a given interleaver size k, the turbo code ensemble C k = {C 1, C 2,... } is the set of codes obtained by selecting all possible interleavers with uniform probability u D D First RSC Encoder p (1) u 0 D D Second RSC Encoder p (2)

45 Page 29/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Graphical Representation of the Parity-Check Matrix The H imposes a set of n k constraints to the n codeword bits Graphical representation via Tanner graphs 10 Codeword bits variable nodes (VNs) V i, i = 1,..., n Check equations check nodes (CNs) C i, i = 1,..., n k An edge connects V i to C j if and only if h j,i = 1 C1 C2 C3 H = check nodes variable nodes V1 V2 V3 V4 V5 V6 10 M. Tanner. A Recursive Approach to Low Complexity Codes. In: IEEE Trans. Inf. Theory (1981).

46 Page 30/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Graphical Representation of the Parity-Check Matrix Graphical representation via Tanner graphs (cont d) Degree of a node = number of edges attached to it Length-l cycle = closed path composed by l edges, with each node in the cycle visited only once Girth of a graph = length of the shortest cycle C1 C2 C3 check nodes length 6 cycle degree= 2 variable nodes V1 V2 V3 V4 V5 V6

47 Page 31/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Regular LDPC Codes Ensemble The C (dv,d c ),n regular unstructured LDPC ensemble is the set of binary linear block codes defined by a Tanner graph with n variable nodes, having degree d v nd v /d c check nodes with degree d c Edge permutation picked uniformly at random C1 C2 C3 C4 C5 C6 C7 C8 C9 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12

48 Page 32/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Irregular LDPC Codes Ensemble An irregular LDPC code graph is often characterized by the distributions of degrees of variable and check nodes The node-oriented variable node degree distribution is denoted by Λ = {Λ i }, i = 1,..., d v,max where Λ i is the fraction of VNs with degree i The node-oriented check node degree distribution is denoted by P = {P j }, j = 1,..., d c,max where P j is the fraction of CNs with degree j C1 C2 C3 C4 C5 degree-5 degree-4 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

49 Page 33/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Irregular LDPC Codes Ensemble Alternatively, edge-oriented degree distributions can be defined The edge-oriented variable node degree distribution is denoted by λ = {λ i }, i = 1,..., d v,max where λ i is the fraction of edges connected to VNs with degree i The edge-oriented check node degree distribution is denoted by ρ = {ρ j }, j = 1,..., d c,max where ρ j is the fraction of edges connected to CNs with degree j C1 C2 C3 C4 C5 degree-5 λ i = iλ i l lλ l, ρ i = ip i l lp l degree-4 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

50 Page 34/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Irregular LDPC Codes Ensemble Degree distributions are often given in polynomial form Λ(x) = i Λ i x i and P(x) = j P j x j λ(x) = i λ i x i 1 and ρ(x) = j ρ j x j 1 We have that λ(x) = Λ (x) Λ (1) and ρ(x) = P (x) P (1) with Λ (x) = dλ(x) dx and P (x) = dp(x) dx.

51 Page 35/158 G. Liva Short Codes Design Introduction October 8, 2014 Low-Density Parity-Check Codes Irregular LDPC Codes Ensemble The C (Λ,P),n irregular unstructured LDPC ensemble is the set of binary linear block codes defined by a Tanner graph with n variable nodes whose degrees are distributed according to Λ m check nodes with degrees are according to P Edge permutation picked uniformly at random C1 C2 C3 C4 C5 degree-5 degree-4 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

52 Page 36/158 G. Liva Short Codes Design Introduction October 8, 2014 Finite-Length Bounds We consider upper and lower bounds as benchmarks Shannon s 1959 sphere-packing lower bound ([Sha59]) on the block error probability of (n, M = 2 k ) codes with equal-energy codewords Gallager s random coding upper bound ([Gal68]) on the average block error probability of (n, M = 2 k ) binary codes See also [DDP98; SS06; PPV10] Block Error Rate 10 0 (3,6) Gallager LDPC Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

53 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

54 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes Moderate-length Codes (k = 1024) Short Codes (k = 256) Very Short Codes (k = 64) 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

55 Page 37/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC Gallager s (3, 6) Code R. Gallager. Low-density parity-check codes Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

56 Page 38/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Accumulator-based Protograph LDPC Codes D. Divsalar et al. Capacity-approaching protograph codes. In: IEEE JSAC (2009) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

57 Page 39/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Protograph Raptor Codes T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes URL: Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

58 Page 40/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) 16-States Turbo Codes with Dithered Relative Prime Interleaver S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

59 Page 41/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Protograph-based Generalized LDPC Codes G. Liva, W. Ryan, and M. Chiani. Quasi-cyclic Generalized LDPC codes with low error floors. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

60 Page 42/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Ultra-Sparse LDPC Codes over F 256 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, dc )-LDPC codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

61 Page 43/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Moderate-length Codes Dimension k = 1024 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Memory-1 Turbo Codes over F 256 G. Liva et al. Short Turbo Codes over High Order Fields. In: IEEE Trans. Commun. (2013) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

62 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes Moderate-length Codes (k = 1024) Short Codes (k = 256) Very Short Codes (k = 64) 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

63 Page 44/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC Gallager s (3, 6) Code R. Gallager. Low-density parity-check codes Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

64 Page 45/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Accumulator-based Protograph LDPC Codes D. Divsalar et al. Capacity-approaching protograph codes. In: IEEE JSAC (2009) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

65 Page 46/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Protograph Raptor Codes T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes URL: Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

66 Page 47/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) 16-States Turbo Codes with Dithered Relative Prime Interleaver S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

67 Page 48/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Protograph-based Generalized LDPC Codes G. Liva, W. Ryan, and M. Chiani. Quasi-cyclic Generalized LDPC codes with low error floors. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

68 Page 49/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Ultra-Sparse LDPC Codes over F 256 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, dc )-LDPC codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

69 Page 50/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Short Codes Dimension k = 256 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Memory-1 Turbo Codes over F 256 G. Liva et al. Short Turbo Codes over High Order Fields. In: IEEE Trans. Commun. (2013) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

70 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes Moderate-length Codes (k = 1024) Short Codes (k = 256) Very Short Codes (k = 64) 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

71 Page 51/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC Gallager s (3, 6) Code R. Gallager. Low-density parity-check codes Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

72 Page 52/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Accumulator-based Protograph LDPC Codes D. Divsalar et al. Capacity-approaching protograph codes. In: IEEE JSAC (2009) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

73 Page 53/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Protograph Raptor Codes T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes URL: Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

74 Page 54/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) 16-States Turbo Codes with Dithered Relative Prime Interleaver S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

75 Page 55/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Protograph-based Generalized LDPC Codes G. Liva, W. Ryan, and M. Chiani. Quasi-cyclic Generalized LDPC codes with low error floors. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

76 Page 56/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Ultra-Sparse LDPC Codes over F 256 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, dc )-LDPC codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

77 Page 57/158 G. Liva Short Codes Design Overview: (Some of the) Best Known Short Codes October 8, 2014 Very Short Codes Dimension k = 64 bits Binary LDPC Codes Binary Turbo Codes Product Codes Non-binary LDPC Codes Non-binary Turbo Codes Memory-1 Turbo Codes over F 256 G. Liva et al. Short Turbo Codes over High Order Fields. In: IEEE Trans. Commun. (2013) Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) 10 5 Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

78 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

79 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding Protograph-based Low-Density Parity-Check Codes Turbo Codes Product and Generalized Low-Density Parity-Check Codes 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

80 Page 58/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Unstructured vs. Structured Ensembles Basics Our definition of LDPC ensemble leads to graphs with limited (if any) structure Unstructured graphs are easy to analyze, but Are difficult to implement in actual (hardware) decoders Allow limited control on edge connectivity properties A refined definition of LDPC ensemble addressing the points above leads to structured LDPC ensembles Repeat-Accumulate-like Codes 11 Protograph Codes 12 Multi-Edge-Type Codes H. Jin, A. Khandekar, and R. McEliece. Irregular repeat-accumulate codes. In: Proc. IEEE Int. Symp. Turbo Codes and Related Topics J. Thorpe. Low-Density Parity-Check (LDPC) Codes Constructed from Protographs. IPN Progress Report T. Richardson and R. Urbanke. Multi-edge type LDPC codes. unpublished

81 Page 59/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles H= Unstructured LDPC Code H= Structured LDPC Code

82 Page 60/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Protograph: small Tanner graph used as template to build the actual code graph Equivalent representation: base matrix Type-1 CN Type-2 CN B = Type-1 VN Type-2 VN Type-3 VN

83 Page 61/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes A protograph can be used to construct a larger Tanner graph by a copy & permute procedure The larger Tanner graph defines the designed code First step: protograph is copied Q times Type-1 CNs Type-2 CNs Type-1 VNs Type-2 VNs Type-3 VNs B 0 = B A

84 Page 62/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Second step: permute edges among the replicas Permutations shall avoid parallel edges Type-1 CNs Type-2 CNs Type-1 VNs Type-2 VNs Type-3 VNs H = B A

85 Page 63/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Second step: permute edges among the replicas Permutations shall avoid parallel edges Type-1 CNs Type-2 CNs 1,1 1,2 2,1 2,2 2,3 Type-1 VNs VNs Type-2 Type-3 VNs H = B A A protograph defines structured LDPC code ensemble: The iterative decoding threshold and distance properties follow from the protograph

86 Page 63/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Second step: permute edges among the replicas Permutations shall avoid parallel edges Type-1 CNs Type-2 CNs 1,1 1,2 2,1 2,2 2,3 Type-1 VNs VNs Type-2 Type-3 VNs H = B A A protograph defines structured LDPC code ensemble: The iterative decoding threshold and distance properties follow from the protograph

87 Page 64/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Depending on code length, the expansion can be done in more steps In each step, girth optimization techniques 1415 are used The final expansion is usually performed by means of circulant permutation matrices (quasi-cyclic code) 1617 Type-1 CNs Type-2 CNs Type-1 VNs Type-2 VNs Type-3 VNs H = B A X.-Y. Hu, E. Eleftheriou, and D.M. Arnold. Regular and irregular progressive edge-growth Tanner graphs. In: IEEE Trans. Inf. Theory (2005). 15 T. Tian et al. Selective Avoidance of Cycles in Irregular LDPC Code Construction. In: IEEE Trans. Commun. (2004). 16 W. Ryan and S. Lin. Channel codes Classical and modern. Cambridge Univ. Press, D. Declercq, M. Fossorier, and E. Biglieri. Channel Coding: Theory, Algorithms, and Applications. Academic Press Library, 2014.

88 Page 65/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Depending on code length, the expansion can be done in more steps In each step, girth optimization techniques 1819 are used The final expansion is usually performed by means of circulant permutation matrices (quasi-cyclic code) 2021 Type-1 CNs Type-2 CNs Type-1 VNs Type-2 VNs Type-3 VNs H = X.-Y. Hu, E. Eleftheriou, and D.M. Arnold. Regular and irregular progressive edge-growth Tanner graphs. In: IEEE Trans. Inf. Theory (2005). 19 T. Tian et al. Selective Avoidance of Cycles in Irregular LDPC Code Construction. In: IEEE Trans. Commun. (2004). 20 W. Ryan and S. Lin. Channel codes Classical and modern. Cambridge Univ. Press, D. Declercq, M. Fossorier, and E. Biglieri. Channel Coding: Theory, Algorithms, and Applications. Academic Press Library, 2014.

89 Page 66/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Punctured (state) and degree-1 variable nodes are allowed, while for unstructured designs they may prevent the iterative decoding convergence process to succeed W.r.t. unstructured ensembles, near-capacity thresholds can be achieved with lower average degrees larger girth Example: Accumulate-Repeat-3-Accumlate (AR3A), R = 1/2, (E b /N 0 ) = db, only 0.3 db from Shannon limit Punctured VN

90 Page 67/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Structured Ensembles Protograph Codes Protograph ensembles allow controlling in finer way the edge connectivity properties of the codes 22 Example: consider the based matrices B 1 = B 2 = They describe sub-ensembles of the unstructured ensemble defined by Λ(x) = 0.2x + 0.4x x x 5, P(x) = 0.333x x 5 with degree-5 VNs punctured. However, (E b /N 0 ) 1 = db while (E b /N 0 ) 2 = + db. 22 G. Liva and M. Chiani. Protograph LDPC codes design based on EXIT analysis. In: Proc. IEEE Global Telecommun. Conf

91 Page 68/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis B = I ch : channel mutual information (MI) at the input of the j-th protograph VN I Ev (i, j): the MI between the message sent by V j to C i and the associated bit, on one of the b i,j edges connecting V j to C i IEv(1, 1) IEc(2, 3) Ich Ich Ich I Ec (i, j): the MI between the message sent by C i to V j and the associated codeword bit, on one of the b i,j edges connecting C i to V j I APP (j): the MI between the APP LLR and the associated codeword bit

92 Page 69/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis Each message (LLR) is modeled as a r.v. M with Gaussian distribution conditioned on the associated codeword symbol r.v. X ( ( )) 1 1 p(m ± 1) = exp m σ2 m 2πσ 2 m 2 We denote by J(σ m) := I(M; X) Note that J(x) = 1 + e (y x 2 /2) 2 2x 2 2πx 2 2σ 2 m log 2 ( 1 + e y ) dy.

93 Page 70/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis The EXIT recursions of unstructured LDPC ensembles 1 VN EXIT function 0.9 Ī Ev = d ( λ d J (J 1 (I ch )) CN EXIT function 0.6 +(d 1) ( J 1 ( Ī Av )) 2 IE,V, IA,C 0.5 Ī Ec 1 d ( ρ d J (d 1) ( J ( ) )) ĪAc Eb/N0 =1.1 [db] 0.1 need to be modified to account for the protograph structure IA,V, IE,C

94 Page 71/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis B = Initialize I ch = J (σ ch ) with σ 2 ch = 8R E b N 0 For punctured VNs, obviously I ch = 0 IEv(1, 1) IEc(2, 3) Ich Ich Ich

95 Page 72/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis B = IEv(1, 1) IEc(2, 3) Variable to check update j, i, if b i,j 0, I Ev (i, j) ( J b s,j [J 1 (I Av (s, j))] 2 + s i ( + (b i,j 1) [J 1 (I Av (i, j))] 2 + [J 1 I (j) ch )] 2 ) Ich Ich Ich j, i set I Ac (i, j) = I Ev (i, j)

96 Page 73/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis B = IEv(1, 1) IEc(2, 3) Check to variable update j, i, if b i,j 0, I Ec (i, j) ( 1 J b i,s [J 1 (1 I Ac (i, s))] 2 + s j ) [ ] 2 + (b i,j 1) J 1 (1 I Ac (i, j)) Ich Ich Ich j, i, set I Av (i, j) = I Ec (i, j)

97 Page 74/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 LDPC Codes: Protograph Ensembles Extrinsic Information Transfer Analysis B = IEv(1, 1) IEc(2, 3) APP-LLR mutual information evaluation ( ( I APP (j) J b s,j [J 1 (I Av (i, j))] 2 + [J 1 s I (j) ch )] 2 ) The steps are iterated until I APP (j) 1, j, or a maximum number of iterations (I max) is reached Ich Ich Ich The iterative decoding threshold, (E b /N 0 ), is the smallest signal-to-noise ratio such that I APP (j) 1 j

98 Page 75/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Repeat-Accumulate Accumulator-based LDPC codes admit simple protograph representation Very low decoding thresholds with low average degrees large girth A clear example is given by repeat-accumulate (RA) codes 23 Information bits Repetition-3 (Tail-biting) Accumulator Parity bits 23 D. Divsalar et al. Constructing LDPC codes from simple loop-free encoding modules. In: Proc. IEEE Int. Conf. Commun. (ICC)

99 Page 76/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Example: AR3A Protograph Accumulate-Repeat-3-Accumulate (AR3A) Protograph 24 Punctured VN d min Threshold, (E b /N 0 ) Encoding O(log n) db O(n) 24 D. Divsalar et al. Capacity-approaching protograph codes. In: IEEE JSAC (2009).

100 Page 77/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Example: AR4A Protograph Accumulate-Repeat-4-Accumulate (AR4A) Protograph 25 Punctured VN d min Threshold, (E b /N 0 ) Encoding O(log n) db O(n) 25 D. Divsalar et al. Capacity-approaching protograph codes. In: IEEE JSAC (2009).

101 Page 78/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based ARA Protograph Thresholds R AR3A, (E b /N 0 ) AR4A, (E b /N 0 ) Shannon Limit 7/ db db db 4/ db db db 3/ db db db 2/ db db db 1/ db db db 1/ db db db 1/ db db db 1/ db db db 1/ db db db 1/ db db db

102 Page 79/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Ensembles with Positive Typical Minimum Distance ARA codes suffer for d min growing sublinearly with n By limiting the number of degree-2 VNs in the protograph to be strictly less than the number of parity VNs, it is possible to guarantee a linear growth of d min Accumulate-Repeat-Jagged-Accumulate (ARJA) codes 26 d min (E b /N 0 ) Encoding O(n) db O ( n 2) Punctured VN 26 D. Divsalar, S. Dolinar, and C. Jones. Construction of protograph LDPC codes with linear minimum distance. In: IEEE International Symposium on Information Theory

103 Page 80/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Dimension k = 1024 bits (3,6) Gallager LDPC ARJA AR3A 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

104 Page 81/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Dimension k = 256 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

105 Page 82/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Accumulator-based Dimension k = 64 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

106 Page 83/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Raptor-like Serial concatenation of a high-rate protograph-based outer LDPC code, and a protograph-based Luby-Transform code 27 ( Bo 0 B = Although the construction targets short block lengths, the outer code parity-check matrix density prevents from obtaining large girths at very short block lengths B LT ) 27 T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes URL:

107 Page 84/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Raptor-like Large flexibility of rates, with thresholds within 0.5 db from the Shannon limit B = B A R (E b /N 0 ) Shannon Limit 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db 6/ db db Other constructions 28 allow to trade threshold for better error floor performance 28 T.-Y. Chen et al. Protograph-Based Raptor-Like LDPC Codes URL:

108 Page 85/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Raptor-like Dimension k = 1024 bits (3,6) Gallager LDPC ARJA AR3A Protograph Raptor 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

109 Page 86/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Raptor-like Dimension k = 256 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

110 Page 87/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Protograph Ensembles: Raptor-like Dimension k = 64 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

111 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding Protograph-based Low-Density Parity-Check Codes Turbo Codes Product and Generalized Low-Density Parity-Check Codes 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

112 Page 88/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Turbo codes, in their parallel concatenation form, are excellent candidates in the short block length regime Turbo codes with 16-states component codes provide the best trade-off between minimum distance and decoding threshold 2930 Interleaver design is crucial 31 FB/FFW Polynomial (Octal) (E b /N 0 ), R = 1/2 Notes 27/ db 16-states 23/ db 16-states 15/ db 8-states 29 C. Berrou, A. Glavieux, and P. Thitimajshima. Near Shannon limit error-correcting coding and decoding: Turbo-codes. In: Proc. IEEE Int. Conf. Commun. (ICC) H. El-Gamal and Jr. Hammons AR. Analyzing the turbo decoder using the Gaussian approximation. In: IEEE Trans. Inf. Theory (2001). 31 S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC

113 Page 89/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Factor Graph Turbo codes factor graphs 32 are characterized by large girths 32 N. Wiberg. Codes and Decoding on General Graphs. PhD thesis. Linköping University, 1996.

114 Page 90/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Interleavers The interleaver is the main responsible for a large girth and for a large spread (essential for large d min ) Still, d min = O(log n) Among the best-known constructions Dithered-Relative-Prime (DRP) 33 Quadratic permutation polynomial (QPP) - LTE S. Crozier and P. Guinand. High-performance low-memory interleaver banks for turbo-codes. In: IEEE VTC O. Takeshita. On maximum contention-free interleavers and permutation polynomials over integer rings. In: IEEE Trans. Inf. Theory (2006).

115 Page 91/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Dithered Relative Prime Interleavers Relative-prime interleavers are characterized by excellent spread Though, they are too regular large multiplicity of minimum-weight codewords local dither to reduce multiplicity M bits 1 i 2 i 3 i 4 i 5 i... k/m i Relative-prime interleaver 1 o 2 o 3 o 4 o 5 o... o k/m Input word Input dither Output dither Output word

116 Page 92/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Dimension k = 1024 bits (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

117 Page 93/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Dimension k = 256 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

118 Page 94/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 The Parallel Concatenated Convolutional Code Ensemble Dimension k = 64 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

119 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding Protograph-based Low-Density Parity-Check Codes Turbo Codes Product and Generalized Low-Density Parity-Check Codes 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

120 Page 95/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Product Codes Basics Product Codes 35 are the ancestors of any iterative coding scheme Simplest case: 2-dimensional product code C = C 1 C 2 Obtained from two binary linear block codes C 1, C 2 with parameters (n 1, k 1 ) and (n 2, k 2 ) k Information bits are organized in a k 2 k 1 array U (with k 1 k 2 = k) Each row of U is then encoded via the (n 1, k 1 ) binary linear block code C 1 The resulting k 2 n 1 array is then encoded column-wise through an (n 2, k 2 ) binary linear block code C 2, leading to an n 2 n 1 array C with the structure [ C = U P (1) P (2) P (12) ] 35 P. Elias. Error-free coding. In: IRE Trans. on Inform. Theory (1954).

121 Page 96/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Product Codes Graphical Representation Decoding takes place iteratively, row and column-wise (block turbo codes) Component code SISO decoding based on its trellis (BCJR) or on modified-chase algorithms k2 k1 information array U n2 n1 codeword array C Column Check Nodes codeword of C parity of C2, P (2) parity of C1, P (1) checks-on-checks, P (1,2) Row Check Nodes codeword of C2

122 Page 97/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Product Codes Distance Properties Code parameters: n = n 1 n 2 k = k 1 k 2 R = (k 1 /n 1 )(k 2 /n 2 ) = R 1 R 2 The minimum distance is d min = d 1 d 2 The multiplicity of codeword with minimum Hamming weight is given by A min = A (1) d 1 A (2) d 2, i.e., the minimum distance multiplicity of the product code is the product of the minimum distance multiplicities of the component codes!

123 Page 98/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Product Codes Distance Properties Example: (1024, 676) product code based a (32, 26) extended Hamming code for both rows and columns (IEEE ) The (32, 26) extended Hamming code has weight enumerator function (WEF) given by A 1 (X) = X X X X X X X X X X X X X 28 + X 32. Component codes minimum distance is d 1 = d 2 = 4 resulting in a product code minimum distance d min = 16 The multiplicity of codewords with Hamming weight 16 is A min = =

124 Page 99/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Product Codes Distance Properties The high multiplicity of minimum-weight codewords leads to high error floors P B 1 2 A minerfc ( d min R E b N 0 ) (n, k) C 1 C 2 d min A min (256, 121) eh (16, 11) eh (16, 11) (256, 165) eh (16, 11) SPC (16, 15) (256, 225) SPC (16, 15) SPC (16, 15) (1024, 676) eh (32, 26) eh (32, 26) (1024, 806) eh (32, 26) SPC (32, 31) (1024, 961) SPC (32, 31) SPC (32, 31) The large multiplicity is mainly due to the highly-regular graph structure...

125 Page 100/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Basics Generalized LDPC (G-LDPC) 3637 codes extends both LDPC and product code constructions Tanner graph includes generalized check nodes based on arbitrary linear block codes Ha = A Hb = C1 C2 C3 C4 C5 single-parity-check Ca Cb Ca V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 36 M. Tanner. A Recursive Approach to Low Complexity Codes. In: IEEE Trans. Inf. Theory (1981). 37 M. Lentmaier and K. S. Zigangirov. Iterative decoding of generalized low-density parity-check codes. In: Proc. IEEE Int. Symp. Inf. Theory (ISIT)

126 Page 101/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Structured Designs One can target G-LDPC codes having same component codes as product codes, but with a graph structure that prevents large minimum weight multiplicity 38 Protograph G-LDPC codes 39 as protograph LDPC codes, but with generalized check nodes (32,6) extended Hamming code (32,6) extended Hamming code 38 M. Lentmaier et al. From Product Codes to Structured Generalized LDPC Codes. In: Proc. Chinacom G. Liva, W. Ryan, and M. Chiani. Quasi-cyclic Generalized LDPC codes with low error floors. In: IEEE Trans. Commun. (2008).

127 Page 102/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Example - Union Bounds Product and G-LDPC codes based on (32, 26) extended Hamming component codes For the G-LDPC code it is possible to design a graph leading to the same girth and minimum distance of the product code Two G-LDPC ensembles: The complete ensemble, and the one expurgated from all codes having d min < 16 It is possible to show that at least 50% of the codes in the original ensemble have d min = 16 or more Block Error Rate Truncated UB, Product Code UB G-LDPC Expurgated Ensemble UB G-LDPC Non-Expurgated Ensemble Divsalar Bound, G-LDPC Expurgated Ensemble Eb/N0 [db]

128 Page 103/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Example Product and G-LDPC codes based on (32, 26) extended Hamming component codes For the G-LDPC code graph, girth equal to the product code one G-LDPC obtained by circulant permutation matrix expansion (quasi-cyclic code) Block Error Rate Product Code G-LDPC Code Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

129 Page 104/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Dimension k = 1024 bits (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

130 Page 105/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Dimension k = 256 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

131 Page 106/158 G. Liva Short Codes Design Binary Codes for Iterative Decoding October 8, 2014 Generalized LDPC Codes Dimension k = 64 bits (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

132 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

133 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding Low-Density Parity-Check Codes Parallel Concatenated Convolutional Codes 5 Beyond Iterative Decoding 6 Summary and Open Challenges

134 Page 107/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary LDPC Codes Basics Defined 40 by M N sparse parity-check matrix H via the equation xh T = 0 where x F n q and the elements of H belong to F q, q > 2 We are mainly interested in q = 2 p Example: parity-check matrix over F 16 [ ] α 3 α 7 α 3 0 H = α 0 0 α 12 α 11 where the field elements are expressed as powers of the primitive element α and where the field primitive polynomial is p(x) = 1 + x + x 4 40 M. Davey and D. MacKay. Low density parity check codes over GF(q). In: IEEE Commun. Lett. (1998).

135 Page 108/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary LDPC Codes Basics Tanner graph representation Codeword symbols variable nodes (VNs) V i, i = 1,..., N Non-Binary Check equations check nodes (CNs) C i, i = 1,..., M An edge connects V i to C j if and only if h j,i 0 C1 C2 C3 H = α 2 α 0 0 α α 5 0 α 11 0 α α 2 α α 10 check nodes variable nodes V1 V2 V3 V4 V5 V6 Binary image of the code: in the codeword, replace each symbol in F q with its length-p, p = log 2 q, binary representation Next, we will always refer to the binary block length n = Np bits

136 Page 109/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary LDPC Codes Belief Propagation Decoding Messages are vectors of q probabilities (probability mass functions, p.m.f.s) At a degree-d CN, the p.m.f.s incoming on d 1 edges are convolved to produce the extrinsic estimate (p.m.f.) of the remaining edge Decoding complexity scales as quadratically in q. It can be reduced to O(q log 2 q) by performing the CN operation via fast Fourier transform 41 ˆm out 1 ˆm out 2 ˆm out 3 = ˆm out 1 ˆm out 2 m in 1 m in 2 m out 3 = m in 1 m in 2 m in 1 F F F 1 m in 2 m out 3 m = p(0),p(1),p( ),p( 2 ),...,p( q 1 ) 41 L. Barnault and D. Declercq. Fast decoding algorithm for LDPC over GF(2 q ). In: Proc. IEEE Inf. Theory Workshop (ITW)

137 Page 110/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary LDPC Codes Ultra-Sparse Ensembles: Cycle LDPC Codes Non-Binary LDPC codes constructed on relatively-large fields (q 64) perform particularly well when the graph is ultra-sparse 42 Constant variable node degree d v = 2 Uniform check node degree Ultra-sparse LDPC Codes are actually instances of cycle codes 43 C1 C2 C3 C4 C2 C3 girth 2 girth 2 C1 C4 Tanner graph perspective Cycle (circuit) code perspective 42 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, d c)-ldpc codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008). 43 S. Hakimi and J. Bredeson. Graph theoretic error-correcting codes. In: IEEE Trans. Inf. Theory (1968).

138 Page 111/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Graphs and Cages Thanks to the sparse nature of the graph, large girths can be attained even for short block lengths Optimum graphs can be either found in literature 44, or constructed by means of progressive-edge-growth techniques P.K. Wong. Cages A survey. In: J. Graph Theory (1982). 45 A. Venkiah, D. Declercq, and C. Poulliat. Design of Cages with a Randomized Progressive Edge Growth Algorithm. In: IEEE Commun. Lett. (2008). 46 W. Chen, C. Poulliat, and D. Declercq. Structured High-Girth Non-Binary Cycle Codes. In: Proc. of APCC

139 Page 112/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Graphs and Cages A graph G is composed by a set V = {v i } of m vertices and by a set E = {e j } of n edges The order of a graph is its number m of vertices For a (v, g)-graph, where g is the girth, with m 0 (v, g) = m m 0 (v, g) v(v 1) r 2 v 2 if g = 2r + 1, 2(v 1) r 2 v 2 if g = 2r. The (v, g)-graph of smallest order is referred to as (v, g)-cage In general the bound is not achieved with equality even for cages When the bound is achieved with equality, the (v, g)-cage is referred to as Moore (v, g)-graph

140 Page 113/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Graphs and Cages (a) The Robertson graph. (b) The Balaban-11 cage. (c) The Petersen graph. (d) The Coxeter graph.

141 Page 114/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Graphs and Cages Example: (120, 40) Non-Binary LDPC Code over F 256 based on the Petersen graph GPP Turbo Code Turbo Code, GF(256) The code turns to be actually a rate-1/3 non-binary turbo code G. Liva et al. Short Turbo Codes over High Order Fields. In: IEEE Trans. Commun. (2013) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

142 Page 115/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Distance Properties The cycle code ensemble is characterized by d min = O(log n) independently from the field order Beyond girth optimization, the minimum distance of the binary image of the code can be kept sufficiently large by a proper choice of the parity-check matrix coefficients

143 Page 116/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization Choosing the coefficients in a parity-check matrix which optimizes the minimum distance of the binary image of the code is a formidable task Less complex (sub-optimum): optimize the coefficients row-wise 47 Example: Parity-check matrix over F 256, p(x) = x 8 + x 4 + x 3 + x [ ] α 8 α 80 α 0 0 H = α 0 α 41 α 122 α 113 The first row imposes the equation c 1 α 8 + c 2 α 80 + c 3 α 0 = 0, i.e. [c 1, c 2, c 3 ] h T 1 = 0 h 1 = [α 8, α 80, α 0] (non-binary SPC code) 47 C. Poulliat, M. Fossorier, and D. Declercq. Design of regular (2, d c)-ldpc codes over GF(q) using their binary images. In: IEEE Trans. Commun. (2008).

144 Page 117/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization Let us analyze the binary code associated with the non-binary SPC code with parity-check matrix h 1 = [ α 8, α 80, α 0] Companion matrix of p(x) = 8 i=0 p ix i = x 8 + x 4 + x 3 + x M = p 0 p 1 p 2 p 3 p 4 p 5 p 6 p 7 = Denote by c 1 the 8-bits representation of the symbol c 1, and by s 1 the 8-bits binary image of s 1 = c 1 α i. Then s 1 = c 1 M i

145 Page 118/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization The binary code associated with the non-binary SPC code having h 1 = [ α 8, α 80, α 0] has parity-check matrix given by [ ] H b r = M 8 M 80 I i.e H b r = This is the parity-check matrix of a (24, 16) binary linear block code

146 Page 119/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization Distance spectrum of the code with parity-check matrix given by [ ] H b r = M 8 M 80 I via MacWilliams identity 48 A(x) = 1 ( ) 1 x 2 (1 + n k x)n B 1 + x where A(x) is the WEF we are interested in, and B(x) is the one of the dual code (having H b r as generator matrix) 48 J. MacWilliams and N. Sloane. The theory of error-correcting codes. North Holland Mathematical Libray, 1977.

147 Page 120/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization Distance spectrum of the code with parity-check matrix given by [ ] H b r = M 8 M 80 I is A(x) = x x x x x x x x x x x x x x x x x x 21 + x 23. Thus, d min = 4 and A min = 40

148 Page 121/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Coefficient Optimization List of good row coefficients, F 256, check node degree 3 Coefficients, [ α i α j α 0] d r,min A r,min [α 8 α 183 α 0] 4 36 [ α 10 α 82 α 0] 4 37 [ α 16 α 167 α 0] 4 37 [ α 41 α 127 α 0] 4 37 [ α 41 α 128 α 0] 4 37 [ α 42 α 128 α 0] 4 37 [ α 72 α 80 α 0] 4 36 [ α 72 α 245 α 0] 4 37 [ α 86 α 213 α 0] 4 37 [ α 86 α 214 α 0] 4 37

149 Page 122/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Example: New CCSDS Telecommand Standard (128, 64) code for satellite (CCSDS) telecommand Requirement: very low undetected error rate Competitor: binary protograph code α α α α 0 0 α α α α α α α α H = α α α α α α 80 α α α 169 α α α α α α α α α α α 0

150 Page 123/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Example: New CCSDS Telecommand Standard 10 0 Protograph (NASA-CCSDS) LDPC Code, GF(256) 10 2 Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

151 Page 124/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Low-Rate via Multiplicative Repetition Non-Binary LDPC codes admit large flexibility in the code rate A low-rate non-binary LDPC code can be obtained from a higher-rate one by simply repeating some codeword symbols, and multiply them by a randomly-picked non-zero field element Rate-1/3 code graph K. Kasai et al. Multiplicatively repeated nonbinary LDPC codes. In: IEEE Trans. Inf. Theory (2011) Rate-1/9 code graph Non-binary LDPC codes are almost rate-less

152 Page 125/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Cycle LDPC Codes Example: Low-Rate via Multiplicative Repetition k = 1024 codes Non-binary LDPC over F 256, mother code has rate R = 1/ AR4A, R =1/6 G-LDPC Code, R =1/4 G-LDPC Code, R =1/6 LDPC Code GF(256), R =1/6 Block Error Rate Sphere-Packing Bound, R = 1/4 Random Coding Bound, R = 1/6 Sphere-Packing Bound, R = 1/6 Random Coding Bound, R = 1/ Eb/N0 [db]

153 Page 126/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Protographs for Moderate-Order Fields For moderate-order fields (e.g., 4 q 32), the cycle code construction tends to exhibit high error floors and poor thresholds In 49, protograph-based non-binary LDPC codes have been proposed for moderate field orders Example: rate-1/2 protograph over F 16, (E b /N 0 ) = 0.35 db 49 L. Dolecek et al. Non-Binary Protograph-Based LDPC Codes: Enumerators, Analysis, and Designs. In: IEEE Trans. Inf. Theory (2014).

154 Page 127/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Protographs for Moderate-Order Fields Non-binary LDPC code over F 16 with protograph LDPC Code, GF(256) Turbo Code (16 states) Protograph LDPC Code, GF(16) 10 2 Block Error Rate 10 3 Coefficients have not been optimized Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

155 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding Low-Density Parity-Check Codes Parallel Concatenated Convolutional Codes 5 Beyond Iterative Decoding 6 Summary and Open Challenges

156 Page 128/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary Turbo Codes Basics Turbo codes on high-order fields 50 can be constructed, with a structure that closely follows that on their LDPC codes counterpart 51 Memory-1 (in field symbols), time-variant convolutional codes Turbo encoder Detail of the RSCC encoder u fi f (1) g (1) D ui gi wi vi Si = pi D Si 1 = pi 1 Π g (2) f (2) p (1) pi D p (2) 50 J. Berkmann and C. Weiss. On dualizing trellis-based APP decoding algorithms. In: IEEE Trans. Commun. (2002). 51 G. Liva et al. Short Turbo Codes over High Order Fields. In: IEEE Trans. Commun. (2013).

157 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding Page 129/158 October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Decoding Decoding over the q-states trellis of the component codes q branches leaving/entering a state in each section Decoding complexity scales as O(q 2 ) Can be reduced to O(q log2 )by decoding through fast Fourier transform52 (n2, k2 ) binary linear code antipodal modulation C2 BPSK Si 0 sequence modulator c 2 Cq PC Fq Si 1 M x 2 X n en the conventional52product code encoder and perspective Fig. 2. On Example of a trellistrellis-based section for a non-binary code with parityj. Berkmann C. Weiss. dualizing APPSPC decoding algorithms. lation one. check matrix in the form (4). Trans. Commun. (2002). ML D ECODING OF P RODUCT C ODES code class introduced in Section II ef) ML decoding can be implemented by IV. S YMBOL - WISE MAP D ECODING In this section, we turn the decoding problem into a symbolwise MAP decoding for the class of product codes introduced In: IEEE

158 Page 130/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary Turbo Codes Memory-1 Time-Variant Recursive Convolutional Codes The codes admit a non-binary protograph LDPC code representation V0 C0 V1 C1 V2 accumulators on F q The protograph ensemble is a sub-ensemble of the unstructured (3, 2) regular LDPC ensemble Same distance properties d min = O(log n) Similar decoding threshold. E.g., on F 256 the rate 1/3 protograph ensemble possesses (E b /N 0 ) = db vs. (E b /N 0 ) = db of the unstructured one

159 Page 131/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Interleaver Design and Graph Interpretation For binary turbo codes, a large-spread regular interleaver is not an option, due to the large multiplicity of minimum-weight codewords that arises from the combination of the regular interleaver structure and of the periodic trellis for the component codes Thus, the interleaver shall sacrifice spread for irregularity (randomness) For non-binary turbo codes, the component codes do not have anymore a periodic trellis (time-variant FB/FFW coefficients), thus large-spread regular interleavers can be used C. Radebaugh, C. Powell, and R. Koetter. Wheel codes: Turbo-like codes on graphs of small order. In: Proc. IEEE Inf. Theory Workshop (ITW)

160 Page 132/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Interleaver Design and Graph Interpretation As for non-binary LDPC codes, non-binary turbo codes can be defined by the graph of a cycle code, with specific structure The graph structure directly specifies the interleaver Tail-biting trellis (1st component code) Information bits Petersen Graph (Cage) C2 C1 C0 C9 C0 C4 C3 C8 C3 C4 C7 C1 C5 C6 C2 Tail-biting trellis (2nd component code)

161 Page 133/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Interleaver Design and Graph Interpretation As for non-binary LDPC codes, non-binary turbo codes can be defined by a graph of a cycle code, with specific structure The graph structure directly specifies the interleaver Tail-biting trellis (1st component code) Information bits Petersen Graph (Cage) C2 C1 C0 C9 C0 C4 C3 C8 C3 C4 C7 C1 C5 C6 C2 Tail-biting trellis (2nd component code)

162 Page 134/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Interleaver Design and Graph Interpretation As for non-binary LDPC codes, non-binary turbo codes can be defined by a graph of a cycle code, with specific structure The graph structure directly specifies the interleaver Tail-biting trellis (1st component code) Information bits Petersen Graph (Cage) C2 C1 C0 C9 C0 C4 C3 C8 C3 C4 C7 C1 C5 C6 C2 Tail-biting trellis (2nd component code)

163 Page 135/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Turbo Codes based on Memory-1 Time-Variant RCCs Interleaver Design and Graph Interpretation As for non-binary LDPC codes, non-binary turbo codes can be defined by a graph of a cycle code, with specific structure The graph structure directly specifies the interleaver Tail-biting trellis (1st component code) Information bits Petersen Graph (Cage) C2 C1 C0 C9 C0 C4 C3 C8 C3 C4 C7 C1 C5 C6 C2 Tail-biting trellis (2nd component code)

164 Page 136/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary Turbo and LDPC Codes, F 256 Dimension k = 1024 bits (3,6) Gallager LDPC ARJA AR3A Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) Block Error Rate Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

165 Page 137/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary Turbo and LDPC Codes, F 256 Dimension k = 256 bits 10 0 Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

166 Page 138/158 G. Liva Short Codes Design Non-Binary Codes for Iterative Decoding October 8, 2014 Non-Binary Turbo and LDPC Codes, F 256 Dimension k = 64 bits 10 0 Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

167 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

168 Page 139/158 G. Liva Short Codes Design Beyond Iterative Decoding October 8, 2014 Ordered Statistics Decoding Principle Compute, on-the-fly, a subset of the candidate codewords among which the one at minimum Euclidean distance from the observation is selected 54 Complete decoding algorithm, though not maximum-likelihood Performance depends on the cardinality of the candidates subset, and on how the set is formed Can be applied in principle to any block code However, since complexity is not linear in n, application is limited to short codes 54 M. Fossorier and S. Lin. Soft-decision decoding of linear block codes based on ordered statistics. In: IEEE Trans. Inf. Theory (1995).

169 Page 140/158 G. Liva Short Codes Design Beyond Iterative Decoding October 8, 2014 Ordered Statistics Decoding Algorithms Example: Most Reliable Basis (MRB) 55. Given the order i, 1. Find the k most reliable channel observations in y and collect them in a vector v 2. Perform Gaussian elimination on the generator matrix G with respect to their positions, obtaining a systematic generator matrix G sys with respect to v 3. Encode c 0 = vg sys 4. Generate the possible error patterns of length k and weight w i 5. For the generic lth error pattern e l, encode c l = (v + e l )G sys 6. Select the codeword in {c l } at minimum Euclidean distance from y The complexity is linear in the number of error patters, N e = i l=0 ( k l ), and polynomial in k The larger i, the closer to ML, to higher the complexity. Typical values of i are [2, 4] 55 Y. Wu and C. Hadjicostis. Soft-decision decoding using ordered recodings on the most reliable basis. In: IEEE Trans. Inf. Theory (2007).

170 Page 141/158 G. Liva Short Codes Design Beyond Iterative Decoding October 8, 2014 Ordered Statistics Decoding Examples MRB decoding with i = 4 (128, 64) extended BCH code, d min = 22 Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) BCH Code, OSD Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

171 Page 142/158 G. Liva Short Codes Design Beyond Iterative Decoding October 8, 2014 Ordered Statistics Decoding Examples MRB decoding with i = 4 (128, 64) extended BCH code, d min = 22 (128, 64) non-binary LDPC code Block Error Rate (3,6) Gallager LDPC ARJA AR3A Protograph (NASA-CCSDS) Protograph Raptor Turbo Code (16 states) G-LDPC (Irregular) LDPC Code, GF(256) Turbo Code, GF(256) BCH Code, OSD LDPC Code, GD(256), OSD Sphere-Packing Bound Random Coding Bound Eb/N0 [db]

172 Page 143/158 G. Liva Short Codes Design Beyond Iterative Decoding October 8, 2014 Efficient Symbol-wise MAP Decoding Examples Performed by exploiting an efficient trellis representation available for certain product codes J.K. Wolf. Efficient maximum likelihood decoding of linear block codes using a trellis. In: IEEE Trans. Inf. Theory (1978) G. Liva, E. Paolini, and M. Chiani. On Optimum Decoding of Certain Product Codes. In: IEEE Commun. Lett. (2014) Block Error Rate Product Code - Iterative Product Code - Symbol-by-Symbol MAP Product Code - Truncated UB CC - Symbol-by-Symbol MAP CC Ensemble - Truncated UB Eb/N0 [db]

173 Outline 1 Introduction 2 Overview: (Some of the) Best Known Short Codes 3 Binary Codes for Iterative Decoding 4 Non-Binary Codes for Iterative Decoding 5 Beyond Iterative Decoding 6 Summary and Open Challenges

174 Page 144/158 G. Liva Short Codes Design Summary and Open Challenges October 8, 2014 The Short Code Design Toolkit Benchmarking Solutions Code Class Performance (Waterfall) Performance (Error floor) Complexity Flexibility (rates) Notes Binary LDPC Good (Tunable) Good (Tunable) Low Excellent - Binary Turbo Excellent? Low Good High-rate design? Product/G-LDPC Uninspiring Good (Tunable) Low Limited - Non-Binary Excellent Excellent High Excellent log-domain Turbo/LDPC decoding? In general, short code design still relies heavily on optimization of known code classes A good asymptotic decoding thresholds seems to be essential for good waterfall performance, but it shall be attained while maintaining the graph sparse (large girth) and reasonable minimum distance Non-binary LDPC and turbo codes satisfy these heuristic principles, and are nowadays probably the best solution from a performance view-point

175 Page 145/158 G. Liva Short Codes Design Summary and Open Challenges October 8, 2014 Open Challenges Just some In general, finite-length design tools that can be efficiently used for the design in the short/moderate length regime are missing. They shall be able to capture both the finite-length scaling of belief propagation, as well as the error floor behavior, with limited computational complexity states turbo codes have great potential at short blocks, but their error floor performance shall be improved, as well as their performance at high code rates 3. Regarding non-binary codes, would be of some importance to devise an efficient log-domain decoding algorithm (beyond extended min-sum algorithms) 4. This tutorial is quite iterative-centric: other solutions?

176 Page 146/158 G. Liva Short Codes Design Summary and Open Challenges October 8, 2014 Open Challenges Feedback? S Encoder Modulator Feed-back Policy Channel D Decoder Soft Demodulator