Finding Small Stopping Sets in the Tanner Graphs of LDPC Codes

Transcription

1 Finding Small Stopping Sets in the Tanner Graphs of LDPC Codes Gerd Richter University of Ulm, Department of TAIT Albert-Einstein-Allee 43, D Ulm, Germany Abstract The performance of low-density parity-check (LDPC) codes over the binary erasure channel for a low erasure probability is limited by small stopping sets. The frame error rate over the additive white Gaussian noise (AWGN) channel for a high signal-to-noise ratio is limited by small trapping sets and by the minimum distance of the LDPC code, which is equal to the codeword with the minimal Hamming weight. In this paper, we present a simple and fast algorithm based on the error impulse method to find small stopping sets in the Tanner graphs of LDPC codes. Since a codeword with small Hamming weight can be represented by a special stopping set, the algorithm can also be used to find the minimum distance of LDPC codes. Furthermore, we use the size of the small stopping sets and the minimum distance of the code to calculate asymptotic performance bounds for LDPC codes transmitted over the binary erasure and the AWGN channel, respectively and compare the bounds with simulations. 1 Introduction Low-density parity-check (LDPC) codes were originally invented by Gallager in 1963 [6]. He showed that LDPC codes are capable of reaching a performance close to the channel capacity at low complexity, when they are decoded by an iterative decoding algorithm, the so-called belief propagation or sum-product algorithm. After being forgotten for more than 30 years, they were rediscovered in 1996 by Mackay and Neal [12] and also by Wiberg [22]. Because of their good performance and their comparably low decoding complexity, they became serious competitors to turbo codes [1]. Gallager considered only regular LDPC codes, i.e., codes that are represented by a sparse parity-check matrix with a constant number of ones in each row and in each column. Later, it was shown [14], [16], [10] that the performance of LDPC codes in the waterfall region can be improved by using irregular LDPC codes. The drawback of irregular LDPC codes is that they exhibit an error floor that is caused by small stopping sets for the binary erasure channel (BEC) [4] and by codewords with small Hamming weight and by small trapping sets for the AWGN channel [15]. Hence, many different construction methods are given to lower the error floor of irregular LDPC codes [7], [20], [5]. But also some regular LDPC codes, e.g., the Margulis construction, show a relatively high error floor [13]. The minimum distance of a code can be used to predict the error floor under ML-Decoding. Since Vardy proves in [21] that the minimum distance cannot be computed in polynomial time, a suboptimal algorithm to estimate an upper bound for the minimum distance of linear codes is introduced by using the error impulse method [2]. This algorithm is improved for LDPC codes by using bit reversing [8] or by using two error impulses [3]. In this paper, we introduce a very fast and simple algorithm by modifying the algorithm described in [3] that finds codewords with low Hamming weight as well as stopping sets with small size. It is less complex than the algorithm with bit inverting and it finds more codewords with low Hamming weight and is less complex than the algorithm introduced in [3]. We use the number of small stopping sets and the number of codewords with low Hamming weight to predict the asymptotic performance for the BEC and for the AWGN channel, respectively. The paper is organized as follows: In Section 2, some basic definitions are given, while Section 3 deals with the algorithm to find small stopping sets. After comparing the new algorithm with existing algorithms in Section 4, we show simulation results for different LDPC codes and the asymptotic bounds in Section 5. Finally, we conclude the paper in Section 6. 2 Definitions and Notations Every binary linear code of length n and dimension k can be represented as Tanner graph [19] or bipartite graph. This graph consists of two sets of nodes connected by edges. One set, the variable nodes,

2 correspond to the n columns of the parity-check matrix H and the other set, the check nodes, represent the rows of the parity-check matrix. The number of rows in H is denoted by m, where m n k. A one in the parity-check matrix in row j and in column i corresponds to an edge between the i-th variable node v i and the j-th check node c j. A check node c j is called a neighbor of a variable node v i, if there exists an edge between c j and v i. The number of edges incident to the i-th variable node v i is called the variable node degree d(v i ), which is equal to the number of ones in column i. Similarly, the number of edges connected with the j-th check node c j is called the check node degree d(c j ) and is equal to the number of ones in row j. This is demonstrated in the following example. Example: Assume the parity-check matrix H of an LDPC code of length n = 10 and dimension k = 5 is given by H = The bipartite or Tanner graph representing this LDPC code is shown in Fig. 1. Fig. 1. v 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 v 9 v 10 An irregular bipartite graph Let d vmax and d cmax denote the maximum variable node and check node degree, respectively, and let λ i and ρ i represent the fraction of edges emanating from variable and check nodes of degree i. Then we can define λ(x) = d vmax i=2 λ i x i 1 and ρ(x) = c 1 c 2 c 3 c 4 c 5 d cmax i=2 ρ i x i 1 as the variable node degree distribution and the check node degree distribution, respectively. Di et al. [4] pointed out the crucial role of stopping sets in the erasure decoding algorithm for LDPC codes [9]. A stopping set is defined as follows: Definition 1 A stopping set S is a subset of V, the set of variable nodes, such that all neighbors of the variable nodes in S are connected to S at least twice. The size of the stopping set s is the cardinality of S. It can be seen in Fig. 1 that the set {v 2,v 7,v 9 }, which is marked by bold lines, is a stopping set. It is shown in [4] that the set of erasures, which remains when the iterative erasure decoding algorithm stops is equal to the unique maximum stopping set. Similarly to the weight distribution W(x), we define the stopping set size distribution S(x) as follows: W(x) = S(x) = n w i x i i=0 n s j x j, j=0 where s j gives the number of stopping sets with size j and w i the number of codewords with Hamming weight i. Furthermore, we define a distance set by: Definition 2 A distance set D is a subset of V, the set of variable nodes, such that all neighbors of the variable nodes in D are connected to D an even number of times. Clearly, every distance set D is also a stopping set S. It is obvious that every distance set with cardinality s represents also a codeword with Hamming weight s. Furthermore, the cardinality of the smallest distance set D (except the empty set) is the minimum distance d of the LDPC code represented by the bipartite graph. We can use the first terms of the weight distribution W(x) to estimate the frame error rates (FERs) over the AWGN channel for a high E b /N 0 and Maximum- Likelihood decoding by ( d+v ( ) ) Eb P B (E b /N 0 ) 0.5 w i erfc R i, N 0 i=d (1) where R is the rate of the LDPC code and v a nonnegative small integer variable. Similarly, the asymptotic FERs over the BEC P B (ǫ) can be estimated by: P B (ǫ) s min+u j=s min s j ǫ j, (2) where ǫ is the bit erasure probability, s min is the minimal stopping set size (except the empty set), and u is a non-negative small integer variable.

3 3 Modified Algorithm In this section, we introduce a simple and fast algorithm to find stopping sets of size smaller than s max in the bipartite graph and codewords with Hamming weight smaller than s max. We use a maximum number of iterations I for the belief propagation algorithm and a threshold t { 1,...,T } for declaring bit h as erasure, if the reliability R h of bit h is at most t. The algorithm can be summarized as follows: For all ( n 2) combinations of j and l Set R h = 1 for h = {1,...,n}. Set R j and R l to a high negative value For i = 1 to I. Run the belief propagation decoder with the sig-min approximation for one iteration. Set the threshold t = 1. while (no stopping set found and t T ) Declare all bits, for which R h t as erasures and all other bits as zero and decode it with the erasure decoding algorithm until a stopping set is found or the all-zero codeword is recovered. If a stopping set is found and if the size is smaller than s max, check if the stopping set was already found. Store it as a stopping set otherwise and as a valid codeword, if the stopping set is a distance set. t = t + 1 End of while End (for i = 1 to I) End (for all combinations of j and l) Since the erasure decoding algorithm is not very complex, the complexity of this algorithm is dominated by the total number of iterations of the belief propagation decoding algorithm, which is equal to I (n 2). Note that we only use the sig-min approximation, which can be implemented with integer variables. The asymptotic complexity of this algorithm is of order O(n 3 ) because one iteration of the belief propagation has a linear complexity in n. The algorithm can easily be extended by using three error impulses, which is useful to find more codewords and stopping sets especially for randomly constructed LDPC codes without degree-2 variable nodes. This results in an asymptotic complexity of order O(n 4 ). 4 Comparison of the Algorithms In this section, we compare the modified algorithm presented in Section 3 with the algorithms described in [3] and in [8]. Table I, Table II, and Table III show the first terms of the weight distribution found with the three different algorithms. The first row stands for the Hamming weights of the codewords, while the other rows represent the multiplicities of the codewords with the corresponding Hamming weight. All codes are irregular LDPC codes with n = 1000, R = 1/2 and a variable node degree distribution λ(x) = 0.283x x x 8. Code 1 and code 2 are constructed with the Progressive Edge-Growth (PEG) algorithm [7], where code 1 has the smallest minimum distance and code 2 the largest minimum distance of 50 LDPC codes constructed with different random seeds. The third code is constructed in an iterative process by using an improved PEG algorithm introduced in [17], searching for stopping sets with size smaller than 19 by using the algorithm presented in Section 3 and prohibiting the construction algorithm to choose connections that lead to such stopping sets. We use a maximum number of 500 iterations for the algorithm described in [3]. For the algorithm explained in [8] we use the program Mindist from [11] and run it with speedup parameter 3 because the simulation time is very long. For the algorithm introduced in Section 3 we use 10 and 100 iterations, respectively and a maximum threshold T = 10. Ham. weight Algorithm [3] Algorithm [8] Iterations Iterations TABLE I WEIGHT DISTRIBUTION OF CODE 1 Ham. weight Algorithm [3] Algorithm [8] Iterations Iterations TABLE II WEIGHT DISTRIBUTION OF CODE 2 Ham. weight Algorithm [3] Algorithm [8] Iterations Iterations TABLE III WEIGHT DISTRIBUTION OF CODE 3 It can be observed that all three algorithms find exactly the same number of codewords with Hamming weight w H 20 for code 1. For code 2 and for code 3 we see that the modified algorithm with 100 iterations finds more codewords than the other two algorithms. Furthermore, it is shown in Table I, Table II, and Table III that all codewords with w H 20 that

4 are found with the other algorithms can also be found with the algorithm introduced in this paper with only 10 decoding iterations. The modified algorithm is less complex than the algorithm introduced in [3] (even if we use the same number of decoding iterations) since no loop over the impulse strength is done in the algorithm introduced in Section 3. The loop over the threshold can be nearly neglected since the complexity of the erasure decoding algorithm is quite low. Furthermore, the modified algorithm uses the sig-min approximation, which can be implemented with integer variables. The simulation time for the modified algorithm with I = 100 is approximately 100 times faster than the simulation time of the program Mindist from [11] with speedup parameter 3. Another advantage of the modified algorithm is that it does not only find codewords with a small Hamming weight, but also stopping sets of small size. This can be used to predict the asymptotic performance of LDPC codes transmitted over the BEC and decoded with the algorithm described in [9]. FER Code 1 Code 2 Code 3 Regular Code 1 bound Code 2 bound Code 3 bound E b /N 0 [db] Fig. 2. FERs over the BEC (n = 1000, R = 1/2) Simulation Results In this section, we compare the asymptotic bounds calculated by (2) with the performance over the BEC and the asymptotic bounds determined by (1) with the FERs over the AWGN channel. Therefore, we use the codes from Table I, from Table II, and from Table III. The stopping set size distributions S 1 (x), S 2 (x), and S 3 (x) of code 1, code 2, and code 3 are S 1 (x) = 1 + x 9 + x , S 2 (x) = 1 + 3x x x x x , and S 3 (x) = 1+29x x x x x Fig. 2 shows the FERs of the 3 different LDPC codes over the BEC and the asymptotic bounds calculated by (2) with u = 5. Furthermore, the FERs of a regular LDPC code with λ(x) = x 2 and ρ(x) = x 5 can be seen in Fig. 2, where the size of the smallest stopping set s min, which was found with the modified algorithm, is s min = 30. All simulations were done with the efficient erasure decoding algorithm described in [9]. It can be seen that all three irregular codes show an error floor that approaches the calculated bounds for a low erasure probability ǫ. Contrariwise, the regular code does not show an error floor in the simulated region, since s min is quite large. Fig. 3 shows the performance of the three irregular codes over the AWGN channel as well as the asymptotic bounds calculated by (1) with v = 5. The weight distributions can be read in Table I, in Table II, and in Table III. Again, the performance of the regular LDPC code is added in Fig. 3. All simulations were done with the shuffled belief propagation decoder described in [23], with a maximum number of 500 iterations, and with the linear approximation described in [18]. FER Code 1 Code 2 Code 3 Regular Code 1 bound Code 2 bound Code 3 bound E b /N 0 [db] Fig. 3. FERs over the AWGN channel (n = 1000, R = 1/2) Here, we can see that the asymptotic performance of code 1 is approximately as calculated by (1), while the error floors of code 2 and code 3 start earlier than the calculated bounds. This is due to the fact that the error floor is mainly caused by small trapping sets, which prevent the iterative decoder to find a valid codeword. The regular LDPC code does not show an error floor in the simulated region because the minimum distance d is high and the probability of small trapping sets is low, when an LDPC code without degree-2 variable nodes is constructed with a pseudorandom algorithm. The upper bound on the minimum distance of the regular LDPC code found with the algorithm proposed in this paper is d 36. Of course we cannot guarantee that all stopping sets with small size are found with the modified algorithm. However, during the simulations no stopping set with s 25 prevented the decoder to find the valid codeword that was not found by the algorithm introduced in Section 3 with I = 100 iterations.

5 6 Conclusions In this paper, we introduced a fast and simple algorithm to find small stopping sets in the bipartite graph and codewords with small Hamming weight of LDPC codes. This algorithm is less complex than existing algorithms and works very well for irregular LDPC codes. We calculated asymptotic bounds for the frame error rates over the binary erasure channel and over the AWGN channel with the stopping set size distribution and with the Hamming weight distribution found by the algorithm. Simulations showed that these asymptotic bounds can be used to predict the frame error rates for a transmission over the binary erasure channel for a low erasure probability. Furthermore, the asymptotic bounds can be used as lower bounds for the frame error rates for a transmission over the AWGN channel. Acknowledgments This work was supported by the German research council Deutsche Forschungsgemeinschaft (DFG) under Grant Bo 867/12. The author would like to acknowledge the DFG for their support. References [1] C. Berrou, A. Glavieux, and P. Thitimajshima. Near Shannon limit error-correcting coding and decoding: Turbo-codes(1). In IEEE International Conference on Communications, pages , Geneva, Swizerland, May [2] C. Berrou and S. Vaton. Computing the minimum distance of linear codes by the error impulse method. In IEEE Intern. Symposium on Inf. Theory, Lausanne, Switzerland, July [3] F. Danehgaran, M. Laddomada, and M. Mondin. An algorithm for the computation of the minimum distance of LDPC codes. Submitted to ETT, [4] C. Di, D. Proietti, I. E. Telatar, T. J. Richardson, and R. L. Urbanke. Finite-length analysis of low-density parity-check codes on the binary erasure channel. IEEE Trans. Inf. Theory, 48(6): , June [5] L. Dinoi, F. Sottile, and S. Benedetto. Design of variablerate irregular LDPC codes with low error floor. In IEEE International Conference on Communications, Seoul, Korea, May [6] R. G. Gallager. Low-Density Parity-Check Codes. M.I.T. Press, Cambridge, [7] X. Y. Hu, E. Eleftheriou, and D. M. Arnold. Regular and irregular progressive edge-growth Tanner graphs. IEEE Trans. Inf. Theory, 51(1): , January [8] X. Y. Hu, P. C. Fossorier, and E. Eleftheriou. On the computation of the minimum distance of low-density parity-check codes. In IEEE International Conference on Communications, volume 2, pages , Paris, France, June [9] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman. Efficient erasure correcting codes. IEEE Trans. Inf. Theory, 47(2): , February [10] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman. Improved low-density parity-check codes using irregular graphs. IEEE Trans. Inf. Theory, 47(2): , February [11] D. J. C. MacKay. wol.ra.phy.cam.uk/mackay/codes/data.html. In Online Database of Low-Density Parity-Check Codes, [12] D. J. C. MacKay and R. M. Neal. Near Shannon limit performance of low-density parity-check codes. Electron. Lett., 32: , August [13] D. J. C. MacKay and M. S. Postal. Weaknesses of margulis and ramanujan-margulis low-density parity-check codes. Electronic Notes in Theoretical Computer Science, 74, [14] D. J. C. MacKay, S. T. Wilson, and M. C. Davey. Comparison of constructions of irregular codes. 36th Allerton Conference Communications, Control, and Computing, September [15] T. Richardson. Error floors of LDPC codes. In 41st Annual Allerton Conference on Communications, Control, and Computing, Allerton, England, October [16] T. J. Richardson, M. A. Shokrollahi, and R. Urbanke. Design of capacity-approaching irregular low-density parity-check codes. IEEE Trans. Inf. Theory, 47(2): , February [17] G. Richter and A. Hof. On a construction method of irregular LDPC codes without small stopping sets. In IEEE International Conference on Communications, Istanbul, Turkey, June [18] G. Richter, G. Schmidt, M. Bossert, and E. Costa. Optimization of a reduced-complexity decoding algorithm for LDPC codes by density evolution. In IEEE International Conference on Communications, Seoul, Korea, May [19] R. M. Tanner. A recursive approach to low complexity codes. IEEE Trans. Inf. Theory, IT-27: , September [20] T. Tian, C. R. Jones, J. D. Villasenor, and R. D. Wesel. Selective avoidance of cycles in irregular LDPC code construction. IEEE Trans. Inf. Theory, 51(1): , January [21] A. Vardy. The intractability of computing the minimum distance of a code. IEEE Trans. Inf. Theory, 43: , November [22] N. Wiberg. Codes and Decoding on General Graphs. PhD thesis, Linköping University, Sweden, [23] J. Zhang and M. P. C. Fossorier. Shuffled iterative decoding. IEEE Trans. Commun., pages , February 2005.