UCSD ECE154C Handout #21 Prof. Young-Han Kim Thursday, June 8, Solutions to Practice Final Examination (Spring 2016)

Transcription

1 UCSD ECE54C Handout #2 Prof. Young-Han Kim Thursday, June 8, 27 Solutions to Practice Final Examination (Spring 26) There are 4 problems, each problem with multiple parts, each part worth points. Your answer should be as clear and readable as possible. Please justify any claim that you make.. Lossless source coding (3 points). Consider a source S that produces independent and identically distributed symbols from the alphabet {A, B, C} with p A =.5, p B =.3, and p C =.2. (a) Find a binary Tunstall code that encodes the source sequence into 3 binary code symbols at a time. (b) What is the rate of this code, i.e., the average number of encoded source symbols per code symbol? (c) Suppose now that the Tunstall code is followed by a binary Huffman code, forming a variable-to-variable-length code. What is the rate of this code, i.e., the ratio of the average number of source symbols to the average number of code symbols? Solution : (a) We need 2 3 = 8 source sequences for the Tunstall coding. However, since we have three symbols, each branching in the Tunstall coding procedure yields 2 new source sequences, keeping the total number odd. Hence, we can only use 7 source strings for the encoding. The Tunstall coding procedure is shown below. A.5 B.3 C AA AB AC BA BB BC One possible encoding is as follows: AA AB AC BA BB BC C.

2 (b) The average number of encoded source symbols is given by L s = 2 ( ) +.2 =.8. Alternatively, the average number of encoded source symbols can be calculated by adding up the probabilities at the nodes of the graph, giving L s = =.8. Thus, the rate of this code is.8/3 =.6. (c) One possible way to do the Huffman coding is shown below. AB.5.3 BA.5.55 AA.25. C.2 AC..45 BB BC.6 This gives the encoding AA AB AC BA BB BC C. 2

3 The average number of code symbols is given by L c = 2 ( ) + 3 ( ) + 4 (.9 +.6) = 2.7. Alternatively, the average number of code symbols can be computed by adding the probabilities at the internal nodes, so we have L c = = 2.7. Thus the rate of this code is given by L s / L c = A binary code (4 points). Consider a binary code with the following four codewords:,,, and. (a) What is the rate of this code? Justify your answer. (b) Is this code linear? Justify your answer. (c) How many errors is this code guaranteed to correct? (d) Suppose that this code is used over a binary symmetric channel with crossover probability p [, /2]. What is the conditional probability of an undetected error, given that the codeword was sent? Solution : (a) The block length n is 7, and there are four codewords, so 2 k = 4, giving k = 2. Thus, the rate of the code is k n = 2 7. (b) The sum of the second and third codewords is, which is not a codeword. Thus the code is not linear. (c) By looking at all possible Hamming distances between distinct codewords, we see that the minimum distance of this code is d min = 3. Thus, this code is guaranteed to correct one bit error. (Note : This code is not linear; therefore, in order to find d min, it is not enough to merely look at the distances of one particular codeword from other codewords, or at the Hamming weights of all codewords. We actually have to check all ( 4 2) = 6 distances.) (d) An undetected error will occur if the received vector is the same as one of the other three codewords. This can happen in the following ways. i. The second, third and fourth bits are flipped. In this case, we get the all-zero codeword as output. ii. The first, fifth, sixth and seventh bits are flipped. In this case, we get the all-one codeword as output. 3

4 iii. The first six bits are flipped. output. In this case, we get the third codeword as By adding the probablities for these individual cases, we see that the conditional probability of an undetected error, given was sent, is P e = p 3 ( p) 4 + p 4 ( p) 3 + p 6 ( p). 3. Linear parity check codes (5 points). Consider a binary linear code defined by the generator matrix G =. (a) Find the parity check matrix of the form H = [ A I ]. (b) What is the minimum distance of this code? Justify your answer. (c) Find all the patterns of 4 erasures that this code can fill in correctly. Suppose now that a new code is formed by puncturing the last bit of all codewords. (d) Find the parity check matrix for the new code of the form H = [ B I ]. (e) What is the minimum distance of the new code? Solution : (a) Replacing the first row of G by the (modulo 2) sum of the second and third rows, the second row by the sum of the first and second rows, and the third row by the sum of all three rows, we get a new generator matrix G =. Note : This transformation is equivalent to pre-multiplying G by the full-rank matrix M =. Since G is now of the form [ I A t], we can now readily write down H as H = [ A I ] =. 4

5 (b) Notice that H does not have two identical columns; thus, d min 3. Also, the first, fourth and sixth columns of H sum to zero, and are thus linearly dependent. So, d min = 3 for this code. (c) Recall that an erasure pattern can be filled in uniquely, if and only if the corresponding columns of H are linearly independent. Based on this, the following 2 patterns of 4 erasures can be uniquely filled in by this code: (, 2, 3, 4), (, 2, 3, 5), (, 2, 3, 6), (, 2, 3, 7), (, 2, 4, 5), (, 2, 4, 7), (, 2, 5, 6), (, 2, 6, 7), (, 3, 4, 7), (, 3, 5, 7), (, 3, 6, 7), (, 4, 5, 7), (, 5, 6, 7), (2, 3, 4, 5), (2, 3, 4, 6), (2, 3, 4, 7), (2, 4, 5, 6), (2, 4, 6, 7), (3, 4, 5, 7), (3, 4, 6, 7), (4, 5, 6, 7). Alternatively, recall that an erasure pattern cannot be filled in uniquely, if and only if the location of the erasures is a superset of the location of s in some nonzero codeword. Using the generator matrix G, the codewords can be enumerated as follows:. Based on the codewords, we see that the following patterns of four erasures cannot be filled in: (, 3, 5, 6), (2, 3, 5, 6), (3, 4, 5, 6), (3, 5, 6, 7), (, 2, 5, 7), (2, 3, 5, 7), (2, 4, 5, 7), (2, 5, 6, 7), (, 2, 4, 6), (, 3, 4, 6), (, 4, 5, 6), (, 4, 6, 7), (, 3, 4, 5), (2, 3, 6, 7). (d) Observe that the last bit of the codewords occur only in the last parity relation. So, if we remove the last row and last column of H, we will get a parity check matrix for the punctured code. Thus, a parity check matrix for the punctured code is given by H p =. This parity check matrix is of the required form H p = [ B I ]. (e) Observe that the second and fifth columns of H p are identical, i.e., linearly dependent. This shows that the punctured code has d min = Convolutional codes (3 points). Consider a binary convolutional code with the fol- 5

6 lowing encoder structure and initial state : y x. z z z (a) What is the rate of this code? Justify your answer. (b) Draw a trellis diagram for this code corresponding to the first 5 input symbols. (c) Find the free Hamming distance d free of this code. We now increase the rate of the code by puncturing under the pattern (, X,, X,, X,...). For example, a codeword... in the original code becomes... in the punctured code. The next 7 questions are on this punctured code. (d) What is the rate of this code? Justify your answer. (e) Draw a trellis diagram for this code corresponding to the first 5 input symbols. (f) Find the free Hamming distance d free of this code. (g) Find the codeword corresponding to the input sequence. How about? (h) Let y()y(2)y(3) be the codeword corresponding to the input x()x(2)x(3). Find y(3) and y(6) in terms of x(), x(2),.... More generally, what is y(3k), k =, 2,..., in terms of the input symbols? (i) Suppose that the sequence is received when this code is used for a binary symmetric channel. Find the codeword nearest to this sequence in Hamming distance. What is the corresponding input sequence? (j) Suppose that the sequence??? is received when this code is used for a binary erasure channel. Find the codeword by filling in the erasures. Repeat this problem for the received sequence???. Now, consider the binary 4-state convolutional code represented by the following en- 6

7 coder structure and initial state : z c a b z c 2 c 3 Here, a and b are the binary input symbols, and c, c 2, and c 3 are the binary output symbols. (k) What is the rate of this code? Justify your answer. (l) Find c 3 (k), k =, 2,..., in terms of the input symbols a(), b(), a(2), b(2),.... (m) (Difficult.) Is the free Hamming distance d free of this code smaller than, equal to, or larger than that of the punctured code in part (f)? Justify your answer. (Hint: It may be useful to compare the answers to parts (h) and (l).) Solution : (a) Each input bit corresponds to two code bits, thus the rate is /2. (b) 7

8 For the branches going out of each state, the top branch corresponds to input, and the bottom branch corresponds to input. (c) By inspecting the trellis, we can find d free for this code, constraining the path to diverge from state and end at state. The relevant path is shown in bold in the following trellis diagram. We thus have d free = 5. (d) Every two input bits correspond to three code bits, thus the rate is 2/3. (e) 8

9 For the branches going out of each state, the top branch corresponds to input, and the bottom branch corresponds to input. (f) By inspecting the trellis, we can find d free for this code, constraining the path to diverge from state and end at state. The relevant path is shown in red in the following trellis diagram. We thus have d free = 3. (g) From the trellis diagram in part (e), we see that the codeword corresponding to the input sequence is, and the codeword corresponding to the input sequence is. (h) y(3) is the same as the fourth bit of the unpunctured code, and thus we have, from the encoder structure, that y(3) = x(2) x() (since the initial state is.) Similarly, y(6) is the 8 th bit of the unpunctured code, and thus, y(6) = x(4) x(3) x(2). In general, y(3k) is the same as the (4k) th bit of the unpunctured code, and is thus given by y(3k) = x(2k) x(2k ) x(2k 2). (i) We can use Viterbi decoding to find the codeword closest in Hamming distance. 9

10 The relevant path is shown in red in the following trellis diagram. Therefore, the closest codeword is, which is at a Hamming distance of from the received sequence, and the corresponding input sequence is. (j) The erasures can be filled in by analyzing the possible paths at each step in the trellis. From inspection, the codeword corresponding to the first erasure pattern is, since none of the other 7 possible ways of filling in the erasures produces a valid codeword. The path corresponding to the correct codeword for the first received sequence is shown in red in the following trellis diagram. From inspection, the correct codeword corresponding to the second erasure pattern is, since none of the other 7 possible ways of filling in the

11 erasures produces a valid codeword. The path corresponding to the correct codeword for the second received sequence is shown in red in the following trellis diagram. (k) Every two input bits correspond to three code bits, thus the rate is 2/3. (l) From the encoder diagram, we have (m) Similar to part (l), we have Similar to part (h), we have c 3 (k) = a(k) b(k) b(k ). c (k) = a(k) a(k ), and c 2 (k) = a(k) a(k ) b(k ). y(3k 2) = x(2k ) x(2k 3), and y(3k ) = x(2k ) x(2k 2) x(2k 3). If we map c (k) y(3k 2), c 2 (k) y(3k ), c 3 (k) y(3k), a(k) x(2k ), and b(k) x(2k), we see that this code is, in fact, the same as the punctured code above. Thus, d free of this code is also equal to 3.

12

13

14

15

16

17

18