CSEP 561 Error detection & correction David Wetherall djw@cs.washington.edu
Codes for Error Detection/Correction ti ti Error detection and correction How do we detect and correct messages that are garbled during transmission? Application The responsibility for doing this cuts across the different layers Network But we re mostly thinking about links Transport Link Physical djw // CSEP 561, Winter 2012 2
Typical implementation ti of the layers djw // CSEP 561, Winter 2012 3
Problem Fading and noise in the received signal can cause us to wrongly decode the symbols/bits! We must be able to recover when this occurs! Tx signal Rx signal 1? 0? djw // CSEP 561, Winter 2012 4
Approach -- Coding Basic approach: add redundant data Error detection codes allow errors to be recognized Error correction codes allow errors to be repaired Simple, motivating error detection scheme: Just send two copies. Differences imply errors. Question: Can we do any better? Answer: Yes stronger protection with fewer bits! But we can t catch all inadvertent errors, nor malicious ones Added measure (later): retransmit messages djw // CSEP 561, Winter 2012 5
Deep-dive on decoding di models Three different models: 1. Hard-decoding Pick a 0 or 1 as the decoded bit, e.g., a1 Simplest, and mostly what we will use 2. Soft-decoding Pick and record confidence, e.g., a 1 with p=0.6 Helps check/reconstruct message with coding 3. Erasures Declare lost data when confidence is low, e.g., lost Pulls out good portion and reconstruct message with coding 1? 0? djw // CSEP 561, Winter 2012 6
Codes for Error Detection/Correction ti ti A scheme maps D bits of data into D+R bits i.e., it uses only 2 D distinct t bit strings of the 2 D+R possible. D message data bits R check bits (systematic) The sender computes the check bits based on the data. The receiver also computes check bits for the data it receives and compares them with the check bits it received. Mismatches detect errors. And mapping to the closest valid codeword can correct errors. Detection/correction schemes are characterized in two ways: Overhead: rate is fraction of useful information, D/(D+R) Ex: 1000 data bits + 100 code bits = rate of 10/11, 10% overhead The errors they detect/correct Ex: all single-bit errors, all bursts of fewer than 3 bits, etc. djw // CSEP 561, Winter 2012 7
The Hamming Distance Errors must not turn one valid codeword into another valid codeword, or we fail to detect/correct them. Hamming distance of a code is the smallest number of bit differences that turn any one codeword into another e.g, code 000 for 0, 111 for 1, Hamming distance is 3 For code with distance d+1: d errors can be detected, e.g, 001, 010, 110, 101, 011 For code with distance 2d+1: d errors can be corrected, e.g., 001 000 djw // CSEP 561, Winter 2012 8
Error Bounds Hamming distance Code turns data of n bits into codewords of n+k bits Hamming distance is the minimum bit flips to turn one valid codeword into any other valid one. Example with 4 codewords of 10 bits (n=2, k=8): 0000000000, 0000011111, 1111100000, and 1111111111 Hamming distance is 5 Bounds for a code with distance: 2d+1 can correct d errors (e.g., 2 errors above) d+1 can detect d errors (e.g., 4 errors above) djw // CSEP 561, Winter 2012
Simplest Error Detection ti Code Parity Start with n bits and add another so that the total number of 1s is even (even parity) e.g. 0110010 01100101 Easy to compute as XOR of all input bits Will detect an odd number of bit errors But not an even number Does not correct any errors (Unless the errors are erasures!) djw // CSEP 561, Winter 2012 10
Error Detection Parity (1) Parity bit is added as the modulo 2 sum of data bits Equivalent to XOR; this is even parity Ex: 1110000 11100001 Detection ti checks if the sum is wrong (an error) Simple way to detect an odd number of errors Ex: 1 error, 11100101; detected, sum is wrong Ex: 3 errors, 11011001; detected sum is wrong Ex: 2 errors, 11101101; not detected, sum is right! Error can also be in the parity bit itself Random errors are detected with probability ½ djw // CSEP 561, Winter 2012
Checksums Used in Internet protocols (IP, ICMP, TCP, UDP) Basic Idea: Add up the data and send it along with sum Algorithm: Mouthful for sum : checksum is the 1s complement of the 1s complement sum of the data interpreted 16 bits at a time (for 16-bit TCP/UDP checksum) 1s complement e nit: flip all bits to make a number negative, so adding requires carryout to be added back. Q: What kind of errors will/won t checksums detect? djw // CSEP 561, Winter 2012 12
Error Detection Checksums Checksum treats data as N-bit words and adds N check bits that are the modulo 2 N sum of the words Ex: Internet 16-bit 1s complement checksum Properties: Improved error detection ti over parity bits Detects bursts up to N errors Detects random errors with probability 1-2 N Vulnerable to systematic errors, e.g., added zeros djw // CSEP 561, Winter 2012
Example: Internet checksum Sending: 1. Arrange data in 16-bit words 2. Put zero in checksum word, add 0001 f203 f4f5 f6f7 +(0000) ------ 2ddf0 3. Add any carryover back to get 16 bits 4. Negate (complement) to get final checksum now will add to zero ddf0 + 2 ------ ddf2 220d djw // CSEP 561, Winter 2012
Example: Internet checksum Receiving: 1. Arrange data in 16-bit words 2. Checksum will be non-zero, add 0001 f203 f4f5 f6f7 + 220d ------ 2fffd 3. Add any carryover back to get 16 bits fffd + 2 ------ ffff 4. Result of 0xffff (alternate form of 0) means checksum correct Success! djw // CSEP 561, Winter 2012
Internet t checksum properties Catches all error bursts up to 15 bits, most 16 bits Random errors detected with prob. 1 2^-16 Fails to catch transpositions, insertion/deletion of zeros These are typically hardware/software bugs not random errors djw // CSEP 561, Winter 2012 16
Fletcher a better checksum Includes a positional component Now sensitive to order of data slightly more computation, but well worth it djw // CSEP 561, Winter 2012 17
CRCs (Cyclic Redundancy d Check) Stronger protection than checksums Used widely in practice, e.g., Ethernet/802.11 CRC-32 Algorithm: Given n bits of data, generate a k bit check sequence that gives a combined n + k bits that are divisible by a chosen divisor C(x) Based on mathematics of finite fields numbers correspond to polynomials, use modulo arithmetic e.g, interpret 10011010 as x 7 + x 4 + x 3 + x 1 djw // CSEP 561, Winter 2012 18
CRC Example Adds bits so that transmitted frame viewed as a polynomial is evenly divisible by a generator polynomial Start by adding 0s to frame and try dividing Offset by any reminder to make it evenly divisible djw // CSEP 561, Winter 2012 19
Error Detection CRCs (2) Based on standard polynomials: Ex: Ethernet 32-bit CRC is defined by: Computed with simple shift/xor circuits Stronger detection than checksums: E.g., can detect all double bit errors Not vulnerable to systematic errors djw // CSEP 561, Winter 2012
Error correcting codes Errors create clusters around codewords Correct by mapping to nearest codeword Works for d errors if distance <= 2d+1 Hamming dist = 3 Single errors Valid codeword d Error correction djw // CSEP 561, Winter 2012 21
Example (3, 1) repetition i code has codewords d 000 and 111 Decoding: 000 0 (no error) 001 0 010 0 011 1 100 0 101 1 110 1 111 1 (no error) Corrects up to 1 error; better not have a double error! (unless there is soft-decoding or erasures!) Other error-correcting codes are more complicated djw // CSEP 561, Winter 2012 22
Error Correction Hamming code HW2 explores Hamming code gives a simple way to add check bits and correct up to a single bit error: Check bits are parity over subsets of the codeword Recomputing the parity sums (syndrome) gives the position of the error to flip, or 0 if there is no error (11, 7) Hamming code adds 4 check bits and can correct 1 error djw // CSEP 561, Winter 2012
Example: Hamming (7,4) code 7 bits per message: 4 bits of data (d), 3 check bits (p) Put check bits in positions that are powers of 2 Bit position: 1 2 3 4 5 6 7 (check bits underlined) (i 1) Check bit i is parity of positions with 2 (i-1) term Design is arranged to make decoding easy Bit # 1 2 3 4 5 6 7 Sent bit p 1 p 2 d 1 p 3 d 2 d 3 d 4 p 1 Yes No Yes No Yes No Yes p 2 No Yes Yes No No Yes Yes p 3 No No No Yes Yes Yes Yes djw // CSEP 561, Winter 2012
Example: Hamming (7,4) code Sender: compute check bits from data and send E.g., data = 0101, message= 0 _ 1 0 1 p 1 = 0+1+1 = 0, p 2 = 0+0+1 = 1, p 3 = 1+0+1 = 0 Sent message = 010 0 0 101 1 Receiver: recompute check bits (=syndrome) Syndrome as binary number gives location of error E.g., receive = 0 1 0 0 1 0 1 (no errors) 0+0+1+1 = 0, 1+0+0+1 = 0, 0+1+0+1 = 0 0 0 0 E.g. receive = 0 1 0 0 1 1 1 (one error in position 6) 0+0+1+1 = 0, 1+0+1+1 = 1, 0+1+1+1 = 1 110 1 djw // CSEP 561, Winter 2012
Error Correction Convolutional codes Operates on a stream of bits, keeping internal state Output stream is a function of all preceding input bits Bits are decoded with the Viterbi algorithm 1 1 1 0 1 1 1 0 1 Popular NASA binary convolutional l code (rate = ½) used in 802.11 djw // CSEP 561, Winter 2012
Error Correction LDPC codes Low-Density Parity Check codes are codes for large blocks defined with a sparse generator matrix Invented by Gallagher (PhD thesis, ~1963) Promptly forgotten, revived ~10 years ago Decoding is iterative with belief propagation (ML) Best performing code, used in modern systems Use in 802.11n, DVB, WiMAX, LTE, power-line, etc. djw // CSEP 561, Winter 2012
Summary of Detection/Correction codes Detection (often at link/network/transport layers) Parity, simple example Checksums, but weak CRCs, widely used Correction (often at physical and application layers) Hamming codes, simple example Convolutional codes Reed-Solomon / BCH Low-density Parity Check (LDPC) codes future djw // CSEP 561, Winter 2012 28
Patterns of Errors Matter Q: Suppose you expect a bit error rate (BER) of about 1 bit per 1000 sent. What fraction of packets would be corrupted if they were 1000 bits long (and you could detect all errors but correct none)? djw // CSEP 561, Winter 2012 29
Patterns of Errors A: It depends on the pattern of errors Bit errors occur at random Packet error rate is about 1 0.999 1000 = 63% Errors occur in bursts, e.g., 100 consecutive bits every 100K bits Packet error rate 2% djw // CSEP 561, Winter 2012 30
Real Error Models Random, e.g., thermal noise as in AWGN Occasional error, handled well by Hamming Bursty, e.g., wires, if there is an error it is likely l a burst If there is an error it is likely a burst Due to physical effects, e.g., connectors, interference Difficult to handle: some messages are a lot wrong vs. most message are only a little wrong Strategies for bursty errors: Use a code designed to handle them, e.g., CRCs, Reed-Solomon Use interleaving to convert burst to random errors djw // CSEP 561, Winter 2012 31
Interleaving a neat tti trick HW2 explores Compute check (parity) bit across items, not per item Tolerates burst errors, at the cost of added latency djw // CSEP 561, Winter 2012 32