Error Control Techniques for Interactive Low-bit Rate Video Transmission over the Internet. Injong Rhee Department of Computer Science North Carolina State University
Video Conferencing over Packet- Switching Networks. Packet Loss Compressed video frame is packetized and transmitted. Packet loss can occur quite frequently due to congestion and user mobility.
Motion Compensated Coding Most video codecs use it. Each P-frame depends on its previous frames. I-frame is periodically transmitted. Temporal Dependency Intra Frame (I-frame) Inter Frame (P-frame)
When a packet is lost,. Packet Loss Intra Frame (I-frame) Inter Frame (P-frame) Error Propagation Conventional Solution : Transmit I-frame more often. However, compression efficiency reduces.
Myth: retransmission is not useful for interactive video TIME F0 F1 F2 F3 F1 (retransmit) lost NACK retransmit (F1 arrives after its display) F0 F1 F2 F3 Conventionally, late packets are discarded.
Retransmission stops error propagation. F0 F1 F2 F3 F1 (retransmit) lost TIME NACK retransmit F1 is recovered here F0 F1 F2 F3 Don t discard: Use late packets to recover their reference frames
Handling retransmitted late packets in decoder Inverse Quantization/DCT Current Reference Current Frame Info Frame Diff.erence Frame + Retransmitted Information. Old Reference Frame Motion Prediction Compensation Display
Error Propagation
Recovery from Error Propagation
Error Recovery Technique Recovery from Error Spread using Continuous Update (RESCU) Focus on error spread/propagation Continuous Update Retransmission (this paper) Forward error correction Combined with Layer coding (this paper)
Extending Packet Deadline 1Add more frame buffers at the decoder. 2Adjust Temporal Dependency Distance (TDD) of a frame. TDD: frame intervals from that frame to its temporally dependent frame.
Add more reference frame buffers. Cascaded Buffering Reference Frame 2 Reference Frame 1 Base Reference Frame Prediction Errors + MVs Prediction Errors + MVs Contains Full Image Minus: More computation and buffers at the decoder. Plus : No change in the codec and picture pattern.
Adjust Temporal Dependency Distance Periodic TDDs (PTDD) Changing Dependency. Intra Inter Inter Periodic f PTDD can be dynamically adjusted. +Reduced Buffers & computations. No protection for non-periodic frame.
Continuous Updates Retransmission & Forward Error Correction Retransmission Retransmit lost packets within PTDD. Retransmission delay requires larger PTDD. Forward Error Correction send parity packets over a PTDD period. Periodic frame Periodic frame Parity Packet for periodic frame
Protecting non-periodic frames RESCU + Quality Assurance Layering HP HP HP HP LP LP LP LP. HP stream is sent with forward error correction. RESCU contributes to reducing bit rates.
protecting non-periodic frames Change temporal dependency Periodic Inter Inter Periodic Errors in non-periodic frames do not propagate. As PTDD increases, Compression efficiency decreases.
Compression Efficiency (MPEG-4 Class A, AKIYO) AVG PSNR (db) H.261 RESCU RESCU + QAL Intra-H261 AVG Bytes/Frame
Error Resilience (H.261 v.s. RESCU)
RESCU + QAL
Error Resilience various loss groups RESCU + QAL Intra-H.261 PSNR RESCU H.261 Loss Rate
Conclusions RESCU accommodates delays in recovery (retransmission or FEC). error resilience for interactive video. high compression efficiency. need I-frames less frequently. Good for bandwidth-constrained, lossy networks (e.g., mobile wireless networks). Much more work to be done.
Error Resilience Retransmission over various network delays 50 ms 100 ms 150 ms 200 ms 250 ms 300 ms