Distributed Video Coding

Transcription

1 Distributed Video Coding Bernd Girod Anne Aaron Shantanu Rane David Rebollo-Monedero David Varodayan Information Systems Laboratory Stanford University

2 Outline Lossless and lossy compression with receiver side information Shifting the complexity of video encoding to the decoder Error-resilient video transmission B. Girod: EE398B: Image Communication II Distributed Video Coding 2

3 Outline Lossless and lossy compression with receiver side information Shifting the complexity of video encoding to the decoder Error-resilient video transmission B. Girod: EE398B: Image Communication II Distributed Video Coding 3

4 Lossless Compression with Side Information X Encoder R H(X Y) Decoder X Statistically dependent Y Y Side Information X Encoder R? Decoder X Statistically dependent B. Girod: EE398B: Image Communication II Distributed Video Coding Y Side Information 4

5 Lossless Compression with Side Information X Encoder R H(X Y) Decoder X Statistically dependent Y Y X Encoder R H(X Y) Decoder X Statistically dependent Y [Slepian, Wolf, 1973] B. Girod: EE398B: Image Communication II Distributed Video Coding 5

6 B. Girod: EE398B: Image Communication II Distributed Video Coding 6

7 Towards Practical Slepian-Wolf Coding Convolution coding for data compression [Blizard, 1969] Convolutional source coding [Hellman, 1975] Syndrome source coding [Ancheta, 1976] Coset codes [Pradhan and Ramchandran, 1999] Trellis codes [Wang and Orchard, 2001] Turbo codes [García-Frías and Zhao, 2001] [Bajcsy and Mitran, 2001] [Aaron and Girod, 2002] LDPC codes [Liveris, Xiong, and Georghiades, 2002] B. Girod: EE398B: Image Communication II Distributed Video Coding 7

8 Rate-Adaptive Slepian-Wolf Coding Turbo Encoder Parity bits Encoder Buffer Turbo Decoder X X Rate-adaptive turbo codes Request bits Y 0.5 Rate Slepian-Wolf bound Rate = H(X Y) L = 8192 bits Total simulated bits = H(X Y) B. Girod: EE398B: Image Communication II Distributed Video Coding 8

9 Lossy Compression with Side Information X Encoder R X Y (d) Decoder X Y [Wyner, Ziv, 1976] For MSE distortion and Gaussian statistics, rate-distortion functions of the two systems are the same. [Zamir, 1996] The rate loss R * (d) R X Y (d) is bounded. Y X Encoder R * (d) Decoder X B. Girod: EE398B: Image Communication II Distributed Video Coding Y 9

10 Practical Wyner-Ziv Coding Wyner-Ziv Encoder Wyner-Ziv Decoder X Quantizer Q Slepian- Wolf Encoder Slepian- Wolf Decoder Q Minimum Distortion Reconstruction X ' Y Y B. Girod: EE398B: Image Communication II Distributed Video Coding 10

11 Non-Connected Quantization Regions Example: Non-connected intervals for scalar quantization q = 1 q = 2 q = 3 x Decoder: Minimum mean-squared error reconstruction with side information conditional centroid f ( x y) XY xˆ ( ) 2 xˆ = arg min E X xˆ y, q x B. Girod: EE398B: Image Communication II Distributed Video Coding 11

12 Outline Lossless and lossy compression with receiver side information Shifting the complexity of video encoding to the decoder Error-resilient video transmission B. Girod: EE398B: Image Communication II Distributed Video Coding 12

13 Interframe Video Coding Predictive Interframe Encoder Y Predictive Interframe Decoder X X Y Side Information B. Girod: EE398B: Image Communication II Distributed Video Coding 13

14 Low Complexity Encoder X Wyner-Ziv Intraframe Encoder Wyner-Ziv Interframe Decoder Y X [Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002] [Aaron, Zhang, Girod, Asilomar 2002] Side Information B. Girod: EE398B: Image Communication II Distributed Video Coding 14

15 B. Girod: EE398B: Image Communication II Distributed Video Coding 15

16 Pixel-Domain Wyner-Ziv Video Codec Intraframe Encoder Interframe Decoder WZ frames X Scalar Quantizer Slepian-Wolf Codec Turbo Encoder Buffer Turbo Decoder Reconstruction X Request bits Side information Y Key frames I Conventional Intraframe coding Conventional Intraframe decoding Interpolation/ Extrapolation I B. Girod: EE398B: Image Communication II Distributed Video Coding 16

17 Pixel-Domain Wyner-Ziv Video Codec Decoder side information generated by motioncompensated interpolation PSNR 30.3 db After Wyner-Ziv Decoding 16-level quantization bpp 11 pixels in error PSNR 36.7 db B. Girod: EE398B: Image Communication II Distributed Video Coding 17

18 Pixel-Domain Wyner-Ziv Video Codec Decoder side information generated by motioncompensated interpolation PSNR 24.8 db After Wyner-Ziv Decoding 16-level quantization 2.0 bpp 0 pixels in error PSNR 36.5 db B. Girod: EE398B: Image Communication II Distributed Video Coding 18

19 Stanford Camera Array Courtesy Marc Levoy, Stanford Computer Graphics Lab B. Girod: EE398B: Image Communication II Distributed Video Coding 19

20 Distributed Light Field Coding Wyner-Ziv Cameras Conventional Cameras WZ-ENC WZ-ENC Distributed Encoding WZ-DEC WZ-DEC Geometry Reconstruction Joint Decoding Rendering [Zhu, Aaron, Girod, 2003] B. Girod: EE398B: Image Communication II Distributed Video Coding 20

21 Distributed Light Field Coding Wyner-Ziv, Pixel-Domain JPEG-2000 Rate: 0.11 bpp PSNR 39.9 db Rate: 0.11 bpp PSNR 37.4 db B. Girod: EE398B: Image Communication II Distributed Video Coding 21

22 DCT-Domain Domain Wyner-Ziv Video Codec WZ frames W DCT Intraframe Encoder Scalar Quantizer Turbo Encoder For each transform band k Buffer Request bits Turbo Decoder Interframe Decoder X k X k Side information Recon Y k IDCT W DCT Key frames I Conventional Intraframe coding Conventional Intraframe decoding Y Interpolation/ Extrapolation I B. Girod: EE398B: Image Communication II Distributed Video Coding 22

23 Rate-Distortion Performance - Salesman Encoder Runtime Pentium 1.73 GHz machine Interframe B ~ 100% WZ DCT ~ 7% WZ Pixel ~ 6% 3 db I frame ~ 18% 6 db Every 8 th frame is a key frame Salesman QCIF sequence at 10fps 100 frames B. Girod: EE398B: Image Communication II Distributed Video Coding 23

24 Rate-Distortion Performance Hall Monitor 3 db Every 8 th frame is a key frame Hall Monitor QCIF sequence at 10fps 100 frames 8 db B. Girod: EE398B: Image Communication II Distributed Video Coding 24

25 Salesman at 10 fps DCT-based Intracoding 149 kbps PSNR Y =30.0 db Wyner-Ziv DCT codec 152 kbps PSNR Y =35.6 db GOP=8

26 Hall Monitor at 10 fps DCT-based Intracoding 156 kbps PSNR Y =30.2 db Wyner-Ziv DCT codec 155 kbps PSNR Y =37.1 db GOP=8

27 Outline Lossless and lossy compression with receiver side information Shifting the complexity of video encoding to the decoder Error-resilient video transmission B. Girod: EE398B: Image Communication II Distributed Video Coding 27

28 Systematic Lossy Source/Channel Coding Wyner-Ziv Encoder Encoder Analog Channel Digital Channel Side info Wyner-Ziv Decoder Decoder Information theoretic optimality conditions [Shamai, Verdú, Zamir, 1998] Enhancing analog image transmission using digital side information [Pradhan, Ramchandran, 2001] Lossy source-channel coding of video waveforms [Rane, Aaron, Girod, 2004, 05, 06] B. Girod: EE398B: Image Communication II Distributed Video Coding 28

29 Systematic Lossy Error Protection (SLEP) Analog Channel Input Video Video Encoder Channel Video Decoder With Error Concealment Side Information Video With Errors Wyner-Ziv Encoder Wyner-Ziv Decoder Output Video B. Girod: EE398B: Image Communication II Distributed Video Coding 29

30 SLEP using H.264/AVC Redundant Slices H.264/AVC ENCODER H.264/AVC DECODER Input Video Encode Primary Pic Entropy Decoding Q -1 T -1 + Output Video Encode Redundant Pic (Requantize) Motion Vecs + Coding Modes Error-prone Channel Motion Vecs + Coding Modes Encode Redundant Pic (Requantize) Side info MC QP RS Encoder Parity Slices QP Erasure Decoding Entropy Decoding Decode Redundant Slice Recovered motion vectors for erroneously received primary slices WYNER-ZIV ENCODER WYNER-ZIV DECODER B. Girod: EE398B: Image Communication II Distributed Video Coding 30

31 RS Encoding across Redundant Slices Redundant Slice Redundant Slice Redundant Slice k n redundant slice byte filler byte parity byte Redundant Slice SLEP parity symbols SLEP parity symbols Transmit only SLEP slices in Wyner-Ziv bit stream Use Reed-Solomon code as a Slepian-Wolf code B. Girod: EE398B: Image Communication II Distributed Video Coding 31

32 RS Decoding across Redundant Slices Regenerated Redundant Slice Regenerated Redundant Slice Recovered Redundant Slice Regenerated Redundant Slice SLEP parity symbols SLEP parity symbols k n B. Girod: EE398B: Image Communication II Distributed Video Coding 32

33 408 kbps, error resilience bit rate = 40 kbps Symbol error probability = 5 x 10-4 Error-free After error propagation QP = db SLEP Error with 40 concealment kbps redundant FEC QP only = db

34 1 Mbps Symbol error probability = 2 x kbps FEC 100 kbps Wyner-Ziv bit stream PSNR: 32.5 db Recovered 53.7 % of lost macroblocks PSNR: 38.0 db Recovered 96.6 % of lost macroblocks

35 Rally, 1 Mbps, 3% packet loss 80 kbps FEC 33.4 db 80 kbps Wyner-Ziv bit stream 38.1 db Recovered 67.5 % of lost macroblocks Recovered 97.1 % of lost macroblocks

36 Distributed Video Coding: Why Do We Care? Chance to Reinvent Compression from Scratch Entropy coding Quantization Signal transforms Adaptive coding Rate control... Powerful New Tool in the Compression Tool-Box Very low complexity encoders Compression for networks of cameras Error-resilient transmission of signal waveforms Digitally enhanced analog transmission Unequal error protection without layered coding Random access... B. Girod: EE398B: Image Communication II Distributed Video Coding 36

37 Further interest: B. Girod, A. Aaron, S. Rane,, D. Rebollo-Monedero Monedero,, "Distributed Video Coding," Proceedings of the IEEE, Special Issue on Video Coding and Delivery. January