Congestion-aware Rate Allocation For Multipath Video Streaming Over Ad Hoc Wireless Networks Xiaoqing Zhu, Sangeun Han and Bernd Girod Information Systems Laboratory Department of Electrical Engineering Stanford University
Outline Multipath video streaming over ad hoc wireless networks Congestion-aware rate allocation Encoder distortion model Network congestion model Layered representation vs. partial repetition Optimization: gradient descent Video streaming results X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 2
Video Streaming Over Wireless Ad Hoc Network Ad hoc wireless network: No fixed infrastructure Each node can be a source, destination or relay Limited bandwidth and power, adverse channel conditions Real-time media applications: Search-and-rescue, highway automation, etc. High rate and delay constraints X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 3
Multipath Video Streaming Benefits of multipath video streaming: Robustness to correlated packet losses [J. Apostolopoulos,1999] Resilience to link failure [S. Mao et al., 2003] Expansion of available bandwidth [E. Setton et al. 2004] Need rate allocation among different paths for scalable video bitstreams X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 4
Problem Statement Rate allocation of embedded video bitstreams over multiple paths, one group of frames at a time Objective: to minimize expected video distortion at the decoder Constraints: Total rate constraint Upper and lower limit for each video frame Transmission rate constraint on each link X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 5
Wavelet Video Coder Original video frames 0 1 2 3 4 5 6 7 LLH LL L LH LH H H H HH H H HH HH H HH HH H Temporal Wavelet Transform Spatial Spatial Wavelet Transform Embedded Quantization & Entropy Coding [Taubman & Zakhor, 1994] [Ohm, 1994] [Choi & Woods, 1999] [Hsiang & Woods, 1999]... Scalable bitstream X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 6
Encoder Distortion Model Distortion-rate function modeled as sum of exponentials d (l) enc (r) = i=1 N c i exp(-α i r) Sample fit of the model x 10 7 x 10 5 16 14 data model 2.5 data model Total Distortion 12 10 8 6 Total distortion 2 1.5 1 4 2 0.5 0 0 1000 2000 3000 4000 5000 6000 7000 Total number of bits Low frequency band RD fit 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Total number of bits x 10 4 High frequency band RD fit X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 7
Network Congestion Model Independent delay distribution on each path Congestion dominated by the bottleneck link M/M/1 queuing model for delay distribution P m = exp(-β m (C m -R avg,m )T playout )/B Playout deadline Packet size Residual channel capacity on the bottleneck link correction factor for overestimate of the drop rate X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 8
Bitstream Partition: Layered Representation Non-overlapping information on each path Higher efficiency More susceptible to packet loss D (l) enc(r a, R b ) = d (l) enc(r a + R b ) D R Path a encoder Path b decoder X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 9
Bitstream Partition: Partial Repetition Repeating the starting part of the bitstream on both paths Sacrifice coding efficiency More robust to channel errors and packet loss D (l) enc(r a, R b ) = d (l) enc(max(r a, R b )) D R Path a encoder Path b decoder X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 10
Optimizing Expected Decoder Distortion To find the optimal rate allocation matrix R : Minimize : D dec (R) = f (D (l) enc(r), P m (R)) Subject to: R ml R min (l) R ml R max (l) R avg, m =(V fps /L) L l=1r ml C m m = 1,, M, l = 1,, L D dec (R) depends on encoder distortion, packet drop rate on each path, and the bitstream partition strategy Optimization method: Gradient descent with the Lagrangian multiplier technique Heuristic: proportional rate allocation according to residual capacity on each path: R ml / (C m R avg,m ) for all video frames X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 11
Video Streaming Experiments Video sequence: Foreman QCIF video sequence at 30 fps, 300 frames Scalable video bistream from 3-D wavelet coding One-level temporal Haar transform Three-level spatial bi-orthogonal 9/7 decomposition Network simulation in NS-2 A network with 15 randomly placed nodes M = 2 disjoint paths, each with 4 hops Link capacities calculated based on Shannon s formula for AWGN Cross traffic randomly fills up to 70% of link capacity Propagation delay and random packet loss ignored X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 12
Video Results: Playout Deadline = 350ms Sequence : Foreman QCIF (Y) Sequence length : 300 Frame rate : 30 fps GOP length: 16 frames Temporal transform: 1-level Haar Spatial transform: 3-level bi-9/7 Playout deadline : 350 ms Packet Size: 1000 bytes Simulation time: 10,000 sec. No. of realizations: ~30 PSNR (db) 34 32 30 28 26 24 22 Foreman QCIF, Playout Deadline T = 350 ms 20 encoder layered opt 18 repetitive opt layered heur repetitive heur 16 50 100 150 200 Rate (kbps) 250 300 350 X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 13
Video Results: Sample Sequence T = 350 ms, Layered representation Optimal Scheme: Rate: 302.29 kbps Heuristic scheme: Rate: 306.44 kbps X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 14
Video Results: Playout Deadline = 150ms Sequence : Foreman QCIF (Y) Sequence length : 300 Frame rate : 30 fps GOP length: 16 frames Temporal transform: 1-level Haar Spatial transform: 3-level bi-9/7 Playout deadline : 150 ms Packet Size: 1000 bytes PSNR (db) Simulation time: 10,000 sec. No. of realizations: ~30 34 32 30 28 26 24 22 Foreman QCIF, Playout Deadline T = 150 ms 20 encoder layered opt 18 repetitive opt layered heur repetitive heur 16 50 100 150 200 Rate (kbps) 250 300 350 X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 15
Video Results: Sample Sequence T = 150 ms, Partial repetition Optimal scheme: Rate: 222.99 kbps Heuristic scheme: Rate: 225.67 kbps X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 16
Conclusions Proposed a rate-allocation scheme for streaming scalable video representation over multiple paths in an ad hoc wireless network Incorporate both the encoder performance and network congestion in the optimization of expected decoder distortion Compare two bitstream partition strategies: layered representation and partial repetition Experimental results show improvement of 0.5dB in received video PSNR over a heuristic scheme for varying playout deadline and both bitstream partition strategies. X. Zhu et al. : Rate allocation for video streaming over ad hoc network ICIP 2004 17