Ph.D. Oral Defense Distributed Rate Allocation for Video Streaming over Wireless Networks Xiaoqing Zhu Tuesday, June, 8 Information Systems Laboratory Stanford University Wireless Home Video Networking SDTV HDTV Mbps 6 Mbps Mbps Microwave Media gateway Mbps Cordless phone X. Zhu: Distributed Rate Allocation for Video over Wireless
What s Wrong with TCP? Link Speed: Mbps Throughput :. Mbps File Transfer Size:.7MB Shared :. Mbps (~ 8% Channel Time) Shared :. Mbps (~ 6% Channel Time) Link Speed: Mbps Throughput : Mbps Video Source @ Mbps X. Zhu: Distributed Rate Allocation for Video over Wireless Review Outline Network and media heterogeneity Related work Media-aware distributed rate allocation System model and optimization Practical protocol design Performance evaluation Extension to wireless video multicast Rate adaptation via scalable video coding (SVC) Comparison with TFRC-based heuristics X. Zhu: Distributed Rate Allocation for Video over Wireless
Network Heterogeneity Channel Time X. Zhu: Distributed Rate Allocation for Video over Wireless TCP Throughput over Wireless Simulation in NS, for 8.a network Stream Stream, alone Mbps 6 - Mbps Stream Throughput (Mbps) Stream, alone Stream, shared Stream, shared [Heusse et al. ] Nominal Speed of Second Link (Mbps) X. Zhu: Distributed Rate Allocation for Video over Wireless 6
Flow Starvation with TCP Stream TCP Throughput Stream 6 8 7 9 Stream Rate (Mbps) 8 7 6 Stream Stream Stream [Xu et al. ] X. Zhu: Distributed Rate Allocation for Video over Wireless 7 Media Heterogeneity BusCIF PSNR (db) CitySD FountainHD Rate (Mbps) Different video has different utility of rate, depending on resolution, codec choice, and video content How to best trade off their quality? X. Zhu: Distributed Rate Allocation for Video over Wireless 8
Subjective Evaluation HD/SD Display Subject Control Panel Evaluation of quality combinations of HD/SD image pairs Mean-opinion-scores (MOS) from 8 viewers, data sets X. Zhu: Distributed Rate Allocation for Video over Wireless 9 MOS of Allocation Results HD stream C : Mbps Increasing C C : - Mbps SD stream o: subjective x: weighted MSE + : MSE : TFRC X. Zhu: Distributed Rate Allocation for Video over Wireless
MOS of Allocation Results HD stream C : Mbps C : - Mbps SD stream Media-aware allocation improves viewing experience X. Zhu: Distributed Rate Allocation for Video over Wireless Related Work Rate allocation for wired network TCP congestion control [V. Jacobson 988] TCP-Friendly Rate Control (TFRC) [Floyd, Fall 999] Pricing-based mathematical framework [Kelly 997-98] [Chiang 7] Video streaming over wireless networks Flow control via multiple TFRC connections [Chen, Zakhor, -6] Media-aware centralized channel time allocation [Kalman, van Beek, Girod ] X. Zhu: Distributed Rate Allocation for Video over Wireless
Review Outline Network and media heterogeneity Related work Media-aware distributed rate allocation System model and optimization Practical protocol design Performance evaluation Extension to wireless video multicast Rate adaptation via scalable video coding (SVC) Comparison with TFRC-based heuristics X. Zhu: Distributed Rate Allocation for Video over Wireless Wireless Network Model Stream s Link utilization: Interference set L l Utilization over interference set: X. Zhu: Distributed Rate Allocation for Video over Wireless
Video Distortion Model Scene cuts Distortion D [Stuhlmüller et al. ] Rate R Rate Dependent on video content and encoder settings Parameters updated for every GOP (~. s) X. Zhu: Distributed Rate Allocation for Video over Wireless Optimization Objective weight of stream importance distortion of stream upper limit total utilization Motivated by results from subjective viewing test Convex objective function with linear constraints X. Zhu: Distributed Rate Allocation for Video over Wireless 6
Congestion price update: Distributed Solution Video rate update: Residual utilization Distortion D Rate R Accumulated congestion price: X. Zhu: Distributed Rate Allocation for Video over Wireless 7 Protocol Design Update congestion price: Advertise video rate Report accumulated congestion price Exchange link state with neighbors: Video packet Acknowledgment Link state message X. Zhu: Distributed Rate Allocation for Video over Wireless 8
Overhead vs. Convergence Time Crew Mbps Mbps Mbps 6 X. Zhu: Distributed Rate Allocation for Video over Wireless 9 Adaptation to Stream Arrival/Departure Capacity (Mbps) Price Λ Allocated Rate (Mbps) PSNR (db) Demo Fountain vs. Crew Fountain Crew Time (s) Convergence time: - seconds X. Zhu: Distributed Rate Allocation for Video over Wireless
Fountain Crew Start: 7.8 db Start: 8. db End:. db End: 8.8 db X. Zhu: Distributed Rate Allocation for Video over Wireless Adaptation to Video Scene Cut Capacity (Mbps) Price Λ Allocated Rate (Mbps) PSNR (db) Fountain vs. Fountain/Crew Fountain Fountain/Crew Time (s) X. Zhu: Distributed Rate Allocation for Video over Wireless
Adaptation to Link Speed Drop Capacity (Mbps) Price Λ Allocated Rate (Mbps) PSNR (db) Crew vs. Crew Crew, Mb Crew, 6Mb Time (s) X. Zhu: Distributed Rate Allocation for Video over Wireless Varying Link Speed: Dijana vs. Raven Stream, media-aware Mbps 6 - Mbps Stream, TFRC Stream, TFRC Stream, media-aware Demo Improvement of average video quality:.7-.8 db X. Zhu: Distributed Rate Allocation for Video over Wireless
TFRC Allocation X. Zhu: Distributed Rate Allocation for Video over Wireless Media-Aware Allocation X. Zhu: Distributed Rate Allocation for Video over Wireless 6
TFRC PSNR:. db Media-Aware PSNR: 6.8 db X. Zhu: Distributed Rate Allocation for Video over Wireless 7 Varying Link Speed: Multi-Hop Network 6 - Mbps PSNR (db) Fountain vs. Cyclists media-aware TFRC Link Speed (Mbps) PSNR (db) 9 Dijana vs. Raven. -. db.9 -. db media-aware TFRC Link Speed (Mbps) 6 8 Crew vs. Crew 6 Raven vs. Fountain PSNR (db) 7 6.7 -. db media-aware TFRC Link Speed (Mbps) PSNR (db).8 -.9 db media-aware TFRC Link Speed (Mbps) X. Zhu: Distributed Rate Allocation for Video over Wireless 8
Fairness Among Streams Stream Stream 8 6 Rate (Mbps) 7 6 Fountain Stream Stream Stream 7 9 TCP TFRC Media-Aware Stream X. Zhu: Distributed Rate Allocation for Video over Wireless 9 Fairness Among Streams Stream Stream Stream Stream Rate (Mbps) Fountain Stream Stream Stream Stream TCP TFRC Media-Aware X. Zhu: Distributed Rate Allocation for Video over Wireless
Protocol Scalability Overhead (%) 7 6 video ACK LSM update 6 7 8 Grid Size X. Zhu: Distributed Rate Allocation for Video over Wireless Review Outline Network and media heterogeneity Related work Media-aware distributed rate allocation System model and optimization Practical protocol design Performance evaluation Extension to wireless video multicast Rate adaptation via scalable video coding (SVC) Comparison with TFRC-based heuristics X. Zhu: Distributed Rate Allocation for Video over Wireless
Video Multicast over Wireless Need rate adaptation at each peer X. Zhu: Distributed Rate Allocation for Video over Wireless Optimization Framework Stream s Objective: 9 F l,c l R s l Constraints: Limit utilization within each interference set Rate of child limited by rate of parent 7 Distributed solution: 6 8 Interference set L l X. Zhu: Distributed Rate Allocation for Video over Wireless
Scalable Video Coding in H.6 Enhancement Layer (EL) Base Layer (BL) T T T T T T T T T Each frame contains one BL packet and one EL packet Rate adaptation by dropping EL packets Frames X. Zhu: Distributed Rate Allocation for Video over Wireless Rate Adaptation with H.6/SVC 6 CIF Sequences @ 6fps PSNR (db) 8 City Crew Soccer Harbor.6.8.. Rate (Mbps).6.8 X. Zhu: Distributed Rate Allocation for Video over Wireless 6
Allocation over Single Tree: Traces Mbps 6 8 Mbps Convergence time: ~ seconds X. Zhu: Distributed Rate Allocation for Video over Wireless 7 Allocation over Single Tree: PSNR per Peer media-aware TFRC Mbps 6 7 6 - Mbps PSNR (db) PSNR (db) PSNR (db) 8 8 8 Demo 6 Link Speed (Mbps) 8 8 8 7 Link Speed (Mbps) X. Zhu: Distributed Rate Allocation for Video over Wireless 8
Video Quality of Peer TFRC: 9. db Media-aware:. db X. Zhu: Distributed Rate Allocation for Video over Wireless 9 Allocation over Two Trees. -.6 db. -.6 db 9 7 6 8. -. db 6 - Mbps X. Zhu: Distributed Rate Allocation for Video over Wireless
Summary of Contributions Media- and network-aware optimization framework Incorporates different video utility functions Accommodates heterogeneous wireless link speeds Captures impact of traffic contention among neighboring links Practical distributed allocation protocol Cross-layer design: fast convergence via rate advertising Approximates subjectively preferred allocation results Outperforms media-unaware TFRC Extension to video multicast over wireless Graceful quality adaptation at intermediate nodes via SVC Higher average video quality than TFRC-based heuristics X. Zhu: Distributed Rate Allocation for Video over Wireless Main Publications Overview and tutorial articles X. Zhu and B. Girod, Video Streaming Over Wireless Networks, Invited Tutorial, Proc EUSIPCO 7 E. Setton, T. Yoo, X. Zhu, A. Goldsmith and B. Girod, Cross-layer Design of Ad Hoc Networks for Real-Time Video Streaming, IEEE Wireless Communications Magazine, August, Invited Paper Media-aware rate allocation X. Zhu and B. Girod, "Subjective Evaluation of Multi-User Rate Allocation for Streaming Heterogeneous Video Contents over Wireless Networks, Proc. ICIP 8, to appear X. Zhu, T Schierl, T. Wiegand, and B. Girod, "Video Multicast over Wireless Mesh Networks with Scalable Video Coding (SVC)", Proc. VCIP 8 X. Zhu, P. Agrawal, J. P. Singh, T. Alpcan, and B. Girod, "Rate Allocation for Multi-User Video Streaming over Heterogenous Access Networks", ACM Multimedia 7, Best Student Paper Award X. Zhu, P. van Beek and B. Girod, "Distributed Channel Time Allocation and Rate Adaptation for Multi-User Video Streaming over Wireless Home Networks, Proc. ICIP 7 X. Zhu and B. Girod, "Distributed Rate Allocation for Video Streaming over Wireless Networks with Heterogeneous Link Speeds", Proc. ISMW 7 X. Zhu, J. P. Singh, and B. Girod, "Joint Routing and Rate Allocation for Multiple Video Streams in Ad Hoc Wireless Networks", Proc. PV 6 X. Zhu and B. Girod, "Media-Aware Multi-User Rate Allocation over Wireless Mesh Networks", Proc. OpComm 6 X. Zhu and B. Girod, "Distributed Rate Allocation for Multi-Stream Video Transmission over Ad Hoc Networks", Proc. ICIP X. Zhu, S. Han and B. Girod, "Congestion-Aware Rate Allocation for Multipath Video Streaming over Ad Hoc Wireless Networks", Proc. ICIP X. Zhu: Distributed Rate Allocation for Video over Wireless
Main Publications (cont d) Cross-layer design for video over wireless S. Adlakha, X. Zhu, B. Girod and A. Goldsmith, "Joint Capacity, Flow and Rate Allocation for Multiuser Video Streaming over Wireless Ad-Hoc Networks", Proc. ICC 7 X. Zhu and B. Girod, "Analysis of Multi-User Congestion Control for Video Streaming over Wireless Networks", Proc. ICME 6 X. Zhu, E. Setton and B. Girod, Congestion-Distortion Optimized Video Transmission over Ad Hoc Networks, Journal of Signal Processing: Image Communications, September E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Scheduling of Video over Wireless Ad Hoc Networks", Proc. ISCAS X. Zhu, S. Rane and B. Girod, "Systematic Lossy Error Protection (SLEP) for Video Transmission over Wireless Ad Hoc Networks", Proc. VCIP X. Zhu and B. Girod, "A Distributed Algorithm for Congestion-Minimized Multi-Path Routing over Ad Hoc Networks",Proc. ICME T. Yoo, E. Setton, X. Zhu, A. Goldsmith and B. Girod, "Cross-Layer Design for Video Streaming over Wireless Ad Hoc Networks", Proc. MMSP E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Multipath Streaming of Video over Ad Hoc Wireless Networks", Proc. ICME E. Setton, X. Zhu and B. Girod, "Minimizing Distortion for Multipath Video Streaming over Ad Hoc Networks", Proc. ICIP X. Zhu: Distributed Rate Allocation for Video over Wireless Other Contributions Application to surveillance camera networks P. Baccichet, X. Zhu, and B. Girod, "Network-Aware H.6/AVC Region-of-Interest Coding for a Multi-Camera Wireless Surveillance Network", Proc. PCS 6 X. Zhu, E. Setton and B. Girod, "Content-Adaptive Coding and Delay-Aware Rate Control for A Multi-Camera Wireless Surveillance Network", Proc. MMSP X. Zhu, E. Setton and B. Girod, "Rate Allocation For Multi-Camera Surveillance over an Ad Hoc Wireless Network", Proc. Picture Coding Symposium, Proc. PCS Light field compression C.-L. Chang, X. Zhu, P. Ramanathan, and B. Girod, "Light Field Compression Using Disparity- Compensated Lifting and Shape Adaptation", IEEE Trans. Image Processing, April 6 X. Zhu, A. Aaron and B. Girod, "Distributed Compression for Large Camera Arrays", Proc SSP C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Inter-View Wavelet Compression of Light Fields With Disparity-Compensated Lifting", Proc. VCIP C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Shape-Adaptation for Light Field Compression", Proc. ICIP B. Girod, C.-L. Chang, P. Ramanathan and X. Zhu, "Light Field Compression Using Disparity- Compensated Lifting", Proc. ICASSP X. Zhu: Distributed Rate Allocation for Video over Wireless