University of North Carolina at Chapel Hill Multimedia Networking Research at UNC Adaptive, Best-Effort Congestion Control Mechanisms for Real-Time Communications on the Internet Kevin Jeffay F. Donelson Smith Gunner Danneels (Intel) Mark Parris Michele Clark Peter Nee http://www.cs.unc.edu/research/dirt 1 What are we doing? Trying to understand how broken the Internet is today Trying to understand how to design real-time multimedia applications for the Internet Why are we doing this? We want to understand if we should spend our efforts building a better Internet, making smarter applications, or both 2 Multimedia Networking Research at UNC How are we doing this? Developing real-time communications and computation middle-ware Building real-time applications with experimental communications software Evaluating their performance on controlled and production networks Running long-term performance studies on the Internet Adaptive, best-effort congestion control for real-time communications Outline Our driving problem realizing distributed, immersive, virtual laboratories The UNC nanomanipulator system The continuous media congestion control problem 2-Dimensional media scaling techniques Experimental results for Internet videoconferencing 3
Distributed, Immersive, Virtual Laboratories Advanced scientific instruments have computer- based or computer-enhanced interfaces Distributed, Immersive, Virtual Laboratories Example Atomic Force Microscopes Treating these systems as distributed systems enables... Better user interfaces Remote operation of instruments Multi-user and collaborative operation Sharing of instruments and specialized computing equipment Distributed, Immersive, Virtual Laboratories Distributed, Immersive, Virtual Laboratories CCD Image Computer Enhanced Image The UNC nanomanipulator system A virtual environment interface to a scanning- probe microscope Provides telepresence on sample surfaces scaled 1,,:1 7 8
Distributed Virtual Laboratories Networking challenges Distributed Virtual Laboratories Networking challenges Graphics Engine & Host 8 Kbps ( touch mode ) 81 Kbps ( scan mode ) 8 Kbps ( touch mode ) 81 Kbps ( scan mode ) 2 Mbps (max) Intel-based Microscope Controller Internetwork Intel-based PHANToM Controller Visual Feedback AFM Control Processing 1. Gbps (max) Internetwork User Interface & Application Processing 9 Kbps Atomic Force Microscope Force Feedback 9 3D Graphics Processing Hand Tracking & Feedback Control 1 Distributed Virtual Laboratories nm distribution scenarios Scientific collaboration over Integrated Services networks Equal distribution of graphics, tracking, and microscope hardware The North Carolina GigaPOP Duke OC-8 MCNC OC-8 NCSU OC-8 Sonet Node Sonet Node Sonet Node UNC OC-8 Sonet Node Distributed Virtual Laboratories nm distribution scenarios Scientific collaboration over Integrated Services networks Scientific collaboration over high-speed, best effort internetworks Co-located graphics & microscope hardware, remote tracking & user interface Co-located microscope hardware, tracking & user interface, remote graphics engine 11 12
Distributed Virtual Laboratories nm distribution scenarios Scientific collaboration over Integrated Services networks Scientific collaboration over high-speed, best effort internetworks Educational outreach over the Internet Co-located graphics & microscope hardware, remote tracking & user interface Adaptive, best-effort congestion control for real-time communications Outline Our driving problem realizing distributed, immersive, virtual laboratories The UNC nanomanipulator system The continuous media congestion control problem 2-Dimensional media scaling techniques Experimental results for Internet videoconferencing 13 1 Cont. Media Congestion Control Effect of packet loss on UNC campus Cont. Media Congestion Control Effect of packet loss across 1 hops 1
Cont. Media Congestion Control The UNC approach Cont. Media Congestion Control The UNC approach Legacy LANs Router Operating principle: Packet- Switched Internetwork Router Legacy LANs Network elements that cannot reserve, or support real-time allocation of resources, will persist for the foreseeable future. Focus on adaptive, best-effort transmission... Treat the network as a black box Assume only that sufficient bandwidth exists for some useful execution of the system... with real-time media control at the endpoints Adaptive, best-effort congestion control for real-time communications 17 Transmission Reception Congestion in the small: delay-jitter Elastic queueing to manage the trade-off between low latency playout and gap-rate Congestion in the large: packet loss Display Initiation Time Adaptive media scaling and packaging to decrease network queueing (latency) and minimize packet loss Congestion Control The nature of congestion x 18 Outline Our driving problem realizing distributed, immersive, virtual laboratories The UNC nanomanipulator system The continuous media congestion control problem 2-Dimensional media scaling techniques Experimental results for Internet videoconferencing 19 Switch Fabric What causes congestion? Switch Fabric Did our multimedia stream(s) cause the network to be congested? Are there simply too many connections competing for too little bandwidth? 2
Congestion Control The nature of congestion Canonical Adaptive Congestion Control Video bit-rate scaling Switch Fabric Switch Fabric How can we make the best use of the (time varying) bandwidth that is available to our streams? How can we determine what this bandwidth is? How can we track how it changes over time? How can we match our application s output to the available bandwidth? 21 Temporal scaling Reduce the resolution of the stream by reducing the frame rate Spatial scaling Reduce number of pixels in an image Frequency scaling Reduce the number of DCT coefficients used in compression Amplitude scaling Reduce the color depth of each pixel in the image Color space scaling Reduce the number of colors available for displaying the image 172 21-9 -1-8 -18-3 -8-2 -2-3 -8 2 - - -3 - - - -8-8 3 23-1 - - -3-2 -9 11 3 3 1-1 1 3 2 2 1 19 7-1 1-1 1 22 UNC Adaptive Congestion Control 2-Dimensional media scaling Canonical approach to congestion Reduce (video) bit-rate Alternate approach View congestion control as a search of a 2-dimensional bit-rate x packet-rate space Scale bit- and packet-rates simultaneously to find a sustainable operating point 2, 1, 1, Bit-Rate (in kbits/s) High-Quality Video (8, bytes/frame) Medium-Quality Video (, bytes/frame) Low-Quality Video (2, bytes/frame) Audio (2 bytes/sample) 3 Packet-Rate (in packets/s) 23 Non-Real-Time Real-Time Bit- and Packet-Rate Scaling An analytic model of media scaling Service Capacity constraints the network is incapable of supporting the desired bit rate in any form Access constraints Inbound Data Movement & Packet Processing Other Outbound Links Adaptor Outbound Media Access & Xmission Time the network can not support the desired bit rate with the current packaging scheme 2
Bit- and Packet-Rate Scaling An analytic model of media scaling Bit- and Packet-Rate Scaling An analytic model of media scaling Non-Real-Time Other Outbound Links Non-Real-Time Other Outbound Links Inbound Outbound Inbound Outbound Real-Time Service Data Movement & Packet Processing Adaptor Media Access & Xmission Time Reduce the packet-rate to adapt to an access constraint Change the packaging or send fewer video frames Primary Trade-off: higher latency (potentially) 2-Dimensional Scaling Example The Recent Success heuristic Initial operating point: (high quality, 12 fps) First adaptation: (high quality, 1 fps) congestion persists Second adaptation: (medium quality,, 1 fps) congestion relieved First probe: (medium quality,, 12 fps) Second probe: (medium quality,, 1 fps) Bit-Rate (in bits/s) Packet-Rate (in packets/s) 2 27 Real-Time Service Data Movement & Packet Processing Adaptor Media Access & Xmission Time Reduce the packet-rate to adapt to an access constraint Reduce the bit-rate to adapt to a capacity constraint Send fewer video frames or fewer bits per video frame Primary Trade-off: lower fidelity 2-Dimensional Media Scaling Finding a sustainable operating point The search space can be pruned by eliminating points that lead to inherently high latency points that lead to high latency given the state of the network 2, 1, 1, Bit-Rate (in kbits/s) 3 Packet-Rate (in packets/s) 2 28
2-Dimensional Media Scaling Dealing with effects of fragmentation 2-Dimensional Media Scaling Does it it work? The problem A sender can only (directly) effect the message rate,, not the packet rate Does fragmentation render message-rate scaling obsolete? Packet-Rate (packets/sec) 18 1 1 12 1 8 2 Message-Rate (mesg/sec) Audio Video 1 9 13 17 2 21 2 29 33 37 128 1 9 1 3 7 8, bytes frame, bytes frame 2, bytes frame 1 12 8 38 Bit-Rate (kbits/sec) Internet Campus-sized internets? The Internet? 29 3 2-Dimensional Media Scaling Does it it work? 2-D Scaling on the UNC Campus Performance with no media scaling Experiments Local LAN Adaptation Scheme 1 Adaptation Scheme 2 Baseline UDP transmission, no adaptations 1-Dimensional media scaling (video bit-rate scaling) Audio and video media scaling & packaging Internet Metrics Remote Reflector UVa, CMU, U. Washington Delivered media frame rate (throughput) Packet loss Media stream latency Adaptations performed over 8 7 3 2 1 8 7 3 2 1 Video FPS (actual) Audio FPS (actual) 1 2 2 3 3 Audio Delivered Latency Audio Induced Latency 1 2 2 3 3 1 2 2 3 3 time 31 32 Audio Latency (ms) 1 8 2 8 7 3 2 1 Lost 1 2 2 3 3 Video Delivered Latency Video Latency (ms)
2-D Scaling on the UNC Campus Performance with video scaling only 2-D Scaling on the UNC Campus Video scaling v. v. no adaptation 8 7 Video FPS (actual) Audio FPS (actual) 1 Lost 8 7 Video FPS (actual) Audio FPS (actual) 8 7 Video FPS (actual) Audio FPS (actual) 8 3 3 3 2 1 2 1 2 2 1 2 2 3 3 1 2 2 3 3 1 1 Lost 1 Lost 1 2 2 3 3 8 7 Audio Delivered Latency Audio Induced Latency 1 2 2 3 3 8 Video Delivered Latency 7 8 2 8 2 8 7 1 2 2 3 3 Audio Delivered Latency Audio Induced Latency 8 7 1 2 2 3 3 Audio Delivered Latency Audio Induced Latency 3 3 2 1 2 1 3 Audio Latency (ms) 3 1 2 2 3 3 Audio Latency (ms) 1 2 2 3 3 Video Latency (ms) 33 2 1 1 2 2 3 3 2 1 1 2 2 3 3 3 2-D Scaling on the UNC Campus Performance with 2-dimensional scaling 2-Dimensional Media Scaling Does it it work? 8 7 Video FPS (actual) Audio FPS (actual) 1 Lost 8 3 Internet 2 1 2 8 7 3 2 1 1 2 2 3 3 Audio Delivered Latency Audio Induced Latency 1 2 2 3 3 8 7 3 2 1 1 2 2 3 3 Video Delivered Latency 1 2 2 3 3 Campus-sized internets yes! It solves the first-mile/last-mile problem The Internet? well... Does our necessary condition for success hold? Does it hold often enough to be useful? How much room is there for 2-D scaling in most codecs? Audio Latency (ms) Video Latency (ms) 3 37
2-D scaling evaluation on the Internet Media scaling in Intel s ProShare TM TM codec 2-D scaling evaluation on the Internet ProShare with no media scaling Bit-Rate 3 3 2 2 Video Audio 8 7 3 1 2 1 3 2 1 1 2 3 8 7 1 2 3 8 7 1 2 3 Packet-Rate ProShare operating points 38 3 2 1 1 2 3 Audio Latency (ms) 3 2 1 1 2 3 Video Latency (ms) 39 2-D scaling evaluation on the Internet ProShare with 2-dimensional media scaling 2-D scaling evaluation on the Internet 2-dimensional v. v. 1-dimensional media scaling 3 3 Video Audio 8 7 3 3 Video Audio 3 3 Video Audio 2 2 2 2 2 2 3 1 2 1 1 1 1 2 3 1 2 3 1 2 3 1-D 1 2 3 2-D 8 8 8 8 7 7 7 7 3 3 3 3 2 2 2 2 1 1 1 1 1 2 3 1 2 3 1 2 3 1 2 3 Audio Latency (ms) Video Latency (ms) 2-D 3 1-D
Sustainability Results Adaptive methods on the Internet Results of an Internet performance study from UNC to UVa Repeated trials from 1 am to 7 PM weekdays Trials separated by at least two hours Scattered over three months Time Slot Sustainable Not Sustainable 1:-12: 7% 33% 12:-1: % % 1:-1: 8% 92% 1:-18: 2% 7% 18:-2: % % Percentage 39% 1% Adaptive, best-effort congestion control for real-time communications Outline Our driving problem realizing distributed, immersive, virtual laboratories The UNC nanomanipulator system The continuous media congestion control problem 2-Dimensional media scaling techniques Experimental results for Internet videoconferencing Adaptive, best-effort congestion control for real-time communications Summary Real-time applications must be adaptive to be effective on the Internet Simple middleware adaptations are sufficient for accommodating most Internet pathologies within the intranet Biasing how a bit-stream is partitioned into packets is more effective than reducing the bit-stream Original Media Sender Will best-effort techniques scale? Router-based congestion control Router QoS Connection Setup Receiver Media Adaptations QoS Capable Network Recursively apply endpoint media adaptations in the network Delay-jitter management adaptations Congestion/flow control adaptations Router QoS Connection Setup Sender Media Adaptations Final Media Receiver Compare performance against CBQ gateways RED packet discard for TCP Delete Oldest & Advance discard for multimedia 7