RTP Transport & Extensions Extended RTCP reporting Timely feedback from receivers to senders RTP Retransmissions Support for Source-specific Multicast (SSM) 2010 Jörg Ott, Varun Singh 66 RTP as a Transport Protocol Transport functionality Should support applications to determine and adapt to varying network conditions In a more or less timely manner (perceived less stringent requirements on congestion control) RTP offerings Ordering, time-keeping Loss, jitter, RTT detection Irregular reporting intervals at course granularity via RTCP Course-grained RTCP report information Hints only: up to the applications to react And you always need two of them Use other transports or do-it-yourself 2010 Jörg Ott 67
RTP over TCP Promise: reliability + congestion control + NAT/firewall-friendly Simple basic mechanisms Providing packet boundaries within the TCP stream Possibly multiplexing of RTP and RTCP (built into other protocols before; see RTSP) Potential issues TCP offers in sequence delivery: head of line blocking TCP is reliable: will deliver all data without regard for delay TCP offer congestion control: will yield variable transport bit rate irrespective of the codec needs Question: (when) will it work? Partial answer: TCP is widely used for streaming, so something must work 2010 Jörg Ott 68 RTP over TCP (2) Eli Brosh, Salman Abdul Baset, Dan Rubenstein, Henning Schulzrinne: The Delay-Friendliness of TCP, ACM SIGMETRICS, 2008. 2010 Jörg Ott 69
Congestion Control: RTP over DCCP DCCP recap Connection-oriented transport protocol Datagram service: no reliability, no ordering Congestion control profiles (e.g., TCP-friendly Rate Control, TFRC) RTP transmission straightforward RTP and RTCP packets map to DCCP packets DCCP provides a congestion-controlled packet flow Requires (sufficiently) adaptive codecs Experimental results (2007) showed that the examined TFRC profiles are insufficient for voice UDP and even TCP perform better See: Vlad Balan, Lars Eggert, Saverio Niccolini and Marcus Brunner. An Experimental Evaluation of Voice Quality over the Datagram Congestion Control Protocol. Proc. IEEE Infocom, Anchorage, AL, USA, May 6-12, 2007. 2010 Jörg Ott 70 Congestion Control: RTP and ECN Network response to congestion: packet drop Works ok with TCP: retransmissions allow for repairing UDP/RTP: packet losses may be immediately visible to the receiver Alternative: Explicit Congestion Notification (ECN) Endpoint negotiate support for ECN Routers mark packets they would otherwise have dropped Receivers provide feedback to sender about observed loss rate Mechanisms prevent receivers from cheating Requires modifications at the network and transport layer Specific RTP requirements Safe fallback: preserve communication if ECN fails No media clipping Topology-dependent considerations 2010 Jörg Ott 71
RTP and ECN Negotiating ECN capability As part of the media stream negotiation Initiating ECN use Probe e2e connectivity for ECN packets Occasionally mark RTP packets with ECT marks Do not mark RTCP packets Success report via RTCP ECN feedback packets Ongoing RTP session Regular use of ECT marks on all RTP packets Feedback reporting RTCP XR reports for detailed and summary reports on ECT-marked packets RTCP XR ECN nonce report 2010 Jörg Ott 72 RTP Transport & Extensions Extended RTCP reporting Timely feedback from receivers to senders RTP Retransmissions Support for Source-specific Multicast (SSM) 2010 Jörg Ott, Varun Singh 73
Extended RTCP Reporting (XR) Provide more detailed feedback (and feed forward) Infer network characteristics (point-to-point and multicast) Provide detailed voice quality information Incorporate many statistics in RTCP packets Lost and duplicate packets Exact packet receipt times Receiver reference time and reception information for RTT measurements Statistics summary VoIP metrics: Burst, gaps, delay,... Detailed reports may get large: thinning reports Report only on every 2 T -th packet (T = 0,, 15) Indicate the thinning factor T in the packet 2010 Jörg Ott 74 General report header RTCP XR 0 1 2 3 7 8 15 16 31 V P reserved RT = XR = 207 Length SSRC Report Blocks Specific report blocks Block Type Type-specific Length Type-specific block contents 2010 Jörg Ott 75
RTCP XR: Detailed Packet Reporting (1) Report (individual) lost and duplicate packets Runlength encoding or bit maps of sequences ( chunks ) Run length: Bit vector: 0 1 2 3 7 8 15 16 31 BT={1,2} Null chunk: 0x0000 rsvd. Start sequence # Chunk #1 Chunk #n-1 0 R 1 T SSRC of source reported Length End sequence # Chunk #2 Chunk #n # packets lost (R=0) or received (R=1) Bit vector (0 = lost, 1 = received packet) Also: post-repair Loss (RFC 5725) and reiceiver-side discarded packets. 2010 Jörg Ott 76 RTCP XR: Detailed Packet Reporting (2) Record individual packet reception times Ideally obtained as close to the incoming interface as possible Middle 32 bits of the NTP timestamp 0 1 2 3 7 8 15 16 31 BT=3 rsvd. Start sequence # T SSRC of source reported Reception time of packet #start Length End sequence # Reception time of packet #(start+1) % 65536 Reception time of packet #(end-2) % 65536 Reception time of packet #(end-1) % 65536 2010 Jörg Ott 77
RTCP XR: Receiver Side RTT Calculation Operation similar to RTCP SR+RR mechanism Receivers report sending and selective reception timestamps, too 0 1 2 3 7 8 15 16 31 Receiver Reference Time Report BT=4 reserved Length SSRC of source reported NTP timestamp (most significant word) NTP timestamp (least significant word) Delay since Last RR Report BT=5 reserved SSRC #1 Last RR #1 Length Delay since Last RR #1 2010 Jörg Ott 78 RTCP XR: Statistic Summary + VoIP Metrics Detailed report on reception statistics for a certain packet interval BT=6 Lost, duplicate packets Min, max, mean jitter + standard deviation VoIP Metrics (BT=7) Lost packets (network) + discarded packets (local jitter buffer = late packets) Identification of (loss/discard) bursts and (loss/discard) gaps Burst: first,, last lost packet in a sequence with loss rate > threshold (Gmin) Gap: Runs of packets which are not in a burst Gap + Burst duration (ms) and respective packet loss rate 1111111111011111111111111000101011001111110110011111111111110111111101 Gap Burst Gap 2010 Jörg Ott 79
Delays RTT delay End system delay (estimated) Signal information Signal + noise level Call quality RTCP XR: VoIP Metrics R factor, extended R factor + MOS listening, conversational Configuration parameters Gmin, packet loss concealment, jitter buffer operation (adaptiveness) Jitter buffer parameters Delay, maximum delay (observed), absolute maximum delay (buffer size) 2010 Jörg Ott 80 RTP Transport & Extensions Extended RTCP reporting Timely feedback from receivers to senders RTP Retransmissions Support for Source-specific Multicast (SSM) 2010 Jörg Ott, Varun Singh 81
RTCP Feedback Issues Senders provide regular information about media stream Seems ok Receivers transmit RTCP at somewhat regular intervals RTCP RRs provide long-term statistics on reception quality Senders can adapt transmission strategy to receiver observations Different codecs, data rate, etc. BUT: No short-term feedback possible Error repair or mitigation impossible Not suitable for congestion control Problem: Value of receiver feedback decreases over time Repair more expensive at later times Artifacts become noticeable to the user 2010 Jörg Ott 82 Approach: RTCP-based Feedback New Profile for RTP: AVPF RFC 4585 Idea: Packet losses are usually rare Provide statistical chance of virtually immediate feedback from receiver(s) to sender Keep the basic RTCP properties Eliminate Tmin Work most efficiently with unicast Also scale to moderate group sizes 2010 Jörg Ott 83
Overview Regular RTCP operation (depicted w/o randomization, i.e. T = Td) T T t Allow (at most every other) RTCP packet to be sent earlier t Allow to reduce the number of regular RTCP packets (w/o affecting RTCP rate) t 2010 Jörg Ott 84 RTCP Feedback Timing T_dither_max = f (group size,...) Last RR (tp) t0 t_e Next RR scheduled (tn) Event detected Immediate/Early RTCP 2010 Jörg Ott 85
Delay calculation T_dither_max = 0 if grp size = 2 l * T otherwise Simulated guess: l = 0.5 Better approach: use RTT measurements! But those are only available to senders Mixed operation (using Td and RTT) will not work. 2010 Jörg Ott 86 Modes of Operation Immediate FB mode Early RTCP mode Regular RTCP mode 2 Report every relevant event immediately Report many of the events but not all Group size Just regular RTCP packets Send feedback + regular RTCP packets 2010 Jörg Ott 87
RTCP Types of Feedback ACK Mode Positive acknowledgements for received packets Restricted to point-to-point operation NACK Mode Negative acknowledgments e.g. for missing packets or other events Scalable with suppression technique Other types of feedback conceivable Transport layer feedback packets (Generic NACK) Identifies missing or received packets Payload-specific feedback packets Specific to certain codecs (e.g. video) Picture / frame loss indication, reference picture selection Application feedback packets 2010 Jörg Ott 88 RTCP Feedback Packet Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 V=2 P FMT PT length SSRC of packet sender SSRC of media source : Feedback Control Information (FCI) : : : Example: Generic NACK Packet 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 PID BLP 2010 Jörg Ott 89
RTCP Feedback Packet Format (2) Example: Slice lost indication First Number PictureID Example: Reference Picture Selection 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 PB 0 Payload Type Native RPSI bit string defined per codec... Padding (0) 2010 Jörg Ott 90 Example for Statistical Feedback Applicability of feedback depends on many parameters Group size, RTP & RTCP bandwidth, application requirements 256 kbit/s video stream, 30 frames per second, 1500 bytes MTU Single sender, > 3 receivers (i.e. 3.75% RTP bandwidth for receivers) H.263+ with approximately 1 packet per frame 5% packet loss, equally distributed, receiver independence Statistically yields 3 losses every two seconds per receiver 3.75% * 256 kbit/s = 9.6 kbit/s for all receivers Assuming 120 bytes (= 960 bits) per RTCP packet: 10 packets / s If every receiver reports every loss event: 6 7 receivers on average If reporting every other loss event is sufficient: ~14 receivers Increases further if losses are correlated in some fashion 2010 Jörg Ott 91
Codec-specific Feedback Extensions Motivated by needs of different video applications Ask for reference frames from a video source (e.g., in a mixer) For video mixing and switching Signal the tradeoff between temporal and spatial resolution At a given bit rate RFC 5104 Full-Intra Request (FIR) Asks a sender to transmit an independently encoded reference frame Temporal-spatial tradeoff Chooses a trade-off point on a 5-bit scale Use a notification for reliability Temporary Maximum Media Stream Bit Rate (TMMBR) Indicates a bit rate limit in b/s Request and notification for reliability 2010 Jörg Ott 92 RTP Transport & Extensions Extended RTCP reporting Timely feedback from receivers to senders RTP Retransmissions Support for Source-specific Multicast (SSM) 2010 Jörg Ott, Varun Singh 93
RTP Retransmissions Explicit repair mechanism for RTP streams Works for applications with acceptable higher latency E.g. media streaming Applicable to point-to-point and small group scenarios Used with RTCP feedback extensions Approach Original RTP stream Augmented by retransmission RTP stream Mapped to different RTP sessions or sender SSRCs Use always different sessions for multicasting Keeps the retransmission scheme backward compatible Does not confuse RTCP statistics Works with all payload types Allows for multiple payload types in a session RFC 4588 2010 Jörg Ott 94 RTCP Retransmission Packet Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 V=2 P FMT PT length SSRC of packet sender SSRC of media source OSN +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Original RTP Packet Payload 2010 Jörg Ott 95
Summary: Applying RTP Adaptive real-time applications Tunable feedback loop for individual and group communications From reporting per 5s and more to event-driven to once per RTT Long-term adaptation Codec choice Packetization size FEC, interleaving Sender RTP Media stream (coded media, FEC, repair) RTCP Sender Reports Timing, synchronization Data rate, packet count Traffic characteristics 3 rd Party Qos Monitor Receiver Short-term adaptation Retransmissions Retro-active FEC Congestion control Adaptive source coding RTCP Receiver Reports Long-term rough statistics Detailed statistics Instant event notifications Congestion information Dejittering, sync, playout Monitoring + reporting Instant event notifications Local error concealment 2010 Jörg Ott 96 Adaptation Examples Rate control Switching between codecs (4 64 kbit/s) Variable bit-rate codecs Audio (e.g., Adaptive Multirate, AMR): ~2 12 kbit/s Video (e.g., Scalable Video Coding, SVC, H.264) Adjust packet sizes (packet rate, overhead) Error control Wireless vs. wired networks Different strategies depending on whether losses are due to errors or congestion Media-specific redundancy coding (adaptive) Dynamically adapt FEC based upon feedback Maximize quality for a fixed rate: split rate budget into data and FEC Increase the transmission rate by adding FEC (congestion!) Combined error and rate control 2010 Jörg Ott 97