Multimedia Streaming Protocols RTSP/RTP RTCP. Prof. Lin Weiguo Copyleft 2009~2017, School of Computing, CUC

Transcription

1 Multimedia Streaming Protocols RTSP/RTP RTCP Prof. Lin Weiguo Copyleft 2009~2017, School of Computing, CUC Dec 2017

2 RTSP RTP RTCP SDP 2 Advanced Windows Network Programming

3 RTSP Real Time Streaming Protocol RFC Advanced Windows Network Programming

4 Introduction } RTSP is like a network remote control for multimedia servers } RTSP establishes and controls streams of continuous media } Streaming data is carried out of band (interleaving is possible) 4 Advanced Windows Network Programming

5 User Control of Streaming Media: RTSP HTTP } does not target multimedia content } no commands for fast forward, etc. RTSP: } client-server application layer protocol } user control: rewind, fast forward, pause, resume, repositioning, etc What it doesn t do: } doesn t define how audio/video is encapsulated for streaming over network } doesn t restrict how streamed media is transported (UDP or TCP possible) } doesn t specify how media player buffers audio/video 5 Advanced Windows Network Programming

6 RTSP: out of band control FTP uses an out-of-band control channel: } file transferred over one TCP connection. } control info (directory changes, file deletion, rename) sent over separate TCP connection } out-of-band, in-band channels use different port numbers RTSP messages also sent out-of-band: } RTSP control messages use different port numbers than media stream: out-ofband. } port 554/8554 } media stream is considered in-band. 6 Advanced Windows Network Programming

7 Protocol Properties } RTSP is text based and HTTP like } Transport independent (can use UDP or TCP) } Unlike HTTP commands, can be sent in either direction } RTSP is a stateful protocol. Server maintains session state. 7 Advanced Windows Network Programming

8 RTSP Connections } RTSP messages can be sent over: } Persistent transport connections used for several requestresponse transactions } One connection per request/response transaction } Connectionless mode } Transport connection!= Session } Server can send message to client only in persistent connection 8 Advanced Windows Network Programming

9 Pipelining } A client that supports persistent connections or connectionless mode MAY "pipeline" its requests (i.e., send multiple requests without waiting for each response). A server MUST send its responses to those requests in the same order that the requests were received. 9 Advanced Windows Network Programming

10 RTSP Reliability } If using reliable transport message is sent just once. } If using unreliable transport, RTSP will retransmit if it does not receive ACK } Timeout is initially set to 500 ms } Can re-compute timeout based on RTT like TCP } For retransmission CSeq is not incremented } Timestamp is used to overcome retransmission ambiguity problem 10 Advanced Windows Network Programming

11 RTSP URI rtsp://media.example.com:554/twister/audio } rtsp identifies reliable protocol } rtspu identifies unreliable protocol } Default port is 554/8554 } Presentation: twister } Stream: audio 11 Advanced Windows Network Programming

12 RTSP Message } Text based messages } Messages use Unicode (ISO 10646) character set encoded using UTF-8 } Lines are terminated by CRLF } Messages can be request or response by either client or server } Message structure is HTTP like 12 Advanced Windows Network Programming

13 Request Message Method URI Version CRLF Header1: Value CRLF Header2: Value CRLF CRLF Body } First line is Request line } URI is always absolute (unlike HTTP can be relative) } URI can be * } Example OPTIONS * RTSP/ Advanced Windows Network Programming

14 Response Message Version Status-Code Reason-Phrase CRLF } Status-Codes are HTTP like } 1xx: Informational } 2xx: Success } 3xx: Redirection } 4xx: Client error } 5xx: Server error } Reason-Phrase is the textual description 14 Advanced Windows Network Programming

15 Working of RTSP Client Browser HTTP Get Stream URL Server Web Server OPTION DESCRIBE SDP Info Initiation Media Player RTCP Message SETUP PLAY RTP Media Stream Media Server Handling PAUSE TEARDOWN Termination 15 Advanced Windows Network Programming

16 RTSP Methods } DESCRIBE Get (low level) description of media object } OPTIONS Get available methods } SETUP Establish transport } PLAY Start playback, reposition } PAUSE Halt delivery, but keep state } RECORD Start recording } ANNOUNCE Change description of media object } REDIRECT Redirect client to new server } GET_PARAMETER Retrieves presentation parameter value } SET_PARAMETER Sets presentation parameter value } TEARDOWN Remove state 16 Advanced Windows Network Programming

17 RTSP Methods Overview method direction object requirement DESCRIBE C->S P,S recommended ANNOUNCE C->S, S->C P,S optional GET_PARAMETER C->S, S->C P,S optional OPTIONS C->S, S->C P,S required (S->C: optional) PAUSE C->S P,S recommended PLAY C->S P,S required RECORD C->S P,S optional REDIRECT S->C P,S optional SETUP C->S S required SET_PARAMETER C->S, S->C P,S optional TEARDOWN C->S P,S required Overview of RTSP methods, their direction, and what objects (P: presentation, S: stream) they operate on 17 Advanced Windows Network Programming

18 OPTIONS rtsp://computing.cuc.edu.cn/test.mp3 RTSP/1.0\r\n CSeq: 2\r\n User-Agent: LibVLC/1.1.9\r\n \r\n Options method querys available methods from the server. 18 Advanced Windows Network Programming

19 RTSP/ OK\r\n Cseq: 1\r\n Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, SCALE, GET_PARAMETER\r\n \r\n An OPTIONS request may be issued at any time, e.g., if the client is about to try a nonstandard request. 19 Advanced Windows Network Programming

20 DESCRIBE rtsp:// /test.mp3 RTSP/1.0\r\n CSeq: 2\r\n Accept: application/sdp\r\n User-Agent: LibVLC/1.1.9\r\n \r\n The DESCRIBE method retrieves the description of a presentation or media object identified by the request URL from a server. It may use the Accept header to specify the description formats that the client understands. 20 Advanced Windows Network Programming

21 RTSP/ OK\r\n CSeq: 3\r\n Date: Sun, Dec :14:43 GMT\r\n Content-Base: rtsp:// /test.mp3/\r\n Content-type: application/sdp\r\n Content-Length: 544 \r\n \r\n v=0 o= IN IP s=mpeg-1 or 2 Audio, streamed by the LIVE555 Media Server i=test.mp3 t=0 0 Content-Type: SDP a=tool:live555 Streaming Media v a=type:broadcast SDP(Session Description Protocol ) a=control:* a=range:npt= a=x-qt-text-nam:mpeg-1 or 2 Audio, streamed by the LIVE555 Media Server a=x-qt-text-inf:test.mp3 m=audio 0 RTP/AVP 14 c=in IP b=as:259 a=control:track1 The server responds with a description of the requested resource. The DESCRIBE reply-response pair constitutes the media initialization phase of RTSP. 21 Advanced Windows Network Programming

22 SETUP rtsp:// /test.mp3/track1 RTSP/1.0 CSeq: 4 User-Agent: LibVLC/1.1.9 Transport: RTP/AVP;unicast;client_port= The SETUP request for a URI specifies the transport mechanism to be used for the streamed media. The Transport header specifies the transport parameters acceptable to the client for data transmission; the response will contain the transport parameters selected by the server. 22 Advanced Windows Network Programming

23 RTSP/ OK CSeq: 4 Date: Sun, Dec :14:43 GMT Transport: RTP/AVP;unicast;destination= ;source= ;client_port= ;server_port= Session: 0C11DED9 The server generates session identifiers in response to SETUP requests. 23 Advanced Windows Network Programming

24 PLAY rtsp:// /test.mp3/ RTSP/1.0 CSeq: 5 User-Agent: LibVLC/1.1.9 (LIVE555 Streaming Media v ) Session: 0C11DED9 Range: npt= The PLAY method tells the server to start sending data via the mechanism specified in SETUP. A client MUST NOT issue a PLAY request until any outstanding SETUP requests have been acknowledged as successful. The PLAY request positions the normal play time to the beginning of the range specified and delivers stream data until the end of the range is reached. 24 Advanced Windows Network Programming

25 RTSP/ OK CSeq: 5 Date: Sun, Dec :14:43 GMT Range: npt= Session: 0C11DED9 RTP-Info: url=rtsp:// /test.mp3/track1;seq=51676;rtptime= PLAY requests may be pipelined (queued); A server MUST queue PLAY requests to be executed in order. That is, a PLAY request arriving while a previous PLAY request is still active is delayed until the first has been completed. 25 Advanced Windows Network Programming

26 client_port=1554 server_port= Advanced Windows Network Programming

27 27 Advanced Windows Network Programming

28 Messages in Action Following messages play central role: 1. SETUP 2. PLAY 3. PAUSE 4. TEARDOWN 28 Advanced Windows Network Programming

29 SETUP } Specifies the transport mechanism to be used for the streamed media } Transport header specifies the transport parameters acceptable to the client } Response contains transport parameters selected by the server } Server generates session identifiers in response to SETUP requests } Client can issue SETUP to change parameters for already streaming media 29 Advanced Windows Network Programming

30 SETUP Message Client -> Server: SETUP rtsp:// /test.mp3/track1 RTSP/1.0 CSeq: 4 User-Agent: LibVLC/1.1.9 Transport: RTP/AVP;unicast;client_port= Server -> Client: RTSP/ OK CSeq: 4 Date: Sun, Dec :14:43 GMT Transport: RTP/AVP; RTP: RTCP: unicast;destination= ;source= ;c lient_port= ;server_port= Session: 0C11DED9 30 Advanced Windows Network Programming

31 PLAY } Tells server to start sending data via mechanism specified in SETUP } Plays from beginning to end of range specified } Pauses at end as if PAUSE has been issued } Requests may be pipelined } Range header can have time parameter } Legal to have PLAY without Range header } Scale header can be used to change viewing rate } Can be used for fast forward or rewind 31 Advanced Windows Network Programming

32 PLAY Message Client -> Server: PLAY rtsp:// /test.mp3/ RTSP/1.0 CSeq: 5 User-Agent: LibVLC/1.1.9 Session: 0C11DED9 Range: npt= Server -> Client: RTSP/ OK CSeq: 5 Date: Sun, Dec :14:43 GMT Range: npt= Session: 0C11DED9 RTP-Info: url=rtsp:// /test.mp3/track1;seq=51676;rtptime= Advanced Windows Network Programming

33 PAUSE } Causes stream delivery to halt } Can pause entire presentation or selected stream } Server resources continue to be reserved } Server will terminate session after timeout period expires (specified in SETUP) } Can contain Range header with one value to specify when to pause } Must fall in one of the PLAY ranges 33 Advanced Windows Network Programming

34 PAUSE Message Client -> Server: PAUSE rtsp:// /test.mp3/ RTSP/1.0 CSeq: 7 User-Agent: LibVLC/1.1.9 Session: 0C11DED9 Server -> Client: RTSP/ OK CSeq: 7 Date: Sun, Dec :14:45 GMT Session: 0C11DED9 34 Advanced Windows Network Programming

35 TEARDOWN } Stops delivery of stream } Frees up resources on the server } Can stop single stream or entire presentation } Example Client -> Server: TEARDOWN rtsp:// /test.mp3/ RTSP/1.0 CSeq: 8 User-Agent: LibVLC/1.1.9 Session: 0C11DED9 Server -> Client: RTSP/ OK CSeq: 8 Date: Sun, Dec :14:46 GMT 35 Advanced Windows Network Programming

36 GET_PARAMETER } The GET_PARAMETER request retrieves the value of a parameter of a presentation or stream specified in the URI. The content of the reply and response is left to the implementation. GET_PARAMETER with no entity body may be used to test client or server liveness ("ping"). } Example Client -> Server: GET_PARAMETER rtsp:// /test.mp3/ RTSP/1.0 CSeq: 6 User-Agent: LibVLC/1.1.9 Session: 0C11DED9 Server -> Client: RTSP/ OK CSeq: 6 Date: Sun, Dec :14:43 GMT Session: 0C11DED9 LIVE555 server needs a GET_PARAMETER Message every 90s so that it knows the client is still alive, otherwise it will break the session. 36 Advanced Windows Network Programming

37 OPTIONS } Options method queries available methods from the server. } An OPTIONS request may be issued at any time, e.g., if the client is about to try a nonstandard request. It does not influence server state. } Example Client -> Server: OPTIONS rtsp://cloud.cuc.edu.cn/test.mp3 RTSP/1.0 CSeq: 2 User-Agent: LibVLC/1.1.9 Server -> Client: RTSP/ OK CSeq: 2 Date: Sun, Dec :14:43 GMT Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER 37 Advanced Windows Network Programming

38 DECSCRIBE } The DESCRIBE method retrieves the description of a presentation or media object identified by the request URL from a server. It may use the Accept header to specify the description formats that the client understands. } Example Client -> Server: DESCRIBE rtsp://cloud.cuc.edu.cn/test.mp3 RTSP/1.0 CSeq: 3 User-Agent: LibVLC/1.1.9 (LIVE555 Streaming Media v ) Accept: application/sdp 38 Advanced Windows Network Programming

39 DECSCRIBE MESSAGE - SDP Server -> Client: RTSP/ OK CSeq: 3 Date: Sun, Dec :14:43 GMT Content-Base: rtsp:// /test.mp3/ Content-Type: application/sdp Content-Length: 393 Session Description Protocol v=0 o= IN IP s=mpeg-1 or 2 Audio, streamed by the LIVE555 Media Server i=test.mp3 t=0 0 a=tool:live555 Streaming Media v a=type:broadcast a=control:* a=range:npt= a=x-qt-text-nam:mpeg-1 or 2 Audio, streamed by the LIVE555 Media Server a=x-qt-text-inf:test.mp3 m=audio 0 RTP/AVP 14 c=in IP b=as:259 a=control:track1 39 Advanced Windows Network Programming

40 Header Field request-header Accept DESCRIBE request Range PLAY request entity-header Cseq All request&response RTP-Info PLAY response Session SETUP to TEARDOWN request&response Transport SETUP request&response 40 Advanced Windows Network Programming

41 State Transition TEARDOWN INIT SETUP TEARDOWN TEARDOWN READY PLAY PAUSE PLAYING SETUP,PLAY 41 Advanced Windows Network Programming

42 Example } rtsp://cloud.cuc.edu.cn/test.mp3 } Live555 + VLC Initiation Handling Termination 42 Advanced Windows Network Programming

43 OPTIONS REQUEST Request URL Sequence number Client Player 43 Advanced Windows Network Programming

44 OPTIONS REPLY Available Methods 44 Advanced Windows Network Programming

45 DESCRIBE REQUEST 45 Advanced Windows Network Programming

46 DESCRIBE REPLY -SDP 46 Advanced Windows Network Programming

47 SETUP 47 Advanced Windows Network Programming

48 PLAY 48 Advanced Windows Network Programming

49 PAUSE 49 Advanced Windows Network Programming

50 TEARDOWN 50 Advanced Windows Network Programming

51 RTP Real-Time Transport Protocol RFC 3550: RTP: A Transport Protocol for Real-Time Applications 51 Advanced Windows Network Programming

52 RTP Overview } is a real-time streaming protocol for IP networks } usually runs on top of UDP } is an Internet standardized packet format for transporting continuous audio-video data over Internet } was developed by the Audio-Video Transport Working Group of IETF } the standard was published as RFC 1889 in 1996 and then superseded by RFC 3550 in 2003 } RTP has several profiles and payload types for different kinds of audio or video streams (e.g. MPEG-1/2/4, H.26[1234] etc.) } the RTP RFC describes also RTCP (Real Time Control Protocol) for monitoring QoS parameters } the default port is Advanced Windows Network Programming

53 RTP characteristics } provides end-to-end delivery service for real-time data, in unicast and multicast sessions } offers synchronization services (timestamping), packet identification and loss detection (sequence numbering) and delivery monitoring/feedback (through RTCP) } does not provide in-order and reliable delivery of packets } does not provide timely delivery of packets, nor QoS guarantees } is independent of the transport protocol (TCP, UDP, DCCP, SCTP etc.) } a RTP session carries one multimedia stream; a RTP session is identified by a pair of triplets (IP address, RTP port, RTCP port) which are negotiated at setup using RTSP and SDP 53 Advanced Windows Network Programming

54 RTP Packet Encapsulation RTP Header RTP Payload Header RTP Payload 54 Advanced Windows Network Programming

55 RTP packet header } Fixed 12 Bytes (96 bits) bit offset Version P X CC M PT Sequence Number 32 Timestamp 64 SSRC identifier CC CSRC identifiers... Profile-specific extension header ID Extension header length CC Extension header Advanced Windows Network Programming

56 Header fields } Version (2 bits) - RTP version number, always 2; } Padding (1 bit) - if set, the packet contains padding bytes at the end of the payload; the last byte of padding contains how many padding bytes should be ignored; } extension (1 bit) - if set, the RTP header is followed by an extension header; } CSRC count (4 bits) - number of CSRCs (contributing sources) following the fixed header; } Marker (1 bit) - the interpretation is defined by a profile; } Payload Type (7 bits) - specifies the format of the payload and is defined by an RTP profile; } Sequence Number (16 bits) - the sequence number of the packet; the sequence number is incremented with each packet and it can be used by the receiver to detect packet losses; } Timestamp (32 bits) - reflects the sampling instance of the first byte of the RTP data packet; the timestamp must be generated by a monotonically and linearly increasing clock; } Synchronization Source (SSRC) (32 bits) - identifies the source of the real-time data carried by the packet; } Contributing Sources (CSRC) (32 bits) - identifies a maximum of 15 additional contributing sources for the payload of this RTP packet. 56 Advanced Windows Network Programming

57 RTP header extensions } the Marker and PayloadType fields are defined by a profile and the profile may even redefine the octet containing these 2 fields } additional fixed fields can be added after the fixed header by a profile } if the X bit in the RTP header is 1, a variable-length header extension (for which the first 32 bits have a specific structure) follows the fixed header; is intended for limited use, experimenting and can be ignored by non interested applications 57 Advanced Windows Network Programming

58 RFC 3551:RTP Profile for Audio and Video } Section 4 Audio MPA MPA denotes MPEG-1 or MPEG-2 audio encapsulated as elementary streams. The encoding is defined in ISO standards ISO/IEC and The encapsulation is specified in RFC } Section 5 Video 5.6 MPV MPV designates the use of MPEG-1 and MPEG-2 video encoding elementary streams as specified in ISO Standards ISO/IEC and , respectively. The RTP payload format is as specified in RFC 2250, Section Advanced Windows Network Programming

59 Payload Type Definitions } Section 6 : Payload Type Definitions PT Enc Name PCMU 1 reserved 2 reserved 3 GSM 4 G723 5 DVI4 6 DVI4 7 LPC 8 PCMA 9 G L16 11 L16 12 QCELP 13 CN 14 MPA 15 G DVI4 17 DVI4 18 G729 Table 4: Payload types (PT) for audio encodings PT Enc Name unassigned 25 CelB 26 JPEG 27 unassigned 28 nv 29 unassigned 30 unassigned 31 H MPV 33 MP2T 34 H unassigned reserved unassigned dynamic dyn H Table 5: Payload types (PT) for video and combined encodings 59 Advanced Windows Network Programming

60 On Receiving RTP packet } check SSRC } new source? } existing source? which one? } check payload type } has format been changed? } which decoder should I use? 60 Advanced Windows Network Programming

61 You are Here Encoder Decoder Middlebox Sender Receiver Network 61 Advanced Windows Network Programming

62 Application-Level Framing } Expose details to applications } Let application decide what to do with a packet, not transport protocol 62 Advanced Windows Network Programming

63 How to send/recv? Let the application decide, not the protocol stack. Tennenhouse, Clark, Architectural Considerations for a New Generation of Protocols, Proceedings of Sigcomm Advanced Windows Network Programming

64 Application Knows Best } How to reorder packets } Whether to ignore loss } Which packet to retransmit 64 Advanced Windows Network Programming

65 Application Data Unit (ADU) } Can be processed individually, even out-of-order } Unit of error-recovery } If part of an ADU is lost, the whole ADU is considered lost } 8-Bit PCM audio: 1 ADU = 1 Byte } MPEG1 Video: 1 ADU = 65 Advanced Windows Network Programming

66 How to chop data into packets? } Every received packet should be useful (even in very lossy environments) } Ideally, 1 ADU in 1 packet 66 Advanced Windows Network Programming

67 MPEG over RTP RFC-2250 RTP Payload Format for MPEG1/MPEG2 Audio/Video 67 Advanced Windows Network Programming

68 RTP Payload Header 12 bytes 4 bytes Rest of IP packet RTP Header RTP Payload Header RTP Payload MPEG-1? 2? Temporal Reference I? P? B? Begin of Slice? End of Slice? 68 Advanced Windows Network Programming

69 RTP Header 4 bytes 4 bytes 4 bytes n Ver: version, P: padding, X: extension, CC: CSRC count, M: marker, PT: payload type, sequence number, media timestamp, SSRC 69 Advanced Windows Network Programming

70 RTP Header } Media Timestamp: 32 bits } the instant the first byte in this packet is captured } 90 khz timestamp (90,000 = 1 second) 70 Advanced Windows Network Programming

71 RTP Header } Marker Bit: } 1 if packet contains the last byte of a frame 71 Advanced Windows Network Programming

72 RTP Header } Payload Type: 7 bits } 32 for MPEG-1/2 video; 33 for MPEG-2 TS } 14 for MPEG audio 72 Advanced Windows Network Programming

73 MPEG Video-specific header } This header shall be attached to each RTP packet after the RTP fixed 4 bytes header MBZ T TR A N N S B E P FB V BFC FF V FFC From RFC-2250: RTP Payload Format for MPEG1/MPEG2 Video 73 Advanced Windows Network Programming

74 RTP Payload Header 4 bytes } MBZ (5 bits) } Unused. Must be Advanced Windows Network Programming

75 RTP Payload Header } T (1 bit) } 1 if there is a MPEG-2 Extension Header after this header. 75 Advanced Windows Network Programming

76 RTP Payload Header } Temporal Reference (10 bits) } The frame number of the current frame within the GOP. 76 Advanced Windows Network Programming

77 RTP Payload Header } AN (Active N bit for error resilience)bit and N bit } Set to 0 for MPEG-1. } Set to 1 when the following bit (N) is used to signal changes in the picture header information for MPEG-2 payloads. 77 Advanced Windows Network Programming

78 RTP Payload Header } S (1 bit) } Is there a sequence header in this packet? } Repetition of sequence header is useful for resynchronization. 78 Advanced Windows Network Programming

79 RTP Payload Header } BS (1 bit) and ES (1bit) } BS(Beginning-of-slice) is 1 if the first byte of this payload is a slice header. } ES(End-of-slice) is 1 if the last byte of this payload is the end of a slice. 79 Advanced Windows Network Programming

80 RTP Payload Header } Picture Type (3 bits) } I (1), P (2), B (3), D (4). 80 Advanced Windows Network Programming

81 RTP Payload Header } Motion Vectors Information } FBV: full_pel_backward_vector } BFC: backward_f_code } FFV: full_pel_forward_vector } FFC: forward_f_code Obtained from the most recent picture header, and are constant for each RTP packet of a given picture 81 Advanced Windows Network Programming

82 Fragmentation Rules } Sequence header: at the start of payload } GOP header: at the start of a payload (or follows Sequence header) } Picture header: at the start of a payload (or follows Sequence/GOP header) 82 Advanced Windows Network Programming

83 Fragmentation Rules } A slice must be either } First data in the packet, or } Follows integral number of slices } A slice may be fragmented if exceeds the size of a packet 83 Advanced Windows Network Programming

84 MP3 (MPEG-1, layer 3) Audio } MP3 audio can be encoded in two ways: } RFC 2250 //Payload type :14 } RFC 3119 //Payload type: Dynamic } RFC 2250 describes the general MPEG-1 video/audio ADU framing (fixed 4 bytes) This header shall be attached to each RTP packet at the start of the payload and after any RTP headers for an MPEG1/2 Audio payload type MBZ Frag_offset Frag_offset: Byte offset into the audio frame for the data in this packet. } Problem: MP3 frames are not self-contained! 84 Advanced Windows Network Programming

85 MP3 Frame Structure } Each frame contains the header (Including the 4 byte MPEG header, optional 2 bytes CRC and 9, 17 or 32 bytes (depending on mono/stereo and MPEG 1 or 2) of side info } MP3 frames have a fixed length } Data of one ADU may span multiple frames } Problem: if one packet lost multiple ADUs lost 85 Advanced Windows Network Programming

86 MP3 RFC 3119 Re-Arrangement } Idea: re-arrange data such that each packet is selfcontained (i.e., decodable) } Effects: } Better error resilience, but } Variable length packets and } Re-arrangement needs to be undone for decoder 86 Advanced Windows Network Programming

87 RFC 3119: Interleaving } Interleaving: interleave cycle of size 8 } Advantage: Consecutive packet losses have less effect } Disadvantage: Send & receive latency is increased 87 Advanced Windows Network Programming

88 Other Thoughts } Packet losses on the Internet are often correlated, forming lost packet trains } What can be done to decorrelate losses? } How to measure audio and video quality? } Objectively computed: PSNR (in db) } Subjective tests: MOS (range: 1 bad to 5 excellent) 88 Advanced Windows Network Programming

89 RFC 2250 versus RFC 3119 } Frame-level loss simulation } R: random loss } G: Gilbert model } Note: Gilbert model produces correlated losses ( packet trains ) 89 Advanced Windows Network Programming

90 MP3 Sender/Receiver Structure Server RFC 2250 MP3 frames RTP packets MP3 frames ADU frames Interleaved ADU frames Client RFC 3119 RTP packets ADU frames Interleaved ADU frames 90 Advanced Windows Network Programming

91 Packet Size } 1 MTU is 1500 bytes } IP Header size = } UDP Header size = } RTP Header size = } RTP Payload Header size = } Payload size = 91 Advanced Windows Network Programming

92 Live555 MP3 over RTP 92 Advanced Windows Network Programming

93 MP3 File Format (VBR) 4 B 413 B 4 B MPEG Audio Frame Header Info MPEG Audio Frame Header MP3 Data ID3v1 1 st Frame No Audio data 2 nd Frame n th Frame 93 Advanced Windows Network Programming

94 1 st RTP Packet 12 bytes 4bytes (Zero) Rest of IP packet RTP Header Marker=1 RTP Payload Header RTP Payload 4 bytes 413bytes 4 bytes 413bytes 4 bytes MPEG Audio Frame Header (Zero) MPEG Audio Frame Header (Zero) MPEG Audio Frame Header MP3 Data Drop 1 MPEG frame 94 Advanced Windows Network Programming

95 1 st RTP Packet 95 Advanced Windows Network Programming

96 2 nd RTP Packet 96 Advanced Windows Network Programming

97 Packets list 97 Advanced Windows Network Programming

98 RTCP (Real-Time Control Protocol) Described by the RTP RFC Advanced Windows Network Programming

99 RTCP (Real-Time Control Protocol) } is described by the RTP RFC } has 2 basic functions: } provides feedback statistics on the QoS parameters (like Round-Trip- Time, delay, jitter, packet losses etc.) for the participants to a RTP session } carries canonical end-point identifiers (CNAME) to all session participants as the source identifier (SSRC) may change in case of a conflict and many SSRC can correspond to the same CNAME (a SSRC is unique only within a RTP session) to keep track of each participant } uses as port the next highest odd-number following the evennumber port of RTP } the RTCP traffic must not be above 5% of the RTP traffic in a session 99 Advanced Windows Network Programming

100 RTCP packet types (reports) } SR (sender reports) the reports sent by active senders of real-time data (audio, video) } RR (receiver reports) the reports sent by receivers of realtime data (audio, video) } SDES source description messages, including CNAMEs } BYE end of participation } APP application-specific functions Multiple RTCP packets (reports) can be concatenated in a compound RTCP packet. 100 Advanced Windows Network Programming

101 Compound RTCP Packets } A compound RTCP packet contains multiple RTCP packets of the previous types. } Example:

102 RTCP header } version (2 bits) the same as for RTP header } padding (1 bit) - the same as for RTP header } count (5 bits) the number of reception report blocks contained in this packet } type (8 bits) the packet type (193 NACK, 200 SR report, 201 RR report, 202 SDES packet, 203 BYE packet, 204 APP packet) } length (16 bits) the length of the RTCP packet in 32 bit words minus one, including the header and padding 102 Advanced Windows Network Programming

103 RTCP SR packet 103 Advanced Windows Network Programming

104 RTCP RR packet 104 Advanced Windows Network Programming

105 RTCP SDES packet 105 Advanced Windows Network Programming

106 RTCP BYE packet 106 Advanced Windows Network Programming

107 RTCP Receiver Report Source description 107 Advanced Windows Network Programming

108 RTCP Receiver Report Goodbye 108 Advanced Windows Network Programming

109 SDP Session Description Protocol RFC Advanced Windows Network Programming

110 SDP } is a protocol used to describe media objects and presentations } usually, in multimedia streaming, SDP messages are sent in RTSP requests } a SDP message contains information about the session, the media streams included in the session and information necessary to receive the media (e.g. IP addresses, ports, formats etc.) } is standardized by IETF first in 1998 and then as RFC 4566 in Advanced Windows Network Programming

111 Field format } A session is described by a series of fields, one per line. The form of each field is as follows. } <character>=<value> Where <character> is a single case-significant character and value is structured text whose format depends upon attribute type. Values are typically a UTF-8 encoding. Whitespace is not allowed immediately to either side of the =. } Within an SDP message there are three main sections } Session description } Timing description } Media description Each message may contain multiple timing and media descriptions. Names are only unique within the associated syntactic construct, i.e. within the session, time, or media. 111 Advanced Windows Network Programming

112 Session Description v= (protocol version) o= (originator and session identifier) s= (session name) i=* (session information) u=* (URI of description) e=* ( address) p=* (phone number) c=* (connection information -- not required if included in all media) b=* (zero or more bandwidth information lines) One or more time descriptions ("t=" and "r=" lines) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute lines) Zero or more media descriptions Optional values are specified with =* and each field must appear in the order shown below. 112 Advanced Windows Network Programming

113 Time and Media Description Time description t= (time the session is active) r=* (zero or more repeat times) Media description, if present m= (media name and transport address) i=* (media title) c=* (connection information optional if included at session level) b=* (zero or more bandwidth information lines) k=* (encryption key) a=* (zero or more media attribute lines) 113 Advanced Windows Network Programming

114 Attributes } SDP uses attributes to extend the core protocol. Attributes can appear within the Session or Media sections and are scoped accordingly as session-level or media-level. New attributes are added to the standard occasionally through registration with IANA. Attributes take two forms: } A property form: a=<flag> conveys a property of the session. } A value form: a=<attribute>:<value> provides a named parameter. 114 Advanced Windows Network Programming

115 Time Formats } Absolute times are represented in Network Time Protocol format (the number of seconds since 1900). If the stop time is 0 then the session is "unbounded." If the start time is also zero then the session is considered "permanent." Unbounded and permanent sessions are discouraged but not prohibited. Intervals can be represented with Network Time Protocol times or in typed time: a value and time units (days ('d'), hours ('h'), minutes ('m') and seconds ('s')) sequence. } Thus an hour meeting from 10am on 1 August 2010, with a single repeat time a week later at the same time can be represented as: t= r= Or using typed time: t= r=7d Advanced Windows Network Programming

116 A SDP message example v=0 o=streamingserver IN IP s=movie.avi c=in IP t=0 100 a=range:npt= m=video 0 RTP/AVP 96 b=as:1514 a=rtpmap:96 MP4V-ES/90000 a=fmtp:96 profile-level-id=1 a=cliprect:0,0,352, Advanced Windows Network Programming

117 RTSP/SDP REPLY 117 Advanced Windows Network Programming

118 Reference } } Ch7, Multimedia networking, Computer Networking: A Top-Down Approach,5th Ed, J.F. Kurose, K.W. Ross, 2009 } } } } Advanced Windows Network Programming