VISP: on Implementing a Video Streaming Protocol for the Internet

Transcription

1 VISP: on Implementing a Video Streaming Protocol for the Internet Michele Bracuto Computer Science Department University of Bologna Mura Anteo Zamboni Bologna Italy Tel Fax bracuto@cs.unibo.it Marco Furini Λ Computer Science Department University of Piemonte Orientale Spalto Marengo Alessandria Italy Tel Fax furini@mfn.unipmn.it Marco Roccetti Computer Science Department University of Bologna Mura Anteo Zamboni Bologna Italy Tel Fax roccetti@cs.unibo.it ABSTRACT A recently proposed mechanism introduced a new approach to support interactive video streaming applications over the Internet. It acts on the video QoS in order to avoid possible problems caused by the network jitter. Simulations results proved the effectiveness of this mechanism. An exhaustive discussion on the implementation of this mechanism is the contribution of this paper. A client-server protocol, named VISP (VIdeo Streaming Protocol), is designed to support interactive video streaming applications over the Internet. VISP is not a video compression scheme and it is not a transport protocol, but an application protocol that controls the video transmission to provide interactivity. Several features have been added in order to complete the protocol and tests confirm that VISP is effective in supporting interactive video streaming applications over the Internet. KEY WORDS QoS Protocols for IP Multimedia, Multimedia Tools and Architectures, Multimedia Networking and Communications, Multimedia Streaming 1 Introduction In recent years, the transmission of video streams has become very popular over the Internet. Videoconferencing, distance learning, on-line games, pay-per-view are some examples of applications that generate a video stream. The transmission of a video stream is strictly correlated to the time-factor and this poses a critical problem, because the Internet was designed for non-realtime critical traffic. Hence, no timing-guarantees are provided to the supported applications and, for this reason, the applications achieve a QoS that is far from what desired. However, despite these QoS problems, there is a large use of video applications over the Internet and this will increase for several reasons: the amount of video in the Internet is increasing as well as the use of high-speed connections (fiber optics or ADSL) to access the Internet from Λ Corresponding author. furini@mfn.unipmn.it houses, and bandwidth and disk storage costs are decreasing. This scenario will facilitate the distribution of multimedia contents. Several video applications provide interactivity to the end-users (pause-resume, jumps to the future and past of the video stream) by using a protocol that allows client/server interactions. While being attractive, interactive applications are more difficult to support than noninteractive applications. In fact, to provide interactivity, an additional constraint must be met: the end-to-end delay experienced by the application traffic should not be noticeable to the end users. Conversely, this constraint is not present in non-interactive applications (e.g., an Internet radio), where large end-to-end delays don t cause any QoS problems to the supported applications. The importance of the end-to-end delay is highlighted byseveralstudies[1,2,3,4,5,6]thatinvestigatedtheeffects of this delay on human perception and they showed that interactive applications are well supported if the endto-end delay is kept within a threshold along the lifetime of the application. Conversely, if the end-to-end delay goes above this threshold, the interactions between end-users are seriously compromised. Hence, the threshold represents a bound for the human perceptions: below this bound the users do not perceive the end-to-end delay and hence the interactions are well supported; above this bound the endto-end delay is noticeable to the end-users and hence the interactions are not well supported. It is to note that the value of this threshold is not fixed [3], but depends on the characteristics of the application and on the level of interactivity requested by the end-users. For instance, a threshold of 150 ms ensures full satisfaction to the end-users of an interactive audio applications[7]. This means that, if the end-to-end delay goes above 150 ms the users experience a bad service, while for values lower than 150 ms, the users can interact without any problems. Among the components that compose the end-to-end delay, the network delay is the most variable component. In fact, data travel from source to destination along a path, composed of links and routers. In best-effort networks, this path is usually shared among traffic generated by other applications and it may happen that a network resource along 1

2 the path is busy, causing the data to be delayed until the resource is available. Since, the network delay is very variable (network jitter) along the application lifetime, it is necessary to design an adaptive mechanism that takes into consideration this variability. Otherwise, the receiver may experience QoS problems. For instance, in a video streaming application the receiver should continuously play out the arriving stream, according to a pre-defined number of frames per second. Due to the network jitter, some video frames may be delivered later than expected, causing problems to the receiver s play out as video frames may not be available for play out when needed. This could cause video play out interruptions (a common situation for users that watch video streams while being connected to the Internet through a low capacity modem). In literature there are mechanisms that mask the network jitter using buffering or smoothing techniques [4, 8, 9, 10]. Roughly, these mechanisms use a start-up delay to mask the network jitter. While being effective in supporting QoS applications, these techniques cannot be used to support interactive applications, as they increase the overall end-to-end delay. A new approach to handle the network jitter is introduced in [11, 12], where it is pointed out that the ideal scenario to provide interactivity is when client and server are directly connected (i.e., no network involved). Roughly, the idea is to play out a video stream in the actual scenario (client and server connected through the Internet) while attempting to simulate the play out in the ideal scenario. Actual and ideal play out are kept close by a mechanism that acts on the video QoS by dropping frames. The approach has been evaluated through a trace-driven simulation and results showed that without using the approach, the network jitter compromises the QoS application and that the new approach introduces great benefits as the QoS is slightly affected while the actual play out is kept very close to the ideal play out. The contribution of this paper is the design of VISP (VIdeo Streaming Protocol), a protocol based on the approach proposed in [11, 12]. VISP is a client-server application protocol designed to support interactive video streaming applications over the Internet. Note that VISP is not a video compression scheme (in fact, it does not include any encoding mechanism), and it is not a transport protocol, but a protocol designed to control the transmission of a video stream in order to provide interactive features to the supported applications. VISP is written in C-language and uses the RTP protocol to transmit the video frames over the Internet. Another contribution of this paper is the introduction of VCR-like functions and periodic clocks synchronization in VISP. VCR-like functions allow the user to better interact with a video server and periodic clocks synchronization is shown to be necessary in order to accommodate different clocks speeds between client and server. To investigate the effects of the clocks skew, several tests have been performed and results show that, even a small difference between client and server clocks, may compromise user interactions with the server. The overall VISP evaluation shows that it well supports interactive video streaming applications over the Internet. The remainder of this paper is organized as follows. In section 2 we describe the main characteristics of the VISP protocol and we present a brief overview of the mechanism upon which VISP is based. In section 3 we present the VISP implementation, highlighting the main features of the VISP protocol and how they are implemented; In section 4 we present experimental results that investigate the effects of the possible difference in speed between client and server clocks. Conclusions are drawn in section 5. 2 VISP description In this section we present our proposed protocol VISP (VIdeo Streaming Protocol), a client-server application protocol designed to support interactive video streaming applications over the Internet. As known, interactive video streaming applications are difficult to support in best-effort networks, as these networks do not provide any type of timing-guarantees to the supported applications. In [11, 12] a new approach to support these applications is proposed and simulations results show that the approach is effective. For this reason, we designed VISP using this approach. It is to point out that VISP is not a video compression scheme (video can be encoded with any mechanisms) and it is not a transport protocol (video frames are transmitted with the RTP protocol). In the following, after a brief description of the approach proposed in [11, 12], we explain i) the reasons why a periodic clocks synchronization is necessary along the application lifetime, ii) how the video is transmitted over the network and, iii) the meaning of the VCR-like functions introduced in VISP. 2.1 Mechanism overview In this section we briefly present an overview of the mechanism proposed in [11, 12] to support interactive video streaming applications over the Internet. As we already stated, the presence of the network jitter may compromise the achieved QoS, as it may cause the end-to-end delay to go above the acceptable threshold (called NIT, Natural Interaction Threshold). Needless to say, if the network is not present (i.e., client and server are directly connected), the network jitter is not present, too. In [11, 12] this scenario is referred to as the ideal scenario (with ideal play out), that is in contrast with the actual scenario (with actual play out, where client and server are connected through the Internet). The mechanism acts on the video QoS in order to keep the actual play out close to the ideal play out.

3 VTD t t' NIT t -t 1 Acceptable VTD Figure 1. Video stream transmission. Figure 2. VTD while playing out the video stream. This is done by using a timestamp mechanism that measures the time difference between the actual and the ideal play out. By supposing the clocks at the sender and at the receiver side synchronized, the time difference is measured through a metric, called VTD (Video Time Difference), which is periodically measured at the receiver side. In essence, the goal of the mechanism is to maintain the VTD within the NIT value. In fact, if the VTD is not noticeable to the users (VTD<NIT), the interactive application is well supported. If the network causes the VTD to go above the acceptable threshold, the mechanism acts on the video QoS (i.e., by dropping some video frames) and reports the VTD within the NIT. In the following T S (i) and T R (i) denotes the ideal and the actual play out time of the frame i, respectively. Transmission Algorithm Each transmitted frame is marked with a timestamp, which represents the ideal play out time of the frame, according to the following rules: 1. The timestamp of the first video frame represents the time at which the video frame is transmitted. If we denote this time with t, it follows that T S (1) = t. 2. A frame i (i >1) is marked with T S (i) =T S (i 1) + ff, whereff =1=ffi and ffi is the number of frame that must be displayed every second. Video Play out algorithm The receiver retrieves video frames from the network, temporarily stores them into its local buffer and then plays out these video frames according to the following rules: 1. Video play out starts when the first video frame arrives at the receiver side, say at time t 0 ; Hence, T R (1) = t 0 ; 2. The receiver plays out the frames at fixed period (i.e., one frame every ff =1=ffi time units); 3. In the buffer, the frame with the lowest timestamp is selected for play out; 4. Once selected, a frame i is removed from the buffer and is played out at time T R (i) = T R (i 1) + k ff (k 1) only if: a) T R (i) T S (i) and b) t t' Figure 3. Video stream transmission using the mechanism. T S (i) >T S (prev(i)), whereprev(i) is the most recently frame that has been played out. If conditions a) and b) are not met, then frame i is discarded and a new frame selection must be done (by applying rule 3); In networks that provide QoS guarantees, such as sufficient bandwidth, no packet loss and no network jitter, video frames are played out at T R, computed with k = 1, which represents the ideal scenario (video frames are played out at a rate of ff time units). Conversely, in the Internet it may happen that k 1, as depicted in Fig. 1, where a sender, at time t, starts transmitting video frames every ff time units. At time t 0, the receiver plays out frame 1. Due to network problems, frame 2 is played out at time t 0 +3 ff, instead of t 0 + ff. In this case, k =3. Needless to say, if a video frame is played out with k 1 the continuity of the video play out is compromised and the play out time of all the successive frames is affected, as well as the VTD. In Fig. 2 is possible to note the effects of these problems on the measured VTDs (a hypothetical NIT value is also depicted in order to compare the VTD with the NIT). Since, frame 2 is played out later than expected, its VTD is equal to VTD(2) = t 0 t +2ff. If VTD(2) >NITthen VTD(j) NIT, for each j 2. This situation poses serious problems if the supported application has interactive features, as all the frames, but the first, have a VTD above the NIT. For this reason, the VTD must be reported within the acceptable NIT. In [11, 12] it is shown that the VTDcan be reduced of ρ time units, by dropping a number of frames, say m, that corresponds to ρ time units (i.e. m ff = ρ), where ff =1=ffi and ffi is the number of frames played every second. Since the amount of time that exceeds the NIT is known (V TD NIT), the VTD can be reported within the NIT, by discarding a number of frames that corresponds to the time quantity VTD NIT. The mechanism works as follows. When the receiver finds out that the VTD is above the NIT (for instance when playing out frame 2), it sends to the sender the value

4 VTD Acceptable VTD NIT t -t Figure 4. VTD measured while playing out the video stream when using the dropping frame mechanism. To be effective, the mechanism proposed in [11, 12] needs the server and the client clocks synchronized. In the prototype implementation described in [11, 12], an initial clocks synchronization activity is carried out prior to the video stream transmission, and simulations results proved the effectiveness of the approach. Simply, clocks synchronization was supposed and not investigated, as several mechanisms can be used to this task (GPS, hand-shake protocol, etc.). During the VISP implementation, we noticed that, although the initial clocks synchronization is necessary, it is not sufficient. In fact, possible problems may arise if server and client clocks have different speeds. In Fig. 5 we report a possible situation where the server clock goes slower than the client clock. Upon the initial synchronization, the server transmits the video frames at a rate of ff time units. At the client side, the client plays out the video frame with the same rate (ff time units). However, since the server clock goes slower, the ff time units at the client side do not correspond to the ff time units at the server side. For instance, if the clocks skew is 1 ms every second, in a 100 minutes movie, the final difference will be equal to 6 seconds (and usually, to provide interactivity, 150 ms is the maximum threshold). Hence, clocks skew compromises our proposed VTD conα Server Client α Figure 5. Server clock goes slower than the client clock. ρ VTD(2) NIT (Fig.3). The sender uses this information to compute the number of frames (m) that has to be discarded in order to report the VTD within the NIT. In this case, it is necessary to drop 2 frames; the sender discards frame 7 and frame 8 and, just after frame 6, transmits frame 9, frame 10 and so on. Fig.4 shows the effects of the mechanism on the VTD. The benefits start when playing out frame 9: if the mechanism is not used, frame 9 is played out with VTD(9) greater than NIT (Fig. 2), but with the mechanism, VTD(9) is lower than NIT (Fig. 4). Moreover, using the mechanism, all the frames transmitted after frame 9 are within the NIT. The mechanism has been evaluated with several simulations, and results presented in [11, 12] show that it is effective in supporting interactive QoS applications. A QoS evaluation was also presented and results showed that the mechanism only slightly affects the QoS of the transmitted video stream. 2.2 Clocks Synchronization trol mechanism as the measured VTD does not correspond to the real situation. In fact, if the computed VTD is affected by clocks synchronization problems and goes above the NIT, then the client supposes, and informs the server, that network problems happened. At the other side, the server will drop some video frames. However, this frame dropping is not needed, as the VTD is above the NIT because client and server clocks are no longer synchronized. In the example depicted in Fig. 5, when frame 2 arrives, the client believes that network problems happened while transporting frame 2 and hence it informs the server that frame 2 is late and the server will proceed with frame dropping. It is easy to understand that this is an uncorrect behavior that may affect the video QoS even though it is not necessary. A similar problem happens if the client clock goes slower than the server clock. In this case, the client may not realize that the VTD is above the NIT and the server may overflow the client buffer. For this reason, in addition to the initial clocks synchronization, it is necessary to periodically synchronize the server and the client clocks, in order to avoid possible problems that may occur if the clocks run with different speeds. This means that a periodic clocks synchronization must be done in order to prevent the problems we have mentioned previously. In section 3.1 we present some implementation details concerning the clocks synchronization used in VISP. 2.3 Media Transport VISP does not define how video frames are encapsulated for transmission over the Internet, but it uses the RTP protocol to deliver these data to the client. The Real-Time Protocol (RTP) [13] is a protocol defined to transmit multimedia contents over the Internet. It is a public-domain standard for encapsulating the multimedia segments. Once the client begins to receive the requested multimedia file, the client begins to render the file. Figure 6 shows the RTP packet format, which is used to transmit the video. VISP uses the RTP packet fields to mark each transmitted frame with a sequence number and with a timestamp. The sequence number is necessary, since video frames are transmitted over the Internet and hence pack-

5 V P X CC M Payload type Sequence number Time-stamp SSRC (Synchronization Source Identifier) CSRC (Contributing Source Identifier 1) CSRC (Contributing Source Identifier...) CSRC (Contributing Source Identifier N) Data Figure 6. RTP Packet Format. ets may arrive out of order; with the sequence number the client can order the arriving packets and can correctly play out the video stream. The timestamp field is used to inform the client of the video frame ideal play out time. Another RTP field is used, and it is the payload type. In fact, VISP is not a video compression mechanism and, hence, it can support applications with video encoded with different mechanisms. The used video compression is communicated to the client through the payload field of the RTP header. The use of the payload allows the server to change the encoding mechanism while transmitting a video stream without interrupting the delivery of the video stream. In fact, the client uses the payload field to determine the necessary decoding mechanism and the server sets the payload field according to the encoding mechanism used. For instance, with motion-jpeg videos the payload field must be set to 26, while for MPEG videos the payload field must be set to VCR functions Several works have considered the problem of adding VCR functions (rewind, fast forward, pause and stop) to streaming video applications (for instance, [14, 15, 16]). These mechanisms require additional resources or employ different encoding mechanisms. Unfortunately, in common Internet environments, additional resources, such as additional bandwidth, bigger buffers or large storage devices at the client may not be available. Our approach introduces VCR functions in VISP without requiring additional resources. By selecting a button in the client interface, a vcr-message is created and sent to the server. This message informs the server about the requested vcr operation. Upon the vcr-message reception, the server has to modify the video stream transmission, according to the requested operation: ffl PAUSE: The server stops the video stream transmission and holds the current video frame number in order to resume the video play out; ffl STOP: The server stops the video stream transmission and the connection is closed; ffl FAST-FORWARD: A FF operation is performed by skipping frames at the sender. This means that the sender, while transmitting the same number of frames per second, delivers a stream that corresponds to a larger sequence of the movie. This allows the client to play out a fast movie. This technique is easy to implement since the sender, after receiving a FF request, starts transmitting every n-th frames, where n represents the speed factor requested by the client. Note that, to perform this operation, no additional bandwidth is requested and that the video QoS depends only on the requested speed factor. ffl REWIND: The same of the fast-forward, but the video is played out backward. Skipping frame technique is trivial to implement when video is encoded with intra-frame technique (as Motion JPEG). If video is encoded using an inter-frame technique (as MPEG), the sender can use one of the dropping algorithms proposed in [11, 12, 17] to skip video frames. 3 VISP Implementation In this section we present details of the VISP implementation. VISP is written in C-language and, as we already stated, is a client-server application protocol designed to support interactive video streaming applications over the Internet, based on the mechanism proposed in [11, 12]. As we already pointed out, a clocks synchronization mechanism is fundamental. After the synchronization, video stream transmission can take place. The server retrieves the video (from an external source, storage, etc.), encodes it and transmits it to the client. On the other side, the client retrieves the video from the network, decodes it and plays it out. During the video stream play out, the client monitors the network by computing the VTD and if there are network problems, the client informs the server to drop video frames in order to keep the actual play out close to the ideal play out. The VISP protocol does not take into account how the video is retrieved and encoded at the server side. In fact, VISP is not an encoding video mechanism, but it is a protocol that controls the video stream transmission in order to provide interactive features to the client. Hence, video can be encoded with any techniques. However, since VISP may drop video frames when requested, it has to be provided with dropping algorithms that take into account the characteristics of the encoded mechanism used. In the current implementation, VISP is provided with dropping frame algorithms designed to handle Motion-JPEG [18] and MPEG [19] video streams. Hence, it can currently support motionjpeg and mpeg video streaming applications. In the following we describe details of the VISP implementation.

6 Message Type Sequence number Extra Information Figure 7. RTP Packet Format. 3.1 Implementing Clocks Synchronization We already mentioned that a clocks synchronization is necessary in order to implement the mechanism proposed in [11, 12]. Clocks synchronization can be implemented with several alternative mechanisms. For example, GPS-based mechanisms and sophisticated represent some possible approaches. The selection of a mechanism must take into account the number of clocks synchronizations that has to be done along the application lifetime. In fact, most of the proposed mechanisms require additional resources or introduce an excessive computational and communication overload. This may arise efficiency problems if the clocks synchronization has to be done frequently. Based on these considerations, we use a simple virtual clock approach in the current implementation of VISP. Roughly, the client periodically sets its clock (T R ), by using the timestamp of the video frame, which contains T S. The virtual clock approach has been selected in the current VISP implementation because it does not need additional resources and it does not need any additional messages. A possible drawback is that the virtual clock accuracy does not take into account the network transfer delay. One important motivation behind our choice is that in a LAN scenario this delay is usually negligible and hence the virtual clock approach can be effective to test the VISP protocol. In particular, we want to investigate if the clocks skew may compromise the interactions between the client and the server and hence, if a periodic clocks synchronization may be really beneficial. To this aim, several experiments have been performed and results are presented in the next section. Needless to say, in the Internet the network transfer delay may be considerable and hence the virtual clock accuracy may not be sufficient. For this reason, it may be necessary to provide VISP with a different clock synchronization mechanism, as one of those presented in [20]. The virtual clock approach is implemented through the following code, where Tsis the ideal play out time and Tr is the actual play out time of the frame seq. void Synch(u_int16 seq, u_int32 ts) { gettimeofday(&now, NULL); vc_time = (double)ts; lc_time = tv2dbl(now); offset_t = lc_time - vc_time; tr = lc_time - offset_t; 3.2 Implementing VCR Functions The client is provided with an interface that allows to request vcr functions. By selecting a button and a speed factor, a vcr-message is created. The message format is reported in Fig. 7. Here, the Message type field is used to denote the operation requested to the sender (PAUSE, FAST, REW, STOP). The Sequence number is not currently used and the Extra Information field is used to send the speed factor to the server. Upon receiving the vcr-message, the server modify the video stream according to the client request, in particular: ffl PAUSE: The server stops the video stream transmission and holds the current video frame number in order to resume the video play out; ffl STOP: The server stops the video stream transmission and connection is closed; ffl REWIND/FAST-FORWARD: The sender, while transmitting the same number of frames per second, delivers a stream that corresponds to a larger sequence of the movie. Roughly, the sender, after receiving a REW/FF request, starts transmitting every n-th frames, where n is the number specified in the vcr-message (Extra Information). As already stated, any skipping frame technique is easy to implement when video is encoded with intra-frame technique (as Motion JPEG). In this case, the speed factor can be any number. Conversely, if video is encoded using an inter-frame technique (as MPEG), the sender can only skip video frame using one of the dropping algorithms proposed in [11, 12]. All these algorithms provides different speed factors. Depending on the speed factor requested by the user, a dropping algorithm is selected (that is, the one that provides a speed factor closest to the one selected by the user). 3.3 Client Implementation The client task is to retrieve the video frames from the network, to decode and to play them out. Video frames are temporarily stored in its buffer, which is also used to order the video frames that have arrived in the wrong order (this may happen because RTP uses the UDP transport protocol and hence no guarantees are given to the video frames transmission). The client is also in charge of computing the VTD in order to find out possible network problems. For each played out frame, the VTD is easily computed by subtracting the video frame timestamp, which represents the ideal play out time, from the current clock, which is the actual play out time. The computed VTD is compared to the NIT and if the VTD is greater than NIT, the client sends the value VTD NIT to the server.

7 3.3.1 Video play out At regular times (depending on the video frame rate), the client selects from its buffer the frame with the lowest timestamp. Video frames are inserted into the client buffer ordered by their timestamp. In this way the video frame candidated to be played out is the video frame at the head position. The video frame at head position in the buffer is played out only if the following conditions hold: 1. The timestamp of the selected video frame is greater that the timestamp of the video frame that has been played out most recently; 2. The timestamp of the selected video frame is lower or equal to the current clock. In the following we present the code that performs this video frame selection. If the selected frame (h buffer) has a timestamp lower than the current clock, it is discarded and the following video frame is considered (h buffer = h buffer->next). struct pkt_q *GetFrame(u_int32 clock) { struct pkt_q *next_frame; next_frame = NULL;... while(h_buffer && (h_buffer->ts <= clock)) { if( h_buffer->seq > last_played) { next_frame = h_buffer; h_buffer = h_buffer->next; last_played = next_frame->seq; break; else { struct pkt_q *tmp = h_buffer; h_buffer = h_buffer->next; Discard( tmp ); return next_frame; Network monitoring The client periodically computes the VTD, as defined in [11, 12]. The VTD value is computed during the play out process. It can be computed for each video frame that is played out. However, to avoid an excessive client overhead computation (useless if the network jitter is not considerable), the client can specify the interval time between consecutive VTD computation. In our implementation the VTD is computed every 60 seconds. In essence, VTD represents the time difference between the ideal and the actual video play out. If the VTD goes above the NIT threshold, the client sends a message to the sender, informing it about the delay that the network has introduced. The message format is reported in Fig. 7. The Message type field is used to denote the operation requested to the sender (in this case the server has to drop video frames). The Sequence number is used to notify the sender the video frame with the VTD above the NIT and the Extra Information field is used to send the computed VTD NIT to the server. In the following we present the code that performs the operation just described. If the control on the VTD is enabled, the VTD is computed and compared with the NIT. If VTD>NIT an RTP control message (CONTROL PT) is sent to the server with the frame number and the value VTD-NIT. if(checkvtd) { VTD = CurrentClock - frame->ts; if( VTD > NIT ) { chdr.seq = htons(frame->seq); chdr.type = htonl(drop); chdr.info = htonl(vtd-nit); RTPSend(CONTROL_PT, &chdr, sizeof(struct c_hdr)); 3.4 Server Implementation The server transmits the video stream over the network. Based on the video frame rate, the server encapsulates each video frame in an RTP packet and sends the packet towards the client. Header fields (Fig. 6) are filled according to the following: ffl PT (Payload Type): It contains a code that represents the type of information transported by the packet; ffl Timestamp: used to mark the video frame. This value is always incremented, according to the video stream frame rate. It is incremented by the ratio between the frequency of the transported data and the number of frame per second (e.g., frequency / fps), which corresponds to a number of seconds equal to 1 / fps. For instance, if we are transmitting a Motion-JPEG video encoded with 25 fps, the timestamp is incremented by 3600 units (90000 / 25), which corresponds to 0.04 seconds (40 ms). The server is also in charge of adapting the video stream transmission upon client request. This request is done when the client finds out network problems and is communicated to the server with an RTP message. The video stream adaptation is done through a dropping phase.

8 3.4.1 Dropping phase As shown in [11, 12], the VTD can be reduced by discarding video frames at the server side. An RTP message, containing a dropping frame request, is received by the server. This message contains the time quantity that exceeds the NIT (i.e., VTD-NIT) and this value (NDelay), along with the video frame rate FPS and the video frequency FR, allow the server computes the number of video frames it has to discard, as follows: FrameToDrop = ceil(ndelay / (FR/FPS)) After computing the number of video frames it has to discard, the server drops video frames according to the encoding mechanism used. In this paper we do not discuss details concerning the dropping frames algorithms, as we used the algorithms proposed in [11, 12]. However, we point out that with Motion-JPEG videos (intraframe encoding mechanism) it is possible to drop video frames without any problems, as all the video frames have the same importance. Conversely, if the video is encoded with inter-frame mechanism (like MPEG) a consideration of the frame type must be done in order to avoid a possible domino effect, when discarding a frame. Readers can refer to [11, 12, 17] for details regarding MPEG dropping frames algorithms. Roughly, after computing the F ramet odrop value, the server avoids the transmission of F ramet odrop consecutive frames. Since all the transmitted frames have a timestamp that represents the ideal play out time (and not the transmission time), the server has to compute the ideal play out time of each transmitted frame. The frame transmitted, after discarding F ramet odrop frames, has an ideal play out time (TS) equal to FrameToDrop times FR/FPS, which represents the time that elapses between the transmission of two consecutive video frames, as show in the following. TSinc = FR/FPS; if(frametodrop > 0) { TS = TSinc * (FrameToDrop + 1); 4 Experimental Results In this section we evaluate VISP through several experiments conducted over our 100 Mbit/sec Department LAN. The goal of our experiments is to show the effectiveness of the mechanism [11, 12] as well as to show that a periodic clocks synchronization must take place in order to avoid problems caused by the clocks skew. To show that a periodic clocks synchronization is necessary we use a virtual clock approach to synchronize the client and server clocks. Among all the possible clocks synchronization mechanisms, we use a virtual clock approach because it does not need additional messages or resources. Delay (ms) VTD Frames Figure 8. VTD Results: VTD control mechanism deactivated, clocks synchronization mechanism deactivated. Delay (ms) NIT VTD Frames Figure 9. VTD Results: VTD control mechanism activated, clocks synchronization mechanism deactivated. Further, in a LAN the accuracy of the virtual clock approach is good because the network transfer delay in a local network may be considered as negligible. To investigate what are the effects of client and server clocks skew we forced the client and server clocks to have different speeds. Experiments show that clocks synchronization is necessary at the beginning of the video stream transmission, but it is also necessary along the application lifetime. In Fig. 8, 9, 10, 11 we present results obtained while transmitting the movie Jurassic Park (for the sake of conciseness, only the results regarding the first 5 minutes transmission are shown). During all the conducted experiments, the client clock was forced to be 25 ms faster than the server clock, and the NIT value was set equal to 80 ms (even though 100 ms was sufficient to provide interactivity). In Fig. 8, we show that, when the VTD control mechanism and the clocks synchronization mechanism are both kept deactivated, the VTD value progressively increases, thus yielding a total amount of 98% of the video frames with a VTD value above the NIT. This is a quite expected result which may be explained due to the combination of the lack of a VTD control mechanism and the increase of the clocks skew between sender and receiver.

9 240 NIT VTD 240 NIT VTD Delay (ms) Delay (ms) Frames Frames Figure 10. VTD Results: VTD control mechanism activated, clocks synchronization mechanism activated with a frequency of 20 Hertz. Figure 11. VTD Results: VTD control mechanism activated, clocks synchronization mechanism activated with a frequency of 30 Hertz. In Fig. 9, we analyze the consequences of applying our VTD control mechanism in a scenario where no clocks synchronization activity is carried out on a periodical basis. In this case, the VTD is kept within the NIT and only 3.97% of the frames have a VTD value larger than the NIT (here, most of the frames have a VTD equal to NIT). However, since the VTD is kept within the NIT by dropping video frames, our VTD control mechanism needs to drop almost 2.7% of the transmitted video frames. This is a quite surprising result as the video frame transmission has been carried out on a high speed LAN. Simply put, the problem here is that nothing has been done in this experiment to avoid the clocks skew which can be considered as the main cause of the consistent frame loss which has been experienced. To obtain more information about this problem, we have repeated the same experiment illustrated above in a scenario where the VTD control mechanism and the simple virtual clocks synchronization mechanism, illustrated previously, were both kept activated. In particular, the clocks synchronization activity was carried out at a frequency of 20 Hertz (Fig. 10) and 30 Hertz (Fig. 11). As shown in Fig. 10, we see that the VTD control mechanism is able to guarantee that almost all the video frames have a VTD value within the NIT (only a percentage equal to 0.03 of frames have a VTD value above the NIT), and in addition only 0.02% of the video frames are dropped due to the operations performed by the VTD control mechanism. In essence, this result confirms that the main motivation behind the noticeable frame loss of the previous experiment was due to the fact our VTD control mechanism tried to react to the clocks synchronization problem rather than to network problems. To confirm this result, we carried out a final experiment where the frequency of the periodical clocks synchronization activity was increased to 30 Hertz. In this case, as shown in Fig. 11, almost all the video frames have a VTD lower than NIT, and almost no frame is dropped to obtain this positive result. As a final consideration, we wish to mention also that we carried out also a set of experiments where the server clock was forced to be faster than the client clock. Summing up, the main result we have obtained is that in this case a noticeable phenomenon of buffer overflow occurs (with a consequent frame loss of approx. to 3%) if no clocks synchronization activity is carried out on a periodical basis. Very similar results have been obtained while transmitting different other movies (Big, Beauty and the Beast, The Simpsons). 5 Conclusions In this paper we presented VISP, a client-server application protocol designed to support interactive video streaming applications over the Internet. VISP is a protocol that controls the video stream transmission in order to provide interactive features to the client. VISP is based on a recently proposed mechanism that acts on the video QoS in order to avoid possible problems caused by the network jitter. VISP provides the client with the ability of performing VCR-like functions and is provided with a clocks synchronization mechanism in order to periodically synchronize client and server clocks. In fact, we noticed that clocks skew may compromise the mechanism upon which VISP is based. For this reason a clocks synchronization mechanism has been introduced. VISP has been evaluated though several experiments and results confirmed that a periodic clocks synchronization is necessary in order to avoid possible problems caused by the clocks skew and also confirm that the mechanism proposed in [11, 12] is effective. The effectiveness of the mechanism along with the introduced VCR-like functions and the periodic clocks synchronization candidate VISP as a protocol to support interactive video streaming applications over the Internet.

10 References [1] S. Kadur, F. Golshani, G. Millard: Delay-jitter control in multimedia application, Multimedia Systems, No. 4, pp , [2] G. Karlsson: Quality Requirements for Multimedia Network Services, Proceedings of Radiovetenskap och kommunikation -96, pp , Lulea, Sweden, June 3-6, [3] T. Kurita, S. Iai, N. Kitawaki: Effects of transmission delay in audiovisual communication, Electronics and Communications in Japan, 77(3):63 74, [4] R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne: Adaptive Playout Mechanisms for packetized audio applications in wide-area networks, IEEE Computer and Communications Societies on Networking for Global Communications, pp , Loas Alamitos, CA, June [5] M. Roccetti, V. Ghini, G. Pau, P. Salomoni, M.E. Bonfigli: Design and Experimental Evaluation of an Adaptive Playout Delay Control Mechanism for Packetized Audio for Use over the Internet, Multimedia Tools and Applications, Kluwer Academic Publishers, Vol. 14, N. 1, pp , May [6] M. Claypool, J. Riedl: End-to end quality in multimedia applications, Chapter 40 in Handbook on Multimedia Computing, [7] S. Sitaram, A. Dan: Multimedia Servers: Application, Environments, and Design, Morgan Kaufmann Publishers, San Francisco [8] W-C. Feng, F. Jahanian, S. Sechrest: Optimal buffering for the delivery of compressed prerecorded video, Multimedia Systems, Vol. 5, No. 5, pp , [9] W-C. Feng, S. Sechrest: Smoothing and buffering for the delivery of compressed prerecorde video,proceedings of the IS&T/SPIE Symposium on Multimedia Computer and Networking, pp , February [10] J.D. Salehi, Z.L. Zhang, J. Kurose, D. Towsley: Supporting Stored Video: Reducing Rate Variability and End-to-End Resources Requirements through Optimal Smoothing, IEEE/ACM Transactions on Networking, Vol. 6, No.4, pp , [11] M. Furini, M. Roccetti: Design and Analysis of a Mechanism for supporting Interactive Video Streaming Applications over the Internet, Proc. of the 6th IASTED/IMSA Conference, Kauai, Hawaii, pp , August [12] M. Furini, M. Roccetti, Interactive MPEG Video Streaming over IP networks: A Performance Report, Proc. of the Communications and Computer Networks (CCN2002) Conference, MIT, Cambridge, USA, pp , November [13] H. Schulzrinne: RTP: The real-time transport protocol, in MCNC 2nd Packet Video Workshop, volume 2, Research Triangle Park, North Carolina, December [14] J.P. Shenoy, M.V. Harrick: Efficient Support for Interactive Operations in Multi-resolution Video Servers, ACM/Springer Multimedia Systems Journal 7(3), (July 1999) [15] J.D. Sicar, J. Salehi, J. Kurose, D. Towsley: Providng VCR capabilities in Large-Scale Video Server, Proceedings of ACM Int. Conference on Multimedia. San Francisco, CA, (October 1994). [16] M.S. Chen, D.D. Kandlur: Downloading and Stream Conversion: Supporting Interactive Playout of Videos in a Client Station, Proceedings of the Int. Conference On Multimedia Computing and Systems (ICMCS), Washington D.C., (May 1995) [17] M. Furini, D. Towsley: Real-Time traffic transmission over the Internet, IEEE Transaction on Multimedia, vol. 3, No. 1, pp , March [18] B. Furht: A survey of multimedia compression techniques and standards. Part I. JPEG standards, Real Time Imaging, vol. 1, pp , [19] ISO/IEC, Information Technology-Coding of Moving Pictures and Associated Audio for Digital Storage Media up to about 1.5 Mbit/s, International Organization for Standardization, [20] N. Laoutaris, I. Stavrakakis, Intrastream Synchronization for Continuous Media Streams: A Survey of Playout Schedulers, IEEE Network Magazine, Vol. 16, No. 3, May [21] Z.L. Zhang, S. Nelakuditi, R. Aggarwal, R.P. Tsang: Efficient Selective Frame Discard Algorithms for Stored Video Delivery across Resource Constrained Networks, Proc. IEEE INFOCOM 99, NYC, pp , March [22] D. Wijesekera, J. Srivastava: Quality of Service (QoS) Metrics for Continuos Media, Multimedia Tools and Applications, Vol. 2, No.3, pp , Sept [23] S. Cen, C. Pu, J. Walpole: Flow and congestion control for Internet streaming applications, Proc. SPIE/ACM Multimedia Computing and Networking 98, San Jose, CA, pp , January 1998.

11 [24] M. Claypool, J. Tanner, The Effects on Jitter on the Perceptual Quality of Video, ACM Multimedia, pp , November [25] D. Ferrari: Delay Jitter Control Scheme for Packet- Switching internetworks, Computer Communications, pp , July [26] M. Podolsky, M. Vetterli, S. McCanne: Limited retransmission of real-time layered multimedia, IEEE Workshop on multimedia Signal Processing, Los Angeles, CA, December [27] D. Stone, K. Jeffay: An Empirical study of delay jitter management policies, ACM Multimedia Systems, Vol. 2, n.6, pp , January [28] Y. Wang, Q. Zhu: Error Control and Concealment for Video Communications: A Review, Proc. of the IEEE, Vol. 86, pp , May 1998.