A Hierarchical Fair Service Curve Algorithm for Link-Sharing, Real-Time and Priority Services. Ion Stoica Hui Zhang T. S. Eugene Ng.

Transcription

1 A Hierarchical Fair Service Curve Algorithm for Link-Sharing, Real-Time and Priority Services Ion Stoica Hui Zhang T. S. Eugene Ng September 1997 CMU-CS School of Computer Science Carnegie Mellon University Pittsburgh, PA os mijm mssmam u An earlier version of this paper appeared in Proceedings of ACM SIGCOMM : This research was sponsored by Defense Advance Research Projects Agency (DARPA) under contract numbers N C-8528 and N K-0002, and by a National Science Foundation (NSF) Career Award under grant number NCR Additional support was provided by Intel Corp., MCI, and Sun Microsystems. Views and conclusions contained in this document are those of the authors and should no be interpreted as representing the official policies, either expressed or implied, of DARPA, NSF, Intel, MCI, Sun, or the U.S. government. 3T! -J gjsw JS.-SCW»g *«ai ÄdlssttSESi yvsp

2 Keywords: Resource management, scheduling, link-sharing, real-time, fairness.

3 Abstract In this paper, we study hierarchical resource management models and algorithms that support both link-sharing and guaranteed real-time services with decoupled delay (priority) and bandwidth allocation. We extend the service curve based QoS model, which defines both delay and bandwidth requirements of a class, to include fairness, which is important for the integration of real-time and hierarchical link-sharing services. The resulting Fair Service Curve link-sharing model formalizes the goals of link-sharing and real-time services and exposes the fundamental tradeoffs between these goals. In particular, with decoupled delay and bandwidth allocation, it is impossible to simultaneously provide guaranteed realtime service and achieve perfect link-sharing. We propose a novel scheduling algorithm called Hierarchical Fair Service Curve (H-FSC) that approximates the model closely and efficiently. The algorithm always guarantees the performance for leaf classes, thus ensures real-time services, while minimizing the discrepancy between the actual services provided to the interior classes and the services defined by the Fair Service Curve link-sharing model. We have implemented the H-FSC scheduler in the NetBSD environment. By performing simulation and measurement experiments, we evaluate the link-sharing and real-time performances of H-FSC, and determine the computation overhead.

4 1 Introduction The emerging integrated services networks will support applications with diverse performance objectives and traffic characteristics. While most of the previous research on integrated services networks has focused on guaranteeing QoS, especially real-time requirements, for each individual session, several recent work [1, 6, 12] has argued that it is also important to support hierarchical link-sharing service. With hierarchical link-sharing, there is a class hierarchy associated with each link that specifies the resource allocation policy for the link. A class represents some aggregate of traffic streams that are grouped according to administrative affiliation, protocol, traffic type, or other criteria. Figure 1 shows an example class hierarchy for a 45 Mbps link that is shared by two organizations, CMU and University of Pittsburgh (U. Pitt). Below each of the two organization classes, there are classes grouped based on traffic types. Each class is associated with a bandwidth, which is the minimum amount of service this class's traffic should receive when there are enough demands. There are several important goals for hierarchical link-sharing service. First, each class should receive certain minimum bandwidth if there are enough demands. In the example, CMU's traffic should receive at least 25 Mbps bandwidth during a period when the aggregate traffic from CMU has a higher arrival rate. Similarly, if there are resource contentions between traffic classes within CMU, the video traffic should get at least 10 Mbps. In the case when there are only audio and video streams from CMU, the audio and video traffic should receive all the bandwidth that is allocated to CMU (25 Mbps) if the demand is high enough. That is, if certain traffic classes from CMU do not have enough traffic to fully utilize its minimum guaranteed bandwidth, other traffic classes from CMU will have a higher priority to use this excess bandwidth than traffic from U. Pitt. While the above policy specifies that CMU audio and video traffic classes should use the excess bandwidth unused by the data traffic, there is still the issue of how the excess bandwidth is distributed between the audio and video traffic classes. A second goal of hierarchical link-sharing service is then to have a proper policy to distribute the excess bandwidth unused by a class to its sibling classes. In addition to the two goals mentioned above, it is also important to support real-time

5 10 Mbps 2Mbps Distinguished Lecture Distinguished Lecture Figure 1: An Example Link-Sharing Hierarchy. and priority services within the framework of hierarchical link-sharing. Since real-time service guarantees QoS on a per session basis, a natural way to integrate real-time and hierarchical link-sharing services is to have a separate leaf class for each real-time session. In the example, the CMU Distinguished Lecture video and audio classes are two leaf classes that correspond to real-time sessions. Finally, we would like to support priority service in the sense that delay (both average delay and delay bound) and bandwidth allocations are decoupled. For example, even though the CMU Distinguished Lecture video and audio classes have different bandwidth requirements, it is desirable to provide the same low delay bound for both classes. Such a decoupling of bandwidth and delay allocation is also desirable for interior or leaf classes that correspond to traffic aggregates. For example, one may want to provide a lower average delay for packets in CMU's audio traffic class than those in CMU's data traffic class. A number of algorithms have been proposed to support hierarchical link-sharing, realtime, and priority services. However, as discussed in Section 7, they all suffer from important limitations. The fundamental problem is that with all three services, multiple requirements need to be satisfied simultaneously. This is very difficult and sometimes im-

6 possible to achieve due to conflicting requirements. This problem is exacerbated by the fact that there is no formal definition of hierarchical link-sharing service that specifies all the requirements. In this paper, we consider an ideal model that can precisely define all the important performance goals of real-time, hierarchical link-sharing, and priority services. The basic building block of the framework is the concept of service curve, which defines a general QoS model taking into account both bandwidth and priority (delay) requirements. In this architecture, each class in the hierarchy is associated with a service curve. An ideal Fair Service Curve link-sharing model is to (a) simultaneously guarantee the service curves for all nodes in the hierarchy, and (b) distribute the excess bandwidth unused by a class to its sibling classes fairly. Since the service curves for class nodes are guaranteed simultaneously, the QoS for both individual sessions (leaf nodes in the hierarchy) and traffic aggregates (interior and possibly leaf nodes in the hierarchy) are satisfied. In addition, delay and bandwidth allocation can be decoupled by choosing different shapes of service curves. Therefore, the fair service curve link-sharing model gives a precise definition of the linksharing service that simultaneously satisfies all the important goals of real-time and linksharing services. Unfortunately, as will be shown in the paper, the ideal model cannot be realized at all times. In spite of this, the model serves two important purposes. First, unlike previous models, the new model explicitly defines the situations when all performance goals cannot be simultaneously satisfied, thus exposing the fundamental tradeoffs among conflicting performance goals. Second, the model serves as an ideal target that a scheduling algorithm should approximate as closely as possible. With the ideal service model defined and the fundamental tradeoffs exposed, we propose an algorithm called Hierarchical Fair Service Curve (H-FSC) that achieves the following three goals: guarantee the service curves of all leaf class nodes, minimize the short-period discrepancy between the total amount of services provided to interior node class and its service curve, allocate the excess bandwidth to sibling classes, with bounded fairness

7 Notice that we made the architecture level decision that whenever there is a conflict, the performance guarantees of the leaf class nodes take priority. We believe this is the right tradeoff as the performance of leaf classes are most related to the performance of individual applications. In particular, since a session is always a leaf class, guaranteed real-time services can be provided on a per session basis with this framework. The rest of the paper is organized as follows. Section 2 presents the Fair Service Curve link-sharing model and discusses the fundamental tradeoffs in approximating this model. Section 3 presents our solution, the Hierarchical Fair Service Curve (H-FSC) scheduler, followed by a discussion on its implementation complexity in Section 4. We analyze the delay and fairness properties of H-FSC in Section 5, and evaluate its performance based on both simulation and measurement experiments in Section 6. We discuss related work in Section 7 before concluding the paper in Section 8. 2 Fair Service Curve Link-Sharing Model In this section, we first define the service curve QoS model and motivate the advantage of using non-linear service curves to decouple delay and bandwidth allocation. We then extend the concept of fairness to service curve based schedulers. Finally, we present the ideal Fair Service Curve link-sharing model and discuss the fundamental tradeoffs involved in designing a scheduler that approximates the model. 2.1 Service Curve Based QoS Model As discussed in Section 1, we will use the service curve abstraction proposed by Cruz [4, 5] as the building block to define the idealized link-sharing model. A session i is said to be guaranteed a service curve Si(-), if for any time t 2, there exists a time ti < t 2, which is the beginning one of session Vs backlogged periods (not necessarily including t 2 ), such that the following holds Sr(t 2 -h) <Wi{t U t 2 ), (1) where w i (t 1,t 2 ) is the amount of service received by session i during the time interval (i 1? t 2 ]. For packet systems, we restrict t 2 to be packet departure times.

8 In the case in which the server service curve is not concave, one algorithm that supports service curve guarantees is Service Curve Earliest Deadline first (SCED) [11]. With SCED, a deadline is computed for each packet using a per session deadline curve A( - ) and packets are transmitted in increasing order of their deadlines. The deadline curve A(-) is computed such that in an idealized fluid system, session Vs service curve will be guaranteed if by any time t, at least D{(t) amount of service is provided to session i. Based on Eq. (1), it follows that Di{t) = mm(s t (t - U) + Wiih)), (2) where the minimization is over all the beginnings of session Vs backlogged periods t^s, and Wi(ti) = Wi(0,ti) is the total amount of service session i receives till time ti. This gives the following iterative algorithm to compute A(')- When session i becomes backlogged for the first time, A(-) is initialized to its service curve -%( ). Subsequently, whenever session i becomes backlogged again at time t a after an idling period, Di(-) is updated according to the following: Di(t) = min(di(t), S z (t - t a ) + Wl {t a )), V t > Dr\ Wi {t a )). (3) The reason for which A(-) is defined only for t > D~ l {ci) is because this is the only portion that is used for subsequent deadline computation. Since A(-) may not be an injection, its inverse function may not be uniquely defined. Here, we define D~ 1 (y) to be the smallest value x such that Di(x) = y. Based on A(-)> tne deadline for a packet of length L\ at the head of session Vs queue can be computed as follows, d i = D7 1 (w i (t) + L*) (4) The guarantees specified by service curves are quite general. For example, the guarantees provided by Virtual Clock and various Fair Queueing algorithms can be specified by linear service curves with zero offsets. 1. Since a linear service curve is characterized by only one parameter, the slope or the guaranteed bandwidth for the session, the delay requirement 1 ln theory, Fair Queueing and its corresponding fluid algorithm GPS can support more general service curves than linear curves [10, 15]. However, in practice, such a resource assignment has a number of limitations. See Section 7 for a detailed discussion.

9 cannot be specified separately. As a consequence, even though delay bounds can be provided by algorithms guaranteeing linear service curves, there is a coupling between the guaranteed delay bound and bandwidth, which results in inflexible resource allocation. With non-linear service curves, both priority (delay) and bandwidth allocation are taken into account in an integrated fashion, yet the allocation policies for these two resources are decoupled. This will increase the resource management flexibility and the resource utilization inside the network. To illustrate the advantage of decoupling delay and bandwidth allocation with non-linear service curves, consider the example in Figure 2, where a video and an FTP session share a 10 Mbps link served by a SCED scheduler. Let the video source sends 30 8KB frames per second, which corresponds to a required bandwidth of 2 Mbps. The remaining 8 Mbps is reserved by a continuously backlogged FTP session. For simplicity, let all packets be of size 8 KB. Thus, it takes roughly 6.5 ms to transmit a packet. Let both video and FTP sessions be active at time 0. Then the sessions' deadline curves are also their service curves. First, consider the case in Figure 2(a) where linear service curves are used to specify the sessions' requirements. The arrival curve A,(-) represents the cumulative number of bits received by session i. The deadline of a packet of session i arriving at time u is computed as the time t such that S(t) equals A(u). As can be seen, the deadlines of the video packets occur every 33 ms, while the deadlines of the FTP packets occur every 8.2 ms. This results in a delay of approximately 26 ms for a video packet. In the second scenario as illustrated in Figure 2(b), we use two-piece linear service curves for characterizing the sessions' requirements. The slope of the first segment of the video session's service curve is 6.6 Mbps, while the slope of the second segment is 2 Mbps. The inflection point occurs at 10 ms. The FTP session's service curve is chosen such that the entire remaining capacity is used. As can be seen, the delay of any video packet is no more than 10 ms in this case. It is important to note that the reduction in the delays for video packets does not come for free: as a result, the delays for FTP packets increase. However, this is acceptable since throughput rather than per packet delay is more important to the FTP session. While in theory any non-decreasing functions can be used as service curves, in practice only linear or piecewise linear functions are used for reasons of simplicity. In general, a concave service curve will result in a lower average and worst case delay for a session than 6

10 a linear or convex service curve with the same guaranteed assymptotic rate. However, it is impossible to have concave service curves for all sessions and still reach high average utilization. Intuitively, this is easy to understand as priority is relative and it is impossible to give all sessions high priority (low delay). Formally, the SCED algorithm can guarantee all the service curves if and only if, Si(t) < S(t) holds for any t > 0 where S(t) is the amount of service the server provides during a time period of t. That is, the sum of the service curves over all sessions should be no more than the server's service curve. 2.2 Service Curve and Fairness While the service curve is very general in specifying the minimum amount of service (both bandwidth and priority) guaranteed to a session or a class, it does not specify how the excess service, which is the extra capacity of the server beyond that is needed to guarantee the service curves of all active sessions, should be distributed. It is possible to have different scheduling algorithms that provide the same service curve guarantees but use different policies for distributing excess service. For example, while Virtual Clock and Weighted Fair Queueing (WFQ) can provide identical linear service curve guarantees, they have different fairness properties. In particular, with Virtual Clock, it is possible that a session does not receive service for an arbitrary long period because it receives excess service in a previous time period. On the contrary, the maximum period that an active session does not receive service in a WFQ server is bounded. While the fairness property has been extensively studied for scheduling algorithms that only use sessions' rates as parameters and there are several formal definitions of fairness properties, such as the relative fairness given by Golestani [8] and the worst-case fairness given by Bennett and Zhang [2], it is unclear what fairness means and why it is important in the context of scheduling algorithms that decouple the delay and bandwidth allocation. In this section, we discuss the semantics of fairness and argue that it is important to have the fairness property even for scheduling algorithms that provides performance guarantees by decoupling the delay and bandwidth allocation. We then give a simple example to illustrate that SCED is an unfair algorithm, but can be extended to be fair. There are two aspects of the fairness property that are of interest: (1) what is the policy of distributing excess service to each of the currently active sessions? (2) whether and to

11 what extent a session receiving excess service in a previous time period will be penalized later? For rate-proportional scheduling algorithms, a perfectly fair algorithm will distribute the excess service to all backlogged sessions proportional to their minimum guaranteed rates. In addition, it will not punish any session for receiving excess service in a previous time period. Generalized Processor Sharing (GPS) is such an idealized fair algorithm. For scheduling algorithms based on general service curves, a fair algorithm should (a) distribute excess service according to a well defined policy, and (b) not penalize a session that uses excess service. Though these two aspects of the fairness property are usually considered together in a formal fairness definition, they are actually orthogonal issues. While different policies can be used to distribute excess service, in this paper we simply distribute excess service according to the service curves. It is the second aspect of the fairness property, i.e., a session that receives excess service in a previous time period should not be penalized, that we would like to emphasize in this paper. There are two reasons why it is important to have such a fair scheduler. First, even in a network that supports guarantees, it is still desirable to let end systems to statistically share the fraction of resources that are either not reserved and/or not currently being used. A network service should encourage a source to opportunistically send more traffic than the minimum guaranteed amount, provided that the guarantees for all other sessions are not affected by the extra traffic. That is, a network should not penalize a session that uses more service than guaranteed if the additional service it uses is the excess service allotted by the server. Fairness is also important when we want to construct a hierarchical scheduler to support hierarchical link-sharing. In [1], it has been shown that the accuracy of link-sharing and delay bounds provided by Hierarchical Packet Fair Queueing (H-PFQ) is closely tied to the fairness property of PFQ server nodes used to construct the H-PFQ scheduler. While the SCED algorithm can guarantee all the service curves simultaneously, as long as the server service curve is not concave, it does not have the fairness property. Consider the example shown in Figure 3(a). Session 1 and 2 have two-piece linear service curves Si(-) and S 2 (-)i respectively, where

12 and Si(t) at, if t < T ßt, if t > T (5) S 2 (t) = { ßt, if t < T " (6) erf, if t > T In addition, let the server rate be one, and assume the followings hold: a < ß, i.e., S'i(-) is convex and S^-) is concave, a + /? < 1, i.e., both service curves can be guaranteed by using SCED, and 2ß > 1, i.e., it is not possible to guarantee the peak rates of both sessions simultaneously. Also, for simplicity, assume that the packets are of unit length, and once a session becomes active it remains continuously backlogged. Under these assumptions, the deadline of the k-th. packet of session i under SCED is simply Sf 1 (k) + f s, where t\ is the time when session i becomes active. Similarly, the deadline of the last packet of session i that has been transmitted by time t (t > t\) is S~ 1 (wi(t l s,t)) + t\. Note that since session i is not active until t l s, we have Wj(t) = Wi(0,t\) + Wi( s,t) = Wi(t l s,t). Now consider the scenario in which session 1 becomes active at time 0 and session 2 becomes active at time t 0. Since session 1 is the only session active during the time interval [0, to], it receives all the service provided by the server, i.e., u>i(t) = t, for any 0 < t < t 0 (see Figure 3(b)). Also, the deadline of the last packet of session 1 that has been transmitted by time t 0 is S^^i^o)) = ^(to). Next, consider at time to, when the second session becomes active (see Figure 3(c)). Since the deadline of the A;-th packet of session 2 is S^ik) + t 0 and packets are served in increasing order of their deadlines, it follows that as long as S'^1(A;) -\-to < Si 1 (t 0 ), only the packets of session 2 are transmitted. Thus, session 1 does not receive any service during the time interval (i 0, ^], where t\ is the smallest time such that S2 1 {w 2 (ti)) + to > S^fto). As shown in Figure 3(c), for any time t, Wi(t) > S\(t) and w 2 (t) > S 2 (t t 0 ) hold, i.e., the SCED algorithm guarantees the service curves of both sessions. However, SCED punishes session 2 for receiving excess service during [0, to] by keeping it from receiving service during (t 0, ii]. This behavior makes it difficult to use SCED in a hierarchical server. To see why, consider a simple two-level hierarchy where the bandwidth is shared by two classes, characterized by the service curves S\(-), and S 2 (-), respectively. Then, if one of 9

13 class l's child classes becomes active at some point between t 0 and ti, it will not receive any service before 1? no matter how "important" this session is! It is interesting to note that in a system where all the service curves are simple lines, SCED reduces to the well-known Virtual Clock discipline. While Virtual Clock is unfair [10, 16], there exists algorithms (such as the various PFQ algorithms) that not only provide the same service curve guarantees as Virtual Clock but also achieve fairness. In PFQ algorithms, each session is associated with a virtual time function that represents the normalized amount of service that has been received by the session. The algorithm then achieves fairness by minimizing the differences among the virtual time functions of all sessions. Since Virtual Clock is a special case of SCED, it is natural to use the same idea for achieving fairness in SCED with general service curves. This is achieved by associating with each session a generalized virtual time function, and servicing the session that has the smallest virtual time function. While we will describe the detailed algorithm in Section 3, we use the example in Figure 3(d) to illustrate the concept. The main modification to SCED would be to use S 2 {t d 0 ) in computing the packets' deadlines for session 2, instead of S 2 (t t 0 ). It can be easily verified that if Si(t) = rit and S 2 (t) = r 2 t, where r x and r 2 are the rates assigned to sessions 1 and 2 respectively, the above algorithm results in identical behaviors as in WFQ. Figure 3(d) shows the allocation of the service time when this discipline is used. Note that, unlike the previous case, session 1 is no longer penalized when session 2 becomes active. In summary, fairness can be incorporated into service curve based schedulers such that (a) the excess service is distributed according to the service curves of active sessions, and (b) a session using excess service will not be penalized later. Unfortunately, this does not come for free. As shown in Figure 3(d) the service curve of session 2 is violated immediately after time t 0. This underlines the difficulty of simultaneously achieving fairness, while guaranteeing the service curves. In fact, as we will see in the next section, in general this is not possible. 2.3 Fair Service Curve Link-Sharing Model As discussed at the beginning of the paper, the important goals of hierarchical link-sharing are: guaranteed QoS for each class, priority or decoupled delay and bandwidth allocation 10

14 among classes, and proper distribution of excess bandwidth. Since the service curve abstraction provides a general definition of QoS with decoupled delay and bandwidth allocation, and can be extended to include fairness property for the purpose of excess bandwidth distribution, it is natural to use service curves to define the performance goals of link-sharing and real-time services. In a Fair Service Curve linksharing mode there is a service curve associated with each node in the link-sharing hierarchy. The goal is then to (1) satisfy the service curves of all nodes simultaneously, and (2) distribute the excess service fairly as defined in Section 2.2. Note that (1) is a general requirement that subsumes both link-sharing and real-time performance goals. A realtime session is just a leaf node in the hierarchy, and its performance will be automatically guaranteed if the Fair Service Curve link-sharing model is realized. Unfortunately, with non-linear service curves, there are time periods when either (a) it is not possible to guarantee the service curve for all classes, or (b) it is not possible to simultaneously satisfy both the service curves and fairness property. To see why (a) is true, consider the hierarchy in Figure 4(a). For simplicity, assume the service curve assigned to an interior class is the sum of the service curves of all its children. z\lso, assume all sessions are continuously backlogged from time 0 except session 1, which is idle during [0,t] and becomes backlogged at time t. During [0,2], since session 1 is not active, its entire service is distributed to session 2 according to the link-sharing semantics. At time t, session 1 becomes active. In order to satisfy session l's service curve, at least Si(At) service need to be allocated for session 1 for any future time interval (t,t + At]. However, as shown in Figure 4(b), since the sum of all the service curves that need to be satisfied during (t,t + At] is greater than the server's service curve, it is impossible to satisfy all the service curves simultaneously during this period. Since decoupling delay and bandwidth allocation is equivalent to specifying a non-linear service curve, this translates into a fundamental conflict between link-sharing and real-time service when the delay and bandwidth allocation is decoupled. To see the fundamental conflict between fairness and real-time requirements with decoupled delay and bandwidth allocation, consider the example in Figure 3 again. As shown in Figure 3(d), if fairness is to be provided, the service curve of session 2 will be violated, i.e., w 2 (t) < S2{t to), for some t > t 0. This is because after t 0 both sessions receive service at a 11

15 rate proportional to their slope, and since immediately after time t 0 their slopes are equal, each of them is served at a rate of 1/2, which is smaller than ß, the service rate required to satisfy S 2 (-). Finally, it is worth to note that when all service curves degenerate to lines, this algorithm reduces to WFQ. Therefore, there are time periods when the Fair Service Curve link-sharing model cannot be realized. In spite of this, the model serves two purposes. First, unlike previous models, this model explicitly defines the situations when all performance goals cannot be simultaneously satisfied. This exposes the fundamental architecture tradeoff decisions one has to make with respect to the relative importance among the conflicting performance goals. Second, the model serves an ideal target that a scheduling algorithm should approximate as closely as possible. We believe that a scheduler should guarantee the service curves of the leaf classes all the time while trying to minimize the discrepancy between the service allocated to each interior class and its fair service according to the model. 3 Hierarchical Fair Service Curve (H-FSC) In this section, we propose a new scheduling algorithm called Hierarchical Fair Service Curve (H-FSC) that closely approximates the ideal Fair Service Curve link-sharing model as defined in the previous section. 3.1 Overview of the Algorithm The scheduling is based on two criteria: the real-time criteria that ensures the service guarantee of all leaf classes, and the link-sharing criteria that aims to satisfy service curves of interior classes and fairly distribute the excess bandwidth. The real-time criteria is used to select the packet only if there is a potential danger that the service guarantees for leaf nodes are violated. Otherwise, the link-sharing criteria is used. Such a policy ensures the real-time guarantee of the leaf classes while at the same time minimizing the discrepancy between the actual services received by interior nodes and those defined by the ideal link-sharing model. With H-FSC, each leaf class i maintains a triplet (e,-, <,?;;), while each interior class j maintains only v h where e,- and d { represents the eligibility time and the deadline associated 12

16 with the first packet of class i's queue, and Vi and Vj are virtual times for the classes. The deadlines are assigned such that if the deadlines of all packets of a session are met, its service curve is guaranteed. The eligibility times are used to arbitrate which one of the two scheduling criteria to use for selecting the next packet. The packet at the head of session i's queue is said to be eligible if e 4 < t, where t is the current time. Eligibility times are computed such that at any given time when there are eligible packets in the system, there is a danger that the deadline of at least one packet is to be violated if the link-sharing instead real-time criteria is used, i.e., there is a potential conflict between linksharing and real-time goals. Since the real-time goal is more important, whenever there are eligible packets, the algorithm will always use the real-time criteria, which is to select, among all eligible packets, the one with the smallest deadline. At any given time when there are no eligible packets, i.e., there are no possible conflicts between link-sharing and real-time goals, the algorithm will apply the link-sharing criteria recursively, starting from the root class and stopping at a leaf class, selects, among all child classes, the one with the smallest virtual time. While deadline and eligibility times are associated only with leaf classes, virtual times are associated with both interior and leaf classes. The virtual time of a class represents the normalized amount of service that has been received by the class. In a perfect fair system, the virtual times for all sibling classes should be identical. The objective of the link-sharing criteria is then to minimize the discrepancies between virtual times for sibling classes. The pseudo code of H-FSC is given in Figure 5. In computing eligibility time, deadline, and virtual time, the algorithm uses three curves, one for each parameter: the eligible curve Ei(-), the deadline curve Di(-), and the virtual curve K( - )- The exact algorithms to update these curves are presented Section 3.2 and Section 3.3. There are several noteworthy points about the algorithm. First, while H-FSC needs to use two packet selection criteria to support link-sharing and real-time services, the other hierarchical algorithm, Hierarchical Packet Fair Queueing (H-PFQ) [1], selects packet solely based on the link-sharing criteria, and yet, it can support both link-sharing and real-time services. This is because H-PFQ guarantees only linear service curves, and it is feasible to guarantee all linear service curves simultaneously in a class hierarchy. In contrast, H- FSC supports decoupled delay and bandwidth allocation by guaranteeing non-linear service curves. As we have shown in Section 2, it is infeasible to guarantee all non-linear service 13

17 curves simultaneously in a class hierarchy. Consequently, H-FSC uses two separate criteria for each of the link-sharing and real-time goals, and employs the mechanism of eligibility time to determine which criteria to use. Second, the algorithm uses three types of time parameters: deadlines, eligibility times, and virtual times. While leaf nodes maintain all three parameters, the interior nodes maintain only the virtual time parameter. This is because deadlines and eligibility times are used for the purpose of guaranteeing the service curves, and H-FSC provides guarantees service curves only for leaf classes. On the other hand, virtual times are used for the purpose of hierarchical link-sharing that involves the entire hierarchy, and therefore are maintained by all classes in the hierarchy. A third point to notice is that while all three parameters are time values, they are measured with respect to different clocks. Deadlines and eligibility times are real times in the sense that they are measured with respect to the physical real-time clock. The absolute values are important as they need to be compared with the real-time clock. In contrast, the virtual time of a class is measured with respect to the total amount of service provided by its parent class. 2 The relative differences between virtual times of sibling classes are more important than the absolute values of the virtual times. Finally, we note that in addition to the advantage of decoupling delay and bandwidth allocation by supporting non-linear service curves, H- FSC provides tighter delay bounds than H-PFQ even for class hierarchies with only linear service curves. The key observation is that in H-PFQ, packet scheduling is solely based on link-sharing criteria, which needs to go recursively from the root class to a leaf class when selecting the next packet for transmission. The net effect is that the delay bound provided to a leaf class increases with the depth of the leaf in the hierarchy [1]. In contrast, with H-FSC, the delay bound of a leaf class is determined by the real-time packet selection criteria, which considers only the leaf classes. Therefore, the delay bound is independent of the class hierarchy. 3.2 Eligible Time and Deadline In this section, we present the algorithm to compute the deadline and the eligible time for each leaf class. 2 For simplicity of notation, the parent of the root class is the server itself. 14

18 For each leaf class i, the algorithm maintains two curves, one for each parameter: the eligible curve E(-) and the deadline curve D(-). In addition, it keeps a variable a, which is incremented by the packet length each time a class i packet is selected using the real-time criteria. Thus c; represents the total amount of service that the class has received when selected under the real-time criteria. Like SCED, the deadline curve A(') is initialized to its service curve S{(-), and updated each time session i becomes active at time t a according to the following: Di(t) = min(a(*), Si(t - t a ) + c-), V t > D- 1^). (7) This is the same as Eq. (3) except that c,- is used instead of tu,-. Since c 8 - does not change when the session receives service via the link-sharing criteria, the deadlines of future packets will not be affected due to the fact that the session receives excess service from the linksharing hierarchy (see Figure 6). This is the essence of the "non-punishment" aspect of the fairness property. While deadlines are used to guarantee service curves for leaf classes, eligibility times are used to arbitrate which one of the two scheduling criteria is to be applied to choose the next packet for service. The key observation is that with non-linear service curves, sometimes it is not possible to achieve perfect link-sharing and guarantee all service curves at the same time. A typical situation is when a session i with a concave service curve becomes active at t a, joining sessions that have convex service curves. Before session i joins, the other sessions receive the excess service, but their deadline curves are not updated. When session i becomes active, if the sum of the slopes of all active sessions' deadline curves at time t is larger than the server rate, it is impossible to satisfy the service curves of all sessions. The only solution is to have the server allocate active sessions "enough" service in advance using the real-time criteria such that the server has sufficient capacity to satisfy the service curves of all sessions when new sessions become active. However, whenever a packet is served using the real-time criteria but another packet has a smaller virtual time, there is a departure from the ideal link-sharing distribution. Therefore, to minimize the discrepancy from the ideal link-sharing model, we want to serve packets using the link-sharing criteria whenever there is no danger that the guarantees for leaf classes will be violated in the 15

19 future. In H-FSC, eligibility times are used to arbitrate which one of the two criteria is to be applied to select the next packet. To give more insight on the concept of eligibility, let E(t) be the minimum service that all active sessions should receive by time t, such that irrespective of the arrival traffic, the aggregate service time required by all sessions during any future time interval (t,t'] cannot exceed R x ( ' - t), i.e., cannot exceed the servercapacity, R. Note that this is a necessary condition: if the active sessions do not receive at least E(t) service by time t, then there exists a scenario in which the service curve of at least one session will be violated in the future. Intuitively, the worst case scenario occurs when all sessions are continuously active after time t [?]. Because the above condition holds for any future time t' : we have E(t) = A-(0 + [max( (A(O-A(0) (8) + (A-(<')-A-(0)-Äx(*'-f))] +, iev(t) where D* represents the deadline curve of a passive session i that becomes active at time t, and [x] + denotes max(x,0). The above equation reads as follows. In the worst case, when all passive sessions become active at time t, the maximum service requested by all sessions during the time interval (t,t'] while all of them remain active is: E; e^(t)(a(i') - Di{t)) + J2ieV(t)(D*(t') D*(t)). Since all sessions can receive at most R x (f t) service during the interval (t,f], and since by time t the active sessions should have received at least J2ieA(t) A(^) in order to satisfy their service curves, the above equation follows. Thus, E(t) represents the minimum service that should be allocated to the active sessions by time t using the real-time criteria in order to guarantee the service curves of all sessions in the future. The remaining (excess) service can be allocated by the link-sharing criteria. Further, it can be shown that the SCED algorithm is optimal in the sense that it can guarantee the service curves of all sessions by allocating exactly E(t) service by time t. With this a possible algorithm would be simply to allocate E(t) service by using SCED, and redistributing the excess service according to the link-sharing criteria. The major challenge in implementing such an algorithm is computing E{t) efficiently. Unfortunately, this is difficult for several reasons. First, as shown in Eq. (8), E(t) depends not only on the 16

20 deadline curves of the active sessions, but also on the deadline curves of the passive ones. Since according to Eq. (7), the deadline curve depends on the time when a session becomes active, this means that we need to keep track of all these possible changes, which in the worst case is proportional to the number of sessions. Second, even if all deadline curves are two-piece linear, the resulting curve E(i) can be n piece-wise linear, which is difficult to maintain and implement efficiently. Therefore, we choose to trade complexity for accuracy, by overestimating E(t). The first step in the approximation is to note that (see Eq. (7)): D*{t') - D*(t) < Si(t' - t), Vt'>t. (9) By using this inequality and the fact that Y,i Stf) < Äxt, for any t, Eq. (8) becomes: E(t) = E Dtf) + [max( (Dtf) - Dtf) + (D*(0 - A*(0) - Ä x (*' - t))] + i A(t) iea(t) iev(t) < J2 D tt) + [ max ( E (Mf) - Di(t)) + E Stf -t)-rx(t'- t))} + iea(t) * >f iea(t) iev{t) < Di(t) + [max( J2 (Di(f) - A(0) + E Stf -t)- E S tf - *))]+ iea(t) t >< iea(t) iev{t) iea(t)uv(t) = E A(<) + [max( Y, (A(0 - Di(t) - Stf - t)))} + iea(t) * >f iea(t) < E (A(<) + [max(a(^) - Dt(t) - Stf - t))} + ). iea(t) > Finally, we define the session's eligible curve to be Etf = Dtf) + [max(a(0 - Dtf - Stf - *))]+, (10) t'>t Vt>D- l { Ci ). The eligible curve Etf determines the maximum amount of service received by session i at time t by the real-time criteria. Since J2ieA(t) Etf > E{t), we have a sufficient condition. Etf is updated every time session i becomes active by the function update_ec according to the above formula. It is important to note that even though the formula, which applies to algorithms with service curves of arbitrary shape, looks complicated, the eligibility curves are actually quite simple to compute in the specific cases that we are interested in. For example, for sessions with concave service curves the eligibility curve is the same as the deadline curve. Intuitively this is easy to understand as the minimum service rate for 17

21 sessions with concave service curves will not increase in the future, thus there is no need to provide future service for it. Similarly, for sessions with two piece-wise linear convex service curve (first slope a, second slope ß, where ß > a), the eligibility curve is the linear curve with the slope of ß. 3.3 Virtual Time The concept of virtual time was first proposed in the context of Packet Fair Queueing (PFQ) and Hierarchical Packet Fair Queueing (H-PFQ) algorithms to achieve fairness, real-time, and hierarchical link-sharing. In H-FSC, we will use a generalized version of virtual time to achieve hierarchical link-sharing. Each Fair Queueing algorithm maintains a system virtual time v s (-). In addition it associates to each session i a virtual start time s,(-), and a virtual finish time /,-( ). Intuitively, v s (t) represents the normalized fair amount of service time that each session should have received by time t, s t (t) represents the normalized amount of service time that session i has received by time t, and /,-(i) represents the sum between Vi(t) and the normalized service that session i should receive for serving the packet at the head of its queue. Since Si(t) keeps track of the service received by session i by time t, S{(t) is also called the virtual time of session i, and alternatively denoted Vi(t). The goal of all PFQ algorithms is then to minimize the discrepancies among v l (t), s and v(t). In a H-PFQ system, each class keeps a virtual time function and the goal is to minimize the discrepancies among all sibling nodes in the hierarchy. Various PFQ algorithms differ in two aspects, the computation of the system virtual time function, and the packet selection policy. Examples of system virtual time functions are the start time of the packet being currently served [9], the finish time of the current packet being currently served [8], and minimum of the start times of all packets at head of currently backlogged queues [1]. Examples of packet selection policies are: Smallest Start time First (SSF) [9], Smallest Finish time First (SFF) [8], and Smallest Eligible Finish time First [1, 14]. The choice of different system virtual time functions and packet selection policies will affect the real-time and fairness properties of the resulted PFQ algorithm. Similar to H-PFQ, for each class i in the hierarchy, H-FSC maintains a virtual time function Vi(t) that represents the normalized amount of service time that class i has received 18

22 by time t. In H-FSC, virtual times are used by the link-sharing criteria to distribute service along the hierarchies according to the classes' service curves. The link-sharing criteria is used to select the next packet only when the real-time criteria is not used. Since the realtime guarantees for leaf classes are ensured by the real-time packet selection criteria, the effect on performance by having different system virtual time functions and packet selection algorithms in the link-sharing criteria is less critical. In H-FSC we use the SSF policy and the following system virtual time function: vf {v i>min + v it rn ax )/2 1 where v i>min and Vi, max are the minimum and maximum virtual start times among all class i's currently active child classes. By doing this, we ensure that the discrepancy between the virtual times of any two active sibling sessions is bounded (see Section 5). It is interesting to note that by taking v? to be either Uj )rm - or Vi^max results in a discrepancy proportional to the number of sessions. In H-FSC, Vi(t) is iteratively computed by using the previous virtual time function and the session's service curve. Virtual times are updated when a packet finishes service or a class becomes active. The function update_v is shown in Figure 7. Notice that update_v recursively updates the virtual time and the virtual time function by following child-parent link in the hierarchy till it reaches the root or a parent class that is active before time t. In the algorithm, we actually maintain a virtual curve Vi(-), the inverse function of u;(-), instead of >;( ). K(") is updated by using the updatejvc function every time a class becomes active: Vi(t) = min(v-(*),$(* - v s p(t)) + Wi ), V t > vr>,-), (n) where Wi is the total amount of service received by class i by time t, and v*,^ is the system virtual time for class i's parent class. Finally, it is worth noting that in the particular case when Si(-) is a straight line with slope r,-, from Eq. (11) we have Vi(t) = Tit. Then, the virtual time V{ is simply V~ l (wi) = Wi/ri, which is exactly the virtual time of session i in the PFQ algorithms. 4 Implementation Issues and Complexity The functions receive_packet and get_packet described in Figure 5 are called each time an event occurs in the real system, i.e., a packet arrives or departs. In our current imple- 19

23 mentation we maintain two requests per session, one characterized by the eligible time and deadline, called real-time request, and the other characterized by the virtual time, called link-sharing request. For maintaining the real-time requests we can use either an augmented binary tree data structure as the one described in [13], or a calendar queue [3] for keeping track of the eligible times in conjunction with a heap for maintaining the requests' deadlines. While the former method makes possible to perform insertion and deletion (of the eligible request with the minimum deadline) in O(logn), where n is the number of active sessions, the latter method is slightly faster in the average case. The link-sharing requests are stored in a heap based on their virtual times. Besides maintaining the request data structures, the algorithm has to compute the various curves, and update the eligible time, the deadline, and the virtual time. While it is expensive to update general service curves, in practice this complexity can be significantly reduced by considering only piece-wise linear curves. In our model, each session i is characterized by three parameters: the largest unit of work, denoted u^ax, for which the session requires delay guarantees, the guaranteed delay d ax, and the session's average rate r,-. As an example, if a session requires per packet delay guarantees, then u^ax represents the maximum size of a packet. Similarly, a video or an audio session can require per frame delay guarantees, by setting u^ax to the maximum size of the frame. The session's requirements are mapped to a two-piece linear service curve, which for computation efficiency is defined by the following three parameters: the slope of the first segment m, the slope of the second segment m], and the z-coordinate of the intersection between the two segments x t. The mapping {u ax,d ax, n) -> (m},xi,mf) for both concave and convex curves is illustrated in Figure 8. It can be easily verified from Eq. (7) that any deadline curve that is initialized to a service curve of one of the two types discussed above remains a two-piece linear service curve after each updating operation. It is worth noting that although all two-piece linear concave curve exhibits this nice property, this is not true for all convex curves. In fact, it can be shown that only the two-piece linear convex service curve which have their first segment parallel with the ^-coordinate have this propriety (which is our case). Since the first segment of a deadline curve does not necessarily intersect the origin, we need an extra parameter to uniquely characterize a deadline curve. We take this parameter to be the y-coordinate of 20