SCHEDULING REAL-TIME MESSAGES IN PACKET-SWITCHED NETWORKS IAN RAMSAY PHILP. B.S., University of North Carolina at Chapel Hill, 1988

Transcription

1 SCHEDULING REAL-TIME MESSAGES IN PACKET-SWITCHED NETWORKS BY IAN RAMSAY PHILP B.S., University of North Carolina at Chapel Hill, 1988 M.S., University of Florida, 1990 THESIS Submitted in partial fulllment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Urbana-Champaign, 1997 Urbana, Illinois

2 ccopyright by IAN RAMSAY PHILP 1997

3 SCHEDULING REAL-TIME MESSAGES IN PACKET-SWITCHED NETWORKS Ian Ramsay Philp, Ph.D. Department of Computer Science University of Illinois at Urbana-Champaign, 1997 Jane W.S. Liu, Advisor In a real-time network, it is not practical to have one centralized scheduler manage all the network resources, e.g., the transmission links and buer space. Instead, each node has its own scheduler which manages the various resources at that node. In an ideal case, the schedulers are completely independent, and the well-known scheduling and analysis techniques developed for single-node systems can be used, thus greatly simplifying the real-time network design. However the use of independent schedulers may lead to buer overruns or missed deadlines and hence, network failure. This thesis addresses the problems that arise in scheduling realtime messages in a packet-switched network that has multiple schedulers and has limited buer space. In our development of the schedulers and the mechanisms for synchronization between the schedulers, we address the following issues: the complexity of the scheduler, the complexity of the synchronization mechanism, the scheme for admission control, the achievable utilization of the network, the ability of the scheduler to meet diverse real-time requests, and the robustness of the scheduler under unpredictable conditions such as temporary overload. iii

4 Table of Contents Chapter 1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Motivation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Problems Addressed : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Organization of the Thesis : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 2 The Real-Time Packet Switching Problem : : : : : : : : : : : : : : : : : : : : Network Architecture : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Hard vs. Soft Real-Time Trac : : : : : : : : : : : : : : : : : : : : : : : : : : : : Message Models : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Admission Control : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Scheduler Implementation Issues : : : : : : : : : : : : : : : : : : : : : : : : : : : 15 3 Related Work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Trac Models : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Scheduling Algorithms for Switches with Buers : : : : : : : : : : : : : : : : : : Scheduling Sporadic and Aperiodic Messages : : : : : : : : : : : : : : : : : : : : Scheduling Soft-Real-Time VBR Messages : : : : : : : : : : : : : : : : : : : : : : Scheduling Messages Through a Buerless Switch : : : : : : : : : : : : : : : : : : 27 4 Scheduling Hard-Real-Time Periodic Messages : : : : : : : : : : : : : : : : : Weighted-Round-Robin Scheduling : : : : : : : : : : : : : : : : : : : : : : : : : : Budgeted-Weighted-Round-Robin Scheduling : : : : : : : : : : : : : : : : : : : : Performance : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 42 5 Scheduling Sporadic and Aperiodic Messages : : : : : : : : : : : : : : : : : : Background Scheduling : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Slot Swapping : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 49 6 Scheduling Soft-Real-Time Periodic Messages : : : : : : : : : : : : : : : : : : Scheduling, Trac Shaping, and Packet-Dropping Algorithms : : : : : : : : : : : Scheduling Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Trac Shaping : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Packet-Dropping Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : 55 iv

5 6.2 Simulation Results for a Single Hop : : : : : : : : : : : : : : : : : : : : : : : : : Scheduling Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Trac Shaping : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Packet-Dropping Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : Simulation Results for Multiple Hops : : : : : : : : : : : : : : : : : : : : : : : : : Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 67 7 Scheduling Periodic Messages Through Buerless Switches : : : : : : : : : Slot Assignment Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Simulation Results : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Extensions to the Single-Pass Algorithms : : : : : : : : : : : : : : : : : : : : : : Single Swapping : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Allowing Buers : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Relaxing the Deadline : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Distribution of Periods : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 84 8 Conclusion : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 86 Bibliography : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 90 Vita : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 95 v

6 List of Tables 3.1 Real-time scheduling algorithms. : : : : : : : : : : : : : : : : : : : : : : : : : : : Packet delay and buer requirements. : : : : : : : : : : : : : : : : : : : : : : : : U act with C = 100. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : U act with C = 1; 000. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : U act with C = 10; 000. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Completion time of sporadic message with wf 4 ree = 1. : : : : : : : : : : : : : : : : Response time distribution for FCFS, TT, WRR, and BWRR. : : : : : : : : : : Probability of generating a nonfeasible schedule from the aggregate density function. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Buer space required using packet-spreading technique. : : : : : : : : : : : : : : (1; k) failure rates for dierent values of k. : : : : : : : : : : : : : : : : : : : : : : 64 vi

7 List of Figures 1.1 Packet interarrival time at hop 2 is less than 10 ms. : : : : : : : : : : : : : : : : General network architecture. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : An example switch schedule. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Rate-Controlled Static-Priority Queuing. : : : : : : : : : : : : : : : : : : : : : : : Example showing selection of SDR. : : : : : : : : : : : : : : : : : : : : : : : : : : The completion time of the k-th group at the l-th hop depends on the completion time of the k-th group at the (l-1)-th hop and the completion time of the (k-1)-th group at the l-th hop. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Pseudocode for the BWRR scheduler. : : : : : : : : : : : : : : : : : : : : : : : : Replenishing the budget in the BWRR algorithm. : : : : : : : : : : : : : : : : : Worst-case phase and ideal phase for buer usage. : : : : : : : : : : : : : : : : : Spreading packet arrivals to reduce buer requirements. : : : : : : : : : : : : : : Probability density function for c i (a) and (b). Aggregate density function for 20 streams (c). : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Comparison of WRR (a) and FCFS (b) algorithms. : : : : : : : : : : : : : : : : : Achievable utilization with P(failure) 10?3. : : : : : : : : : : : : : : : : : : : : Multihop network topology. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Response times for spread and continuous packet arrivals. : : : : : : : : : : : : : Response times for dierent spread factors. : : : : : : : : : : : : : : : : : : : : : Example MP-TSA schedule. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Success rates for dierent schedule lengths, N = 4, U = 0:95. : : : : : : : : : : : Success rates for dierent switch dimensions, L = 30, U = 0:95. : : : : : : : : : : Success rates for average link utilization, N = 4, L = 30. : : : : : : : : : : : : : : Example of single swapping. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Schedule where single swapping cannot remove conict. : : : : : : : : : : : : : : Success rates for MLF-SDR plus single swapping, N = 4, L = 30. : : : : : : : : : Success rates for switches with buers, U = 0:85, L = 30. : : : : : : : : : : : : : Success rates with relaxed deadline, U = 0:85, L = 30. : : : : : : : : : : : : : : : Success rates with relaxed deadline, N = 8, U = 0:85. : : : : : : : : : : : : : : : Success rates for dierent distributions of the period, N = 8, U = 0:85, L = vii

8 Chapter 1 Introduction Proper functioning of a real-time network requires not only that the real-time messages be delivered reliably to their destinations but also that the messages be delivered by specied times called the deadlines. In a real-time network, it is not practical to have one centralized scheduler manage all the network resources, e.g., the transmission links and buer space. Instead, each node in the network has its own scheduler that manages the resources at that node. The schedulers at each node make decisions about which packets to schedule on the transmission links and how to manage the buer space so that each real-time message is delivered by its deadline to its destination. 1.1 Motivation Ideally, from the points of view of scalability and simplicity, the schedulers at each node operate independently. However, sometimes this may not be possible. Non-real-time issues such as congestion control, which typically requires interaction among schedulers, must also be handled by a real-time network. In addition, certain timing issues requiring clock synchronization or other scheduler interaction may arise. A common approach for scheduling real-time messages is to assign intermediate deadlines to the messages in such a way that if each message meets its intermediate deadlines at the nodes along its way to the destination, it also meets its end-to-end deadline. Ideally, in this approach, once a message's intermediate deadlines have been assigned, the techniques that have been developed for single node systems can be used to schedule message transmissions and perform 1

9 the necessary schedulability analysis. This approach can greatly simplify the design of a realtime network. Unfortunately, the use of independent schedulers may lead to buer overruns or missed deadlines and hence, network failure. This fact is illustrated in the following two examples. The rst example shows how a network failure may occur in a multihop packet-switched network due to lack of buer space. Suppose that the network has two hops and each hop guarantees a relative deadline of 10 ms for the transmission of 20 packets of a particular message stream. If the rst hop is lightly loaded, it may transmit the packets immediately upon arrival. But if the second hop is heavily loaded, the packets may not be transmitted until their deadlines. Even if the average rate of packet transmission at the rst hop is 2,000 packets per second, the number of packets transmitted in a shorter interval may be higher. For example, the rst hop may transmit 2,000 packets in the rst 10 ms and then transmit no packets for 990 ms. If this happens and the second hop only transmits 20 packets every 10 ms, a queue of packets will build up in the second hop, and it may eventually run out of buer space resulting in lost packets and missed deadlines. The second example involves static-priority (SP) scheduling, a well-known scheduling technique in real-time systems, and shows how a failure may occur even if the short-term rate at which packets are transmitted is controlled. As in the rst example, we assume a multihop packet-switched network. In SP scheduling, each message stream declares its period and the maximum number of packets produced in each period. All packets belonging to that message stream are assigned the same xed priority, and at each node the scheduler always transmits the ready packet with the highest priority. The admission-control phase guarantees that if all message streams adhere to their contracted parameters, all messages will be transmitted by their deadlines. However, if a higher priority stream produces more packets than its declared maximum or if the period is shorter than what it declared, lower priority packets may be aected and miss their deadlines. Figure 1.1 shows an example of what can go wrong if independent schedulers are used in a two-hop network. The gure shows the times of packet transmissions at Hop 1. At Hop 1, 20 packets arrive at time 0, and 20 more packets arrive at time 10, i.e., the interarrival time of the two groups of 20 packets is 10 ms. The admission-control process and the SP scheduler guarantee that both groups of packets will be transmitted within 10 ms of their arrival times. However, suppose the rst group of packets is not transmitted until time 10 2

10 Hop time Figure 1.1: Packet interarrival time at hop 2 is less than 10 ms. and that the second group is transmitted by time 15. The interarrival time of the two groups of packets at the second hop is only 5 ms; therefore, lower priority packets at the second hop may miss their deadlines. As we showed in the preceding examples, it is not always possible to have completely independent schedulers. On the other hand, it is desirable to have the schedulers be as independent as possible. Interaction among schedulers such as clock synchronization or explicit message passing schemes that keep track of distributed state complicates the system design and takes up network bandwidth which could otherwise be used for transmitting application messages. 1.2 Problems Addressed This thesis addresses the problem of scheduling real-time messages in a packet-switched network that has multiple schedulers and limited buer space. In our development of the schedulers and the mechanisms for synchronization between the schedulers, we are concerned with the following issues: the complexity of the scheduler, the complexity of the synchronization mechanism, the scheme for admission control, the achievable utilization of the network, the ability of the scheduler to handle diverse real-time requirements, and the robustness of the scheduler under transient overload. The specic problems we address vary along three dimensions: the switch architecture, the types of deadline of the messages, and the message model. We consider two types of switch architecture: one in which the switches have nonzero but nite buer space and the other in which the switches have no buers. We consider both hard and soft deadlines. When a message has a hard deadline, the network must guarantee that all packets in the message are delivered to the destination at or before the deadline. In order to meet this hard deadline, an admission step must be performed before the message is allowed to enter the network, with worst-case 3

11 assumptions made on the number of packets generated. When a message has a soft deadline, it is permissible to occasionally miss a deadline or even to drop a small number of packets. we consider three types of message models: periodic messages in which packets arrive at regular intervals, sporadic messages which arrive at irregular intervals, and aperiodic messages which arrive at irregular intervals and do not have deadlines. Scheduling Hard-Real-Time Periodic Messages The rst problem we addressed is that of scheduling hard-real-time periodic messages in a multihop packet-switched network with nite buer space at each hop. For this problem, we developed the budgeted-weighted-round-robin (BWRR) algorithm. The BWRR algorithm is a modication of the weighted-round-robin (WRR) [29, 10, 23] algorithm which cycles through the message streams transmitting a limited number of packets from each stream, thus guaranteeing each stream a minimum fraction of the transmission bandwidth. The WRR algorithm allows the actual transmission rate to exceed the minimum rate, which, as we shall see, increases jitter and the required buer space. Like the WRR algorithm, the BWRR algorithm ensures a minimum transmission rate, but it also ensures that the maximum transmission rate of each message stream does not exceed its allocated rate. In this way, the BWRR algorithm signicantly reduces the jitter in message delivery and the buer space requirement per message stream. The BWRR algorithm has the advantages of the WRR algorithm: it has a simple hardware implementation; the admission test for determining whether a new message stream is schedulable is simple; it does not require a global clock or any explicit ow control messages; it can achieve high utilization of the network under most practical scenarios; and it reacts predictably under transient overloads. In addition, with the BWRR algorithm, the necessary buer space is predictable and can be xed at system conguration time. Scheduling Sporadic and Aperiodic Messages The second problem we addressed is that of scheduling sporadic and aperiodic messages along with the hard-real-time periodic messages in networks with nite-buer switches. Specically, we show how to integrate the scheduling of sporadic and aperiodic messages in a network where the periodic messages are scheduled according to the BWRR algorithm. For this problem, we developed two solutions. The rst, called background scheduling, simply uses the bandwidth 4

12 not used by the periodic messages for the transmission of sporadic and aperiodic messages, We show that by choosing to allocate at each hop the minimum available bandwidth among all hops along the path of a sporadic or aperiodic message, the minimum delay bound and buer space requirement are attained. By allocating more than this minimum bandwidth at the hops where more bandwidth is available, more buer space is required, but the delay bound is not improved. The second solution, called slot swapping, is able to achieve lower delay than the background scheduling solution, but its admission test and scheduler implementation are much more complex, making a high-speed implementation more dicult. Scheduling Soft-Real-Time Periodic Messages The third problem we addressed is that of scheduling soft-real-time periodic messages in networks with nite-buer switches. One way to schedule soft-real-time trac is to use one of the algorithms developed for hard-real-time trac and perform admission control based on the peak trac rate. However, soft-real-time trac frequently has a variable-bit-rate (VBR), with a peak rate many times higher than the average rate. Thus, in order to achieve a higher network utilization it is desirable to admit soft-real-time trac at a rate closer to its average rate and to handle overload conditions gracefully. We show that the hard-real-time algorithms perform quite poorly for soft-real-time trac admitted at less than the peak rate and that the rst-come-rst-serve (FCFS) algorithm performs best among all the evaluated algorithms. To gracefully handle buer overow, we developed a (1; k) fair packet dropping algorithm The (1; k) dropping algorithm tries to drop no more than one packet in any sequence of k packets from any stream. We show that although the (1; k) dropping algorithm does improve network performance, the same eect can be achieved by slightly lowering the network utilization and using the simple last-in-rst-discard (LIFD) dropping algorithm. Finally, we show that buer space requirements can be minimized by spreading the packet arrivals to the rst switch in the network over the period of the stream. Because we use a FCFS scheduler, reducing buer space at a hop also means reducing delay at that hop. We use this fact to show that by increasing delay outside the network, i.e., by spreading the packet arrivals to the rst switch, the delay inside the network decreases enough to result in lower total end-to-end delay. 5

13 Scheduling Periodic Messages Through Buerless Switches The nal problem we addressed is scheduling periodic real-time messages through buerless switches. For this problem, we assume that when a packet is transmitted through a switch, the switch maintains a physical connection between the appropriate input and output links for the duration of the packet transmission. When two or more packets arrive on dierent input links destined for the same output link a conict occurs and at most one of the packets is successfully transmitted. In order to prevent conicts, tight synchronization is needed between not only the switch and the sending processors but also between the sending processors themselves. In this problem, which we call the the multiple-period time slot assignment (MP-TSA) problem, messages are assigned time slots in which they are transmitted. The problem is to nd a set of time slot assignments that yields a conict-free schedule in which all messages are transmitted by their deadlines. Such a schedule is called a conict-free feasible schedule. In order to determine whether a conict-free feasible schedule of the switch exists for a set of periodic messages with arbitrary periods, we implemented a search program which exhaustively searches for all possible feasible schedules. We randomly generated over 10 5 message sets that had input and output link utilizations less than or equal to 1.0 and found a conict-free schedule for every set. Because the number of possible schedules grows enormously as the switch dimension and schedule length grow larger, performing an exhaustive search in order to nd a conict-free schedule is impractical. This motivated us to look for more ecient algorithms to solve the MP-TSA problem. We have designed four basic heuristic algorithms which make a single pass through the length of the schedule for this purpose. They are based on the earliestdeadline-rst (EDF), minimum-laxity-rst (MLF), and system of distinct representatives (SDR) algorithms. The success rate of an algorithm is the percentage of the time the algorithm nds a conict-free schedule when one exists. We evaluated the success rates of our heuristic algorithms through simulation and found that algorithms based on the MLF algorithm outperform those based on the EDF algorithm and that the algorithms based on the SDR algorithm outperform simpler extensions of the EDF and MLF algorithms. Even the best of our single-pass heuristic algorithms performs quite poorly as the size of the scheduling problem increases. We were thus motivated to nd better algorithms. We developed a more complex algorithm, called MLF with single swapping (MLF-SS), which makes multiple 6

14 passes over the length of the schedule. Although this algorithm performs better than any of the single-pass algorithms, it too suers from poor performance as the size of the problem increases. We then considered two extensions to the MP-TSA problem. In the rst we allowed a small number of buers in the switch. In the second, we extended the message deadlines by one time slot. In both cases, the success rate of our MLF-SS algorithm is signicantly better than when no buers are used and when the deadline is not extended. 1.3 Organization of the Thesis The rest of the thesis is organized as follows. Chapter 2 presents the real-time message switching problem. It presents the network architecture used in the thesis as well as the concept of hard and soft-real-time messages. It introduces the periodic, sporadic and aperiodic message models and discusses many admission control and scheduler implementation issues we are concerned with in designing a network for transmitting real-time messages. Chapter 3 discusses related work. It presents four trac models that have been used to describe real-time trac, including the periodic model used in the thesis. It also presents several algorithms that have been designed for scheduling real-time messages in a multihop packet-switched network. We classify these algorithms into two groups: priority-based and rate-based and discuss the advantages and disadvantages of each. This chapter also presents work related to the scheduling of sporadic and aperiodic messages and related work that is specic to the problem of scheduling soft-real-time trac and dealing with buer overow. It also discusses work related to the buerless switch scheduling problem. Chapter 4 addresses the problem of scheduling hard-real-time periodic messages in a multihop packet-switched network with limited buer space. It presents the BWRR scheduling algorithm and discusses the complexity of the scheduler and the admission-control procedure. This chapter also presents bounds for the end-to-end delay and the per message stream buer space requirement and discusses the achievable utilization of the network. Chapter 5 addresses the problem of scheduling sporadic and aperiodic messages along with the hard-real-time periodic messages discussed in Chapter 4. It presents the background scheduling and slot swapping solutions to this problem. 7

15 Chapter 6 addresses the problem of scheduling soft-real-time VBR trac. This chapter examines the applicability of the hard-real-time scheduling algorithms to the problem and discusses fair handling of overload conditions. It also shows how lower deadlines and jitter can be achieved by careful trac shaping. Chapter 7 addresses the problem of switching periodic messages through buerless switches. It discusses the achievable utilization of the network and the complexity of nding a schedule that transmits all messages by their deadlines. Chapter 8 concludes the thesis with a summary of our results. 8

16 Chapter 2 The Real-Time Packet Switching Problem In this chapter, we rst present the network architecture assumed in the thesis. Next, we present the concept of hard and soft-real-time messages. We then present the periodic, sporadic and aperiodic message models. We are also concerned with many admission control and scheduler implementation issues in designing a real-time network. This chapter also discusses these issues. 2.1 Network Architecture In this thesis, we are concerned with packet-switched networks. Messages submitted to such a network for transmission are rst broken up into packets. A message is completely transmitted when its last packet is received at the destination. Throughout the thesis, we assume that the packets have xed size. The use of xed size packets greatly simplies the design of the switch, resulting in less expensive and faster equipment. The components of these networks are switches. We assume that the switches are nonblocking, i.e., that a switch can route any incoming packet to the appropriate output link without conict. Packets from dierent input links destined for dierent output links do not interere with each other, and thus, the only possible queuing delay occurs at the output links. Figure 2.1b shows a general network whose components are switches (Figure 2.1a). A message is sent from switch S1, which is the source, to switch S5, which is the destination. 9

17 S4 S2 nxn src S1 S5 dst S3 S6 (a) (b) Figure 2.1: General network architecture. With this network model, the delay a message experiences when transmitted from one switch to another switch consists of the following three parts. 1. The queuing delay is the amount of time that a packet is blocked at the output link of a switch. This delay results from the fact that multiple packets may arrive at a single output link at the same time and must therefore be buered. 2. The propagation delay is the time that a single bit takes to traverse a link between two switches. For any pair of switches, the propagation delay is constant and depends on the length of the link between the switches. 3. The transmission delay is the amount of time required to transmit a packet. Since packets are of constant length, the transmission delay is constant and depends on the packet length and the transmission rate. We will ignore the propagation delay in our analysis, keeping in mind that the true delay of a message includes this delay. Hereafter, we call each switch and the output link traversed by a message a hop. A message originates at some source switch S src. The message is broken up into xed size packets and is sent across a path of length L, measured in the number of hops, to its destination switch S dst where the message is nally reassembled. From the perspective of the message, each hop is a one way point-to-point connection between two switches shared by many other messages. The 10

18 C B D D A A B D C A B B D A A B C 4X4 C A C B C D D Figure 2.2: An example switch schedule. end-to-end delay of a message is the sum of the queuing delays and transmission delays along all L hops. As mentioned in Chapter 1, we are interested in two types of switches: switches with nonzero but nite buer space and switches with no buers. The two corresponding types of networks demand dierent solutions. In the former, when two or more packets arrive at the same time destined for the same output link, the packets are buered at the output link. The packets buered at the output link are then scheduled for transmission. In the latter, when a packet is to be sent through a switch, the switch must maintain a physical connection between the appropriate input and output links for the duration of the packet transmission. Exactly one packet can be routed through the switch to a particular output link at a time. If the network attempts to transmit two or more packets through a switch destined for the same output link at the same time, a conict occurs and at most one of the packets is successfully transmitted. In order to prevent conicts, the network must maintain some sort of synchronization between the switches in the network. Here, we assume that the network avoids conicts by maintaining a global clock and that the times for message transmission are predetermined so that no two packets destined for the same output link are sent at the same time. The problem, then, is to determine a schedule for the message transmissions. Figure 2.2 shows an example schedule for a 4 4 switch. Each slot on the input links is labeled with the letter of the output link on which the packet is to be transmitted. As can be seen in the gure, during each time slot one packet arrives on each of the four input links and one packet is transmitted on each of the four output links. There are no conicts in the schedule; therefore the switch is able to successfully transmit all the packets. 11

19 2.2 Hard vs. Soft Real-Time Trac We divide real-time trac into two types: hard-real-time and soft-real-time. Hard-real-time trac requires guarantees on the delivery and delivery time of all packets. In order to achieve a hard guarantee, a message stream must specify its worst-case packet generation rate to the network before the stream can be admitted. During the admission process, the network reserves bandwidth on the links and buer space in the switches required to meet the delivery rate and time guarantees. Once the stream is admitted, as long as the stream adheres to its declared parameters, all deadlines are guaranteed to be met. Soft-real-time trac, on the other hand, does not have hard requirements. It is preferable that the network meet the delay, jitter, and loss requirements, but it is usually permissible to slightly exceed these values. Even with soft-real-time trac, it is desirable to admit a message stream M i to the network only if its delay and loss requirements can be met and its admission does not cause messages from other streams already admitted by the network to exceed their delay and loss requirements. Admission of a message stream with soft-real-time requirements is usually done based on a rate that is close to its average rate, and the network must handle overload conditions gracefully. We choose as our measure for graceful handling of overload conditions the (1; k) failure rate. A failure to deliver a packet occurs when the packet arrives at its destination after its deadline or is dropped in the network because of the lack of buer space. A (1; k) failure is said to occur for a message stream when more than 1 out of any k consecutive packets fail to be delivered to the destination. Network performance is measured by the number of (1; k) failures. The (1; k) failure rate is the number of (1; k) failures divided by the total number of packets transmitted. For example, if k = 10 and the 11th, 30th, 40th, and 41st packets are dropped, the number of failures is 2. Since the rst 10 packets are delivered, the dropping of the 11th packet is not a failure. Also, since packets 20 through 29 are delivered, the dropping of the 30th packet is not a failure. However, when the 40th packet is dropped, only 9 consecutive packets have been received, so a failure occurs. The dropping of the 41st packet also counts as one failure. In our example, if the total number of packets transmitted is 100, the (1; k) failure rate is

20 2.3 Message Models In this section we introduce the periodic, sporadic, and aperiodic message models which we use in the thesis. The Periodic Message Model We assume here that periodic processes generate periodic real-time messages. A periodic process is characterized by three parameters: its period, its worst-case execution time, and its deadline. Since real-time periodic messages are generated by periodic processes, they too t the periodic model. Each periodic message stream M i consists of a periodic sequence of message instances. We call the j-th instance of message stream M i M i;j. When it is not necessary to distinguish between individual instances, we refer to both the message stream and its instances as M i. We characterize a periodic message stream M i by the tuple (c i ; p i ; d i ; j i ; k i ). Here, c i is a probability distribution from which the number of packets c i;j (i.e., the length) of a message instance M i;j is drawn. When c i is deterministic, i.e., each c i;j is the same, or when we are only interested in the maximum value of any c i;j, we will refer to the number of packets generated per period simply as c i. The second parameter, p i, is the minimum length of the interval between the arrivals of any two consecutive instances of M i. The third parameter, d i, is the maximum allowable end-to-end delay of every instance. The fourth parameter, j i, is the maximum allowable jitter. The fth parameter, k i, describes the allowable loss rate of the stream and will be described later. We call p i the period of M i and d i the relative end-to-end deadline, or simply deadline, of M i. In addition to a deadline constraint, each periodic message may also have a constraint on the allowable jitter. Many applications that generate periodic messages at periodically xed intervals of time expect the network to deliver successive message instances with interdelivery time close to the period of the messages. For example, video frames are generated with a period of 33 ms and must be displayed at that rate. Jitter is dened as the maximum dierence in message interarrival times from the stream period. We will subsequently refer to M i in various simpler forms by leaving out some of the above parameters in M i 's specication. For example, we are often interested in nding the achievable value of the end-to-end delay and the maximum jitter of each message stream when given the 13

21 lengths and periods of all the message streams in the network. Therefore, we usually leave out the deadline, jitter, and loss rate in the tuple and simply refer to M i by (c i ; p i ). Unless otherwise stated, we take our basic time unit to be the time required to transmit one packet and call this time a slot. Each packet is transmitted in one slot. The utilization of a periodic P message M i is U i = c i =p i. The utilization of an input or output link l of a switch is U(l) = i U i where the sum is over all messages M i that are sent across link l. We let U be the average utilization of all the input and output links of a particular switch. The Sporadic Message Model A sporadic real-time message is a hard-real-time message instance that may arrive at the source at any time instant. The worst-case transmission time c i and deadline d i of each sporadic message becomes known upon the arrival of the message. We therefore specify the sporadic message by the tuple (c i ; d i ). Upon arrival of a sporadic message, the network executes an acceptance test which determines whether or not the network can deliver the message to its destination by its deadline. If the message is accepted, the network must deliver it by its deadline. If the network cannot guarantee delivery of the message by the deadline, the message is rejected immediately. The goal of the network is to accept as many sporadic messages as possible. The Aperiodic Message Model An aperiodic message may arrive at the source at any time and does not have a hard deadline. An aperiodic message is specied by the single parameter (c i ). The network always accepts aperiodic messages with the goal of minimizing the average response time of the aperiodic messages. 2.4 Admission Control In order to receive real-time service from the network, a message stream must rst go through an admission control process in which the network determines whether it has the needed resources to meet the requirements of the message. For this purpose, the application declares to the network its trac characteristics which we mean specically here the parameters (i.e., period 14

22 and maximum length) and requirements (i.e., deadline, jitter, and loss rate) of each message to be transmitted by the application. Based on the parameters provided by the application, the network admission control process proceeds and decides whether the network has enough available resources to admit each message of the application into the network. Specically, a message stream M i is admitted to the network if and only if it can be scheduled on each hop so that its end-to-end delay is no more than its deadline and its admission does not cause messages from other streams to miss their deadlines. In addition, the admission control process must consider the available buer space within the network which is a scarce resource and must be managed carefully. For hard-real-time messages, the network must ensure that no packets are dropped due to lack of buer space. 2.5 Scheduler Implementation Issues Once a message is admitted to the network, the schedulers must manage the network resources so the guarantees made during the admission control process are met. The following issues need to be carefully addressed during system design. Scheduler Complexity At each hop, the scheduler decides which packet to transmit next on the output link. In a highspeed network, the complexity of the scheduler is critical because it is likely to be the bottleneck which determines the achievable transmission rate of the switch. While a simple scheduling algorithm such as rst-come-rst-serve (FCFS) is likely to achieve the highest transmission rate, it may not be able to provide hard-real-time guarantees. On the other hand, some of the complex scheduling algorithms that have been developed for real-time processor scheduling are likely to incur so much overhead that they unduly limit the achievable transmission rate. A compromise must be found between these two seemingly conicting goals. Buer Space For soft-real-time messages, it may not be practical from a utilization standpoint to guarantee that no soft-real-time packets will be dropped due to lack of buer space. Instead, it is 15

23 sometimes better to allow buer overows, with the switch being responsible for providing fair buer space usage. Complexity of Admission Control Depending on the scheduling algorithm being used, the time taken to determine whether or not a periodic stream will be admitted can vary widely. In a network with a large number of periodic messages entering and leaving the network, it is important that the admission control processing be simple enough so the network can keep up with the incoming requests. Synchronization Between Schedulers In Chapter 1 we showed that it is not always possible to implement a real-time network with completely independent schedulers. On the other hand, it is desirable to have the schedulers be as independent as possible. Including clock synchronization or explicit message passing schemes to keep track of distributed state complicates the system design and uses network bandwidth which could otherwise be used for transmitting application messages. Network Utilization The achievable utilization of the network is the utilization beyond which no new messages can be admitted. In order to admit as many message streams as possible, this utilization threshold should be as high as possible. In other words, the scheduler should be able to meet the real-time constraints of the admitted messages at as high a utilization as possible. Flexibility The real-time network will likely receive diverse requests from the real-time applications. The applications will request for admission messages with dierent periods, lengths, and deadlines. The scheduler should be able to meet these diverse requirements. Robustness During the admission control phase, a decision is made as to whether the network can meet the real-time requirements of a periodic message under the assumption that the message adheres 16

24 to its declared parameters. If the message does not adhere to its parameters, the message has broken its contract with the network, and the network's guarantee no longer holds. However, if one periodic message breaks its contract, other periodic messages which are still adhering to theirs should still receive their guarantees. In other words, one misbehaving periodic message should not adversely aect the performance of others. 17

25 Chapter 3 Related Work In this chapter, we rst discuss four trac models that have been used to characterize realtime trac. We then discuss work closely related to two scheduling problems addressed in the thesis: scheduling hard and soft real-time periodic messages through a switch with nite buer space. Specically, there are several known scheduling disciplines for nite buer switches. We classify these algorithms into two groups: priority-based and rate-based. The priority-based approaches have fairly complex implementations. The rate-based approaches are simpler and more readily lend themselves to high-speed implementations. The BWRR algorithm which we describe in Chapter 4 is a rate-based approach which achieves a smaller end-to-end delay and requires less buer space than other rate-based algorithms. We discuss work related to the scheduling of sporadic and aperiodic messages as well as related work that is specic to the problem of scheduling soft-real-time trac. Finally, we discuss work related to the buerless switch scheduling problem. 3.1 Trac Models The commonly used models for characterizing real-time trac are: the periodic model, the (X min ; X ave ; I) model, the (; ) model, and the leaky bucket model. The Periodic Model The periodic message model was originally described in the context of processor scheduling [27]. As stated in Chapter 2, a periodic message M i is characterized by the tuple (c i ; p i ; d i ; j i ; k i ) or 18

26 more simply (c i ; p i ). We shall use the periodic model for describing real-time trac throughout the thesis. The (X min ; X ave ; I ) Model In the (X min ; X ave ; I) trac model [14, 12], three parameters are used to characterize the trac: X min is the minimum packet inter-arrival time, X ave is the average packet inter-arrival time, and I is the time interval over which X ave is computed. The two parameters X ave and I are used to characterize bursty trac and are not used in the description of hard-real-time trac which is solely described by the parameter X min. The (X min ; X ave ; I) model cannot accurately describe periodic trac because it cannot describe the bulk arrival of c i packets at the beginning of each period. Instead, the model assumes the packets arrive one at a time minimum interarrival time X min. We say that the arrivals are \spread out" in this case. The (; ) Model The (; ) model [9] describes trac in terms of a rate parameter and a burst parameter such that the total number of packets from a stream in any time interval t is no more than + t. Again, the (; ) model cannot accurately describe the arrival of c i packets at the beginning of a period. Leaky Bucket The Leaky Bucket model [41, 34] is characterized by two parameters: a rate r and a depth b. The bucket lls up with tokens at a rate r, with b being the maximum number of tokens in the bucket. The presence of a token in the bucket indicates that a packet may be sent to the network. 3.2 Scheduling Algorithms for Switches with Buers Several algorithms have been proposed for scheduling real-time messages through a switch with nite buer space. We classify them into two groups: priority-based and rate-based. The best known algorithms are listed in Table 3.1. The primary advantage of the priority-based algorithms is that they provide a great deal of exibility in providing dierent delay bounds for 19

27 Priority-Based Fair Queueing (FQ) [10] Virtual Clock (VC) [46] Delay Earliest-Due-Date (D-EDD) [14] Jitter Earliest-Due-Date (J-EDD) [42] Minimum-Laxity-First (MLF) [28] Rate-Controlled Static-Priority queuing (RCSP) [44] Rate-Based Framed-Round-Robin (FRR) [35, 22] Stop and Go (S&G) [15] Timed-Token (TT) [18] Table 3.1: Real-time scheduling algorithms. diverse messages. Priority-based algorithms require the scheduler to maintain a sorted priority queue, which for most priority assignments is expensive and may not be feasible in a high-speed switch. The RCSP algorithm also requires at each output port an expensive regulator which controls the number of packets admitted to the priority queue. The rate-based algorithms have a simpler implementation and are likely more suitable for high-speed switches. The BWRR algorithm which we describe in Chapter 4 is a rate-based algorithm. The dierence between our BWRR algorithm and the other rate-based algorithms is that, for the same cost, the BWRR algorithm achieves a smaller end-to-end delay and requires less buer space. The FRR and S&G algorithms can achieve the same end-to-end delay and buer space requirement as the BWRR algorithm, but they require global clock synchronization to do so. Fair Queueing The goal of the Fair Queueing (FQ) algorithm [10] is to emulate bit-by-bit round robin service among the message streams. Since bit-by-bit round robin is impractical, each packet is assigned a nish time that is the time at which the packet would have been transmitted if the outgoing link had actually been scheduled on the bit-by-bit round robin basis. The scheduler then transmits packets in order of their nish times. If there are N streams being transmitted, each stream gets 1=N of the total bandwidth when averaged over time. Alternatively, streams can 20

28 be given dierent fractions of the total bandwidth by assigning them weights. The weight of a stream is proportional to the number of bits transmitted during each round of round robin service. The biggest drawback of the FQ algorithm is that the scheduler must keep a sorted priority queue of messages, thus making a high-speed implementation impractical. In Chapter 4, we describe the weighted-round-robin (WRR) scheduling algorithm [23] which is similar to the FQ algorithm but has a much simpler implementation. Virtual Clock Just as the FQ algorithm tries to emulate bit-by-bit round robin, the Virtual Clock (VC) algorithm [46] tries to emulate the Time Division Multiplexing (TDM) service discipline. Each packet is allocated a virtual transmission time which is the time the packet would be transmitted if the scheduler were actually doing TDM. The packets are then scheduled according to their virtual transmission times. For example, if a stream requires a bandwidth of 100 packets per second, the packets from that stream are given virtual transmission times 10 ms apart. The VC algorithm has the same drawback as the FQ algorithm: it requires that the scheduler maintain a sorted priority queue. Delay Earliest-Due-Date The Delay Earliest-Due-Date (D-EDD) algorithm [14] is based on the Earliest-Deadline-First (EDF) algorithm [27] in which packets are assigned deadlines and the scheduler transmits packets in order of their deadlines. Liu and Layland proved that on a single link the EDF algorithm achieves 100% link utilization when used to transmit periodic messages whose relative deadlines are equal to their respective periods. The EDF algorithm is optimal for one link, i.e., it nds a schedule whenever such a schedule exists. According to the D-EDD algorithm, each message stream declares its maximum packet arrival rate and the scheduler guarantees a relative deadline on each hop in the path of the stream. The end-to-end relative deadline is simply the sum of the local relative deadlines. When a packet arrives at a hop it is assigned a local (absolute) deadline and is inserted into a sorted priority queue according to the local deadline. The scheduler always transmits the packet with the closest deadline. The local deadline at a hop is assigned based upon the 21