Models and Algorithms for Hierarchical Resource Management in Integrated Services Networks. Other Contributors

Transcription

1 Models and Algorithms for Hierarchical Resource Management in Integrated Services Networks Hui Zhang School of Computer Science University 1998 Hui Zhang 1 Other Contributors Jon Bennett Yanghua Chu Eugene Ng Donpaul Stephens Ion Stoica Dennis Chang Keng Lim Jianliang He Bo Fan 1998 Hui Zhang 2 1

2 Outline Introduction Hierarchical Generalized Processor Sharing (H-GPS) and Hierarchical Packet Fair Queueing (H-PFQ) Hierarchical Fair Service Curve (H-FSC) Simulation & experimental results Conclusion 1998 Hui Zhang 3 Introduction Packet network À statistical sharing of network resources À packets delayed or dropped during congestion Traditional congestion control technique À end system dynamically measures available bandwidth À adapts to network congestion À works only if all sources comply À cannot provide differential or guaranteed QoS QoS approaches today À end-to-end delay or bandwidth guarantee on a per flow basis À different QoS classes: guaranteed, statistical, predicted, controlled load, differential 1998 Hui Zhang 4 2

3 QoS at Different Traffic Granularities Service Providers À MCI vs. UUNET Organizations À CMU vs. University of Pittsburgh À SCS vs. ECE Applications À distributed simulation vs video conferencing À FTP vs. WEB vs. Telnet 1998 Hui Zhang 5 Hierarchical Resource Sharing Link 155 Mbps 100 Mbps 55 Mbps Provider 1 Provider 2 60 Mbps 40 Mbps CMU U.Pitt 30 Mbps 10 Mbps SCS ECE Campus Resource contention/sharing at different levels Resource management policies should be set at different levels, by different entities À resource owner À service providers À organizations À applications seminar video seminar audio Control Distributed Simulation Audio Video WEB 1998 Hui Zhang 6 3

4 Hierarchical Resource Sharing Link 155 Mbps 60 Mbps CMU 30 Mbps SCS 100 Mbps Provider 1 10 Mbps ECE 40 Mbps U.Pitt Guaranteed Service Campus 55 Mbps Provider 2 Controlled Load Service FTP Best-effort Service Telnet seminar video seminar audio Control Distributed Simulation Audio Video WEB 1998 Hui Zhang 7 Packet Scheduling Algorithm Operate at the multiplexing point Control the interactions among À traffic streams in the same class À different performance-oriented traffic classes À different administrative traffic classes 1998 Hui Zhang 8 4

5 Traffic Control Mechanisms and Timescales admission control routing renegotiation rerouting source adaption scheduling buffer management policing connection RTT packet service Scheduling À operates at the smallest timescale À at the multiplexing point 1998 Hui Zhang 9 Hierarchical Resource Scheduling 60 Mbps CMU 30 Mbps SCS seminar video Provider 1 Link 10 Mbps ECE Control U.Pitt 155 Mbps 100 Mbps 55 Mbps seminar audio 40 Mbps Distributed Simulation Provider 2 Audio Campus Video Requirements: À Real-time QoS for leaf classes (i.e., delay, bandwidth) WEB À Link-Sharing service sharing among classes proper distribution of excess service among sibling classes (e.g., fair) À Priority decoupled delay/bandwidth allocation À Statistical sharing encourage adaptation by endsystem 1998 Hui Zhang 10 5

6 Previous Work Class-Based Queueing by Van Jacobson, Sally Floyd À ad hoc link-sharing semantics À no firm guarantees CSZ Algorithm by Clark, Shenker, Zhang À firm guarantee comparable to that provided by GPS À no fine grain link-sharing Neither designed to be scalable with number of classes Fundamental Problem À more difficult to provide QoS in a hierarchy 1998 Hui Zhang 11 Our Approach Start with an idealized model that can capture all the requirements Develop practical algorithms to approximate the ideal model Two models and algorithms: À Hierarchical Generalized Processor Sharing model and Hierarchical Packet Fair Queueing algorithm À Hierarchical Fair Service Curve model and algorithm 1998 Hui Zhang 12 6

7 Hierarchical Generalized Processor Sharing physical resource 100% GPS server 60% 10% 10% 10% 10% 50% 10% Idealized fluid algorithm À service multiple queues simultaneously, in proportional to shares À parent distributes service instantaneously to children 1998 Hui Zhang 13 GPS Example [Parekh 92] Red session has packets backlogged between time 0 and 10 Other sessions have packets continuously backlogged 50% 10% 10% 10% 10% 10% Hui Zhang 14 7

8 H-GPS Example session has packets backlogged between time 0 and 10 Other sessions have packets continuously backlogged 50% 45% 5% 10% 10% 10% 10% 10% Hui Zhang 15 GPS and H-GPS Properties Proven end-to-end delay bounds for guaranteed traffic Adaptation friendly À no punishment in the future for using excess service À scalable technique for estimating excess bandwidth for adaptive applications GPS: guaranteed service for each flow H-GPS: guaranteed service for all classes À leaf class: individual flow À interior class: traffic aggregate À QoS met for all traffic classes simultaneously À ideal semantics for hierarchical resource management 1998 Hui Zhang 16 8

9 Packet vs. Fluid System GPS and H-GPS are fluid systems À multiple queues can be serviced simultaneously À no non-preemption unit Real system are packet systems À one queue is served at any given time À packet transmission is not preempted Design issue: packet algorithms approximating the fluid H-GPS algorithm À accuracy: delay bound, fairness, bandwidth distribution À complexity: practical implementation 1998 Hui Zhang 17 Packet Approximation of Fluid System Standard techniques of approximating fluid GPS À select packet that will finish first in GPS assuming that there are no future arrivals Important properties of GPS À finishing order of packets currently in system independent of future arrivals Implementation based on virtual time À assign virtual finish time to each packet upon arrival À packets served in increasing order of virtual times Various Packet Fair Queueing (PFQ) algorithms À Weighted Fair Queueing (WFQ), Self-Clocked Fair Queueing (SCFQ), Start-time Fair Queueing (SFQ) 1998 Hui Zhang 18 9

10 Approximating GPS with WFQ Fluid GPS system service order Weighted Fair Queueing À select the first packet that finishes in GPS Observation: packet finishes in WFQ no late than in GPS À basis for proving delay bound for WFQ Another observation: some packet finishes service much earlier in WFQ in GPS 1998 Hui Zhang 19 Packet Approximation of H-GPS Idea 1: use similar approach as approximating GPS À select packet finishing first in H-GPS, assuming there are no future arrivals Problem À relative finish order of packets dependent on future arrivals À can not use virtual time implementation 1998 Hui Zhang 20 10

11 H-GPS Example has no packets backlogged between time 0 and 11 Other sessions have packets continuously backlogged Need to make a decision at time 11 50% 45% 5% 10% 10% 10% 10% 10% Hui Zhang 21 Approximating H-GPS: Decision at Time 11 50% 45% 5% 10% 10% 10% 10% 10% Case 1: no future arrivals Case 2: active at time 11 Relative finish order dependent on future arrivals Hui Zhang 22 11

12 Packet Approximation of H-GPS H-GPS 10 GPS 6 4 GPS GPS GPS GPS GPS PFQ Packetized H-GPS 1 10 PFQ 2 PFQ PFQ PFQ PFQ Idea 1 À select packet finishing first in H- GPS assuming there are no future arrivals À problem: finish order in system dependent on future arrivals virtual time implementation won t work Idea 2 À use a hierarchy of PFQ to approximate H-GPS 1998 Hui Zhang 23 H-GPS and H-WFQ Example Two levels of hierarchy 50% 45% GPS order at top level 5% 10% 10% 10% 10% 10% WFQ order at top level 1998 Hui Zhang 24 12

13 H-GPS and H-WFQ Example Arrival process À active at time 5 À all other flows continuously backlogged Top level GPS server Top level WFQ server at [0,4] À make decision assuming there are no future arrivals À select packets in class À only class is active Class active at time 5 À has to endure a long delay 50% 45% 5% 10% 10% 10% 10% 10% Hui Zhang 25 PFQ and H-PFQ Many PFQ algorithms À SCFQ, SFQ, FBFQ, WFQ À how do properties of PFQ relate to H-PFQ? Worst-case Fair Index À measuring the accuracy of PFQ in approximating GPS H 1 α h DH = D + h h= 1 r À tight H-PFQ delay bound small WFI of PFQ All previously proposed PFQ algorithms have large WFI s 1998 Hui Zhang 26 13

14 A More Accurate Approximation of GPS Problem with WFQ À WFQ can be ahead of GPS too much Worst-case Fair Weighted Fair Queueing (WF 2 Q) À a packet is eligible if it starts service in GPS À among all eligible packets, select the one that finishes first in GPS À optimal algorithm in approximating GPS À normalized WFI is one maximal size packet WF 2 Q service order 50% 10% 10% 10% 10% 10% Hui Zhang 27 H-GPS and H-WF 2 Q Example Arrival process À active at time 5 À all other flows continuously backlogged Top level GPS server Top level WFQ server À there are no future arrival À never ahead of GPS by one packet size À class receives service immediately after becoming active 50% 45% 5% 10% 10% 10% 10% 10% Hui Zhang 28 14

15 WF 2 Q+ WFQ and WF 2 Q À need to emulate fluid GPS system À high complexity WF 2 Q+ À provide same delay bound and WFI as WF 2 Q À lower complexity 1998 Hui Zhang 29 Example Hierarchy 1998 Hui Zhang 30 15

16 Uncorrelated Cross Traffic 60ms Delay under H-WFQ Delay under H-SCFQ 40ms 20ms 60ms 40ms 20ms Delay under H-SFQ Delay under H-WF 2 Q Hui Zhang 31 Correlated Cross Traffic 60ms Delay under H-WFQ Delay under H-SCFQ 40ms 20ms 60ms 40ms 20ms Delay under H-SFQ Delay under H-WF 2 Q Hui Zhang 32 16

17 H-GPS and H-PFQ Hierarchical Generalized Processor Sharing model À proven end-to-end delay bounds for guaranteed traffic À ideal semantics for hierarchical resource management QoS met for all traffic classes simultaneously À friendly to bursty and adaptation sources scalable mechanism for estimating available bandwidth by adaptive applications no punishment in the future for using excess service in the past Hierarchical Packet Fair Queueing Algorithm À accurately and efficiently approximates H-GPS model À first algorithm to support real-time, link-sharing simultaneously 1998 Hui Zhang 33 Limitation of GPS and H-GPS Model Service specified by one parameter: rate Delay bound is a function of the rate: B/R À B: maximum burst size À R: guaranteed service rate Coupling between delay and bandwidth allocation À low delay requires higher rate À inefficient resource utilization for low delay/low bandwidth sessions 1998 Hui Zhang 34 17

18 Service Curve QoS Model Each class is assigned a service curve that specifies the minimum amount of service it should receive GPS guarantees a linear service curve Low delay requires higher bandwidth reservation bits Arrival curve Service curve delay 1998 Hui Zhang 35 Service Curve QoS Model Each class is assigned a service curve that specifies the minimum amount of service it should receive Non-linear service curve decouples the delay/bandwidth allocation bits Arrival curve Service curve delay time 1998 Hui Zhang 36 18

19 Fairness Defined in context of Generalized Processor Sharing (GPS) À each session is assigned a share À service is instantaneously distributed in proportion to their shares À excess service unused by a session also distributed proportion to sessions shares Important property: memoryless, non-punishment À a session not punished for extra service it received in the past À important for hierarchical link-sharing À encourages adaptation and statistical multiplexing 1998 Hui Zhang 37 Hierarchical Fair Service Curve Model Each class is associated a service curve Service curves of all classes are guaranteed Excess service is distributed fairly among sibling classes Properties À formally defines link-sharing, real-time, priority requirements À generalized fairness property applicable to arbitrary-shape service curves decouple excess service distribution policy from guaranteed service 1998 Hui Zhang 38 19

20 Fundamental Conflicts With non-linear service curves or priority service À it is impossible to guarantee service curves of all classes À It is impossible to simultaneously provide real-time guarantees and instantaneous fairness 1998 Hui Zhang 39 H-FSC Scheduling Algorithm Approximates H-FSC model by À resolving conflicts in favor of real-time requirements for leaf classes À minimizing discrepancy between service each class should ideally receive and the service it actually receives Based on two criteria À real-time: guarantee service curves of all leaf classes À link-sharing: guarantee service curves of interior classes and distribute excess service fairly À whenever there is a potential of conflict, real-time criteria is used 1998 Hui Zhang 40 20

21 H-FSC Algorithm (e,d,v) (v) (e,d,v) (v) (e,d,v) e - eligible time d - deadline v - virtual time The Algorithm if (there is an eligible packet) /* real-time criteria */ send eligible packet with min. deadline d ; else /* link-sharing criteria */ send packet with min. virtual time v ; Eligible time (e): whenever there are eligible packets there are potential conflict between real-time and link-sharing goals Deadline (d): if all deadlines of packets of a class are met, its service curve guarantee is met Virtual time (v): normalized amount of service received by a class e, d, v are computed based on three curves E( ), D( ), V( ) 1998 Hui Zhang 41 H-FSC Complexity Theoretically, worst case complexity O(log n), but can be improved taking advantage of timestamp properties All service curves are two-piece linear À enough to decouple bandwidth & delay allocation Computing e, d, and v requires one comparison, division E( ), D( ), V( ) are updated only when session becomes active À updating D( ) and V( ) requires in the worst case three multiplications, one division and several additions, subtractions À updating E( ) requires at most one addition 1998 Hui Zhang 42 21

22 Delay and Fairness Properties of H-FSC Service curves of all leaf classes are guaranteed For any class, the difference between the service it should receive in the ideal H-FSC model, and the service it actually receives is bounded 1998 Hui Zhang 43 Simulations: Class Hierarchy 45M bps Link 20M bps A Poisson Poisson Poisson Poisson Poisson 5M bps 5M bps 5M bps 5M bps 5M bps Audio Video FTP ON- OFF 64Kbps 2M bps 5M bps 8M bps Poisson 4M bps 1998 Hui Zhang 44 22

23 Simulations: Delay Results for Audio (H-WF 2 Q+ vs. H-FSC) Packet size = 1280 bits, bandwidth = 64Kbps bits bits 64Kbps 256Kbps Kbps 20m s time 5m s time 20 ms H-WF 2 Q+ H-FSC 15ms 10 ms 5 ms 1998 Hui Zhang 45 Implementation Status 622 Mbps ATM Implementation by Fore Systems Software IP Implementation on NetBSD/PC router À flat hierarchy, 1000 flows, 9 usec/packet À 3 levels hierarchy, 1000 flows, 12 usec/packet À generic interface for control software: Beagle, RSVP Alpha release À University of Utal, Washington University, Sun Microsystem High speed implementation À Ascend GRF router software implementation completed in vbns (joint work with MCI) hardware implementation in process (with Ascend) À Cisco IOS: summer Hui Zhang 46 23

24 1998 Hui Zhang 47 NSF vbns Network 1998 Hui Zhang 48 24

25 Local Darwin Testbed m3 m3 m8 m8 maui maui altamira altamira aspen yosemite yosemite lascaux.12.2 lascaux whiteface m m5 squirrelhill squirrelhill timberline m7 m m2 m4 m Hui Zhang 49 Router Node Architecture Routing In/Out Routing Signaling In/Out Beagle or RSVP Control Delegates DART Other Control Delegates Control API Event Notification Local Resource Manager Control Plane Data In Classifier Classifier Classifier Route Route Lookup Route Lookup Lookup Scheduler Classifier Data Out Data Plane 1998 Hui Zhang 50 25

26 Simple Virtual Network Experiment Experiment set-up À application virtual mesh with FFT distributed across three nodes Audio/video transmission FTP application À background UDP traffic Base-line senario À reservation for individual flows in application mesh À no virtual mesh specific resource management policy Test case senario À two level of resource management hierarchy À virtual mesh specific resource management policy 1998 Hui Zhang 51 Base-line Case: No Hierarchy A/V guarantee met FTP throughput À 1.7 Mbps FFT finish time À 29 second 5RXWHUÃ 5RXWHUÃ )DVWÃ)RXULHUÃ7UDQVIRUP $XGLR9LGHR )LOHÃ7UDQVIHU Ã0ESV /LQN Ã0ESV Ã0ESV 0ESV Ã0ESV $9 )73 ))7 8'3 $GYHUVDU\Ã8'3 5RXWHUÃ 1998 Hui Zhang 52 26

27 Link-sharing at Root Node 1998 Hui Zhang 53 Test Case: Two Levels of Resource Hierarchy A/V guarantee met FTP throughput À 4 Mbps À factor of 2.4 improvement FFT finish time À 27 second Same amount of total resource reserved 5RXWHUÃ 5RXWHUÃ )DVWÃ)RXULHUÃ7UDQVIRUP $XGLR9LGHR )LOHÃ7UDQVIHU Ã0ESV /LQN Ã0ESV Ã0ESV 0HVK 8'3 Ã0ESV Ã0ESV 0ESV $9 )73 ))7 $GYHUVDU\Ã8'3 5RXWHUÃ 1998 Hui Zhang 54 27

28 Link-sharing at Root Node 1998 Hui Zhang 55 Link-sharing at Interior Application Node 1998 Hui Zhang 56 28

29 Experiment: Link-Sharing 10M bps Link 4M bps A FTP FTP FTP FTP 1.5Mbps 1.5Mbps1.5Mbps1.5Mbps 80Kbps 1.44M bps ON- 480Kbps 2M bps OFF 1998 Hui Zhang 57 Ongoing Work High performance Combined-Input-Output buffered switches or routers Implementation with no per flow state Wireless networks with location-dependent errors 1998 Hui Zhang 58 29

30 High Speed Implementation All existing QoS scheduling/queue management algorithms assume output buffering Output-buffered switches/routers have fundamental limits on scalability 1998 Hui Zhang 59 High Speed Implementation (2) High performance switches/routers use Input-Output buffered architecture À multiple contention points À QoS support more difficult 1998 Hui Zhang 60 30

31 Core-Stateless Implementation QoS algorithms require switch to maintain per flow state À expensive to install and maintain À less robust and fault tolerant Core-stateless implementation architecture À only edge maintains per flow state À core routers do not maintain state 1998 Hui Zhang 61 Wireless Networks Location dependent errors GPS and H-GPS do not apply well À backlogged flows in error cannot transmit 1998 Hui Zhang 62 31

32 Darwin: Electronic Value-added Services Market Application combines local resources with network services to create end-product Value-added services integrate communication with computation and storage services Electronic services market has a hierarchical structure. Application Meshes Hierarchical Services Bearer Level 1998 Hui Zhang 63 Resource Virtualization Value Added Service Provider 1 Value Added Service Provider 2 Bitway Provider Mbps 100 Mbps 55 Mbps Bitway Service Provider 1 60 Mbps CMU 30 Mbps SCS Value Added Service Provider 1 10 Mbps ECE 40 Mbps U.Pitt Campus Value Added Service Provider 2 Guaranteed Service Controlled Load Service FTP Best-effort Service Telnet Bitway Service Provider 1 seminar video seminar audio Distributed Simulation WEB Control Audio Video application end point 1998 Hui Zhang 64 32

33 Summary Physical resource can be dynamically partitioned as a set of virtual resources with a hierarchical relationship Different entities (organizations, service providers, and applications) can independently specify entity-specific policies for their portions of the virtual resource hierarchy Admission control and hierarchical scheduling ensure policies are met for all entities À QoS provided for both individual traffic streams and traffic aggregates of different entities Collaborative and competitive resource sharing seamlessly integrated Abstraction applicable to other resources (CPU) also 1998 Hui Zhang 65 References J.C.R. Bennet and H. Zhang, Hierarchical Packet Fair Queueing Algorithms, IEEE/ACM Transactions on Networking, 5(5): , Oct Also in SIGCOMM 96. I. Stoica, H. Zhang, E. Ng, A Hierarchical Fair Service Curve Algorithm for Link-sharing, Real-Time and Priority Services, SIGCOMM 97. D. Stephens, H. Zhang, Implementing Distributed Packet Fair Queueing in a Scalable Switch Architecture, INFOCOM 98. I. Stoica, S. Shenker, H. Zhang, Core-Stateless Fair Queueing: A Scalable Architecture to Approximate Fair Bandwidth Allocation in High Speed Networks, SIGCOMM 98. URL: Hui Zhang 66 33