Quality of Service in US Air Force Information Management Systems

Transcription

1 Quality of Service in US Air Force Information Management Systems Co-hosted by: Dr. Joseph P. Loyall BBN Technologies Sponsored by: 12/11/2009 Material Approved for Public Release.

2 Quality of Service is a Crucial Aspect of Information Management for Tactical Information Dominance QoS management for net-centric information systems strives to provide a predictable high level of mission effectiveness & user satisfaction within available resources Capture QoS aspects of mission requirements Effectively utilize available resources for mission effectiveness Manage the resources that could become bottlenecks Mediate conflicting demands for resources Dynamically reallocate as conditions change 12/11/2009 2

3 Core Information Management Services Sends outgoing s Subscribers Register Subscription Registered Predicates Producers Submission Service Receives incoming s Dissemination Service Broker Service Published Archive Archive Service Inserts s into repository Matches s & registered subscriber predicates Query results Query clients Query predicate Query results Processes queries & getnext calls Query Service Data store = Managed Information Object () 12/11/2009 3

4 Hard Problems in Providing QoS Management for IM Services Data size is not uniform Network routers can assume a maximum transmission unit, which is relatively small s can differ significantly in size and complexity, both in metadata and payload Estimating resource usage and time for processing are difficult to do a priori Processing can vary significantly in time Predicate evaluation and query processing are affected by the number of predicates and subscribers; the complexity of predicates; and the size and complexity of metadata The destination of information is unknown when information enters the system No opportunity for end-to-end QoS management or prioritized treatment based on the importance of its destination or knowledge of its use Two occurrences of fanout A single published can match multiple subscribers and a single query can result in many s that need to be delivered Downstream resource usage (i.e., during dissemination) of a published or a query is extremely difficult to predict and take into consideration in upstream management (i.e., during submission, brokering, and query processing) Many potential places for bottlenecks The diversity of configuration for real deployments determine where the bottlenecks appear, but cannot be defined a priori Many bottlenecks are hidden behind layers of middleware that are used to implement the IM services

5 QoS Enabled Dissemination (QED) is a Layered Architecture Status information Monitoring Svc Collect statistics Store as time series Metrics Metrics Status information Operation Insert task Extract task Client Task Creation Create Tasks Operation task object Sequence creation Task Mgt LQM Enqueue task Assign thread Dequeue task Execute task QED Admin ClientPolicy objects ISQM Service Convert ClientPolicy to KAoS policies Deconflict the set of policies Set properties on seqs Seq. Properties Sequence attributes Fairness strategy Fairness strategy (JMS struct) Policy actions (add, replace, remove) Shaping Svc Decide whether to shape Policy Invoke shaping QoS Administration Aggregate QoS Management Seq. info Policy Mgt Service Manage the KAoS policies Select the policies pertaining to a sequence Metrics Local QoS Management Dissem. Svc LQM Create dissemination tasks Enqueue tasks Dequeue tasks Xlayer Task queues Get thread to assign to task Thread Pool Archive Task Query Task GetNext Task Broker Task Shaped or NULL s Estimate of # results through call return shaping Sub IDs s Client ID Insert Extract Diss. queues, Sub IDs Mockets Client QoS Mechanisms 12/11/2009 5

6 Elements of QED QoS administration Entry of QoS policies and mission information Information Services QoS Manager (ISQM) and policy manager Aggregates QoS policies over users and resources of an IMS Local QoS Managers (LQMs) Enforce QoS policies CPU LQM prioritizes CPU intensive tasks Bandwidth LQM prioritized bandwidth intensive tasks QoS monitoring Statistics to support decision making and QoS visibility Mockets managed transport Options for reliable, ordered, unreliable, and unordered delivery QoS mechanisms Differentiated bins/queues prioritize operations and dissemination tasks Partitioned predicate match predicates based on importance and cost shaping reduce size of s and match client preferences 12/11/2009 6

7 Challenge How QED Addresses the Hard Problems How QED addresses the challenge Data size is not uniform Processing can vary significantly in time Destination is unknown when information enters the system Fanout of published s to many subscribers and many s to query client Bottlenecks hidden in layers of middleware Associate importance with information objects, not packets Incorporate cost of sending, processing, and brokering, e.g., size, in priority calculation and enforcement Incorporate cost of processing (based on factors that can increase it) in priority calculation and task scheduling Base priorities and QoS policies on mission information, inc. user, operation, and information type; Mission planning coupling a publisher and consumer can be captured in mission-level importance Information processing incorporates publisher importance; information dissemination incorporates consumer importance; scheduling changes as an moves through the system Subscriber importance and dissemination cost taken into account in scheduling dissemination, so that s are delivered to important receivers first Context awareness used to order results to disseminate most useful first QED scheduling keeps large result sets from blocking other important operations and s to the extent possible Factor out identifiable and controllable bottlenecks, e.g., move processing out of EJB method bodies and into schedulable tasks and processes. 12/11/2009 7

8 QoS Policies in QED Client Sequence creation Task Mgt LQM QED Admin ClientPolicy objects ISQM Service Seq. Properties Fairness strategy Sequence attributes Policy actions Policy, Seq. ID Fairness strategy Policy Mgt Svc Dissem. Svc LQM Policies define a mapping from a context C to an importance (i) & set of QoS preferences (P): C (i, P) Context is the range of functions over observable properties of Actors (A), Operations (O), Information (M), Resources (R) C = f(a) g(o) h(m) j(r) Policies are maintained in the Policy Management service The ISQM distributes policies & fairness strategies to Local QoS Managers We currently have two LQMs Task management LQM manages CPU intensive tasks, i.e., brokering, query, getnextresultset, & archive Dissemination service LQM manages bandwidth intensive tasks

9 Task queues Thread Pool Registered Predicates CPU Management Using Priority Bins & Task Scheduling Broker task Insert task Extract task Producer Publish Broker task Archive task Get thread to assign to task Dissemination Worker Predicate Evaluation Query Task Creation Task Mgt LQM Archive task Process boundary Query client Query task GetNext task Archive Worker GetNext- ResultSet Query task GetNext task Query Worker Archive/ Query Service Data store CPU intensive operations (brokering, queries, archival) are enqueued as tasks Use a configurable number of bins (queues) Each bin defined by importance (operation,, client) & cost (expected duration of task) Bins allow for linear time insertion (vs. logn insertion time for priority queue) Number of bins determine granularity of control (one bin per task = priority queue) Assign threads to tasks in order of priority Policies define scheduling (thread assignment) strategies Weighted Fair Queuing (WFQ) Each queue i is serviced by a number of threads associated with its relative weight Strict Queuing available threads always serve highest priority tasks & tasks can be starved

10 QED Approach to Managing Bandwidth Contention Bandwidth Mgr Client queues Mocket Client Broker Task Set(subs), Dissemination Svc Client queues Mocket Client Dissemination task Client queues Mocket Client Dissemination service creates dissemination tasks Each diss. task is placed on a queue for its destination s mocket based on its importance Each client queue sends diss. tasks in priority order to BW manager Block on a getpermissiontosend semaphore When bandwidth is available, BW manager semaphore unblocks the next diss. task BW manager calculates the time to send Size of /amount of bandwidth is sent through the appropriate mocket BW manager blocks while is in transit

11 Demonstration of QED in Action The client s name The s the client is receiving, their type and source, how many, their rate, the time of the last received, and the number dropped Shaping (scaling) introduces latency (approx ms) Relatively higher importance s are lower latency cas.image & uas.image s are arriving approx. 1 per second Non-shaped image Scaled image

12 Performance of QED Versus the IM Baseline Mean Receive Rate (Hz) Mean Latency (sec) Base Equal Weighted Strict Base Equal Weighted Strict CPU Bound Type Type > 100 Infinite Network Bound Type N/A > Type > Improved control over information QoS N/A Significantly higher QoS provided to higher importance information Higher throughput, 2x over baseline Lower latency than baseline and low importance s Green = MEDIUM importance Blue = LOW importance Black = undifferentiated service

13 Summary We are developing QoS enabled dissemination capabilities for PSQ IM services Multi-layered approach, with aggregation of QoS requirements Policy driven based on properties of users, operations, information, and resources Monitoring of resource usage and delivered QoS Multi-resource management (CPU and BW) QED provides a necessary capability for the adoption of IM services in tactical military environments Ongoing work is focusing on SOA environments, greater scale and distribution, and additional capabilities, including disruption tolerance 12/11/