Intelligent QoS Grid for Virtualized Workloads Gaurav Gupta Tata Consultancy Services

Transcription

1 Intelligent Grid for ized Workloads Gaurav Gupta Tata Consultancy Services

2 Characteristics of Data Analytics BI Image Processing Multi Media Static Content OLTP BigData NoSQL ECM Cloud IOT ERP Web 2.0 Media Streaming Search HPC Online Gaming Social Networking Random Sequential R/W Latency Secondary R/O Multiplexed Deduplication Parallel Spike Seek Compression ized I/O Secondary Block I/O Data IOPs Backup Mixed Sequential I/O Pattern Capacity Mixed File I/O Parallel Primary I/O blender effect Performance Response time File I/O Access Throughput Storage Communication Database 2

3 Application Lifecycle & Challenges in Environment Workload1 Railway Reservation Festival Emergency Booking Workload1 Workload2 Performance Interference Workload2 Social Media Workload3 Workload3 Corporate Portal World Wide Event Quarterly Results Workload4 Online Retail Product Promotion / Sale Ecosystem Workload4 Resource Pool Hypervisor Workload3 Physical Resource CPU Memory Disk NIC Resource Contention.??? Challenge Un-utilized resource Threshold Solution 1 Add more nodes in the cluster? 2 Machine Movement? Compute Network Storage 3

4 Elastic Controls - Key # of CPU % Memory DMA I/O request size %CPU Pages Exchange Context Switch Concurrence Req1 Compute Node Boundary Layer2 Req2 Req3 Network Layer3 MTU size Max Packet size Max bit rate Network wait time NetIOC Transmission delays Storage Class of Services (CoS) Tag Control Information Prioritization bits (0-7) 802.1P Type of Service (ToS) DSCP (Differential Services Code Point: 0 to 63) IP Precedence (0 7) Disk queue length Disk I/O wait time Seek delay Disk priority % Read-write Rotational delay Prioritization Controls 4

5 Grid An overview Federated Intelligent Distributed architecture Allows grouping of virtual resources & s Scales up to cluster limit Senses the change in workload behavior Dynamically leases virtual resources to others Implements bandwidth-as-a-service model Compute Network Storage Analytic Takes into account the historical usage Prioritizes resource allocation Physical Node(s) 5

6 Grid - Building Blocks Domain Extender Sensor Discover all virtual stacks in the cluster Allows forming domain for sharing virtual resources Senses the change in workload behavior Dynamically releases virtual bandwidth to others in the domain Compute Network Storage Regulator Throttles resource bandwidth across nodes based on change in requirement Physical Node(s) 6

7 Grid Operations Lease Lease Recall Extender Sensor Regulator Sensor Regulator 1 Discovers VMs & 2 3 Tracks resource usage against baseline of each VM 1 Tracks resource usage of the lessor VMs against new baselines Creates Domain Requests Monitor 4 5 Requests Change in tuning 6 2 Requests recall leased bandwidth 3 Adjusts of lease and lessor to original values Borrows resources by rebaseline s in the domain Adjusts the of the lenders & borrower VMs 7

8 Co-existence of workloads through Grid effect (Indicative) Railways reservation Baseline resources Stretched compute on single node in cluster (Leased with in same node) Stretched compute & storage from multi-node in the cluster (Leased from N2 & N3 nodes) Emergency Booking Shrinked compute & stretched storage at multi-node in the cluster (Lease Recalled by N3) Domain1 N1-2VMs, N2-1VM, N3-1VM Social Media Pic and video files sharing during festivals Domain2 N1-N5: 1VM each Corporate Portal Live streaming of press conference on quarterly results Domain3 N1-N5: 1VM each Online Retail Catalog search and order placement Compute Network Storage 8

9 Thank You Gaurav Gupta 9