COMP6511A: Large-Scale Distributed Systems Windows Azure Lin Gu Hong Kong University of Science and Technology Spring, 2014
Cloud Systems Infrastructure as a (IaaS): basic compute and storage resources E.g., Amazon AWS/EC2, VMWare vcloud Platform as a (PaaS): cloud application infrastructure E.g. Google App Engine, Salesforce.com, Windows Azure Software as a (SaaS): cloud applications E.g. Google Docs, Microsoft Office Web Companions, Office 365
The Microsoft Cloud Categories of s s Software s Platform s Infrastructure s
Windows Azure - Overview Windows Azure presents itself as a platform for cloudbased application development For the datacenter: parallel processing, resource management, provisioning, and monitoring For users: cloud-based applications, integrated development environment
Cloud Based Computation with Windows Azure Web Role Worker Role HTTP Load Balancer IIS ASP.NET, WCF, etc. main() { } Agent Agent Fabric Compute Storage Fabric VM
Model Cloud apps are constructed in role-instance Model A role instance is a set of code, configuration, and local data, deployed in a dedicated VM At runtime each Role executes on one or more instances
Windows Azure Web & Worker Roles Worker Role For general development and computation Background processing Web Role Customized for web applications A web role is a worker role with the addition of IIS (i.e., it can do everything a worker role can do as well) Windows Azure currently supports two types of roles: worker role and web role. A service must include at least one role of either type, but may consist of any number of web or worker roles.
Azure Node Structure A node is a management unit of FC Contains an FC Agent in Hyper-V root partition Each role instance runs in Guest OS with GA FA delegates GAs to handle VM status
High-Level Architecture Host Partition Guest Partition Guest Partition Host OS Server Core Virtualization Stack (VSP) s Guest OS Server Enterprise Virtualization Stack (VSC) s Guest OS Server Enterprise Virtualization Stack (VSC) Drivers VMBUS VMBUS VMBUS Hypervisor NIC Disk1 Disk2 Hardware CPU
Windows Azure Compute Virtual Machines The VMs are provided by a cloud-optimized hypervisor s see a 64-bit Windows Server 2008 or Windows Server 2012 interface Development fabric: a desktop replica of Windows Azure
Node Health Monitoring Based on heartbeats, which are typically sent in 15 second intervals Once the index falls below zero, FC attempts to heal node For example, host agent timeout is 10 minutes Timeouts vary depending on node state and operation Worst-case reaction time is timeout interval + heartbeat interval Missing Heartbeats Missed Heartbeat Recovery Initiated Node Health Index Heartbeat Interval Heartbeat Timeout
Windows Azure Storage Types of Storage HTTP Blobs Tables Queues Compute Storage Fabric
Windows Azure Storage The Windows Azure storage services provide storage for binary and text data, messages, and structured data Blob service: storing binary and text data Queue service: storing messages that may be accessed by a client Table service: structured storage for non-relational data Windows Azure drives: mounting an NTFS volume accessible to code running in your Windows Azure service Azure provides scalable, durable cloud storage, backup and recovery solutions for any data, large and small. -- http://azure.microsoft.com/en-us/services/storage/
Windows Azure Storage Tables Table Table Table... Entity Entity Entity... Storage Accounts Property Property Property Name Type Value
Windows Azure Storage Tables: Strengths Massively scalable Perspective: Applied to the right problem, Windows Azure Tables are a beautiful thing But they re not the optimal solution for many other applications Amazon, Google, and others provide similar abstractions
Windows Azure Storage Tables: Limitation Access via REST You can t use ordinary ADO.NET No SQL A new approach for developers to learn No aggregates You can t easily move relational data to or from it No schema Entities and properties are not exactly rows (tuples) and columns (attributes)
SQL Azure (a.k.a. SQL s) SQL Azure Database Others (Future).NET s s SQL Azure Windows Azure s Windows Server Windows Vista/XP Windows Mobile Others
SQL Azure Database SQL Azure Formerly known as SQL Data s (SDS) and SQL Server Data s (SSDS) Huron Data Hub Built on the Microsoft Sync Framework, which is a comprehensive synchronization platform that enables collaboration and offline access for applications, services, and devices with support for any data type, any data store, any transfer protocol, and any network topology. Limitations in functionality and scalability In the future: reporting, analysis, extract/transform/load (ETL) services,
Using Queues An application model To scale, add more of either 1) Receive work Web Role ASP.NET, WCF, etc. Worker Role main() { } 4) Do work 2) Put work in queue 3) Get work from queue Queue
Azure Apps - Scale out
Windows Azure Deployment Step 1: Allocate nodes Across fault domains, across update domains Step 2: Place OS and role images on nodes Step 3: Configure settings Step 4: Configure loadbalancers, Start Roles Step 5: Maintain desired number of roles Failed roles automatically restarted Node failure results in new nodes automatically allocated Fault-tolerant high-performance distributed computing fabric Load-balancers
Image-Based Deployment Host Partition Guest Partition Guest Partition Maintenance OS Guest Partition VHD VHD VHD App1 Package App3 Package App2 Package Host partition differencing VHD Guest partition differencing VHD Guest partition differencing VHD Guest partition differencing VHD HV-enabled Server Core base VHD Server Enterprise Server base Enterprise base VHD VHD Server Core base VHD
How Azure runs an app The Fabric Controller Process app definition files to create VHDs (Virtual Hard Disks) Place role VHDs on compute nodes Boot and configure role instance VMs Start role host processes (IIS, WCF ) Run role instances Response to HTTP request Maintain service health
Bus 2) Discover endpoints 3) Access application Bus Registry Endpoints 1) Register endpoints Organization X Organization Y Access Control Bus
Bus Exposing internal applications on the Internet isn t a comfortable solution Firewalls get in the way, may be slow, expensive Bus: provides a cloud-based intermediary between clients and internal applications Direct or queued It also provides a service registry that clients can use to find the services they need
Azure Apps Overview The Internet The Internet via TCP or HTTP L B L B L B Web Web Site Site (ASPX, ASMX, (ASPX, ASMX, Role WCF) WCF) IIS as Host Tables Queues Storage Blobs Worker Worker Role Managed Interface Call Windows Azure Data Center
Web Patterns User Web Browser Mobile Browser Silverlight WPF Enterprise Enterprise Web Svc ASP.NET ASP.NET (Web ASP.NET Role) (Web ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Web Svc ASP.NET (Web ASP.NET Role) (Web ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Jobs (Worker ASP.NET ASP.NET (Web Role) ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Data Enterprise Data Table Storage Blob Storage Queue Storage Enterprise Identity Private Cloud User Data Data Reference Data Bus Access Control Workflow Identity Public Cloud s
Patterns Grid / Parallel Computing User Web Browser Mobile Browser Silverlight WPF Enterprise Enterprise Web Svc ASP.NET ASP.NET (Web ASP.NET Role) (Web ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Web Svc ASP.NET (Web ASP.NET Role) (Web ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Jobs (Worker ASP.NET ASP.NET (Web Role) ASP.NET Role) (Web ASP.NET Role) (Web Role) (Web Role) Data Enterprise Data Table Storage Blob Storage Queue Storage Enterprise Identity Private Cloud User Data Data Reference Data Bus Access Control Workflow Identity Public s