GETTING STARTED WITH WINDOWS CONTAINERS, DOCKER AND AZURE

Transcription

1 GETTING STARTED WITH WINDOWS CONTAINERS, DOCKER AND AZURE Thomas Maurer Cloud Architect and MVP Samuel Erskine MVP and Author

2 Thomas Maurer Microsoft MVP, MCT Microsoft MVP, MCT Cloud itnetx Author, Trainer & Snr Manager Foodie, Frequent Flyer Gadget Freak

3 AGENDA Agile DevOps World. Windows Containers Basics

4

5 EXPECTATIONS OF CLOUD APPLICATIONS

6 CURRENT CHALLENGES Customer Pain Scalability (lack of) All or nothing updates Downtime during updates Complicated deployment High 3 rd party license costs Developer Pain Long build times Locked into single technology stack High team coordination/overhead Easy to break the monolith Long release cycles

7

8

9

10

11

12

13 Rapid Deployment Fast Iterations Service Separation High Availability Reproducibility On Any Cloud Scalability Test at Scale Reduced Costs

14 WHAT IS A CONTAINER?

15 WHAT IS A CONTAINER? Application OS Hardware

16 WHAT IS A CONTAINER? Traditional virtual machines = hardware virtualization VM VM VM

17 WHAT IS A CONTAINER? Traditional virtual machines = hardware virtualization VM VM VM

18 WHAT IS A CONTAINER? Traditional virtual machines = hardware virtualization VM VM VM

19 WHAT IS A CONTAINER? Containers = Operating system virtualization Applications Kernel Traditional virtual machines = hardware virtualization VM VM VM

20 WHAT IS A CONTAINER? Containers = Operating system virtualization CONTAINER CONTAINER CONTAINER Applications Kernel Traditional virtual machines = hardware virtualization VM VM VM

21 WHAT IS A CONTAINER? Containers = Operating system virtualization CONTAINER CONTAINER CONTAINER Applications Kernel Traditional virtual machines = hardware virtualization VM VM VM

22 WHAT IS A CONTAINER? Containers = Operating system virtualization CONTAINER CONTAINER CONTAINER Windows Server Containers Maximum speed and density CONTAINER CONTAINER CONTAINER Applications Kernel Traditional virtual machines = hardware virtualization VM VM VM

23 WHAT IS A CONTAINER? Containers = Operating system virtualization CONTAINER CONTAINER CONTAINER Windows Server Containers Maximum speed and density CONTAINER CONTAINER CONTAINER Applications Kernel Traditional virtual machines = hardware virtualization Hyper-V Containers Isolation plus performance VM VM VM CONTAINER CONTAINER CONTAINER Kernel Kernel Kernel

24 HOW DO CONTAINERS WORK?

25 HIGH LEVEL ARCHITECTURE Container Development and Management Toolset Container Runtime

26 WINDOWS CONTAINERS Host User Mode System Processes Container Management

27 WINDOWS CONTAINERS Host User Mode System Processes Container Management

28 WINDOWS CONTAINERS Host User Mode System Processes Container Management System Processes

29 WINDOWS CONTAINERS Host User Mode System Processes Container Management System Processes Application Process(es)

30 WINDOWS CONTAINERS Host User Mode System Processes Container Management System Processes Application Process(es)

31 WINDOWS CONTAINERS Host User Mode System Processes Container Management Hyper-V Container System Processes Application Process(es)

32 WINDOWS CONTAINERS Host User Mode System Processes Container Management Hyper-V Container System Processes Application Process(es)

33 WINDOWS CONTAINERS Host User Mode System Processes Container Management Hyper-V Container System Processes Application Process(es) System Processes Application Process(es)

34 WINDOWS CONTAINERS System Processes Host User Mode Container Management Virtual Machine Specifically Optimized To Run a Container Hyper-V Container System Processes Application Process(es) System Processes Application Process(es)

35 A QUICK EXAMPLE

36 STARTUP PERFORMANCE Windows Server Container Under 600 ~1 Milliseconds! Hyper-V Container ~1.75 ~3.3 seconds A virtual A machine virtual machine takes ~5 takes seconds ~3 seconds to over a min HP ProLiant SL250s Gen8, E5-2600, 2 Socket, 8 Core, 128GB RAM, HP SATA SSD

37 STARTUP PERFORMANCE Windows Server Container Under 600 Milliseconds! Hyper-V Container ~1.75 seconds A virtual machine takes ~3 seconds Windows Server Container ~1 second Hyper-V Container ~3.3 seconds A virtual machine takes ~5 seconds to over a min

38 DENSITY Windows Server Container First Container ~150MB ~120MB Additional Containers ~75MB Hyper-V Container First Container ~340MB ~555MB Additional Containers ~280MB ~150MB HP ProLiant SL250s Gen8, E5-2600, 2 Socket, 8 Core, 128GB RAM, HP SATA SSD

39 DENSITY Windows Server Container First Container ~120MB Additional Containers ~75MB Hyper-V Container First Container ~340MB Additional Containers ~150MB Windows Server Container First Container ~150MB Additional Containers ~75MB Hyper-V Container First Container ~555MB Additional Containers ~280MB

40 HOW DO I CREATE A CONTAINER?

41

42 DOCKER CONTAINER MANAGEMENT Registry Registry Host Containers Containers Containers Client Daemon Images Images Images

43 CREATE A CONTAINER docker run -d --name "demo1" microsoft/nanoserver

44 DOCKER RUN Creates and Starts a New Container Runtime options Name (network name and management name) Interactive or Service Network configuration Resource management Volume mappings Isolation level

45 DEMO CREATING A CONTAINER

46 DEVOPS WHY, WHAT & HOW

47 @samerskine Dev view of Enterprise Operations Ops code platform Ops Rules

48 @samerskine Ops view of Devs What is code is behind this? Dev house

49 RESOLVING ISSUES WITHOUT DEVOPS

50 OUR SHARED DEFINITION DEVOPS Automated Testing Continuous Deployment Continuous Integration Release Management Infrastructure as Code Application Performance Management Configuration Management

51 ENTERPRISE ADOPTION OF DEVOPS Hey, I need 5 more features by next week Sure lets work on it together

52 THE NEW DEV, OPS, AND BUSINESS

53 WHAT IS A CONTAINER IMAGE?

54 CONTAINER IMAGE Analogous to a VHD and config file to a virtual machine Created by running a container and capturing changes Changes include files and registry Metadata Name, Creation Data, Command To Execute, Dependences Contents

55 CONTAINER IMAGES

56 CONTAINER IMAGES Image Contents HKLM HKCU License.txt PerfLogs Program Files HKCR HKU Program Files (x86) Users Windows

57 CONTAINER IMAGES Image Contents HKLM inetpub SOFTWARE/ mykey mysite.html Image Contents HKLM HKCU License.txt PerfLogs Program Files HKCR HKU Program Files (x86) Users Windows

58 AUTOMATED IMAGE BUILDING Docker Build and Dockerfiles Method for automated container image build Consumed when running docker build Caches unchanged commands Integrates into Docker Hub Examples IIS FROM microsoft/windowsservercore RUN powershell command Add-WindowsFeature Web-Server Website FROM iis ADD mysite.htm inetpub\mysite.htm

59 DEMO DOCKER BUILD

60 IMAGE REGISTRIES What is a registry? Stores container images Images are Pushed into a registry Images are Pulled from a registry Images are Searched for within a registry

61 IMAGE REGISTRIES Docker Hub and Docker Store Public, Official and Private image repositories Granular access controls with organization support Automated image build support Docker Trusted Registry Enterprise Grade Private Registries Runs on your infrastructure (on-prem or cloud) Active Directory and Role Based Access Controls Docker Registry Open source foundation of Hub and DTR Runs on your infrastructure (on-prem or cloud) as a container and or

62 CONTAINER INFRASTRUCTURE LAYERING Application Linux Containers Windows Server Containers Hyper-V Containers Deployment Packaging Orchestration Dev Cloud IaaS

63 Azure Container Registry

64 WHY: AZURE CONTAINER REGISTRY Keep Your Images Private Stored in Azure with your resources Network-Close Deployed to your targets within the same data center No ingress/egress fees or latency Azure Active Directory Integration Manage registry access using AAD Familiar Open Source CLIs docker login, pull, push

65 CONTAINER DEVOPS WORKFLOW Container Service Source Code Control (SCC) Build/CI, Integrate, Test CD, Deploy Run, Manage Production environments Service Fabric App Services Batch Azure Container Registry coming soon Inner-Loop Run Monitor and Diagnose Code Validate Debug

66 Azure Container Service kubernetes DC/OS Swarm

67

68 ND (P40) and NCv2 (P100s) Dv3 and Ev3 (with nested virtualization)

69 Soon Soon Soon Soon

70 NESTED VIRTUALIZATION Guest VM Nested Virt vcpu Virtualization

71 NESTED VIRTUALIZATION Windows Guest Linux Guest Hyper-V Isolation vcpu vcpu vcpu Virtualization

72 WHAT ABOUT

73 LICENSING

74 PATCHING AND UPDATES Update Container OS Image Pull updated base image Rebuild containers using dockerfiles

75 PATCHING AND UPDATES Update Container OS Image Pull updated base image Rebuild containers using dockerfiles

76 PATCHING AND UPDATES Update Container OS Image Pull updated base image Rebuild containers using dockerfiles KB = new image on Docker Hub

77 PATCHING AND UPDATES Update Container OS Image Pull updated base image Rebuild containers using dockerfiles FROM windowsservercore RUN powershell command Add-WindowsFeature Web-Server KB = new image on Docker Hub

78 PATCHING AND UPDATES Update Container OS Image Pull updated base image Rebuild containers using dockerfiles FROM iis ADD mysite.htm inetpub\mysite.htm FROM windowsservercore RUN powershell command Add-WindowsFeature Web-Server KB = new image on Docker Hub

79 PATCHING AND UPDATES Update as a new layer Download update in container (ala run Windows Update in the container) When container is stopped update is applied as a new layer

80 PATCHING AND UPDATES Update as a new layer Download update in container (ala run Windows Update in the container) When container is stopped update is applied as a new layer

81 PATCHING AND UPDATES Update as a new layer Download update in container (ala run Windows Update in the container) When container is stopped update is applied as a new layer Same Image

82 PATCHING AND UPDATES Update as a new layer Download update in container (ala run Windows Update in the container) When container is stopped update is applied as a new layer Same Image Same Image Same Image

83 PATCHING AND UPDATES Update as a new layer Download update in container (ala run Windows Update in the container) When container is stopped update is applied as a new layer Same Image Same Image Same Image

84 RESOURCE CONTROLS CPU Percent of host CPU container can consume Memory Disk Maximum memory container can consume Maximum IO bandwidth on the system drive Maximum IOPs limit on the system drive Network Platform support for egress caps

85 NETWORKING Fully Manageable with Docker Network creation/enumeration Service Discovery Optimized for Microsoft Cloud Stack Advanced network policy (ACLs, QoS) can be assigned per container endpoint Load Balancing can be handled through the Microsoft Software Load Balancer (Coming Soon) Area of Regular and Continuous Innovation Docker tooling support (Compose) for networking (Limited Support at GA) Integration with other Orchestrators (Kubernetes, Swarm, etc.) Native Overlay network driver Multiple networks (NAT and overlay) per host

86 STORAGE Container Image Not designed for persistent data Not designed for secrets Volumes Enables storage persistence Enables mapping of storage into containers Read-Only or Read/Write Multiple containers on the same host can access the same location Plug-In Architecture Network Storage Containers access SMB shares Accessed though the containers network

87 VOLUME MAPPING Container Host

88 VOLUME MAPPING License.txt PerfLogs Program Files Program Files (x86) Users Windows Container Host ContainerData

89 STORAGE VOLUME MAPPING EXAMPLE License.txt PerfLogs Program Files Program Files (x86) Users Windows Container Host ContainerData

90 VOLUME MAPPING License.txt PerfLogs Program Files Program Files (x86) Users Windows License.txt PerfLogs Program Files Program Files (x86) Users Windows data Container Host ContainerData

91 VOLUME MAPPING License.txt PerfLogs Program Files Program Files (x86) Users Windows License.txt PerfLogs Program Files Program Files (x86) Users Windows data Container Host ContainerData

92 AUTOMATION AND MANAGEMENT

93 COMPOSITION AND ORCHESTRATION Applications typically comprised of multiple containers Containers typically hosted across a cluster of nodes Orchestration tooling automates this

94 COMPOSITION AND ORCHESTRATION Docker Compose Define application as separate containers Manage different containers as a unit Scale parts of application as needed Docker Swarm Aggregate container hosts Supports tagging, affinity/anti-affinity

95 COMPOSITION AND ORCHESTRATION Azure Service Fabric Microservice and orchestration platform Build applications as containers and/or microservices Available on Windows & Linux Built-in cross-container communication Web based management UI Available On-Prem, Azure or other Clouds

96 COMPOSITION AND ORCHESTRATION Kubernetes Open source project started by Google Windows support being added though community partnership spear headed by Apprenda

97 COMPOSITION AND ORCHESTRATION Mesos/Mesosphere + Marathon Aggregates container hosts Web based UI Service Launch and Discovery

98 MANAGEMENT AND MONITORING TOOLS Docker Datacenter Commercially Supported Docker Engines Compute

99 MANAGEMENT AND MONITORING TOOLS Docker Datacenter

100 MICROSOFT OPERATIONS MANAGEMENT SUITE Monitoring Solution for Docker Syslog events Performance metrics Container data

101 BUILD ONCE, RUN ANYWHERE using System; class Program { static void Main() { } }

102 APPLICATIONS IN CONTAINER

103 STATELESS APPLICATIONS Web sites & application workers Batch processing Continuous Integration workflows

104 IMAGE2DOCKER Convert WIM/VHD to Dockerfile Written in PowerShell Open Source Discover Find Installed Programs Show Roles & Features IIS Sites & Configurations Common applications SQL, Apache Generate Dockerfile

105 TAKING IT FURTHER Practice your skills, build out infrastructure Deploy frequently Automate everything Plan to centralize configuration and logging Scale it out Deploy more containers to meet demand Test and look for issues that would block scalability Learn orchestration platforms & load balancing techniques

106 STATEFUL APPLICATIONS Persistent Data Move files out of the container Store on host folder or named volume Runtime State Move out of process Distributed caches Key/value storage Message queues SQL 0MDB

107 STATEFUL APPLICATIONS Benefits Deploy all applications tiers anywhere Build as many test deployments as you need Continuous integration & unit tests Match customer environment Move your container across hypervisors, datacenters, cloud Challenges Doesn t imply scalability Scale up vs Scale Out Scale out requires coordination & synchronization

108 GREAT FOR VMS Hosting low level network & identity services DHCP, Domain controllers Leveraging hardware or driver access Print, GPU Depending on Active Directory for configuration No object in AD Group Policy Running desktop applications Remote Desktop & VDI

109 SOME WINDOWS SERVER NEWS Container-optimized Nano Server with.net Core 2.0 Windows Subsystem for Linux (WSL) on Windows Server Linux Containers on Windows Container Hosts Container Storage Container Orchestration

110 Dutch bank IBAN: NL87INGB HOUSE OF TAILS 70 dogs!!! Safety, food, water, health, blankets, shade, love, fun $15 = 1 month food Donation box near registration area and participate in the raffle for huge rewards!

111

112

113 BREAKING DOWN LARGER APPLICATIONS Identify seams Iterate Prioritize Decompose

114 BREAKING DOWN LARGER APPLICATIONS Development Makes development teams faster Decoupled development, less coordination Freedom to use the best tools for the job Shorter builds & faster deployment Operations Deploy faster with automation Reliable and deterministic Clear list of running containers Less risk, less downtime Containers deliver consistency through dev, test, and production

115 OUR JOURNEY TO NANO SERVER DOCKER CI What is it? Part of the Docker Project For Every Code Submission Compiles the change for both Linux and Windows Uses Jenkins to execute tests against Linux and Windows hosts Windows Server Core Image Azure D3 VM (4 cores, 14GB RAM, 200GB SSD) Takes ~31 min and 21 sec to run Nano Image Early prototype (not checked in yet) Azure D2 VM (2 cores, 7GB, 100GB SSD) Takes ~21min and 39 sec to run Benefit of Nano 30% Reduction in Execution Time! 50% Reduction in Cost

116 NETWORKING & SERVICE DISCOVERY NAT External: Host IP TCP:80 TCP:80 Internal: /12 DNS: musicstore_test sql

117 DEPLOYMENT SIMPLIFICATION Everywhere

118 ACTIVE DIRECTORY IDENTITY FOR CONTAINERS Ready for Enterprise Applications No infrastructure changes to Active Directory required Leverages Group Managed Service Accounts Enables containerized applications to authenticate with SQL Server, file shares, and more Simple and Secure Identity specified at container start time Credentials are not stored within the container image No changes required to move across environments i.e. dev, test, prod, cloud

119 ACTIVE DIRECTORY IDENTITY FOR CONTAINERS 1) Use default accounts for services & tasks (LocalSystem, Network Service) Credential Spec Service: IIS User: LocalSystem Domain\MyWebApp1$ DefaultAccount: Domain\MyWebApp1$ 2) Provide default service account to use when starting container 3) Container connects using service account

120 SECURE DEPLOYMENT AND AUDITING

121 DEPLOYING IIS & SQL WITH DOCKER-COMPOSE version: '2' services: musicstore_test: build: context:. dockerfile: Dockerfile.iis ports: - "80:80" sql: image: microsoft/mssql-server-2014-express-windows