So, I have all these containers! Now what?

Size: px

Start display at page:

Download "So, I have all these containers! Now what?"

Dennis Wade
5 years ago
Views:

1 So, I have all these containers! Now what?

2 Image by Connie Zhou

3 Developer View job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = // Number of tasks }

4 Developer View

5 Binary Developer View What just happened? Config file webbrowsers browsers web borgcfg BorgMaster UIshard shard BorgMaster UI BorgMaster UI shard BorgMaster UIshard shard BorgMaster UI persistent store (Paxos) Scheduler scheduler linkshard shard link link shard link shard link shard Borglet Borglet Borglet Borglet

6 Image by Connie Zhou

7 Developer View

8 Failures task-eviction rates and causes

9 Efficiency Advanced binpacking algorithms Experimental placement of production VM workload, July 2014 available resources one machine stranded resources

10 Efficiency Multiple applications per machine CPI^2 paper, EuroSys 2013 tasks per machine

11 Efficiency Resource reclamation limit: amount of resource requested potentially reusable resources reservation: estimate of future usage usage: actual resource consumption time

12 Observations: 1. If your developers are spending time thinking about individual machines, you re operating at too low-level of an abstraction. You want to operate at the level of applications 2. We get efficiency by: a. sharing resources b. reclaiming unused allocations 3. s make users more productive (Borg paper) Images by Connie Zhou

13 Guestbook App MySQL Client php php php Python memcached

14 s

15 Old Way: Shared Machines No isolation No namespacing Common libs Highly coupled apps and OS app app app app libs kernel

16 Old Way: Virtual Machines Some isolation Inefficient Still highly coupled to the guest OS Hard to manage app app libs libs kernel kernel app app libs libs kernel kernel

17 New Way: s app app libs libs app app libs libs kernel

18 Docker Example PHP & Apache Memcached libs libs MySQL libs libs Debian Server Docker Engine

19 Why containers? Performance Repeatability Quality of service Accounting Portability A fundamentally different way of managing applications Images by Connie Zhou

20 Demo

21 Kubernetes

22 Kubernetes Greek for Helmsman ; also the root of the word Governor Orchestrator for Docker containers Supports multi-cloud environments Inspired and informed by Google s experiences and internal systems Open source, written in Go Manage applications, not machines

23 Concepts Intro Label Service Replication Controller Volume Node

24 Kubernetes Cluster Kubernetes Master Replication Controller Scheduler API Server Kubelet Kubelet Kubelet Proxy Proxy Proxy Kubernetes Node Kubernetes Node Kubernetes Node

25 Cluster Options From Laptop to high-availability multi-node cluster Hosted or self managed On-Premise or Cloud Bare Metal or Virtual Machines Many options, See Matrix for details Kubernetes Cluster Matrix:

26 s Consumers The atom of scheduling for containers Application specific logical host Ephemeral can die and be replaced Web Server Single container pods can be created directly from a container image Volume

27 s Can be used to group containers & shared volumes Github Consumers s are tightly coupled Shared namespace Shared network IP and port namespace Git Synchronizer Node.js App Ephemeral s in pods live and die together Think in terms of services that you usually run on the same machine Volume

28 Volume Bound to the that encloses it Look like Directories to s What and where they are determined by Volume Type Many Volume Type options EmptyDir Lives with the pod Can be backed by Memory (tmpfs)

29 Volume Bound to the that encloses it Look like Directories to s What and where they are determined by Volume Type Many Volume Type options EmptyDir HostPath Maps to directory on host Use with caution /<rootdir> /etc --/usr --/var --/log

30 Volume Bound to the that encloses it Look like Directories to s What and where they are determined by Volume Type Many Volume Type options EmptyDir HostPath nfs (and similar services) NFS

31 Volume Bound to the that encloses it Look like Directories to s What and where they are determined by Volume Type Many Volume Type options EmptyDir HostPath nfs (and similar services) Cloud Provider Persistent Block Storage

32 Sidecar Pattern Github Consumers Git Synchronizer Node.js App Volume

33 Ambassador Pattern Consumers Redis Shards PHP App Redis Proxy localhost

34 Adapter Pattern Monitoring System Redis localhost OR Redis Exporter Volume

35 Labels Dashboard Dashboard show: type = FE type = FE frontend frontend type = FE show: version = v2 version = v2 version = v2 Behavior Benefits Metadata with semantic meaning Membership identifier The only Grouping Mechanism Allow for intent of many users (e.g. dashboards) Build higher level systems Queryable by Selectors

36 Replication Controllers Replication Replication Controller Controller #pods = 2 version = v1 version= v1 frontend frontend version = v1 version = v2 Replication Controller #pods = 1 version = v2 show: version = v2 show: version = v2 Behavior Benefits Keeps s running Gives direct control of #s Grouped by Label Selector Recreates s, maintains desired state Fine-grained control for scaling Standard grouping semantics

37 Replication Controller Replication Controllers Canonical example of control loops Have one job: ensure N copies of a pod if too few, start new ones if too many, kill some group == selector Replicated pods are fungible Replication Controller - Name = nifty-rc Selector = { App : Nifty } Template = {... } NumReplicas = 4 Start 1 more How many? How many? No implied order or identity 3 OK API Server 4

38 Liveness Process Level: Kubelet checks with Docker that is running App Level: User defined health checks: HTTP Health checks (Kubelet calls a Web Hook) Exec (Kubelet runs command in container) TCP Socket (Kubelet attempts to open a socket to the container) Image: IStockPhoto

Services A group of pods that act as one == Service group == selector Client Defines access policy only load balanced for now Gets a stable virtual IP and port Portal (VIP)

39 Services A group of pods that act as one == Service group == selector Client Defines access policy only load balanced for now Gets a stable virtual IP and port Portal (VIP) called the service portal also a DNS name VIP is captured by kube-proxy watches the service constituency updates when backends change Hide complexity - ideal for non-native apps

40 Canary Example Replication Replication Controller Controller version = v1 #pods = 2 frontend frontend version= v1 version = v1 version = v2 type = FE type = FE type = FE show: version = v2 Replication Replication Controller Controller version = v2 #pods = 1 show: version = v2 Service Service VIP Label selectors: Label selector: version type = FE = 1.0 type = Frontend

41 Mapping to Kubernetes MySQL php php python php Client memcache d

42 Developer View (Replication Controller) spec: containers: - name: php-guestbook image: php-guestbook:europython resources: limits: memory: "128Mi" cpu: "500m" ports: - containerport: 80 protocol: TCP replicas:

43 Scheduling Capabilities Predicate based Currently Selection Node Capacity (based on requested resource limits) Prioritisation Nodes that match all predicates are ranked Priority for Node whose already-running pods consume the least resources To Come Resource aware scheduling

44 Kubernetes Status Kubernetes 1.0 as of mid July Formerly announced at OSCON this week Open sourced in June, 2014 won the BlackDuck rookie of the year award Google launched Google Engine (GKE) hosted Kubernetes Roadmap:

45 Google Engine (Beta) Managed Kubernetes (Kubernetes v1) Manages Kubernetes master uptime Manages Updates Cluster Resize via Managed Instance Groups Centralised Logging Google Cloud VPN support

46 Demo - Clusters and resizing

47 Visualizing Kubernetes $ kubectl proxy --www=k8s-visualizer/ Kubelet AuthN APIs Scheduling Master Scheduler REST (pods, services, controllers) Replication Controller Proxy Kubelet Proxy Kubelet Proxy Node3 Node3 Node1

Engine Cluster Scaling Managed Instance Group (MIG) Grouping of identical instances Instances are cluster nodes Nodes Created from Template Includes Credentials and Tokens required to auth to the

48 Engine Cluster Scaling Managed Instance Group (MIG) Grouping of identical instances Instances are cluster nodes Nodes Created from Template Includes Credentials and Tokens required to auth to the Master MIG can be manually scaled New nodes request to be added to cluster Will eventually facilitate node level autoscale monitor Instance Group Manager create/ destroy VMs Instance Template Managed Instance Group

49 Engine Cluster Creation Google Developers Console Google Deployment Manager HashiCorp Terraform resource "google_container_cluster" "primary" { name = europython-gke zone = europe-west1-c initial_node_count = 3 node_config { machine-type = n1-standard-4 } master_auth { username = bilbao password = notverysecret } }

50 Demo - Visualization

Frequently Asked Questions Q. How does Kubernetes handle secrets? docs/secrets.md Q. How will Kubernetes scale pods? docs/proposals/autoscaling.

51 Frequently Asked Questions Q. How does Kubernetes handle secrets? docs/secrets.md Q. How will Kubernetes scale pods? docs/proposals/autoscaling.md (Based on Traffic, predictive analysis or arbitrary data) Q. How will Kubernetes scale nodes? (Likely based on resources, signals from scheduler and pending queue) Q. How can we make Kubernetes highly available docs/availability.md

52 Kubernetes is Open Source We want your help! irc.freenode.net

53 Also tweet questions Or find me at the Google Booth today Questions

Microservices. Chaos Kontrolle mit Kubernetes. Robert Kubis - Developer Advocate,

Microservices. Chaos Kontrolle mit Kubernetes. Robert Kubis - Developer Advocate, Microservices Chaos Kontrolle mit Kubernetes Robert Kubis - Developer Advocate, Google @hostirosti About me Robert Kubis Developer Advocate Google Cloud Platform London, UK hostirosti github.com/hostirosti