Container Orchestration on Amazon Web Services. Arun

Size: px

Start display at page:

Download "Container Orchestration on Amazon Web Services. Arun"

Stephen Hart
6 years ago
Views:

2 Container Orchestration on Amazon Web Services Arun

3 Docker Workflow

6 Development using Docker Docker Community Edition Docker for Mac/Windows/Linux Monthly edge and quarterly stable releases Native desktop or cloud provider experience

7 Swarm-mode: Initialize docker swarm init --listen-addr <ip>: secret <SECRET>

8 Swarm-mode: Add Worker docker swarm join --secret <SECRET> <manager>:2377

9 Swarm-mode: Add More Workers docker swarm join --secret <SECRET> <manager>:2377

10 Swarm-mode: Primary/Secondary Master docker swarm join --manager --secret <SECRET> --listen-addr <master2>:2377 <master1>:2377

11 Docker for AWS CloudFormation template Integrated with AWS Infrastructure Autoscaling Groups (ASG) Elastic Load Balancer (ELB) Elastic Block Store (EBS)

13 Kubernetes Concepts Pods: collocated group of Docker Node Docker containers that share an IP and storage volume Service: Single, stable name for a set of pods, also acts as LB Label: used to organize and select C1 C2 C3 Pod 1 Pod 2 Service web group of objects Replica Set: manages the lifecycle of port 80 port 80 pods and ensures specified number are running nginx Pod 1 nginx Pod 2

14 Core Concepts: Kubernetes Master Node: Machine or VM in the cluster API Server (pods, services, ) Master: Central control plane, provides unified view of the cluster etcd: distributed key-value store used Scheduler Controller Manager to persist Kubernetes system state Worker: Docker host running kubelet Kubelet Proxy Worker etcd etcd etcd (node agent) and proxy services Runs pods and containers Docker Monitored by systemd (CentOS) or monit (Debian)

15 Kubernetes Cluster Internet kubectl Kubelet Proxy Worker Load Balancer Master Docker API Server (pods, services, ) Scheduler Controller Manager Kubelet Proxy Worker etcd etcd etcd Docker

16 kubectl Controls the Kubernetes cluster manager kubectl get pods or minions kubectl create -f <filename> kubectl update or delete kubectl resize replicas=3 replicaset <name>

17 Kubernetes on AWS Single node cluster minikube Multi-node cluster on AWS kops: github.com/kubernetes/kops kube-aws: github.com/kubernetes-incubator/kube-aws Heptio: github.com/aws-quickstart/quickstart-heptio Google Cloud, Azure, Tectonic, OpenShift

19 Amazon EC2 Container Service Cluster Management Container Orchestration Deep AWS Integration

20 Mapping to EC2 Workloads Instances Services Tasks

22 Deep Integration with AWS Specify IAM role used by the containers in a task Services deploy and scale quickly, easily extensible For example, git push can trigger a deployment using CI tool Scale a service up or down based upon CloudWatch alarms Audit through CloudTrail, can track task/role association Application Load Balancer: Define routing rules based on content

24 Mesos Open source cluster manager Developed at UC Berkeley Provides resource isolation and sharing across distributed applications Run distributed systems on the same pool of nodes Hadoop, Spark, Jenkins, Cluster monitoring Tasks isolated via Linux containers

25 Mesos Architecture

26 ZooKeeper Quorum Frameworks Master Master (standby) Master (standby) Executor Slave Slave Slave Slave Marathon Kubernetes Spark Marathon Aurora Task

27 Frameworks Frameworks are targeted at a use case and domain-specific Master node offers resources to each framework Framework accepts the offer and execute applications Framework has scheduler and executor Scheduler registers with the master for offer Executor launched on slave nodes to run the task Passes a description of the task to run

28 Job 1 Job 2 Job 1 Job 2 Scheduler Scheduler <t1, s1, 2GB, 2CPU, > <t2, s4, 2GB, 4CPU, > 3 2 <s1, 2GB, 2CPU, > <s4, 4GB, 4CPU, > 5 Allocation Module Master 4 <m, t2, s4, 2GB, 4CPU, > <m, t1, s1, 2GB, 2CPU, > 4 1 <s1, 4GB, 4CPU, > <s4, 4GB, 4CPU, > 1 Slave Slave Slave Slave

29 DC/OS on AWS CloudFormation template Basic Easy to get started, minimal setup required Great for simple production deployment, demos and testing Limited customization options Advanced Highly customizable More setup work is required

30 Thanks! Arun github.com/javaee-samples/docker-java/tree/master/slides

Package your Java Application using Docker and Kubernetes. Arun

Package your Java Application using Docker and Kubernetes Arun Gupta, @arungupta Docker Captain Java Champion JavaOne Rock Star (4 years) NetBeans Dream Team Silicon Valley JUG Leader Author Runner Lifelong