Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Size: px

Start display at page:

Download "Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme"

Roxanne Melton
5 years ago
Views:

1 CNA2006BE Deep Dive: Architecting Container Services with VMware & Pivotal Developer- Ready Infrastructure Merlin Glynn (VMware) Ramiro Salas (Pivotal) VMworld 2017 Content: Not for publication #VMworld #CNA2006BE

2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. 2

3 Agenda 1 2 Pivotal Cloud Foundry 101 Why do my Developers want it? Kubernetes 101 Why do my Developers want it? 3 Ops: Architecture for Containers Ops: Network & Security Controls 5 Ops: Monitoring & Logging 6 Ops: Platform as Code{} 7 Ops: +PKS 3

4 Pivotal Cloud Foundry 101 Why do my Developers want It?

5 Pivotal Cloud Foundry 101 Developer `cf push` war Here is my source code Run it on the cloud for me I do not care how Root FS Build Pack Staging Drop war let A I URL Request: myapp.foo.com *.foo.com = NSX Edge Vip myapp.foo.com NSX Edge LB Pool Members Routing Routing Routing Availability Zone 1 Availability Zone 2 Availability Zone 3 A I 5

6 Kubernetes 101 Why do my Developers want It?

7 Kubernetes 101 Developer `kubectl apply f myapp.yml` Master etcd Service: nodeport ingress kube-proxy kube-proxy POD URL Request: myapp.foo.com/k8siscool or Docker Registry Worker K8s Cluster Load Balancer distribution POD Worker 7

8 Architecting for Containers 101

Developer Pivotal Cloud Foundry Application Services or Container Services Application Logging &

9 DRI Architect for Agility Architect the right Abstractions Automate Everything Build for Failure Control Virtual Data Center Platform Operator Automation Agility Day 2 Operations Control BOSH Developer Pivotal Cloud Foundry Application Services or Container Services Application Logging & Monitoring Self Service PKS BOSH powered Kubernetes vsphere NSX vsan Wavefront vrli (Dev) vrli (Ops) vrops vrni

10 Ops: Architecting for Availability & Scale VMworld 2017 Content: Not for publication

11 vsphere Fundamentals for Platform Operator Architecting for Availability & Scale Virtual Data Center Org App App Developer Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 AZ2 AZ3 loggre gator Ops Manager (OVA) BOSH 11

12 Physical Fault Domains Cluster Design Best Practices Enable vsphere HA Enable & Tune BOSH HealthMonitor Resurrection Platform Operator Virtual Data Center Org Developer go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator vsphere HA App App Space BOSH Hlth Cell_0 vsphere Cell_1 HA Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 BOSH Agent(s) Monitor AZ1 AZ2 AZ3 loggre gator Ops Manager (OVA) BOSH 12

Physical Fault Domains Cluster Design Best Practices Enable vsphere HA Enable & Tune BOSH HealthMonitor Resurrection Plan For Singletons Externalize DR (vdp, Image, Snapshot, pgdump) Platform

13 Physical Fault Domains Cluster Design Best Practices Enable vsphere HA Enable & Tune BOSH HealthMonitor Resurrection Plan For Singletons Externalize DR (vdp, Image, Snapshot, pgdump) Platform Operator Virtual Data Center BOSH Agent(s) Org App App Developer Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain webdav loggre loggre mysql (blob) mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 DR AZ1 BOSH AZ2 AZ3 loggre gator S3 Compat Storage BlobStore Ops Manager (OVA) DR DR BOSH 13

14 IaaS Multi Tenancy Cluster Design Best Practices Enable vsphere HA Enable & Tune BOSH HealthMonitor Resurrection Plan For Singletons Externalize DR (vdp, Image, Snapshot, pgdump) Platform Operator Virtual Data Center Use s & Scale Clusters as needed CPI Acct 1 Assigned vcenter Perms ACL Pool Limits & Shares Quota AZ1 Foundation 1 Prod Foundation AZ2 Foundation 1 Dev Test UAT Foundation AZ3 Foundation 1 Ops Manager (OVA) C P I C P I CPI Acct 2 Assigned vcenter Perms Pool Limits & Shares AZ1 Foundation 2 AZ2 AZ2 Foundation 2 AZ3 Foundation 2 AZ3 BOSH 14

15 Recovering the Platform BC/DR Best Practices Platform as Code{} Platform Operator BOSH Agent(s) Org App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 BOSH AZ2 AZ3 loggre gator S3 Compat Storage BlobStore Ops Manager (OVA) BOSH 15

16 Recovering the Platform BC/DR Best Practices Platform as Code{} Backup Services for Platform Persistent Data Backup Services for App Service Persistent Data Don t Forget External App Data not managed by Platform Operator BOSH Agent(s) MySql Service Tile Org mysql mysql mysql App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 BOSH AZ2 AZ3 loggre gator Backup Job S3 Compat Storage BlobStore Ops Manager (OVA) BOSH 16

17 Recovering the Platform BC/DR Best Practices Platform as Code{} Backup Services for Platform Persistent Data Backup Services for App Service Persistent Data Don t Forget External App Data not managed by VMotion (Yes) SVMotion (NO) Platform Operator BOSH Agent(s) * Org App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator vmdk * Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 BOSH AZ2 AZ3 loggre gator Backup Job S3 Compat Storage BlobStore Ops Manager (OVA) BOSH 17

18 Multi-Site Platforms BC/DR Best Practices Business Continuity w/ Multi Site GSLB Platform Operator {} VMworld 2017 Health Checks GSLB Health Checks NSX Edge LTM NSX Edge LTM Common Service Mesh Data Content: Not for publication AZ2 AZ3 Ops Manager (OVA) BOSH 18

19 Analytics Logging Operations Automation Monitoring Security Container Registry etcd VMworld 2017 vsphere master VMware PKS worker Kubernetes on BOSH (Kubo) NSX etcd BOSH (PKS) master worker vsan GCP Service Broker Content: Not for publication 19

20 What about PKS? BOSH Deploys KUBO Same BOSH Availability Zone Constructs are available Spread Core K8S Jobs across BOSH Availability Zones Master ETCD Workers Multi Site can be GSLB in much the same way as BOSH Makes Kubernetes Day 1 & Day 2 easy. Does NOT require 20

21 Architecting the Platform BC/DR Best Practices Platform as Code{} Backup Services for Platform Persistent Data Backup Services for App Service Persistent Data Business Continuity w/ Multi Site Architectural Resource(s) VMworld 2017 VMware VVD (Validated Design) Pivotal Lite Reference Architecture DEVELOPER-READY INFRASTRUCTURE Deliver innovation faster to customers Link(s) In Progress Cluster Design Best Practices Enable vsphere HA Enable & Tune BOSH HealthMonitor Resurrection Plan For Singletons Use s & Scale Clusters as needed Content: Not for publication VMotion (Yes) SVMotion (NO) 21

22 Ops: Network & Security Controls

Network Fundamentals for Network Design Best Practices Get Wildcard Certs & DNS Approved {} http://myapp.default-apps.foo.

23 Network Fundamentals for Network Design Best Practices Get Wildcard Certs & DNS Approved {} DNS: *.sys.pcf.foo.com *.default-apps.foo.com Ops Mgr VMworld 2017 () LS: Infra BOSH CF Control Plane GO RTR NSX Edge /26 /22 APPS SSH Diego TCP TCP RTR TCP SSH APPS Brain * * * LS: OSPF LS: ERT Internal Apps Elastic Runtime Cell Cell Single Foundation app.public-apps.foo.com CF ASG A I A I LS: Services # LS: Services # LS: Services # * * * External Services Mysql Rabbit Logical Routing (DLR) Content: Not for publication /24(s) IaaS: vsphere Security Zone A (Hub) 23

24 Network Security & Controls Network Design Best Practices Get Wildcard Certs & DNS Approved Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs On Demand: Developer trigger VM provision Pre-Provisioned: Ops triggers VM provision DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime CF ASG A I Cell A I Cell Single Foundation LS: Services # LS: Services # LS: Services # * * * External Services Mysql Rabbit Logical Routing (DLR) /24(s) IaaS: vsphere Security Zone A (Hub) 24

25 Network Security & Controls Network Design Best Practices Get Wildcard Certs & DNS Approved Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs On Demand: Developer trigger VM provision Pre-Provisioned: Ops triggers VM provision Use Application Security Groups (ASGs), App level egress firewall to & external IP ranges DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime CF ASG A I Cell A I Cell Single Foundation LS: Services # LS: Services # LS: Services # * * * External Services Mysql Rabbit Logical Routing (DLR) /24(s) IaaS: vsphere Security Zone A (Hub) 25

Network Security & Controls Network Design Best Practices Get Wildcard Certs & DNS Approved Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs On

26 Network Security & Controls Network Design Best Practices Get Wildcard Certs & DNS Approved Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs On Demand: Developer trigger VM provision Pre-Provisioned: Ops triggers VM provision Use Application Security Groups (ASGs), App level egress firewall to & external IP ranges Use NSX Edge for Load Balancing, SSL Termination, & Perimeter FW ACLs DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime CF ASG A I Cell A I Cell Single Foundation LS: Services # LS: Services # LS: Services # * * * External Services Mysql Rabbit Logical Routing (DLR) /24(s) IaaS: vsphere Security Zone A (Hub) 26

27 Network Security & Controls Network Design Best Practices Get Wildcard Certs & DNS Approved Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs On Demand: Developer trigger VM provision Pre-Provisioned: Ops triggers VM provision Use Application Security Groups (ASGs), App level egress firewall to & external IP ranges Use NSX Edge for Load Balancing, SSL Termination, & Perimeter FW ACLs Use NSX DLR for Org & Space level segmentation Multiple Isolation Segments Isolation segments allow Operators to group Diego cells and attach to multiple Logical Swicthes. DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime ISO CF ASG IaaS: vsphere Security Zone A (Hub) DNS: *.public-apps.foo.com A I Cell A I Cell LS: Services # LS: Services # LS: Services # * * * Single Foundation External Services Mysql Rabbit /24(s) ISO GO RTR Logical Routing (DLR) LS: Isolation_A CELL CELL CELL /22 Isolation Segment Public Apps IaaS: vsphere Security Zone B (Spoke) 27

Network Security & Controls Network Design Best Practices Use NSX Security Groups for dynamic security principals BOSH Integrated NSX (Dynamic Membership) Ingress & Egress Org/Space Specific FW

28 Network Security & Controls Network Design Best Practices Use NSX Security Groups for dynamic security principals BOSH Integrated NSX (Dynamic Membership) Ingress & Egress Org/Space Specific FW Dynamic LB Pool Membership DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime ISO CF ASG DNS: *.public-apps.foo.com A I Cell A I Cell LS: Services # LS: Services # LS: Services # * * * Single Foundation External Services Mysql Rabbit /24(s) ISO GO RTR Logical Routing (DLR) LS: Isolation_A CELL CELL CELL /22 Isolation Segment Public Apps IaaS: vsphere Security Zone A (Hub) IaaS: vsphere Security Zone B (Spoke) BOSH 28

29 Network Security & Controls Network Design Best Practices Use NSX Security Groups for dynamic security principals BOSH Integrated NSX (Dynamic Membership) Ingress & Egress Org/Space Specific FW Dynamic LB Pool Membership Use Distributed Firewall Policy Leverage Integrated Dynamic Security Groups Control East+West from single policy engine Control App to App at the Org/Space level with Isolation Segments DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime ISO CF ASG DNS: *.public-apps.foo.com A I Cell A {} I Cell LS: Services # LS: Services # LS: Services # * * * Single Foundation External Services Mysql Rabbit /24(s) ISO GO RTR Logical Routing (DLR) LS: Isolation_A CELL CELL CELL /22 Isolation Segment Public Apps IaaS: vsphere Security Zone A (Hub) IaaS: vsphere Security Zone B (Spoke) BOSH 29

30 Network Security & Controls Network Design Best Practices Use NSX Security Groups for dynamic security principals BOSH Integrated NSX (Dynamic Membership) Ingress & Egress Org/Space Specific FW Dynamic LB Pool Membership Use Distributed Firewall Policy Leverage Integrated Dynamic Security Groups Control East+West from single policy engine Control App to App at the Org/Space level with Isolation Segments Use RFC 1918 for Repeatability DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr () BOSH CF Control Plane APPS GO RTR SSH Diego TCP TCP TCP SSH APPS Brain RTR * * * LS: OSPF Internal Apps Elastic Runtime ISO CF ASG DNS: *.public-apps.foo.com A I Cell A {} I Cell LS: Services # LS: Services # LS: Services # * * * Single Foundation External Services Mysql Rabbit /24(s) ISO GO RTR Logical Routing (DLR) LS: Isolation_A CELL CELL CELL /22 Isolation Segment Public Apps IaaS: vsphere Security Zone A (Hub) IaaS: vsphere Security Zone B (Spoke) 30

31 () () Network Security & Controls Network Design Best Practices Platform as Code{} to automate Day 1 & Day 2 ops Platform Operator VMworld 2017 DNS: *.sys.pcf.foo.com *.default-apps.foo.com /26 LS: Infra LS: ERT /22 Ops Mgr DNS: *.sys.pcf.foo.com *.default-apps.foo.com NSX Edge /26 LS: Infra LS: ERT /22 Ops Mgr BOSH CF Control Plane BOSH CF Control Plane APPS APPS GO RTR GO RTR SSH SSH NSX Edge Diego Internal Apps Elastic Runtime Diego TCP Brain RTR * * * TCP TCP TCP APPS APPS Brain RTR * * * Single Foundation Internal Apps Elastic Runtime TCP SSH LS: OSPF SSH TCP LS: OSPF CF ASG IaaS: vsphere Security Zone A (Hub) CF ASG Cell Cell Single Foundation A I Cell A I Cell A I A I LS: Services # LS: Services # LS: Services # * * * Mysql Rabbit LS: Services # LS: Services # LS: Services # * * * External Services External Services Mysql Rabbit Logical Routing (DLR) /24(s) Logical Routing (DLR) Content: Not for publication /24(s) IaaS: vsphere Security Zone A (Hub) 31

32 Network Security & Controls Application Security Groups (ASG): Uses iptables in the Diego Cell Server Controls Egress only at the container source level Can control any IP address as the target Operator Declares in the Platform Org Platform Operator cf create-security-group SECURITY-GROUP PATH-TO-RULES-FILE cf create-security-group dev-mssql mssql.json Prod Mssql Prod Mssql AppA AppB AppC Space [ { "protocol": "tcp", "destination": " /24", "ports": " " }, { "protocol": "udp", "destination": " /24", "ports": " " } ]

33 Network Security & Controls Container to Container Networking: Creates and Overlay (VXLAN) Controls ingress & egress between Ais (containers) Uses CNI Today Flannel Tomorrow NSX-T Developer can Declare in CI/CD cf allow-access SOURCE-APP DESTINATION-APP --protocol PROTOCOL --port PORT cf allow-access AppA Appc --protocol TCP --port 443 Org Developer AppA AppB AppC Space

components need to route to each other Services Deployed on an Overlay Network NSX-T

on external gateway Load Balance to kube-proxy VMworld 2017 Load Balancer kube proxy EXTERNAL

34 What about PKS? KUBO Networking is less Complex Typically multiple smaller K8s Deployments The core Kubernetes components need to route to each other Services Deployed on an Overlay Network NSX-T Enterprise Security Policy Enterprise Tools & Logging Common Ingress Paths: kube-proxy running on external gateway Load Balance to kube-proxy VMworld 2017 Load Balancer kube proxy EXTERNAL SVC Request Content: Not for publication External Service Gateway Image source: 34

35 Network Security & Controls Network Design Best Practices Use NSX Security Groups for dynamic security principals Use Distributed Firewall Policy Control East+West from single policy engine Control App to App at the Org/Space level with Isolation Segments Use Container to Container Networking to allow developer to define fine grained App level security Use RFC 1918 Repeatability Platform as Code{} to automate Day 1 & Day 2 ops VMworld 2017 DEVELOPER-READY INFRASTRUCTURE Deliver innovation faster to customers Network Design Best Practices Use Multiple NSX Logical Switches & Subnets, 1 per Deployment ( Tile) allow Subnet to Service Level ACLs Use Application Security Groups (ASGs), App level egress firewall to & external IP ranges Use NSX Edge for Load Balancing, SSL Termination, & Perimeter FW ACLs Use NSX DLR for Org & Space level segmentation Content: Not for publication Resource(s) KUBO Git Repo VMware & NSX Design Guide Link(s) Coming Soon 35

36 Ops: Monitoring & Logging

37 Monitoring & Logging Developer Platform Operator I need to keep my apps healthy I need self service to my Apps Log s Virtual Data Center I need to instrument my Apps (APM) I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need to audit 37

38 Monitoring & Logging Developer I need to keep my apps healthy I need self service to my Apps Log s Virtual Data Center I need to instrument my Apps (APM) Developer Log Access Routes `cf logs appa` `cf logs`: streams single app s log events for dev to redirect where needed VMworld 2017 Metrics: app correlating App logs, and container Metrics, ~2week retention vrli: Longer term scalable log storage and indexing, dashboards, & alerts Nozzle Metrics Content: Not for publication vrli 38

Monitoring & Logging Developer I need to keep my apps healthy I need self service to my Apps Log s Virtual Data Center I need to instrument my Apps (APM) App & App execution specific Metrics

39 Monitoring & Logging Developer I need to keep my apps healthy I need self service to my Apps Log s Virtual Data Center I need to instrument my Apps (APM) App & App execution specific Metrics tc_server: jdbc_query_failed `cf create service my-apm-endpoint` custom_app_metric: transaction_response_time Future!!! Platform Operator Agents Added to Buildpacks Exposed to developers via CF Service Broker 39

40 Monitoring & Logging Platform Operator I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need to audit vrops Cloud Foundry Metrics (KPIs) vsphere & NSX Metrics (KPIs) vrops Nozzle 40

Monitoring & Logging Platform Operator Alerts Thresholds I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need

41 Monitoring & Logging Platform Operator Alerts Thresholds I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need to audit Dashboards vrops vrli Cloud Foundry Metrics (KPIs) vsphere & NSX Metrics (KPIs) vsphere & NSX Events CF Platform Events vrops Nozzle Syslog Nozzle 41

42 Monitoring & Logging Platform Operator Alerts Thresholds I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need to audit Dashboards vrops vrli Cloud Foundry Metrics (KPIs) vsphere & NSX Metrics (KPIs) vsphere & NSX Events CF Platform Events vrops Nozzle Syslog Nozzle All App Events 42

Logging Per App Only Sidecar App Logging @ Pod level DaemonSet App Logging @ Cluster level Cluster

43 What about PKS? Platform Operator Developer App Logging System Logging OS & Processes not run in Containers App Logging Per App Only Sidecar App Pod level DaemonSet App Cluster level Cluster Logging POD POD POD POD Daemon Deamon Deamon Set Set Set (PODs) LOGGER vrli vrli vrli Dockerd vrli App Cluster level SyslogD Cluster Logging Not handled in K8s API DOCKERD DOCKERD

44 What about PKS? K8s Monitoring Integration w/ Wavefront by VMware Wavefront Integration can be deployed as containers within the K8s Cluster Proxy Heapster Comprehensive Dashboards SaaS APM for the Developer Cluster KPIs for the Operator Integrated with PKS Platform Operator Developer Image source:

45 What about PKS? vrealize Operations & K8s Operator KPIs Single Pane for SDDC & K8s clusters monitoring vrli Integrated Alert on K8s KPIs Entity Relationship Capacity Planning Integrated with PKS Platform Operator 45

46 Ops: Monitoring & Logging I need to keep the Platform healthy I need to plan capacity I need to watch & Alert on KPIs I need to audit Resource(s) Wavefront: KUBO Integration Blue Medora : vrops MP Blue Medora : vrli Pack Platform Operator VMworld 2017 Link(s) DEVELOPER-READY INFRASTRUCTURE Deliver innovation faster to customers Developer I need to keep my apps healthy I need self service to my Apps Log s Virtual Data Center I need to instrument my Apps (APM) Content: Not for publication

47 Ops: Platform as Code{}

48 BOSH 101 Built for Platform Operators Deploys Complex Distributed Systems Kubo Day 1 & Day 2 Ops Initial Deployment Updates/Patches Maintains Health Platform Operator VMworld 2017 Content: Not for publication 48

49 Day 1 & Day 2 Platform as Code{} Declarative NSX_Config: edge_vip_1:3 nsxmgr_endpoint: nsxmgr.vmware.io lswicth_ert_cidr: /22 Platform Operator YAML Drives NSX-V AZ1 AZ2 AZ3 NSX-V (Edge - Load Balancing Logical Switch Firewall Services) 49

50 Day 1 & Day 2 Platform as Code{} Declarative Ert_config: diego_database_instances:3 diego_brain_instances: 3 diego_cell_instances: 9 Platform Operator YAML Org App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 AZ2 AZ3 Drives NSX-V loggre gator NSX-V (Edge - Load Balancing Logical Switch Firewall Services) Ops Manager (OVA) BOSH 50

51 Day 1 & Day 2 Platform as Code{} Declarative Ert_config: diego_database_instances:3 diego_brain_instances: 3 diego_cell_instances: 12 Platform Operator YAML Drives NSX-V Org App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 AZ2 AZ3 loggre gator Cell_3 Cell_3 Cell_3 NSX-V (Edge - Load Balancing Logical Switch Firewall Services) Ops Manager (OVA) BOSH 51

52 Platform as Code{} Declarative Change Controlled Archived Audited Day 1 & Day 2 Platform Operator Drives NSX-V Org App App Space go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr cc uaa brain cc uaa brain cc uaa brain loggre loggre mysql mysql mysql gator gator Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 AZ1 AZ2 AZ3 loggre gator Cell_3 Cell_3 Cell_3 NSX-V (Edge - Load Balancing Logical Switch Firewall Services) Ops Manager (OVA) BOSH 52

53 Day 1 & Day 2 Declarative Platform Operator Platform as Code{} Change Controlled Archived Audited Repeat Drives NSX-V Scale Repair Recovery Repave Rotate Creds NSX Edge LTM NSX Edge LTM Ops Manager (OVA) BOSH 53

54 Day 1 & Day 2 CVE & Update Patching The New Stack Patch at ANY Layer of the Application Stack Address CVE in minutes/hours versus days/weeks Simply re-stage all apps when any layer is patched Platform as Code{} Developer Platform Operator Org ERT Tile BuildPack Stemcells App App Space Restage Applications Vulnerability in Code{} CVE in Root File System of Container CVE Exec Layer: TC Server CVE on the Container Host OS 54

55 What about PKS? CVE & Update Patching The New Stack Stemcells still there Harbor Scans Images for Vulnerability (Clair) Address CVE in minutes/hours versus days/weeks Platform as Code{} Developer Platform Operator Stemcells Restage Applications CVE FOUND!!! Docker Registry Vulnerability in Code{} CVE in Root File System of Container CVE Exec Layer: TC Server CVE on the Container Host OS BOSH 55

56 What about PKS? Platform Operator vra KUBO Can scale. A lot BOSH allows for a repeatable pattern of K8S Clusters as well. Many Development teams Multiple Security Zones for Applications Multi Cluster HA within a DC CI/CD Pattern similar to Developer Developer A B PKS BOSH VCF

57 Ops: Platform As Code{} Declarative Resource(s) Day 1 & Day 2 Change Controlled Archived Audited Repeat Scale Repair Pivotal NSX + Pipeline Recovery Repave Rotate Creds Pivotal Generic Install & Upgrade pipelines DEVELOPER-READY INFRASTRUCTURE Deliver innovation faster to customers Link(s) CVE & Update Patching The New Stack Patch at ANY Layer of the Application Stack Address CVE in minutes/hours Virtual Data Center versus days/weeks Simply re-stage all apps when any layer is patched Platform as Code{} 57

58 Wrapping It up

59 Developer Ready Infrastructure Solves for DevOps Reqs Platform Operator Automation Day 2 Operations Control BOSH Developer Pivotal Cloud Foundry Application Services or Container Services Application Logging & Monitoring Self Service PKS BOSH powered Kubernetes vsphere NSX vsan Wavefront vrli (Dev) vrli (Ops) vrops vrni

60 Developer Ready VMworld VMworld US Key Focus Description CNA1509BU DRI Developer-Ready Infrastructure from VMware & Pivotal CNA1612BU CNA2006BU & Kubo DRI Use Cases: Deploying real-world workloads on Kubernetes and Pivotal Cloud Foundry Deep Dive: Architecting Container Services with VMware and Pivotal Developer Ready Infrastructure CNA2080BU Kubo Deep Dive: How to Deploy and Operationalize Kubernetes CNA3429BU CNA3430BU MGT2871BU Kubo & vrops, vrli Basics of Kubernetes on BOSH: Run Production-grade Kubernetes on the SDDC Your Enterprise Cloud-Native App Platform: An Introduction to Pivotal Cloud Foundry Bridging the Operations Gap Between the Software-Defined Data Center and Pivotal CF for VMware Deployments NET1523BU & NSX Integrating NSX and Cloud Foundry PAR4411PU DRI Emerging Technologies with VMware and Pivotal - presented jointly by VMware, Pivotal and Special Guest Speakers from Cognizant and WWT 60

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

CNA1509BU Developer-Ready Infrastructure from VMware and Pivotal Merlin Glynn, VMware Ramiro Salas, Pivotal #VMworld #CNA1509BU Disclaimer This presentation may contain product features that are currently