Implementing Container Application Platforms with Cisco ACI

Transcription

1

2 BRKDCN-2627 Implementing Container Application Platforms with Cisco ACI Andres Vega Product Manager, Engineering

3 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkdcn Cisco and/or its affiliates. All rights reserved. Cisco Public

4 Agenda Introduction ACI Support for Container Application Platforms OpenShift Integration Cloud Foundry Integration Demo Conclusion

5 Openstack node.js OPEN SOURCE IS THE SOURCE OF TECHNOLOGICAL INNOVATION Linux Kernel - Chris Wright, CTO Red Hat 1M+ projects nginx Kubernetes Apache Project KVM aptomi Spinnaker SPIFFE TensorFlow 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

6 Application Architectural Evolution Service Autonomous Loosely-coupled Microservice Single Purpose Stateless Independently Scalable Automated f() Function Single Action Event Sourced Ephemeral BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 6

7 Pervasive Analytics Pervasive Security Native Support for Container Application Platforms Application Container Orchestration Docker Kubernetes Openshift Pivotal Cloud Foundry Mesosphere Opflex CNI Fast, Secure and Scalable Networking Intent based Infrastructure Automation Cisco ACI Anywhere Physical Virtual Private Any Cloud BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 7

8 Foundational Technologies BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 8

9 Application Centric Infrastructure Any Application Any hypervisor Build to and support open systems and standards Distributed network functions for traffic and service optimization Policy consistency provides for containers running reliably and securely Ease of deploying, scaling and managing BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 9

10 Docker provides a lightweight portable format to ship application images to different environments. Packing up all dependencies at both system and application level, but lacks features for supporting n-tier applications Kubernetes is a free and open source project. It does scheduling, orchestration and runs Docker containers at scale for production workloads, but does not address enterprise security requirements. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 10

11 Native Platform Integration on ACI BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 11

12 Why ACI and Application Container Platforms Turnkey solution for node and container connectivity Flexible policy: Native platform policy API and ACI policies Hardware-accelerated: Integrated load balancing Fast, easy, secure and scalable networking for your Application Container Platform Visibility: Live statistics in APIC per container and health metrics Enhanced Multitenancy and unified networking for containers, VMs, bare metal BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 OpenShift BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 13

14 OpenShift is a enterprise grade feature set layer on top of Docker and Kubernetes, that makes it accessible and easy for the developer to create applications, and a platform for operators that simplifies deployments of containers for both development and production workloads. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 14

15 OpenShift Container Networking and Network Security user request Openshift SDN configures an overlay using Open vswitch (OVS). ROUTER (HAproxy) Network Isolation is performed by use of and in between VXLAN segments. Pods can be assigned to different project networks. All incoming HTTP/S traffic is proxied through a Router Pod. OVS OVS OVS BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 15

16 OpenShift Networking and Network Security with ACI ACI Policy user request APIC Network Policy API Policy applied to OpenShift objects kind: NetworkPolicy apiversion: extensions/v1beta1 metadata: name: allow-orange-to-blue-same-ns spec: podselector: matchlabels: type: blue ingress: - from: - podselector: matchlabels: type: red Network policies supported using standard upstream format but enforced through OpFlex / OVS using APIC Host Protection Profiles. Openshift apps can be moved without modification to/from ACI and non-aci environments. Embedded fabric and virtual switch load balancing: PBR in fabric for external service load balancing OVS used for internal service load balancing OVS OpFlex OVS OpFlex OVS OpFlex VMM Domain for Openshift: Stats per namespace, deployment, service, pod Physical to container correlation BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 16

17 Support for Network Policy in ACI namespace-a Specification of how selections of pods are allowed to communicate with each other and other network endpoints. Network namespace isolation using defined labels directional: allowed ingress pod-to-pod traffic filters traffic from pods in other projects can specify protocol and ports (e.g. tcp/80) In Openshift: Project admin controlled. Capabilities in Openshift moving forward: (automated multitenant network policy, multitenant isolation as default, configure/edit/view policies in UI) namespace-b Policy applied to namespace: namespace-a kind: NetworkPolicy apiversion: extensions/v1beta1 metadata: name: allow-orange-to-blue-same-ns spec: podselector: matchlabels: type: blue ingress: - from: - podselector: matchlabels: type: red BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 17

18 Dual level Policy Enforcement by ACI Native API Default deny all traffic Both Kubernetes Network Policy and ACI Contracts are enforced in the Linux kernel of every server node that containers run on. apiversion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny spec: podselector: {} policytypes: - Ingress - Egress Containers are mapped to EPGs and contracts between EPGs are also enforced on all switches in the fabric where applicable. Both policy mechanisms can be used in conjunction. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 18

19 Mapping Network Policy and EPGs Cluster Isolation Namespace Isolation Deployment Isolation Single EPG for entire cluster. (Default behavior) No need for any internal contracts. Each namespace is mapped to its own EPG. Contracts for inter-namespace traffic. Each deployment mapped to an EPG Contracts tightly control service traffic Key Map EP G NetworkPolicy Contract BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 Architecture Integration specific Updated Platform Upstream GUI + Stats APIC ACI Container Ctrl (ACC) VMM + APIC Model atomic-openshift-node CNI API Server Host Agent (HA) Opflex Agent (OA) Proxy Etcd OVS Datapath Master Node Leaf BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 20

21 Install Procedure 1. Install provisioning tool (provided as RPM and DEB packages). 2. Provision the fabric for OpenShift and save deployment YAML file. 3. Install OpenShift. 4. Apply the deployment YAML file to use ACI-Openshift CNI. 5. Update Openshift Router to use ACI fabric. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 21

22 Installation Prerequisites Before installing the OpenShift integration, the following is assumed: A working ACI fabric install on a version 3.1 or later An existing configured Vmware ESXi VMM Domain An L3 Routed Outside, along with an L3 External Network that serves as external access. Any required route reflector configuration for fabric. A next-hop router connected to the l3 external network capable of doing SNAT and which can be configured with the required routes. For the full installation guide please refer to: Cisco ACI and Kubernetes Integration on Cisco.com: BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 22

23 (1/2) Installation: Pre-provisioning 1. Install the ACI containers provisioning tool (available from Cisco.com as RPM and DEB Packages): 2. Create a sample configuration file to be edited: acc_provision --sample > aci-containers-config.yaml 3. Push to the APIC for the fabric to be provisioned: acc_provision --flavor=openshift-3.6 -c aci-containers-config.yaml -o acicontainers.yaml -a -u [apic username] -p [apic password] Note: This will generate the file aci-containers.yaml that you will use after installing OpenShift. It also create files user- [system id].key and user-[system id].crt that contain the certificate used to connect to APIC. The file contents are security-sensitive. Save these files as you ll need them if you need to change the configuration later and want to avoid disrupting a running cluster because of a key change. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 23

24 Install Procedure Install OpenShift Install Openshift using ansible as in [1], with the following exceptions: 1. Enable CNI, but DO NOT enable Redhat's CNI plugin 2. Open UDP port 8472 on the server s firewall on all nodes [1] BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 24

25 Install Procedure Deploy YAML file Deploy the CNI plugin with the following command on the OpenShift master node: oc apply -f aci-containers.yaml BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 Install Process Fix Openshift Router 1. Remove old router oc delete svc router oc delete dc router 2. Create container networking router oc adm router --service-account=router --host-network=false 3. Expose router service externally oc patch svc router -p '{"spec":{"type": "LoadBalancer"}}' BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 26

27 Troubleshooting - Cluster report Acikubectl command to override EPG mapping and debug the system. --help presents command help file [root@kube-master ~]# acikubectl This tool provides a simple way to manage Kubernetes objects and annotations for the ACI Containers Controller. This offers a simple way to manage the ACI policy for your containers. Usage: acikubectl [command] Available Commands: debug Commands to help diagnose problems with ACI containers get Get a value help Help about any command set Set a value Export Cluster Report Logs acikubectl debug cluster-report -o cluster-report.tar.gz BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 Load-balanced Traffic Logical Path ACI L3Out Interface L3Ext /0 Node1 IP Service Contract PBR Service Graph Node2 IP L3Ext /32 NodeN IP BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 28

29 Cloud Foundry BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 29

30 Demo

31 Cloud Foundry is an open source application development plaform that combines CF Application Runtime and CF Container Runtime technologies to run applications of any language or framework. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 CF Container Network and Network Security Apps must route via GoRouter no source identify is preserved GoRouter represents a performance bottleneck and loss of source identity. ASGs only enforced on egress. Container-to-Container network policies are only enforced in the overlay network. Isolation segments are not applied within the overlay network. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 32

33 ACI CF Container Network and Network Security with ACI External traffic SNAT-ed traffic Container to Container traffic Every App gets an IP LB for East-West in Host, North-South performed in fabric with option to bypass GoRouter. Policy control of ingress traffic by enforcement of ACI contracts at ingress. Preservation of source identity with SNAT address per App for outgoing traffic. Security policies enforced end-to-end for C-to-C, Container to VMs/Baremetal by combination of PCF Network Policy and ACI Policies. Enforcement of Isolation Segments as a network construct in overlay. Flexible mapping of Orgs, Spaces and Apps to ACI EPGs for added security. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 33

34 Architecture BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 34

35 EPG Mapping BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 35

36 Installation Prerequisites Before installing the Cloud Foundry integration, the following is assumed: A working ACI fabric install on a version 3.1 or later An existing configured Vmware ESXi VMM Domain An L3 Routed Outside, along with an L3 External Network that serves as external access. Any required route reflector configuration for fabric. A next-hop router connected to the l3 external network capable of doing SNAT and which can be configured with the required routes. This is a Beta feature in 3.1. For assistance please contact the Cisco Technical Assistance Center. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 36

37 (1/2) Installation: Pre-provisioning 1. Install the ACI containers provisioning tool: 2. Create a sample configuration file to be edited: acc_provision --sample > cf0-prov-config.yml 3. Push to the APIC for the fabric to be provisioned: acc_provision -c mycf0-prov-config.yaml -o mycf0-vars.yaml -f cloudfoundry-1.0 -a Note: This will generate the file aci-containers.yaml that you will use after installing OpenShift. It also create files user- [system id].key and user-[system id].crt that contain the certificate used to connect to APIC. The file contents are security-sensitive. Save these files as you ll need them if you need to change the configuration later and want to avoid disrupting a running cluster because of a key change. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 Install Procedure Deploy CF with ACI Add-ons 1. After deploying Bosh Director, and having created a cloud config file, update the cloud config file using the following command: bosh update-cloud-config \ mycf0-cloud-config.yml \ -o manifest-generation/cloud_config_ops.yml -l mycf0-vars.yaml 2. Upload the required stemcell to BOSH director: export STEMCELL_VERSION=$(bosh int cf-deployment/cf-deployment.yml --path /stemcells/alias=default/version) bosh upload-stemcell 3. Upload the ACI add-ons BOSH release file to Director: bosh upload-release release/aci-containers-release beta1.tar.gz BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 38

39 Install Procedure Deploy CF with ACI Add-ons 4. Choose a DNS name (system domain) for your deployment and ensure that this name resolves to the reserved IP addresses you chose for GoRouter (e.g and ). Also ensure that wildcard DNS resolution is allowed. That is, if your system domain is mycf0.fab15.local, then all names like *.mycf0.fab15.local should resolve to the GoRouter s address. 5. Create a cf-deployment operations file, router-static-cf.yml to assign static address to the GoRoute type: replace path: /instance_groups/name=router/networks/name=default/static_ips? # Replace the addresses below with IPs reserved for GoRouter value: [" ", " "] BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 39

40 Install Procedure Deploy CF with ACI Add-ons 6. Deploy CloudFoundry (remember to replace <your-system-domain>). bosh deploy -d cf cf-deployment/cf-deployment.yml \ -o manifest-generation/cf_ops.yml \ --vars-store=mycf0-vars-store.yml \ -l mycf0-vars.yaml \ -v system_domain=<your-system-domain> 7. Verify that CloudFoundry has been deployed successfully. cf login --skip-ssl-validation -a -u admin -p $(bosh int mycf0-vars-store.yml --path /cf_admin_password) Expected output to look like: API endpoint: Authenticating... OK BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 40

41 Roadmap and Futures BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 41

42 Supported Container Application Platforms Baremetal ESXi KVM Open source Kubernetes Openshift Future Future Pivotal Cloud Foundry 3.1 Beta Future Swarm Future 200 Future Mesosphere Future 400 Future BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 42

43 Roadmap Platform Kubernetes 1.6 Support Features Containers on bare-metal servers PBR and distributed load balancing OVS / OpFlex integration. Platform Pivotal Cloud Foundry Kubernetes Openshift 3.7 ACI 3.0 Today Q2CY18 (F release) Future Platform Openshift 3.6 Kubernetes 1.7 Features Multipod Containers on ESXi Platform Docker EE Mesosphere Feature Multisite BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 43

44 Projects of Interest Istio SPIFFE Open Policy Agent (OPA) Aptomi BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 44

45 Conclusion ACI allows direct access to the ACI policy model, so that Container Orchestration Platforms they can participate as first-class citizens within an ACI fabric. Allow seamless interconnection of containers, VMs, and physical devices on an ACI fabric. Support native policy semantics, so that a container application that is specified using Kubernetes NetworkPolicy will work correctly out of the box. Leverage fabric resources forwarding and policy capabilities to offload and accelarate distributed network functions for scale, performance and density. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 45

46 Demo

47 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkdcn Cisco and/or its affiliates. All rights reserved. Cisco Public

48 Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at Cisco and/or its affiliates. All rights reserved. Cisco Public

49 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 49

50 Thank you

51