Cisco ACI and Pivotal Cloud Foundry Integration 2

Cisco ACI and Pivotal Cloud Foundry Integration New and Changed Information 2 Cisco ACI and Pivotal Cloud Foundry Integration 2 Preparation 2 Pivotal Cloud Foundry Compatibility 2 Preparing for Pivotal Cloud Foundry ACI Integration 2 Deployment 3 Provisioning Cisco ACI to Work with Pivotal Cloud Foundry 3 Deploying PAS with ACI CNI Plug-in Tile 4 Upgrading the ACI CNI Plug-in 6 Removing Cisco ACI Add-ons from Pivotal Cloud Foundry 6 Unprovisioning Pivotal Cloud Foundry from the ACI Fabric 7 Operations 7 Using Cisco ACI-Specific Extensions 7 Collecting Log Files for Support Requests 8

Revised: August 1, 2018 New and Changed Information The following table provides an overview of the significant changes up to this current release. The table does not provide an exhaustive list of all changes or of the new features up to this release. Cisco APIC Release Version Release 3.2(1) Feature Support for Pivotal Cloud Foundry in Cisco ACI Description This release enables the deployment of Pivotal Cloud Foundry in the Cisco ACI fabric. Cisco ACI and Pivotal Cloud Foundry Integration Pivotal Cloud Foundry is a platform as a service (PAAS) that uses Linux containers to deploy and manage applications. It works as an overlay on various infrastructure systems like VMware vsphere and Amazon Web Services (AWS) and operates on the underlying network used by these systems. Beginning with Cisco APIC Release 3.2(1), Pivotal Cloud Foundry is integrated with Cisco Application Centric Infrastructure (ACI). This enables customers to use all Cisco ACI security and policy features with Pivotal Cloud Foundry containers. In the Cisco APIC Release 3.2(1), Cisco ACI integration applies to Pivotal Cloud Foundry deployed on VMware vsphere where the Cisco ACI provides the network fabric for VMware vsphere. This document is a guide to deploying Pivotal Cloud Foundry integrated with Cisco ACI and describes the use of Cisco ACI-specific extensions to Pivotal Cloud Foundry. Preparation Pivotal Cloud Foundry Compatibility Cloud Foundry is compatible with the following software: Cisco APIC Release 3.2(1) Ops Manager 2.1 and Pivotal Application Service (PAS) 2.1.1 Cisco ACI add-ons 0.2.0 Note This document does not include deployment of isolation segments. Preparing for Pivotal Cloud Foundry ACI Integration The following tasks must be completed before you can integrate Pivotal Cloud Foundry with the Cisco ACI. 2

Before you begin It is assumed that you have completed the following tasks: Set up the Cisco ACI fabric to use with a VMware vcenter deployment. See the Cisco ACI and Cisco APIC documentation on Cisco.com. Read and understood the guidelines in the knowledge base article Cisco ACI and OpFlex Connectivity for Orchestrators. Step 3 Step 4 Create a VMware VMM domain in Cisco APIC that uses the desired VMware vcenter data center. Ensure that you have an Attachable Entity Profile (AEP) in Cisco APIC that enables communication on the switch ports that are connected to ESXi hypervisors. Create a VRF that to hold all your endpoints (BOSH Director, Ops Manager, Pivotal Cloud Foundry component VMs, and containers). Create and provision L3Out in Cisco APIC for external communication and associate it with the VRF you created in the previous step. Step 5 Create an external network under the L3Out that allows traffic to all destinations (0.0.0.0/0). Step 6 Step 7 Create a working directory and extract the Cisco ACI add-ons distribution file (dist-generics-cloudfoundryxxxxxx.tgz). Install the acc_provision Debian package. Deployment Provisioning Cisco ACI to Work with Pivotal Cloud Foundry Before you begin Ensure that you have completed the tasks in the section Preparing for Pivotal Cloud Foundry ACI Integration, on page 2 in this guide. Create a provisioning configuration file, using the following example. Change the values in the example to fit your environment. ## Configuration for ACI Fabric aci_config: system_id: mycf0 # Unique ID for this install apic_hosts: # List of APIC hosts to connect to 172.28.184.150 apic_login: 3

username: admin password: myadminpassword vmm_domain: # CloudFoundry VMM domain config encap_type: vxlan # Encap mode: vxlan or vlan mcast_range: # Every VMM must use a distinct range start: 225.20.1.1 end: 225.20.255.255 nested_inside: type: vmware name: fab15vds1 # Your VMware VMM domain name # The following resources must already exist on the APIC, # they are used, but not created by the provisioning tool. aep: esxaep # The AEP for ports/vpcs vrf: # VRF to place the endpoints in name: l3out_1_vrf tenant: common l3out: name: l3out1 # Used for external communication external_networks: - l3out_1_net # Used for external contracts # # Networks used by CloudFoundry # net_config: node_subnet: 10.1.0.1/16 # Subnet for CloudFoundry nodes pod_subnet: 10.2.0.1/16 # Subnet for container IPs extern_dynamic: 150.3.0.1/24 # Subnet for dynamic external IPs extern_static: 150.4.0.1/24 # Subnet for static external IPs node_svc_subnet: 10.5.0.1/24 # Subnet for service graphs service_vlan: 4002 # VLAN used by LoadBalancer services infra_vlan: 4093 # VLAN used by ACI infra In the preceding example, node subnet 10.1.0.0/16 will be used for the BOSH Director virtual machine (VM) and Pivotal Cloud Foundry component VM. Step 3 Configure Cisco APIC and generate a configuration file for cf-deployment. acc_provision -a -u <apic username> -p <apic password> -c mycf0-prov-config.yaml -o mycf0-vars.yaml -f cloudfoundry-1.0 This command configures Cisco APIC for Pivotal Cloud Foundry and generates a file called mycf0-vars.yaml. Make a note of the following values in the file mycf0-vars.yaml: apic_dvs, apic_node_portgroup, apic_node_portgroup, apic_node_subnet, and apic_infra_portgroup. Deploying PAS with ACI CNI Plug-in Tile Create the Ops Manager VM (v2.1) using ovftool or another tool on VMware vsphere. Make sure that this VM's virtual NIC (vnic) is attached to the vsphere portgroup created by acc_provision (apic_node_portgroup in the configuration file produced by acc_provision). Power up this VM and point your browser to the VM s IP address, and provide credentials to log in. 4

Step 3 Step 4 Configure BOSH Director (tile named VMware vsphere) to specify the location of vcenter and create availability zones. Map the default network in Create Networks to the vsphere portgroup created by acc_provision. See the configuration output file produced by acc_provision for the values you will need. vsphere Network Name: apic_dvs/apic_node_portgroup CIDR, Gateway: apic_node_subnet Reserved IP Ranges: Use the IP address for the Ops Manager VM. Step 5 Step 6 Step 7 Download Pivotal Application Service (PAS) and import it as a product but do not deploy it at this stage. Click the PAS tile to configure it. Provide all the form inputs required by your installation. Create an additional network in BOSH Director by completing the following steps: a) On the VMware vsphere (BOSH Director) configuration page, click Create Networks > Add Network b) Provide the following values for the new network: Name: apic-infra Subnet-vSphere Network Name: apic_dvs/apic_infra_portgroup Subnet-CIDR: 169.254.0.0/16 Subnet-Gateway: 169.254.0.1 Step 8 Run the scripts/ops_manager_patch.sh script from the directory that contains the scripts/cisco-pcf-opsman-patch.diff path. This script restarts the Ops Manager after applying the patch. You will lose connectivity to Ops Manager for a few minutes. ops_manager_patch.sh <host> <username> <password> Positional parameters host Name or IP address of OpsManager username Name of local user on OpsManager with sudo privileges password Password of local user Step 9 0 Import the tile ACI CNI Plug-in tile, and add it by clicking on the + icon. Configure the ACI CNI Plug-in tile using the Python script scripts/aci-tile-config.py, providing the output file produced by acc_provision as input. python aci-tile-config.py [-h] host user password config_file positional arguments: host Name or IP Address of Ops Manager user Ops Manager Admin user name password Ops Manager Admin user password config_file Name of config file created by acc_provision optional arguments: -h, --help show this help message and exit 5

1 2 3 To view the configured values, click the tile. Enable the ACI CNI Plug-in by completing the following steps: a) Click Pivotal Application Service. b) Go to Network > Container Network Interface Plugin. c) In the Container Network Interface Plugin area, choose external. On the dashboard, click Apply Changes to install PAS with ACI add-ons. Installation can take a few hours. Upgrading the ACI CNI Plug-in This topic describes how to upgrade the Cisco ACI CNI Plug-in release version for Pivotal Cloud Foundry without having to reinstall Ops Manager and Pivotal Application Service (PAS) components. You can perform this operation when you want to upgrade to new features and bug fixes in Cisco ACI. Undo the Ops Manager patch using the script ops_manager_patch.sh. Use the same release version of the script as your current installation. Apply the new patch to the Ops Manager using the ops_manager_patch.sh script from the new release version that you want to upgrade to. The Ops Manager reboots and loses connectivity for a few minutes. Note You can skip the Ops Manager patch upgrade if there are no changes to the ops_manager_patch.sh file between the releases. Step 3 Step 4 Delete the ACI CNI Plug-in from the Ops Manager UI. Import the latest version of the ACI CNI Plug-in tile that you want to upgrade to and add it to the Ops Manager UI. Provide the acc_provision parameters using the aci-tile-config.py script and follow the regular installation process and apply the changes. Removing Cisco ACI Add-ons from Pivotal Cloud Foundry You can delete Installed Cisco ACI plug-ins by following the instructions in this section. In the Ops Manager GUI, delete the ACI CNI Plug-in tile from Ops Manager and apply the changes. (Optional) To undo the patching of Ops Manager (as described in Step 8 of Deploying PAS with ACI CNI Plug-in Tile), run the following command: 6

ops_manager_patch.sh -u <host> <username> <password> Positional parameters host Name or IP address of OpsManager username Name of local user on OpsManager with sudo privileges password Password of local user Unprovisioning Pivotal Cloud Foundry from the ACI Fabric This section describes how to uprovision Pivotal Cloud Foundry from the Cisco ACI fabric. Before you begin Before unprovisioning the resources allocated to your Pivotal Cloud Foundry installation from your Cisco ACI fabric, ensure that Pivotal Cloud Foundry and BOSH Director have been removed from Ops Manager, and then delete the Ops Manager VM from VMware vsphere. a) Unprovision the fabric. acc_provision -a -d -u <apic username> -p <apic password> -c mycf0-prov-config.yaml -o mycf0-vars.yaml -f cloudfoundry-1.0 Note This command also deletes the Cisco ACI tenant. If you are using a shared tenant, this command is dangerous. Operations Using Cisco ACI-Specific Extensions You can access Cisco ACI-specific Pivotal Cloud Foundry extensions through a Python CLI script, cf-aci.py. Extensions features include EPG annotations and external IP address. The Python CLI script is in the scripts/ directory of the distribution files (dist-generics-cloudfoundryxxxxxx.tgz). Most commands are self- explanatory and take one or two arguments. Run the Python CLI script to access the Cisco ACI-specific Pivotal Cloud Foundry extensions, using the following example:./scripts/cf-aci.py --help Usage: cf-aci.py <command> <command-arguments> Available commands: app-ext-ip <app-name> Get external IP of an app app-vip <app-name> Get virtual IP of an app epg-app <app-name> Get EPG annotation of an app epg-org <org-name> Get EPG annotation of an org epg-space <space-name> Get EPG annotation of a space 7

set-app-ext-ip <app-name> <IP-address> Set external IP on an app set-epg-app <app-name> <EPG-name> Set EPG annotation on an app set-epg-org <app-name> <EPG-name> Set EPG annotation on an org set-epg-space <app-name> <EPG-name> Set EPG annotation on a space unset-app-ext-ip <app-name> Remove external IP of an app unset-epg-app <app-name> Remove EPG annotation of an app unset-epg-org <org-name> Remove EPG annotation of an org unset-epg-space <space-name> Remove EPG annotation of a space Collecting Log Files for Support Requests If problems arise, Cisco support may ask that you submit log files to help them troubleshoot the problems. Follow the steps in this section to collect the log files for Pivotal Cloud Foundry. Step 3 Step 4 Get the list of VM instances in your deployment. bosh vms -d cf Generate report on the desired VM instance (diego_database or diego_cell). bosh ssh -d cf <instance> -c 'sudo/var/vcap/packages/apic-host-report/apic-host-report.sh' Note the report file mentioned in the output. Copy over the report file. bosh scp -d cf <instance>:<path/to/report-file>. 8

2018 Cisco Systems, Inc. All rights reserved.

Americas Headquarters Cisco Systems, Inc. San Jose, CA 95134-1706 USA Asia Pacific Headquarters CiscoSystems(USA)Pte.Ltd. Singapore Europe Headquarters CiscoSystemsInternationalBV Amsterdam,TheNetherlands Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.