Cisco Metacloud Controller Bundle Design and Implementation Guide v1.8.3

Transcription

1 Cisco Metacloud Controller Bundle Design and Implementation Guide v1.8.3 Table of Contents Preface... 3 Introduction... 3 Components... 4 Architecture Overview... 4 Starter Configuration... 4 General Purpose Configuration... 5 High Performance Configuration... 5 Installation... 5 Data Center Requirements... 5 Cisco 2901 Integrated Services Router (Terminal Server)... 5 Installation... 6 Cabling for Remote Console (Out-of-Band) Connectivity... 6 C2901 Async Module... 7 Connecting the 2901 Router... 7 Connecting the 2901 Ethernet Switch Module... 7 Starter Configuration: Cisco 1001-X Series Aggregation Services Router (2) 2.5G Throughput.. 7 Installation... 8 Starter Configuration: Cisco Nexus 9396PX switches with M12PQ module (2)... 8 Installation... 8 Cabling for Starter Configuration... 8 Connecting the ASR1001-X routers and Nexus 9396P Switches... 9 Port Map for Physical Connections:... 9 General Purpose Configuration: Cisco 1001-X Series Aggregation Services Router (2) 10G Throughput... 9 Installation... 9 General Purpose Configuration: Cisco Nexus 9396PX switches with M12PQ module (2)... 9 Installation Cabling for General Purpose Configuration Connecting the ASR1001-X routers and Nexus 9396P Switches Port Map for Physical Connections: High Performance Configuration: Cisco 1002-X Series Aggregation Services Router (2) 20G Throughput Installation High Performance Configuration: Cisco Nexus 9396PX switches with M12PQ module (2) Installation Cabling for High Performance Configuration Connecting the ASR1002-X routers and Nexus 9396P Switches Port Map for Physical Connections: Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 1

2 Cabling for Expansion: Nexus 9332PQ Cabling up the ASR 1000 Series routers, Nexus 9396PX switches and Nexus 9332PQ Switches Installing the the NX-OS Image for Nexus Switches Connecting the UCS C220 Control Plane Servers Initial Configuration for Establishing Remote Console Connectivity Network Information Required Initial Configuration Template Applying the Configuration BIOS Configuration for both Controller and Compute Nodes Cabling Compute Servers to the Nexus 9396 Switches Open Network Ports Administrative Outbound Connections OOB Outbound Connections OOB Inbound Connections Default VLAN IDs Next Steps Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 2

3 Preface OpenStack is a free and open source Infrastructure-as-a-Service (IaaS) cloud computing project released under the Apache License. It enables enterprises and service providers to offer on-demand computing resources by provisioning and managing large networks of virtual machines. Cisco Metacloud, formerly known as Cisco OpenStack Private Cloud, is a software as a service offering from Cisco that allows organizations to take advantage of the speed and agility that come with OpenStack without having to manage or operate the underlying infrastructure components, including OpenStack services. Organizations that adopt Cisco Metacloud experience many of the same benefits they would in a public cloud but enjoy the added flexibility, reliability, and security of their own data centers. Utilizing Cisco Unified Computing System (UCS), Cisco Nexus 9000 Series switches and Cisco Aggregation Services Routers, known as the Controller Bundle, Cisco OpenStack Private Cloud provides a next-generation data center platform that unites computing, network, storage access, and virtualization into a single cohesive system. The Cisco Metacloud Controller Bundle provides a validated architecture for deploying the Cisco Metacloud service, and the ability to grow incrementally as capacity needs change. This combination of the Cisco UCS platform and Cisco OpenStack Private Cloud accelerates your IT by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. This Cisco Validated Design document provides design and implementation guidance for deploying the Cisco Metacloud Controller Bundle to enable a customer s multi-tenant cloud services. Introduction The Cisco Metacloud Controller Bundle is a converged infrastructure solution based on Cisco UCS, Nexus and Aggregation Services Routers that provide a convenient, consistent, pre-configured hardware and network architecture that has been validated for Cisco Metacloud usage. The extensive validation and documentation provided in this Cisco Validated Design clearly lays out the steps involved in the installation of the Controller Bundle solution. The Cisco Metacloud solution comprises the following key components: Cisco Metacloud service subscription Cisco Metacloud Controller Bundle o o o o Cisco UCS C-Series servers for cloud orchestration Cisco Nexus 9300 Series Switches Cisco Aggregate Services Router (ASR) Cisco 2901 Integrated Services Router (ISR) Cisco UCS or other servers, that meets the minimum requirements, for Cisco Metacloud. The Cisco Metacloud Controller Bundle for Cisco Metacloud Design and Implementation Guide version 1.x is the first edition reference architecture with accompanying Bill of Materials (BOM). As the Cisco Metacloud solution is updated to support changes to OpenStack or advanced features that Cisco may offer there will be subsequent editions of this design guide Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 3

4 Components The Cisco Metacloud Controller Bundle for Cisco Metacloud now offers 3 different configurations options and includes the following components: 3 x UCS C220 M4 Servers Provide a 3-node control plane cluster to ensure high availability of key private cloud services. 2 x ASR1000 Series Routers Provide up to 20Gb/s network throughput from the cloud s network to the customer s data center and represents the north/south demarcation of the cloud. South of these devices are managed by Cisco, north is the customer s network. 2 x Nexus 9396PX Switches Provide top of rack (ToR) and aggregation switching, including providing a path for future compatibility with Cisco ACI. Additional pairs of Nexus 9000 Series switches can be added to these to expand server capacity. 1 x 2901 Integrated Services Router Provide out of band (OOB) access to all the servers and networking components of the Metacloud Controller Bundle. Architecture Overview The Cisco Metacloud Controller Bundle for Cisco Metacloud offers 3 configuration options and the following diagram details the physical architecture of the Controller Bundle. Starter Configuration The Starter configuration supports up to 40 compute servers and offers up to 2.5Gbps of bandwidth throughput. The throughput of a configuration includes the total amount of network traffic coming into and out of the ASR. This includes traffic to the cloud from an external customer network or from the cloud to an upstream gateway (north/south traffic), and traffic between subnets within the Availability Zone (east/west traffic. This configuration uses the Cisco ASR1001-X Services router and utilizes 2 of the 6 onboard 1G ports. Please note that although the Cisco ASR1001-X Services router comes with two 10G ports, the 2.5G license does not enable these ports and they will be inactive until an appropriate license is purchased Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 4

5 General Purpose Configuration The General Purpose configuration supports up to 100 compute servers and up to 10Gbps of bandwidth throughput. The throughput of a configuration includes the total amount of network traffic coming into and out of the ASR. This includes traffic to the cloud from an external customer network or from the cloud to an upstream gateway (north/south traffic), and traffic between subnets within the Availability Zone (east/west traffic). This configuration uses the Cisco ASR1001-X Services router and utilizes one of the two onboard 10G ports. Please note that although the Cisco ASR1001-X Services router comes with two 10G ports, the 10G license only enables one of these ports with the other being inactive until an appropriate license is purchased. High Performance Configuration The High Performance configuration supports up to 400 servers and up to 20Gbps of bandwidth throughput. The throughput of a configuration includes the total amount of network traffic coming into and out of the ASR. This includes traffic to the cloud from an external customer network or from the cloud to an upstream gateway (north/south traffic), and traffic between subnets within the Availability Zone (east/west traffic. This configuration uses the Cisco ASR1002-X Services router and utilizes two 10G SPA ports as well as two 10G XFP transceivers. Installation One of the main advantages and key differentiators of the Cisco Metacloud is the ease in which it is operationalized. The high level steps in deploying and configuring the Controller Bundle are listed below. More detailed instructions are provided in a later chapter. 1. Rack and cable the controller bundle, as well as capacity hardware 2. Connect the Cisco 2901 Integrated Services Router to the Cisco Metacloud Operations Center. 3. Perform BIOS configuration on UCS servers 4. Cisco Metacloud operations personnel remotely install, monitor and operate the Cisco Metacloud service. Data Center Requirements The following are requirements and recommendations regarding the environment in which a Controller Bundle configuration is deployed: 1. The ASR1000 Series routers require network access from all sources that will need to access instances in the cloud and should be redundantly connected to two independent network paths. 2. The 2901 ISR Router requires either a public or private IP address and the ability to initiate a VPN connection to the Metacloud Operation Center. A direct access through SSH or console to the 2901 is also required. Please note that this may require changes to the customer s outbound firewall rules and routes. 3. All servers and networking equipment in both the Controller Bundle and the cloud compute capacity should be connected to redundant power sources within the data center rack. 4. A local computer with a console cable appropriate for configuring Cisco devices is required for the initial software configuration of the 2901 ISR Router router. 5. A monitor and keyboard is required to update BIOS settings. Cisco 2901 Integrated Services Router (Terminal Server) Cisco 2901 w/ HWIC-16A and 2 CAB-HD8ASYNC Terminal Server 1 16-Port Async HWIC 1 512MB DRAM 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 5

6 1 256MB Compact Flash 2 High Density 8-port EIA-232 Async Cable ( Octal Cables ) 1 Eight port 10/100/1000 Ethernet switch interface card Installation 1. Install the 16-port Async HWIC module on EHWIC slot 0 (the rightmost slot) of the 2901 router. 2. Install the 8-port Ethernet switch module on EHWIC slot 2 and 3 of router. This module occupies two slots. 3. Mount the 2901 Integrated Services Router with the modules in the rack, and connect the two octal cables to the Async HWIC module. Cabling for Remote Console (Out-of-Band) Connectivity Connecting the 2901 ISR router to the Cisco ASR1000 Series and Cisco Nexus 9396 switch console ports and management ports provides remote console connectivity for Cisco Metacloud Operations. The physical connections required are illustrated below: 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 6

7 C2901 Async Module Two sets of octal cables are used to connect the console ports of the Cisco ASR1000 Series routers and Cisco Nexus 9396PX switches to the Cisco ISR Follow the instructions below to connect the consoles. 1. Connect the connector to the Async module. 2. Connect the cables to the console ports P0: connect to the first ASR1000 Series router console port. P1: connect to the second ASR1000 Series router console port. P2: connect to the first Nexus 9396PX switch console port. P3: connect to the second Nexus 9396PX switch console port. Note: P0, P1, P2 and P3 connections are at the bottom octal cable. These are the only 4 connections required for this setup. Connecting the 2901 Router 1. Connect Gigabit Ethernet G0/0 (RJ45) to an uplink to the Internet for out-of-band (OOB) access. 2. Connect Gigabit Ethernet G0/1 (RJ45) to port 44 of the first Nexus 9396PX switch. Connecting the 2901 Ethernet Switch Module The 8-port switch module is used to connect the management ports of the ASR1000 Series routers and Nexus 9396PX switches. P0: connect to the first ASR1000 Series router s management port (G0). P1: connect to the second ASR1000 Series router s management port (G0). P2: connect to the first Nexus 9396PX switch s management port (mgmt0). P3: connect to the second Nexus 9396PX switch s management port (mgmt0). P4: connect to port 44 of the second Nexus 9396PX switch. Starter Configuration: Cisco 1001-X Series Aggregation Services Router (2) 2.5G Throughput Accessories for each (1) ASR1001-X Series router (2x10GB, 6x1GE ports and 2.5G IPBase License) 1 x Cisco ASR1000 Series 8GB DRAM 1 x Cisco ASR1000 Series IOS XE UNIVERSAL 2 x Cisco 1000BASE-T SFP transceivers 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 7

8 Installation 1. Insert two 1000BASE-T SFP transceivers into ports GE0 and GE1 of the ASR1001-X. 2. Repeat for the 2 nd ASR1001-X router. 3. Mount the two ASR1001-X routers into the rack. Starter Configuration: Cisco Nexus 9396PX switches with M12PQ module (2) Accessories for each (1) Nexus 9396PX switch 1 x Nexus 9300 Accessory Kit 5 x 1000BASE-T SFP 4 x QSFP40G BiDi Short-reach Transceiver 1 x 12-port 40G Uplink Module Installation 1. Insert 4 x 40G QSFP+ BiDi transceivers into 40G ports 9,10,11 and 12 of the uplink module. 2. Insert 5 x 1000BASE-T SFP transceivers into ports 44 to 48 of the base module. 3. Repeat for the 2 nd Nexus 9396PX switch. 4. Mount the two Nexus 9396PX switches in the rack. Cabling for Starter Configuration 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 8

9 Connecting the ASR1001-X routers and Nexus 9396P Switches 1. Connect the two 1G interfaces from each ASR1001-X Series router slot GE0 and GE1 to ports 47 and 48 of each corresponding Nexus 9396PX switch. 2. Connect the Customer router s 1G connections to port 45 and 46 on each corresponding Nexus 9396PX switch. 3. Cross-connect the 4 x 40G connections on ports 9-12 between the two Nexus 9396PX switches. Port Map for Physical Connections: The physical cabling for above-mentioned connections are listed here in the table: Source Source Model Source Slot- Destination Destination Destination Hostname Port Hostname Model Slot-Port ASR1001X-1 ASR1000 Series GE0 Nexus9396PX-1 Nexus 9396PX E1/47 ASR1001X-1 ASR1000 Series GE1 Nexus9396PX-1 Nexus 9396PX E1/48 ASR1001X-2 ASR1000 Series GE0 Nexus9396PX-2 Nexus 9396PX E1/47 ASR1001X-2 ASR1000 Series GE1 Nexus9396PX-2 Nexus 9396PX E1/48 Nexus9396PX-1 Nexus 9396PX G1/45 Customer Device-1 / Port 1 Nexus9396PX-1 Nexus 9396PX G1/46 Customer Device-1 / Port 2 Nexus9396PX-2 Nexus 9396PX G1/45 Customer Device-2 / Port 1 Nexus9396PX-2 Nexus 9396PX G1/46 Customer Device-2 / Port 2 Nexus9396PX-1 Nexus 9396PX 40G 2/9 Nexus9396PX-2 Nexus 9396PX 40G 2/9 Nexus9396PX-1 Nexus 9396PX 40G 2/10 Nexus9396PX-2 Nexus 9396PX 40G 2/10 Nexus9396PX-1 Nexus 9396PX 40G 2/11 Nexus9396PX-2 Nexus 9396PX 40G 2/11 Nexus9396PX-1 Nexus 9396PX 40G 2/12 Nexus9396PX-2 Nexus 9396PX 40G 2/12 General Purpose Configuration: Cisco 1001-X Series Aggregation Services Router (2) 10G Throughput Accessories for each (1) ASR1001-X Series router (2x10GB, 6x1GE ports and 10G IPBase License) 1 x Cisco ASR1000 Series 8GB DRAM 1 x Cisco ASR1000 Series IOS XE UNIVERSAL 1 x Cisco 10GBASE-SR SFP Module Installation 1. Insert a 10GBASE-SR SFP module into port TE0 on each of the ASR1001-X routers. 2. Repeat for the 2 nd ASR1001-X router. 3. Mount the two ASR1001-X routers into the rack. General Purpose Configuration: Cisco Nexus 9396PX switches with M12PQ module (2) Accessories for each (1) Nexus 9396PX switch 1 x Nexus 9300 Accessory Kit 2 x 10GBASE-SR SFP Module 1 x 1000BASE-T SFP 4 x QSFP40G BiDi Short-reach Transceiver 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 9

10 1 x 12-port 40G Uplink Module Installation 1. Insert 4 x 40G QSFP+ BiDi transceivers into 40G ports 9,10,11 and 12 of the uplink module. 2. Insert 2 x 10GBASE-SR SFPs into ports 45 and 47 of the base module. 3. Insert 1 x 1000BASE-T SFP into port 44 of the base module. 4. Repeat for the 2 nd Nexus 9396PX switch. 5. Mount the two Nexus 9396PX switches in the rack. Cabling for General Purpose Configuration Connecting the ASR1001-X routers and Nexus 9396P Switches 1. Connect one 10G interfaces from each ASR1001-X Series router slot TE0 to port 47 on each corresponding Nexus 9396PX switch. 2. Connect the Customer router s 10G connection to port 45 on each corresponding Nexus 9396PX switch. 3. Cross-connect the 4 x 40G connections on ports 9-12 between the two Nexus 9396PX switches Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 10

11 Port Map for Physical Connections: The physical cabling for above-mentioned connections are listed here in the table: Source Source Model Source Slot- Destination Destination Destination Hostname Port Hostname Model Slot-Port ASR1001X-1 ASR1000 Series TE0 Nexus9396PX-1 Nexus 9396PX E1/47 ASR1001X-2 ASR1000 Series TE1 Nexus9396PX-2 Nexus 9396PX E1/47 Nexus9396PX-1 Nexus 9396PX TenG 1/45 Customer Device-1 / Port 1 Nexus9396PX-2 Nexus 9396PX TenG 1/45 Customer Device-2 / Port 1 Nexus9396PX-1 Nexus 9396PX 40G 2/9 Nexus9396PX-2 Nexus 9396PX 40G 2/9 Nexus9396PX-1 Nexus 9396PX 40G 2/10 Nexus9396PX-2 Nexus 9396PX 40G 2/10 Nexus9396PX-1 Nexus 9396PX 40G 2/11 Nexus9396PX-2 Nexus 9396PX 40G 2/11 Nexus9396PX-1 Nexus 9396PX 40G 2/12 Nexus9396PX-2 Nexus 9396PX 40G 2/12 High Performance Configuration: Cisco 1002-X Series Aggregation Services Router (2) 20G Throughput Accessories for each (1) ASR1002-X router (2x10GB, 6x1GE ports and 36G IPBase License) 1 x Cisco ASR1000 Series 4GB DRAM 1 x Cisco ASR1000 Series IOS XE UNIVERSAL 2 x 10GBASE-SR Module and XFP Transceivers Installation 1. Insert two 10G Shared Port Adapters (SPA) into slots 1 and 2 of each ASR1002-X. 2. Insert a single XFP 10G transceiver into each SPA. 3. Repeat for the 2 nd ASR1002-X router. 4. Mount the two ASR1002-X routers into the rack. High Performance Configuration: Cisco Nexus 9396PX switches with M12PQ module (2) Accessories for each (1) Nexus 9396PX switch 1 x Nexus 9300 Accessory Kit 4 x 10GBASE-SR SFP Module 1 x 1000BASE-T SFP 4 x QSFP40G BiDi Short-reach Transceiver 1 x 12-port 40G Uplink Module 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 11

12 Installation 1. Insert 4 x 40G QSFP+ BiDi transceivers into 40G ports 9,10,11 and 12 of the uplink module. 2. Insert 4 x 10GBASE-SR SFPs into ports 45 through 48 of the base module. 3. Insert 1 x 1000BASE-T SFP into port 44 of the base module. 4. Repeat for the 2 nd Nexus 9396PX switch. 5. Mount the two Nexus 9396PX switches in the rack. Cabling for High Performance Configuration Connecting the ASR1002-X routers and Nexus 9396P Switches 1. Connect the two 10G interfaces on slots 1 and 2 of each ASR1002-X Series router to the 10G ports 47 and 48 of each corresponding Nexus 9396PX switch. 2. Connect the Customer router s 10G connections to port 45 and 46 of each corresponding Nexus 9396PX switch. 3. Cross-connect the 4 x 40G connections on ports 9-12 between the two Nexus 9396PX switches Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 12

13 Port Map for Physical Connections: The physical cabling for above-mentioned connections are listed here in the table: Source Hostname Source Model Source Slot- Port Destination Hostname Destination Model Destination Slot-Port ASR1002X-1 ASR1000 Series TenG 0/1/0 Nexus9396PX-1 Nexus 9396PX E1/47 ASR1002X-1 ASR1000 Series TenG 0/2/0 Nexus9396PX-1 Nexus 9396PX E1/48 ASR1002X-2 ASR1000 Series TenG 0/1/0 Nexus9396PX-2 Nexus 9396PX E1/47 ASR1002X-2 ASR1000 Series TenG 0/2/0 Nexus9396PX-2 Nexus 9396PX E1/48 Nexus9396PX-1 Nexus 9396PX TenG 1/45 Customer Device-1 / Port 1 Nexus9396PX-1 Nexus 9396PX TenG 1/46 Customer Device-1 / Port 2 Nexus9396PX-2 Nexus 9396PX TenG 1/45 Customer Device-2 / Port 1 Nexus9396PX-2 Nexus 9396PX TenG 1/46 Customer Device-2 / Port 2 Nexus9396PX-1 Nexus 9396PX 40G 2/9 Nexus9396PX-2 Nexus9396PX-1 Nexus 9396PX 40G 2/10 Nexus9396PX-2 Nexus9396PX-1 Nexus 9396PX 40G 2/11 Nexus9396PX-2 Nexus9396PX-1 Nexus 9396PX 40G 2/12 Nexus9396PX-2 Cabling for Expansion: Nexus 9332PQ Nexus 9396PX 40G 2/9 Nexus 9396PX 40G 2/10 Nexus 9396PX 40G 2/11 Nexus 9396PX 40G 2/12 Cisco Metacloud can support up to 400 physical servers in one pod using Nexus 9396PX switches and a pair of Nexus 9332PQ switches. By introducing the Nexus 9332PQ switches, Metacloud can aggregate up to 10 pairs of Nexus 9396PX switches and connect up to 400 individual servers. If you plan on expanding past 40 servers at any time, it is highly recommended that customers install the Nexus 9332PQ switches during the initial install of Metacloud in order to alleviate downtime during future expansions. Please note: When deploying Nexus 9332PQ switches, the Cisco ASR 1000 Series routers will be connected to the Nexus 9332PQ switches instead of the Nexus 9396PX switches. Also, because the Nexus 9332PQ switches only have 40G ports the ASR 2.5G license will not be supported with expansion and the Nexus 9332PQ Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 13

14 Cabling up the ASR 1000 Series routers, Nexus 9396PX switches and Nexus 9332PQ Switches 1. Connect the two 10G interfaces on slots 1 and 2 of each ASR1000 Series router to the 10G ports 25 and 26 of each corresponding Nexus 9332PQ switch. Please note: Since the Nexus 9332PQ switch only supports 40G ports, you will need to connect in the Cisco 40-Gigabit-to-4x10-Gigabit breakout cables to both ports 25 and 26. These breakout cables will provide 4 x 10G ports in which to connect the ASR 1000 Series routers but do not connect both ASR 1000 Series 10G ports to a single breakout cable. 2. Cross-connect the 4 x 40G connections on ports of each Nexus 9332PQ switches. 3. Cross-connect a 40G connection from each Nexus 9396PX switch to the same port of each 9332PQ switch (port 1 on each Nexus 9332PQ for the first 9396PX switch, port 2 for the second 9396PX switch, port 3 for the third 9396PX switch, etc). Installing the the NX-OS Image for Nexus Switches If any of the Nexus switches have come installed with a Cisco ACI image, customers will need to install the appropriate Cisco NX-OS image from the Cisco website. Cisco NX-OS downloads can be found at but please note that NX-OS image downloads require Cisco.com logins associated with a valid service contract. Once the image is downloaded, place it on a USB drive and insert the USB drive into the appropriate Cisco Nexus switch. From the console type the following commands, using the correct image name where applicable: (none)# copy usb/usb1/usb/usb1/nxos i2.2a.bin bootflash: (none)# reload This command will reload the chassis, Proceed (y/n)? [n]: y Once the switch reboots, you must interrupt the boot process during the boot sequence by typing Ctrl-C and booting the new NX-OS bootimage: Ctrl-C loader> boot nxos i2.2a.bin Connecting the UCS C220 Control Plane Servers 1. For the first controller, locate the onboard MLOM module with two 10G ports. 2. Connect the first port to the 10G port 1 of first Nexus 9396PX switch, and then connect the second port to the 10G port 1 of second Nexus 9396PX switch. 3. Repeat this for the second and third controllers and connect to port 2 and 3 on each switch. Initial Configuration for Establishing Remote Console Connectivity The first-time configuration process involves gathering a few configuration details of the local network, inserting them into a configuration template, and using a computer connected to the console port of the Cisco ISR 2901 to apply the completed configuration. The initial bootstrap process is a one-time manual process. All future maintenances and upgrades will be done remotely by the Cisco Metacloud Operations Center as part of operating the service Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 14

15 Network Information Required 1. A publicly-routable IP Address for the ISR2901 or a private address that the End Customer can translate to a public IP for the purposes for establishing a VPN connection (OOB_WAN_IP) 2. The gateway IP address (OOB_WAN_GW) on Customer s device serves as gateway for the above IP Initial Configuration Template The following configuration template should be completed during the initial turn-up on the ISR2901 router. The initial passwords will be changed by the Cisco Metacloud Operations Center once this setup is completed. The ISR2901 router comes with default username and password as cisco/cisco. 1. Once you log into the ISR 2901 router, perform write erase and reload to boot up with clean config. 2. Select a temporary password and replace the TEMP_PASSWORD in the config with this newly created password. The customer will then provide to the Metacloud Operations team this password so that they may continue configuring the ISR After the ISR 2901 configuration has been completed, the Metacloud Operations team will update the password to match current operating standards. 3. Paste the following config with the actual IP info for OOB_WAN_IP and OOB_WAN_GW. hostname c2901 enable secret <TEMP_PASSWORD> ip cef license boot module c2900 technology-package securityk9 username temp_admin privilege 15 secret <TEMP_PASSWORD> no ip domain lookup ip domain name metacloud.in crypto key generate rsa general-keys modulus 1024 ip ssh version 2 vrf definition wan address-family ipv4 exit-address-family access-list 101 permit any access-list 101 deny ip any any interface GigabitEthernet0/0 vrf forwarding wan ip address <OOB_WAN_IP> ip access-group 101 in no shut ip route vrf wan <OOB_WAN_GW> line con 0 exec-timeout 15 0 logging synchronous login local line 0/0/0 0/0/15 session-timeout 30 exec-timeout 30 0 disconnect-character Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 15

16 logging synchronous no exec transport input ssh transport output none line vty 0 15 exec-timeout 15 0 login local transport input ssh End Applying the Configuration Connect a computers USB port to the Cisco 2901 console port via a Cisco console cable and a USB port adapter, and use a terminal application to access the console interface of the 2901 router. The serial port settings should be configured as: Bits per second: 9600 Data bits: 8 Parity: None Stop bits: 1 Flow control: None Verify successful configuration by pinging an IP address elsewhere on the Internet, such as A successful attempt should look like this: #ping Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to , timeout is 2 seconds: Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms If unsuccessful please verify the info is obtained correctly or contact the Cisco Metacloud Operations Center for assistance.. BIOS Configuration for both Controller and Compute Nodes For each UCS node, the USB keyboard, Video and USB mouse can be connected through the front panel of the server using a Cisco-provided dongle. 1. Access the configuration screen by pressing the F8 key on the following screen Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 16

17 3. When prompted for password, the default username is admin and default password is password. Change the password to Cisco123. The Metacloud Operations Center will change this password once remote console connectivity is established. 4. The BIOS/CIMC settings should be as follows: Address set to DHCP Active-active mode VLAN ID enabled and set to VLAN Press F10 to save. 6. Press ESC and exit the BIOS configuration Cabling Compute Servers to the Nexus 9396 Switches Note: This step can be completed after the Controller Bundle hardware has been set up. All compute servers should be dual connected to the Nexus 9396 switches. Each connection should use the compute server s 10Gb network interface to connect to the Nexus 9396 switches. If using Cisco C-Series servers for compute the 10Gb network ports on the MLOM module should be directly connected to the two Nexus 9396 switches. Information on how to install Cisco C-Series servers can be found on the Cisco website ( If using Cisco servers a Cisco Fabric Interconnect should only be used if using Cisco B-Series servers. All other supported Cisco compute servers should connect directly to the Cisco Nexus 9396 switches. If Cisco B-Series servers are being used for compute they, must be connected to a pair of Cisco Fabric Interconnects. The Cisco Fabric Interconnects should be dual connected to the Nexus 9396 switches. Please refer to the Cisco Fabric Interconnect documentation on the Cisco.com website for how the servers should be connected to the Cisco Fabric Interconnects ( A Port Map is also provided below to aid in ensuring the correct wiring is implemented Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 17

18 Source Hostname Source Model Source Slot-Port Destination Hostname Nexus9396PX-1 Nexus 9396PX TenG 1/4 Compute Server 1 Nexus9396PX-1 Nexus 9396PX TenG 1/5 Compute Server 2 Nexus9396PX-1 Nexus 9396PX TenG 1/6 Compute Server 3 Nexus9396PX-1 Nexus 9396PX TenG 1/7 Compute Server 4 Nexus9396PX-1 Nexus 9396PX TenG 1/8 Compute Server 5 Nexus9396PX-1 Nexus 9396PX TenG 1/9 Compute Server 6 Nexus9396PX-1 Nexus 9396PX TenG 1/10 Compute Server 7 Nexus9396PX-1 Nexus 9396PX TenG 1/11 Compute Server 8 Nexus9396PX-1 Nexus 9396PX TenG 1/12 Compute Server 9 Nexus9396PX-1 Nexus 9396PX TenG 1/13 Compute Server 10 Nexus9396PX-1 Nexus 9396PX TenG 1/14 Compute Server 11 Nexus9396PX-1 Nexus 9396PX TenG 1/15 Compute Server 12 Nexus9396PX-1 Nexus 9396PX TenG 1/16 Compute Server 13 Nexus9396PX-1 Nexus 9396PX TenG 1/17 Compute Server 14 Nexus9396PX-1 Nexus 9396PX TenG 1/18 Compute Server 15 Nexus9396PX-1 Nexus 9396PX TenG 1/19 Compute Server 16 Nexus9396PX-1 Nexus 9396PX TenG 1/20 Compute Server 17 Nexus9396PX-1 Nexus 9396PX TenG 1/21 Compute Server 18 Nexus9396PX-1 Nexus 9396PX TenG 1/22 Compute Server 19 Nexus9396PX-1 Nexus 9396PX TenG 1/23 Compute Server 20 Nexus9396PX-1 Nexus 9396PX TenG 1/24 Compute Server 21 Nexus9396PX-1 Nexus 9396PX TenG 1/25 Compute Server 22 Nexus9396PX-1 Nexus 9396PX TenG 1/26 Compute Server 23 Nexus9396PX-1 Nexus 9396PX TenG 1/27 Compute Server 24 Nexus9396PX-1 Nexus 9396PX TenG 1/28 Compute Server 25 Nexus9396PX-1 Nexus 9396PX TenG 1/29 Compute Server 26 Nexus9396PX-1 Nexus 9396PX TenG 1/30 Compute Server 27 Nexus9396PX-1 Nexus 9396PX TenG 1/31 Compute Server 28 Nexus9396PX-1 Nexus 9396PX TenG 1/32 Compute Server 29 Nexus9396PX-1 Nexus 9396PX TenG 1/33 Compute Server 30 Nexus9396PX-1 Nexus 9396PX TenG 1/34 Compute Server 31 Nexus9396PX-1 Nexus 9396PX TenG 1/35 Compute Server 32 Nexus9396PX-1 Nexus 9396PX TenG 1/36 Compute Server 33 Nexus9396PX-1 Nexus 9396PX TenG 1/37 Compute Server 34 Nexus9396PX-1 Nexus 9396PX TenG 1/38 Compute Server 35 Nexus9396PX-1 Nexus 9396PX TenG 1/39 Compute Server 36 Nexus9396PX-1 Nexus 9396PX TenG 1/40 Compute Server 37 Nexus9396PX-1 Nexus 9396PX TenG 1/41 Compute Server 38 Nexus9396PX-1 Nexus 9396PX TenG 1/42 Compute Server 39 Nexus9396PX-1 Nexus 9396PX TenG 1/43 Compute Server Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 18

19 Source Hostname Source Model Source Slot-Port Destination Hostname Nexus9396PX-2 Nexus 9396PX TenG 1/4 Compute Server 1 Nexus9396PX-2 Nexus 9396PX TenG 1/5 Compute Server 2 Nexus9396PX-2 Nexus 9396PX TenG 1/6 Compute Server 3 Nexus9396PX-2 Nexus 9396PX TenG 1/7 Compute Server 4 Nexus9396PX-2 Nexus 9396PX TenG 1/8 Compute Server 5 Nexus9396PX-2 Nexus 9396PX TenG 1/9 Compute Server 6 Nexus9396PX-2 Nexus 9396PX TenG 1/10 Compute Server 7 Nexus9396PX-2 Nexus 9396PX TenG 1/11 Compute Server 8 Nexus9396PX-2 Nexus 9396PX TenG 1/12 Compute Server 9 Nexus9396PX-2 Nexus 9396PX TenG 1/13 Compute Server 10 Nexus9396PX-2 Nexus 9396PX TenG 1/14 Compute Server 11 Nexus9396PX-2 Nexus 9396PX TenG 1/15 Compute Server 12 Nexus9396PX-2 Nexus 9396PX TenG 1/16 Compute Server 13 Nexus9396PX-2 Nexus 9396PX TenG 1/17 Compute Server 14 Nexus9396PX-2 Nexus 9396PX TenG 1/18 Compute Server 15 Nexus9396PX-2 Nexus 9396PX TenG 1/19 Compute Server 16 Nexus9396PX-2 Nexus 9396PX TenG 1/20 Compute Server 17 Nexus9396PX-2 Nexus 9396PX TenG 1/21 Compute Server 18 Nexus9396PX-2 Nexus 9396PX TenG 1/22 Compute Server 19 Nexus9396PX-2 Nexus 9396PX TenG 1/23 Compute Server 20 Nexus9396PX-2 Nexus 9396PX TenG 1/24 Compute Server 21 Nexus9396PX-2 Nexus 9396PX TenG 1/25 Compute Server 22 Nexus9396PX-2 Nexus 9396PX TenG 1/26 Compute Server 23 Nexus9396PX-2 Nexus 9396PX TenG 1/27 Compute Server 24 Nexus9396PX-2 Nexus 9396PX TenG 1/28 Compute Server 25 Nexus9396PX-2 Nexus 9396PX TenG 1/29 Compute Server 26 Nexus9396PX-2 Nexus 9396PX TenG 1/30 Compute Server 27 Nexus9396PX-2 Nexus 9396PX TenG 1/31 Compute Server 28 Nexus9396PX-2 Nexus 9396PX TenG 1/32 Compute Server 29 Nexus9396PX-2 Nexus 9396PX TenG 1/33 Compute Server 30 Nexus9396PX-2 Nexus 9396PX TenG 1/34 Compute Server 31 Nexus9396PX-2 Nexus 9396PX TenG 1/35 Compute Server 32 Nexus9396PX-2 Nexus 9396PX TenG 1/36 Compute Server 33 Nexus9396PX-2 Nexus 9396PX TenG 1/37 Compute Server 34 Nexus9396PX-2 Nexus 9396PX TenG 1/38 Compute Server 35 Nexus9396PX-2 Nexus 9396PX TenG 1/39 Compute Server 36 Nexus9396PX-2 Nexus 9396PX TenG 1/40 Compute Server 37 Nexus9396PX-2 Nexus 9396PX TenG 1/41 Compute Server 38 Nexus9396PX-2 Nexus 9396PX TenG 1/42 Compute Server 39 Nexus9396PX-2 Nexus 9396PX TenG 1/43 Compute Server Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 19

20 Open Network Ports In order for the Cisco Metacloud service to communicate with the Cisco Metacloud Operations Center, customers must allow connectivity from their network back to Cisco Metacloud. This can be achieved by having the customer s network team allow communication on the following ports in their network. Administrative Outbound Connections Port Number Protocol Source IP Destination IP Function NA ICMP ASR External Network Any of the below Connectivity Testing 443 TCP ASR External Network General Repository 443 TCP ASR External Network CMS 443 TCP ASR External Network PKI services 80, 443 TCP ASR External Network , Apt Repository 80, 443 TCP ASR External Network , Yum Repository 8443 TCP ASR External Network , , Monitoring TCP ASR External Network , Admin Access TCP ASR External Network , Log Monitoring TCP ASR External Network DB Backups 53 TCP/UDP ASR External Network , , DNS Zone Transfer , TCP/UDP ASR External Network Any IP DNS (if not provided) 123 TCP/UDP ASR External Network Any IP NTP (if not provided) 1194 TCP ASR External Network , Management VPN Tunnel OOB Outbound Connections Port Number Protocol Source IP Destination IP Function NA ICMP ISR2901 WAN IP* , Connectivity Testing 500 UDP ISR2901 WAN IP* , OOB VPN Tunnel 4500 UDP ISR2901 WAN IP* , OOB VPN Tunnel NA ESP ISR2901 WAN IP* , OOB VPN Tunnel OOB Inbound Connections Port Number Protocol Source IP Destination IP Function NA ICMP , ISR2901 WAN IP* Connectivity Testing 500 UDP , ISR2901 WAN IP* OOB VPN Tunnel 4500 UDP , ISR2901 WAN IP* OOB VPN Tunnel NA ESP , ISR2901 WAN IP* OOB VPN Tunnel 22 TCP /26 ISR2901 WAN IP* SSH Admin Connectivity * - If a private IP address is used, then use the public NAT address. Default VLAN IDs The following lists the different VLANs that are used in a Metacloud Availability Zone along with descriptions for each. Name IPv4 Type VLAN ID Netmask Description External Private or Public 99 Variable Network external /outside of the ASR s this network is used as the hop between the AZ and the rest of the routed domain. Service Private 1002 /27 The service network that is upstream of the MCPs. Intracluster Private 1001 None required. The Intracluster VLAN will need to be the native, untagged VLAN associated with each of the Ethernet switch ports. The PXE boot of the nodes happens across this VLAN. Storage Private 1003 /23 Each MHV will connect to this network for various services. OOB Private 1000 /23 LOM/Console interfaces of MCP 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 20

21 and MHV. Metacloud Project Private 2000 /24 A dedicated project domain for Metacloud Engineering. Project Network(s) Private Start at 2001 Variable The project network for instances is sized for the maximum number of instance interfaces attached. Next Steps Once End Customer has verified remote connectivity from the Cisco 2901 and all UCS Controller Servers have their BIOS settings in place please contact the Cisco Metacloud Operations Center. Once remote connectivity has been established from the Cisco Metacloud Operations Center, Cisco will be able to fully provision your Cisco Metacloud service. This includes but is not limited to adding your compute servers and configuring Cisco Block Storage or Customer provided storage. Note: Contact information for Cisco Metacloud Operations will be provided after your ordered is placed Cisco and/or its affiliates. All rights reserved. This document is Cisco Public. 21