Planning and Designing Virtual Unified Communication Solution

Transcription

1 Planning and Designing Virtual Unified Communication Solution Xavier Hémery Consulting Systems Engineer CCIE Voice - #17602

2 Does it look familiar?

3 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design Presenter Xavier Hémery Consulting System Engineer 3

4 - Planning and Designing Virtual Unified Collaboration Solution - Abstract Traditionally, Cisco Unified Communications used to be installed on MCS servers. This is changing with server virtualization. This session will discuss how to plan and design a Unified Communications deployment using VMware Virtualization. This topic is sometimes referred as UC on UCS. This session will cover Unified Communications support on the UCS platforms and Specifications-Based Hardware Support platforms which allow for example support on 3 rd party servers. It will also cover how to size a deployment and how to place UC Virtual Machines on the servers, and will provide design considerations around storage design, networking, and redundancy. Prerequisite: Basic understanding of the Cisco Unified Communications solution. Basic networking and data center knowledge. 4

5 This information is subject to change at the sole discretion of Cisco, and Cisco will have no liability for delay in the delivery or failure to deliver any of the products or features set forth in this document. 5

6 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 6

7 Virtualization: Why? Benefits from Hypervisor View Partitioning Isolation Run multiple virtual machines (VM) on single physical server Encapsulation Each VM is isolated from other VM on same server Hardware Independence VM stores systems information in files 7 Run a VM on any server without modification Budget savings with less cabling, less power for cooling etc.

8 Evolution: Physical to Virtual VM for CUCM Pub VM for CUCM Sub VM for Mediasense VM for CCE Physical Servers Virtual Servers (VMs) on Physical Server VM VM VM VM Cisco UC Application UC App UC App UC App UC App MCS Server Hardware ESXi Hypervisor UCS Server Hardware CPU Memory NIC Drive CPU Memory NIC Storage 8

9 Evolution: Physical to Virtual Application Design: Same VM for CUCM Pub VM for CUCM Sub VM for Mediasense VM for CCE Physical Servers Virtual Servers (VMs) on Physical Server Cisco UC Application MCS Server Hardware Media Convergence Servers (MCS) CPUEnd Memoryof NICSale Drivein October 2013 Hardware Design: Different 9 CUCM 10 and above is a VM VM VM VM virtual-only release UC App UC App UC App UC App ESXi Hypervisor UCS Server Hardware CPU Memory NIC Storage

10 Virtualization Overview and Definitions Guest - Cisco Collaboration application VM Hypervisor - VMware vsphere ESXi Host - Physical server Guest Hypervisor Definition Virtual Machine Software allowing to run VMs on a host Support Cisco Collaboration application release 8.0(2)+ in general VMware vsphere ESXi only. Release 4.0, 4.1, 5.0, 5.1, or 5.5 Host Physical Server Cisco UCS or other VMware compatible servers 10

11 OVA Open Virtualization Archive OVA OVA Application VMFS Storage Block Block Block Chunk Block Block Block Chunk Application Block Block VMFS Block Storage Chunk Block Block Block Chunk Partition Alignment OVA (Open Virtualization Archive) Definition File containing VM configuration (e.g. virtual hardware) Support OVAs must be used when installing a VM. Ensure proper VM configuration and application partition alignment 11

12 New vocabulary Cores vs Sockets Sockets = physical processor in server Cores = vcpu = division of processing within a socket Storage Types Storage Area Networks (SAN) EMC or NetApp Fiber Channel (FC) / Fiber Channel over Ethernet (FCoE) - preferred NFS / iscsi more restrictive Direct Attached Storage (DAS) C-Series only Input/Output Operations Per Second (IOPS) Performance measurement to benchmark computer storage devices (DAS & SAN) Core 0 Core 2 Core 1 Core 3 12

13 UC Apps vs. Traditional Business Apps Why we have rules Tier 0 Collaboration Business-critical Tier 0 app higher uptime expectation Degradation = Outage Higher Predictability required Network-centric and People-centric More client/client, More Session-based Different resource usage Real-time less tolerant of loss, delay and jitter Application footprint" not what Server admin used to CPU benchmarks usually don t correlate well Tier 1/2 Tier 3 CRM, ERP, HRIS, directory, groupware, , etc. Blah, blah File/print, utilities, etc. Voice Video IM etc. Blah, blah Less cores, RAM, disk space More MHz and disk IOPS 13

14 Real Time Application Impacts CPU oversubscription = CPU performance issues Disk Alignment = Application Impact < 20ms for Physical Device Latency Disk Latency < 2ms for Kernel Latency 14

15 UC in a Virtualized Environment "DocWiki" PPDIOO for UC virtualization Covers all aspects of UC Virtualization (UC applications, VMware and platform support for UC virtualization Always up to date check back often 15

16 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 16

17 HW Platform Options Packaged Collaboration Solution (PCS) Tested Reference Configuration (TRC) Specs-Based Any vendor 17

18 TRC vs. Specs-Based Packaged Collaboration Solution TRC Specs-Based Business Edition 6000 Business Edition 7000 UC on UCS UC on UCS 3 rd -Party Any Server Choices (Leverage Existing Investment) More More More Assurance Simplicity Ease of Deployment TAC Notes 18

19 UC Virtualization Deployment Models 1. Packaged Collaboration Solution (PCS) Including Business Edition 6000 for Midmarket Including Business Edition 7000 for Enterprise PCS 2. UC on UCS - Tested Reference Configurations (TRC) TRC UCS B/C 3. UC on UCS - Specs-based support Specs-Based Including Hosted Collaboration Solution for Service Provider Including UC on Vblock, FlexPod, ExpressPod for Premise/Managed Enterprise Including UCS E-Series for Mid-Market vcenter required VMware Supported Servers 4. 3 rd -Party Specs-based support Specs-Based Any vendors on VMware HCL Guide at vcenter required 19

20 Tested Reference Configurations (TRC) T Tested R Reference C Configuration Tested by Cisco Guaranteed Application Performance One size fits all for all the Collaboration apps Reference for Specs- Based Fixed-Configuration from Cisco Fixed HW configurations TRC UC on UCS Chassis Based: FC SAN only, Full or Halfwidth blade Business Edition 6000 PCS Business Edition 7000 Software and Hardware Bundle for ease of deployment Rack Mount Based: DAS only, 1 to 2 Rack Unit Tested Reference Configurations (TRC): 20 Rack Mount Based: DAS only, 1 or 2 Rack Unit depending on BE6K versus BE7K

21 Packaged Collaboration Solution Cisco Business Edition Portfolio PCS Purpose-built for Midmarket Cisco Business Edition 6000 (BE6000) Optimized for Up to 1000 users TRC Modular Design for Enterprise Cisco Business Edition 7000 (BE7000) Optimized for users TRC Simplified UC on UCS Offering Runs on Tested Reference Configuration Faster & Easier Quoting as well as Deploying Hardware, Hypervisor & OVAs Factory Installed 21

22 Server Performance Type Full versus Restricted CUCM 8.6, 9.X, 10.X Full UC Performance CPU Restricted UC Performance CPU = Small CUCM 1K CUCM 2.5K CUCM 7.5K CUCM 10K CUCM 1K CUCM 2.5K CUCM 7.5K CUCM 10K Some apps require certain OVAs for certain CPU/TRC. Some UC apps won t have OVA for all TRCs (C220 M3S TRC#2) 22

23 # Apps TRC Business Edition and TRC Platforms today Restricted Performance Full Performance BE6000 HD VM VM VM VM BE7000 VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM BE 6000 HD Server (Small Plus TRC, 16c) BE6000 MD VM VM VM VM VM BE 6000 MD Server (Small TRC, 8c) <1K users ( K devices) VM VM VM VM VM BE7000 (Medium TRC*, 12c) VM VM VM VM Medium TRC*, 8c 1K-5K users (<15K devices) Large TRCs*, 16c #Users 23 Extra-Large TRCs*, 20c >5K users (>15K devices) Legend 8c: 8 cores 2X-Large TRC*, 40c *VM counts are averages/examples only and vary by design.

24 # Apps TRC Business Edition and TRC Platforms 2015 Restricted Performance VM VM VM VM VM VM VM VM VM BE 6000 H Server (Small Plus TRC, 16c) C220 M4 BE6000M BE6000H VM VM VM VM VM BE7000M VM VM VM VM VM BE7000M (Medium TRC*, 12c) Full Performance BE7000H VM VM VM VM VM VM VM VM VM BE7000H (Large TRC*, 20c) C240 M4 BE 6000 M Server (Small TRC, 8c) C220 M4 BE6000S VM VM VM VM BE 6000 S 2921 (Extra Small TRC, 4c) <1K users ( K devices) 1K-5K users (<15K devices) C240 M4 #Users 24 VM VM VM VM VM VM VM VM VM Large TRC*, 20c B200 M4 >5K users (>15K devices) Legend 8c: 8 cores *VM counts are averages/examples only and vary by design.

25 On-Premise Family Model Options Basic IPT to Full Collaboration PCS Cisco Packaged Collaboration Solutions Now Serving Companies of Every Size BE7000 Family Small-scale BE6000S 150 Users 300 Devices Core UC Apps Only Ideal for small-scale Voice/IPT Point to point video BE6000 Family Total Per System Support Medium-scale BE6000M 1000 Users 1200 Devices 4+1 Apps Ideal for medium-scale Voice/IPT Contact Center 3 rd -Party Apps Medium-scale BE6000H 1000 Users 2500 Devices 8+1 Apps Different Capacities Ideal for medium-scale Voice/IPT Conferencing Contact Center 3 rd -Party Apps Large-scale BE7000M Users Devices 4-6 Apps Ideal for large-scale Voice/IPT Contact Center 3 rd Party Apps Large-scale BE7000H Users Devices 8-10 Apps Ideal for large-scale Voice/IPT Conferencing Contact Center 3 rd -Party Apps Customer Size

26 TRCs TRC Shipping 2XL B440 M2 TRC#1 (UCS-B440M2-VCDL1) Full-width Blade, Quad E (40-cores), 256Gb RAM, Diskless UC on UCS XL L M B230 M2 TRC#1 (UCS-B230M2-VCDL1) Half-width Blade, Dual E (20-cores), 128Gb RAM, Diskless C260 M2 TRC#1 (UCS-C260M2-VCD2) 2RU Rack-mount, Dual E (20-cores), 128Gb RAM, DAS B200 M3 TRC#1 (UCUCS-EZ-B200M3 or UCSB-EZ-UC-B200M3) Half-width Blade, Dual E (16-cores), 96Gb RAM, Diskless C240 M3S TRC#1 (UCUCS-EZ-C240M3S) 2RU Rack-mount, Dual E (16-cores), 96Gb RAM, DAS C220 M3S TRC#1 (UCUCS-EZ-C220M3S) 1RU Rack-mount, Dual E (8-cores), 64Gb RAM, DAS C240 M3S TRC#1 (BE7K-K9 / BE7K-K9-XU) 2RU Rack-mount, Dual E (16-cores), 96Gb RAM, DAS S+ C220 M3S TRC#2 (BE6K-STBDL-PLS-K9= / -XU=) 1RU Rack-mount, Dual E (16-cores), 48Gb RAM, DAS S C220 M3S TRC#2 (BE6K-ST-BDL-K9= / -XU= or UCSC-C220-M3SBE=) 1RU Rack-mount, Dual E (8-cores), 32Gb RAM, DAS UC on UCS UC on UCS UC on UCS UC on UCS UC on UCS BE7000 BE6000 HD UC on UCS BE6000 MD UC on UCS More at: 26

27 TRC TRC Allowed Deviations Deviations allowed Component Modifications Allowed? Server Model/Generation Model CPU Quantity and # cores Speed Physical Memory Quantity, RAID, technology DAS Storage Disk size Disk speed C-series (NIC Type, vendor, technology) Adapters C-series (NIC card quantity) B-series (Mezzanine card) Within the same family Yes, if higher Yes, if higher Yes, if higher Yes, if higher Yes 27

28 Specs-Based Virtualization Specs-Based UCS & Third Party servers on VMware hardware compatible list (HCL) Cisco supports UC applications only and not performance of platform Not all UC applications support Specs Based (UCCE) Need to review Docwiki for current support Positioned for customers with extensive virtualization experience Customer assumes responsibility/risk to configure/scale hardware adequately Customer expected to perform troubleshooting via VMware vcenter prior to calling TAC 28

29 Specs-Based - Requirements TRC Specs-Based Limited Cisco UCS servers Limited CPUs UCS C: Limited DAS options UCS B: FC SAN only Any server More CPUs options Flexible DAS & SAN ANY Server (VMware compatible) E5-2400/2600/4600 v1/v2/v GHz (Restricted Performance) E5-2600/4600 v1/v2/v GHz E7-2800/4800/ GHz (Restricted Performance) E7-2800/4800/ GHz E7-2800/4800/8800 v GHz Any Storage compatible with Server and VMware E.g. Flexible DAS config, FC, FCoE, iscsi, NFS NAS UCS C: Limited Options UCS B: Flexible Adapters Optional Flexible adapters Required Any adapters compatible with server and VMware vcenter required (for logs and historical statistics) 29

30 Specs-Based Allowed CPU s Support varies with CPU Family and speed CPU Architecture and Model Full UC Performance CPU Restricted Performance CPU Intel Xeon E7-2800v2, E7-4800v2 or E7-8800v2 Intel Xeon E7-2800, E or E ,5GHz+ 2,4GHz+ 2,0GHz-2,49GHz 2,0GHz-2,39GHz Intel Xeon E5-2600v3 2,5GHz+ 2,0GHz-2,49GHz Intel Xeon E5-2600v2 or E5-4600v2 2,5GHz+ 2,0GHz-2,49GHz Intel Xeon E or E ,5GHz+ 2,0GHz-2,49GHz Intel Xeon E5-2400v2 Not supported 2,0GHz+ Intel Xeon E Not supported 2,0GHz+

31 What about ISR blades? AXP SRE Older - no support UCS Express SREV-910 X Other SREV-9xx Older - limited support - CUCM 8.6 controlled release: US DoD only. - CUC specs-based only - MediaSense 8.5(4)+ UCS E-Series E140D/DP M1 E160D M2 Other UCS E-Series E180D M2 E160D M1 E160DP M1 E140DP M1 E140S M1 E140S M2 Specs-based-only support - Restricted UC Perf. CPU (E5-24xx GHz) - Expect low scale (<4 VMs, few 100 users) due to IOPS bottlenecks - DP: to check if less drive. No support in general (except for CUC/specs-based) - CPU GHz too slow for Specs-based (E3, or not enough available drives/iops.

32 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 32

33 Which Hypervisor Cisco virtualized UC Support Product Description Support VMware Server A program that gets installed on an OS such as Windows or Linux No VMware Workstation Similar to Server, but adds features like snapshots. For developers. No VMWare ESX Bare-metal hypervisor in a linux-like operating system. No VMWare vsphere ESXi ESXi Hypervisor is a bare-metal thin hypervisor that provides real-time performance. Version support UC application dependent. VMWare vcenter Server * Manages multiple ESX/ESXi hosts Yes VMware Player Same as Server. Runs VMs. Can t create VMs. No VMware Fusion Same as Player for Mac OS X No Any other Hypervisor * vcenter required for Specs-Based Yes No 33

34 UC Applications ESXi version support ESXi releases: 4.0, 4.1,5.0, 5.1, 5.5 (varies with UC application) All 10.X UC applications support 5.1, most support 5.5 Application ESXi / vsphere 5.1 ESXi / vsphere 5.5 CUCM 8.0(2) or later 9.X or later CUP/Unified IM&P 8.6(4) or later 9.X or later CUC 8.0(2) or later 9.X or later CCX 8.5(1) SU4 or later 9.X or later vcenter required for Specs-Based Complete list in the docwiki: 34

35 Which ESXi Package Should I Use? Attribute Cisco UC Virt. Hypervisor 5.1/5.5 Cisco UC Virt. Foundation 5.1/5.5 VMware vsphere ESXi 5.1 or 5.5 Standard, Enterprise or Enterprise Plus Edition Availability Package Collaboration Solution only Any Any Price and SKU included Yes Yes Installation Pre-installed on BE7000 server Field-installed Field-installed License & Media Logistics Cisco.com, simplified Cisco.com, simplified Standard vmware.com Support Cisco UC TAC Cisco UC TAC Cisco DC TAC if OEM, otherwise VMware Per-app size Limited (8vcpu per VM) Limited (8vcpu per VM) Higher based on Edition vcenter Integration No Yes Yes Advanced VMware Features (HA, vmotion, N1KV, etc.) Cisco Prime Collaboration Deployment (PCD) No No Yes based on Edition Incompatible Compatible Compatible 3 rd -party Apps allowed? Limited (see BE6000/7000 coresidency policy document) Limited (same as BE6000/7000) 35 Any allowed

36 Which ESXi Package Should I Use? Attribute Price and SKU Cisco UC Virt. Hypervisor 5.1/5.5 5.X License included with BE6000/BE7000 server price (+ESW/UCSS) VMW-VS5-HYP-K9 Cisco UC Virt. Foundation 5.1/5.5 Add-on SKU, extra license cost ($2499+ESW/UCSS) R-VMW-UC-FND5-K9 VMware vsphere ESXi 5.1 or 5.5 Standard, Enterprise or Enterprise Plus Edition Add-on SKU, extra license cost ($3732 to $13,106 + ISV1) VMW-VS5-ST-1A=, VMW-VS5-ET-1A=, VMW-VS5-EP-1A=, etc. Install Pre-installed on BE7000 server Field-installed Field-installed Choices (Leverage Existing Investment) License & Media Logistics Cisco.com, simplified Cisco.com, simplified Standard vmware.com More More Support Cisco TAC Cisco TAC Cisco TAC if OEM, otherwise VMware Per-app size Limited (8vcpu per VM) Limited (8vcpu per VM) Higher based on Edition More Assurance Simplicity Ease of Deployment vcenter Integration No Yes Yes Advanced VMware Features (HA, vmotion, N1KV, etc.) Cisco Prime Collaboration Deployment (PCD) No No Yes based on Edition Incompatible Compatible Compatible 3 rd -party Apps allowed? Limited (see BE6000/7000 coresidency policy document) Limited (same as BE6000/7000) 36 Any allowed

37 UC Applications VMware Feature Support Features vmotion Storage vmotion Copy Clone 2015 Cisco and/or its affiliates. All rights reserved. 37 HA VMware DRS CUCM Y (C) Y (C) Y (C) Y (C) N N CUP / IM & Presence Y (C) N Y (C) Y (C) N N CUC Y (P) N Y (C) Y (C) N N CCX Y (C) Y (C) Y (C) N N N Unified Communications VMware Requirements: Fault Tolerance vmotion Y(C): supported with Caveat: Can be done with live traffic, but slight risks to impact calls. Y(P): Partial: During maintenance window only. Copy / Clone Y(C): supported with Caveat: Shutdown VM first vcenter required for these features VMware Distributed Resource Scheduler (DRS) Not supported at this time. No real benefits since Oversubscription is not supported LEGEND: Y(C) Supported with Caveats Y(P) Partial or Limited N Not Supported

38 High Availability (HA) Design Application HA vs. VMware HA Collaboration application HA Best Practice: VMware HA is allowed but Use Cisco Collaboration application built-in redundancy first VMware HA DAS Storage Support Yes No (requires shared storage) VM file system corruption Protection Yes No Failover Time Faster Longer Additional mechanisms required to ensure failover does not result in an unsupported configuration Fully Supported Yes No No Yes 38

39 Backup Strategy Application vs. VMware Backup Cisco Disaster Recovery System (DRS) VMware Copy / Clone / vdp Backup can be taken when VM is running Yes No (Power Off VM) Restore time Longer Short Best Practice: Can use VMware Copy for fast restore but Always perform a DRS Back Up 39

40 VMware FT Redundancy Blade 2 Blade 2 Not supported at this time Only works with VMs with 1 vcpu Costly (double hardware requirement) VMware FT doesn t provide redundancy if the UC app crashes (both VMs would crash) Best Practice: Use UC application built-in redundancy instead 40

41 VMWare Geographic Redundancy VMware Site Recovery Manager (SRM) Blade 1 Blade 2 WAN Allowed VMware SRM doesn t provide redundancy if issues with VM file system as opposed to the UC app built-in redundancy UC VMs have same IP address in both data centers (needs OTV for example) Best Practice: Storage Replication Use UC application built-in redundancy (clustering over the WAN) 41

42 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 42

43 How to Design - Overview Application Design Overall Application Design Application Sizing 1 2 VM Placement 43

44 Application Design 1 Application Design Overall Application Design - Architecture and Deployment Models - High Availability Design If <1000 users: - Apply BE6K design - CUCST not required If <5000 users: If >5000 users or outside of PA - Preferred Architecture assumptions: (PA) design - Application Sizing (CUCST) - CUCST not required - VM Configuration Selection - Determine Number of VMs Required 44

45 How to Determine the Number of VMs Required Estimate Sizing Design Guides - SRND Validate Sizing for complex deployment Collaboration Sizing Tool (CST) Cisco Collaboration Sizing Tool (CST) available to Partners, Cisco Account Teams, and Cisco Advanced Services 45

46 How to Select a VM Configuration Select Appropriate VM Config Consider: # users and # vcpu VM Config compatible with Hardware? Ye s No Change VM Config or Platform Application VM Config vcpu Restricted UC Performance CPU: BE6000, Small(+) TRC Unified CM 10.0 IM & Presence 10.0 Unity Connection Full UC Performance CPU: BE7000, Medium TRC or larger 10, , , , , , , ,000 full UC 1 20, , , ,000 1 Docwiki: 46

47 VM templates (OVA) Product Scale (users) vcpu vram (GB) vdisk (GB) Server Performance Type Unified CM 10.X Unity Connection 10.x Unified IM & Presence 10.x Unified CCX 10.x 10, x 110 Full only 7, x 110 Full only 2, x 80 Full only 1, x 80 Restricted and Full 20, x 300/500 Full only 10, x 146/300/500 Full only 5, x 200 Full only 1,000 2/1 4 1 x 160 Restricted and Full 15,000 full UC x80 Full only 1,000 full UC x 80 Restricted and Full 400 agents x 146 Full only 300 agents x 146 Full only 100 agents x 146 Restricted and Full Servers with Restricted UC performance CPU can use only small templates Servers with Restricted UC CPU Performance C220M3S TRC#1 TRC#2, TRC#3 More templates at: 47

48 VM Placement VM Placement Goal: Determine the number of physical servers required How to do VM Placement? - Manual placement using the rules in the docwiki - Use VM Placement Tool (VMPT) 2 Resources - Docwiki: - VM Placement Tool (VMPT): 48

49 CPU Resources Physical Core Dual sockets, 6 cores per socket Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Consider the number of PHYSICAL cores on the server Do not consider the number of LOGICAL cores obtained through Hyperthreading CPU Cores Logical Processors 49

50 CPU Resources - Mapping Mapping: One vcpu = One physical core No oversubscription: The number of vcpus can t exceed the number of physical cores vcpu UCM-PUB UCM-SUB1 IM&P-1 CER-1 CCX-1 Cores 50

51 CPU Resources Unity Connection With Unity Connection only: Reserve a spare core for the ESXi scheduler 51

52 Other Hardware Resources Memory No Oversubscription Reserve RAM for ESXi scheduler ESXi 5.1 or earlier: 2 GB ESXi 5.5: 4 GB Storage Ensure enough storage space is available Thick vs. Thin provisioning TRC with DAS: Thick provisioning required Specs-Based or SAN: Thick provisioning recommended, Thin provisioning allowed 52

53 How to mix apps on a server (Co-Residency)? None Limited UC with UC only Full None: No co-res Limited: Co-res with specific applications suite and OVA size UC with UC Only: Co-res with Cisco UC applications only (see previous Full: Co-res among Cisco UC application, Cisco non-uc and 3 rd Party applications Example: MeetingPlace None Virtualization Domain Limited Virtualization Domain Examples: CCE Logger 8.x CCE HDS 8.x Examples: CUCM 8.6(1)- CCX 8.0 CCE Logger 9.x UC with UC only Virtualization Domain Examples: CUCM 8.6(2)+ CCX 8.5+ Most UC apps today Full co-res Virtualization Domain 3rd Party Apps (Cisco or Non Cisco) ) Examples: Cisco Nexus v1k Cisco ISE Cisco vwlc VMware vcenter VMware VSA Windows AD Billing SW

54 Which UC applications supports co-residency? Most UC applications support UC with UC only or Full co-residency Newer UC Releases more likely to support Full co-residency UC Applications Unified CM Unity Connection Unified Presence / IM & Presence Unified Contact Center Express Co-residency Support 8.0(2) to 8.6(1): UC with UC only 8.6(2)+: Full 8.0(2) to 8.6(1): UC with UC only 8.6(2)+: Full 8.0(2) to 8.5: UC with UC only 8.6(1)+: Full 8.0(x): UC with UC only 8.5(x)+: Full Specific information is located on every application page Example CUCM 54

55 Co-residency with 3 rd party applications ( Full Co-Residency) UC on UCS rules also imposed on 3 rd party VMs (e.g. no Oversubscription allowed for 3 rd party apps) The co-residency rules and categories apply to TRC and Specs-Based Number and Type of 3 rd party co-resident applications limited with Cisco Collaboration virtualization SW OEMs (Cisco UC Virt. Hypervisor/Foundation, default ship with BE6000/BE7000) More info in the docwiki and TAC Tech Note:

56 Business Edition and 3rd party Co-residency BE6000 S: Fixed 5 Cisco Collaboration apps only BE6000 M: 4 applications + 1 management per server BE6000 H: 8 applications + 1 management per server BE7000 M and BE7000 H : # applications limited per 1:1 vcpu:core If using Cisco UC Virtualization Hypervisor or UC Virtualization Foundation Business Edition support 3rd-party apps in Collaboration category of Solutions Plus or Cisco Developer Network with a maximum count of 3 3rd-party VMs per server, and 6 in total. Note: For BE7000 M/H using other VMware licence, standard TRC co-residency rules apply 56

57 Design: Collaboration VM Placement Tool (VMPT) Optimized for BE6K, BE7K and UC on UCS DAS TRCs. Future = improve Specs-based support. 2. Implements most rules from & provides Embedded Error Correction 3. Use after SRND or Sizing Tool manually input. Future = launch from Sizing Tool with auto-populate. 4. Exports PDF of your design. Manually convert to UCS/VMware SKUs. Exploring future BOM generation. 5. Post roadmap asks here:

58 VM Placement Example with 5000 users Best Practices Spread VMs across different nodes to minimize failure impact. Provide spare capacity for future growth, server maintenance, or Collaboration application upgrades possibly requiring more resources. 58

59 Agenda Overview Platforms VMWare Selection Sizing and VM placement Planning and Designing UC Virtualization is being simplified Storage Design Network Design Deployment Design 59

60 Storage Design Overview SAN/NAS DAS TRC Specs-Based Any Vendor compatible with server and VMware Meet Performance Requirements FC / FCoE Only Flexible Storage Protocol TRC Fixed BOM Fixed RAID configuration Specs-Based Compatible with server and VMware Meet Performance Requirements Flexible Disk/RAID Configuration 60

61 Storage Array Requirements TRC Specs-Based Storage Vendor Any Vendor compatible with Server and VMware Storage Technology FC only Flexible (FCoE, iscsi, NFS ) Other Requirements Meet Performance Requirements Hypervisor Performance Requirements SAN Kernel Command Latency < 4ms Physical Device Command Latency < 20ms NAS Total Guest Latency < 24ms 61

62 SAN Storage Disk provisioning OVA / OVF preset configuration: Thick Provisioning Lazy Zeroed SAN thin pooling / provisioning: Even though the virtual machine template is set to thick provisioning most customers will ignore it on the SAN level. Therefore the VM will be provided with disk space as needed. SAN storage thin pooling / provisioning is supported: For Your Reference If the SAN is set to ignore the thick provisioning of the virtual machine the customer / partner takes over the responsibility that it will always get the disk space required (OVA reference). In case the VM will run out of disk space due to the lack of SAN storage a DRS system recovery must be done. 62

63 SAN Storage Performance Requirements < 20ms for Physical Device Latency < 2ms for Kernel Latency 63

64 IOPS Guidelines: To help keep Latency low Unified CM BHCA Average IOPS 10K ~35 25K ~50 50K ~100 CUCM upgrades generate 800 to 1200 IOPS in addition to steady state IOPS Presence VM Size Average IOPS 1000 users ~60 Unity Connection VM size Average IOPS Peak IOPS 2 vcpu (5,000 users) ~130 ~715 4 vcpu (10,000 users) ~220 ~870 Unified CCX VM Size Average IOPS Peak IOPS 2 vcpu (300 agents) ~150 ~1500 More details in the docwiki: 64

65 IOPS Calculation Example - Summary 3x CUCM Pub/TFTP 2x CUCM Subs Typical Avg IOPS (steady state) 300 (93%-98% seq. writes) 200 (93%-98% seq. writes) Occasional Avg IOPS (steady state for a few hours) 300 (93%-98% seq. writes) 200 (93%-98% seq. writes) 2x IM&P x CUC 402 (95% seq. writes) 1x DRS 50 1x Upgrade 402 (95% seq. writes) 1,200 (mostly seq. writes) 65 Additional IOPS during Spikes (few minutes) Total: 1,144 Read: 1,032, Write: 442 Total: 1,144 TOTAL 1212 (~95% seq. writes) 2,412 (mostly seq. writes) Read: 1,032, Write: 442 Provide as much information as possible to the Storage Team

66 SAN Array LUN Best Practices HDD Recommendation LUN Size Restriction UC VM App Per LUN LUN Size Recommendation FC class (e.g 450 GB 15K, 300 GB 15K) ~ 180 IOPS Must never be greater than 2 TB Between 4 & 8 (different UC apps require different space requirement based on OVA Between 500 GB & 1.5 TB PUB VM1 SUB1 VM2 UCCX1 VM3 CUP1 VM4 SUB2 VM1 SUB3 VM2 UCCX2 VM3 CUP2 VM4 LUN 1 (720 GB) LUN 2 (720 GB) Single RAID5 Group (1.4 TB Usable Space) HD 1 450gig 15K RPM HD 2 450gig 15K RPM HD 3 450gig 15K RPM HD 4 450gig 15K RPM HD 5 450gig 15K RPM 66

67 DAS Storage configuration with TRCs XL C260 M2 TRC#1 2x [8x 300GB 10K disk in RAID 5] Megaraid with Battery Backup L C240 M3S TRC#1 2x [8x 300GB 15K disk in RAID 5] Megaraid with Battery Backup/Super Cap M C240 M3S TRC#2 1x [12x 300GB 10K disk in RAID 5] Megaraid with Battery Backup M C220 M3S TRC#1 1x [8x 300GB 15K disk in RAID 5] Megaraid with Battery Backup/Super Cap S C220 M3S TRC#2 1x [4x 500GB 7.2K disk in RAID 10] Megaraid with Battery Backup S+ C220 M3S TRC#3 1x [8x 300GB 15K disk in RAID 5] Megaraid with Battery Backup/Super Cap 67

68 DAS Storage Requirements BOM (Disks/RAID controllers) TRC and BE6000 / BE7000 Fixed. Exceptions: Faster/Larger disks ok Specs-Based Flexible RAID configuration Fixed Flexible Thick/Thin Provisioning Thick only Thick recommended Storage Performance Design No IOPS calculation needed Use TRC as reference Or calculate IOPS 68

69 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 69

70 Networking with Virtualization Overview UC Application LAN Management LAN Traffic outside of the ESXi Host UC applications traffic and Management traffic for ESXi Host ESXi vmnic Corresponds to physical NIC NIC teaming support vswitch / vds / Nexus 1000v VM vnic attached to Port Group vnic Virtual NIC of the VM No NIC teaming support on UC on UCS ESXi Host 70

71 Network Design UCS C-Series How many Ethernet ports do I need? To determine number of NIC ports required, use Bandwidth Formulas available in the SRNDs. For Unified CM, calculate BW requirements using the Collaboration SRND: Intra-Cluster Communications Services (ICCS) Database Replication Voice Signaling (SIP/SCCP/MGCP) Voice Media (with MoH, SW MTP, SW CFB, etc ) TFTP Only account for traffic in/out of the host (not within host). Media Traffic (especially Video) typically consumes more Bandwidth With redundancy, need 2x number of ports. TRC provide more NIC than required 71

72 Plan for Network Redundancy C-series Connect to at least 2 upstream physical switches Plan for redundancy when LOM or NIC card fails Redundancy for VM traffic is more important than for ESXi Management traffic Use VMware NIC teaming for redundancy and load sharing ESXi Management VM Traffic CIMC NIC Card (PCI-E) Management Port (CIMC) LOM (LAN-On- Motherboard) 72

73 Best Practice: VMware NIC Teaming for C-series Port Channel Single virtual Port Channel (vpc) Virtual Switching System (VSS) / virtual Port Channel (vpc) required Two Port Channel (no vpc) VSS/vPC not required but No physical switch redundancy since most UC applications have only one vnic EtherChannel EtherChannel vpc Peerlink vmnic0 vmnic1 vmnic2 vmnic3 vswitch Route based on IP hash vmnic0 vmnic1 vmnic2 vmnic3 vswitch1 vswitch2 vnic 1 vnic

74 VMware NIC Teaming for C-series Overview For Your Reference VMware NIC Teaming combines multiple uplink ports (vmnics) in a virtual Switch Port Channel Uplink switch Etherchannel NIC Teaming configuration in vswitch Route based on IP hash Note: Standby connectors cannot be configured 74

75 VMware NIC Teaming for C-series Overview For Your Reference VMware NIC Teaming combines multiple uplink ports (vmnics) in a virtual Switch No Port Channel / No vpc Uplink switch No Etherchannel NIC Teaming configuration in vswitch Route based on the originating virtual port ID Route based on source MAC hash 75

76 Plan for Network Redundancy B-series With B-series: Plan for redundancy in case a Fabric Interconnect or upstream switch fails ETH 1 ETH 2 MGMT MGMT 6100 Fabric A Cluster 6100 Fabric B FEX A FEX B CNA vhba 1 vhba 2 vmnic 1 vmnic2 vswitch or vds vnic 1 vnic 2 vnic 3 vnic 4 76

77 Number of Network Ports available with TRCs B440 M2 TRC#1 Cisco VIC B230 M2 TRC#1 Cisco VIC C260 M2 TRC#1 LOM + NIC B200 M3 TRC#1 Cisco VIC 1240 C240 M3S TRC#1/2 LOM + NIC C220 M3S TRC#1 LOM + NIC C220 M3S TRC#2 LOM only 2x 10GbE ports for LAN+FC 2x 10GbE ports for LAN+FC 6x 1GbE ports for LAN (2 LOM + 4 PCI-E) 4x 10GbE ports for LAN+FC 12x 1GbE ports for LAN (4 LOM + 8 PCI-E) 6x 1GbE ports for LAN (2 LOM + 4 PCI-E) 2x 1GbE ports for LAN (2 LOM) 2x 10GbE ports shared 2x 10GbE ports shared 4x dedicated 1GbE ports 2x/4x 10GbE ports shared 10x dedicated 1GbE ports 4x dedicated 1GbE ports 2x 1GbE ports (shared with ESXi Mgt) For most UC deployments, use all available ports, and this should be enough to handle all traffic, even when accounting for redundancy. 77

78 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 78

79 Installing UC app Prepare Storage Install VMware ESXi Create new VM using an OVA Use the OVA available on cisco.com for virtual HW settings and for Disk Alignment Use the the correct version of the UC app CUCM_10.5_vmv8_v1.8.ova UC App UC App release VM Hardware Version ESXi4:vmv7, ESXi5:vmv8 OVA Version When deploying the VM template, select VM size Install UC application Cisco Collaboration on Virtual Servers: 79

80 Deployment Models Options Again Same design rules with virtualization Same deployment models Same software Mixing MCS and virtualized servers in the same cluster with CSR 10 no longer supported Exception: Services based on USB and Serial Port not supported SMDI for legacy voice mail integration Fixed audio MoH (live audio stream) Workaround: Multicast MoH only using ISR router with an E&M or FXO port MoH Server Multicast enabled Live Audio Source More details in the UC SRND: 80

81 Migration to UCS Overview CUCM 8.0(1) or earlier UC Software Upgrade CUCM 8.0(2) or later Hardware Migration Upgrade to a release that support virtualization if current release doesn t support virtualization. For example, virtualization requires 8.0(2)+ with CUCM, CUC, CUP, CCX. Follow the normal Server Replacement procedure: DRS backup, Install using the same UC release, DRS restore Replacing a Single Server or Cluster for Cisco Unified Communications Manager: 81

82 What if my MCS server is quite old? Bridge Upgrade In place upgrade Bridge Upgrade Hardware Migration CUCM 9.1(2) Upgrade CUCM 10.X Bridge upgrade for old MCS servers which do not support a UC release supported for Virtualization. With a Bridge Upgrade, the old hardware can be used for the upgrade, but the UC application will be shut down after the upgrade. Only possible operation after the upgrade is DRS backup. Examples: MCS-7845H3.0/MCS-7845H1: Supports Bridge Upgrade to CUCM 8.0.(2)-8.6(x) List of Supported Servers for CUCM: 82

83 What if my if I want a short downtime? Jump Upgrade Migrate to UCS in lab before upgrading CUCM 6.1(4), 6.1(5) CUCM 7.1(3), 7.1(5) Hardware Migration CUCM 6.1(4)-6.1(5) CUCM 7.1(3), 7.1(5) Upgrade CUCM 9.1(2) Upgrade Virtualize on older CUCM release Isolated environment Upgrade to 9.1(2) Rebuild VM for partition alignment (install again with DRS backup/restore) Switch over to the new virtualized CUCM cluster (flash-cut) CUCM 10.X 83

84 What if I want a simpler path? Upgrade With Prime Collaboration Deployment to CUCM 10.X CUCM 6.1(5) or later Upgrade & Migration CUCM 10.X Version 10 or higher virtual only Prime Collaboration Deployment Migration Manage the migration of legacy CUCM clusters (as old as 6.1.5) to new VM based servers on 10.0 to facilitate Migration. CUC, CUCM IM/P, UCCX Upgrade UC app software Install cop files (locales or device packs) on a cluster Switch versions Reboot Change or same IP addresses or hostnames on existing 10.x clusters Fresh install a new Unified Communication or IM&P cluster on 10.x Scheduled or immediate execution Steps are customizable (scripts) 84

85 Prime Collaboration Deployment (PCD) PCD is a VMware vapp used for management of Cisco collaboration applications: CUCM 6.X - CUCM - CUC CUCM 7.X CUCM 8.X PCD ESXi Host - CUP / IM&P - CUCCX - Management tasks (Upgrade, Switch Versions, Server Restart, Readdress, Install and Migrate) are based on collaboration application and version of the application VMware vapp is pre-configured virtual machine with OS and PCD application (1.4GB) 85 - pcd_vapp_ucos_ _vmv7_v1.2.ova

86 VM Resizing - VM Configuration changes Example 2,500 users 7,500 users In general, VM configuration changes are allowed if moving to another supported VM configuration (OVA) How to change the VM configuration? vcpu, vram: shutdown VM, make changes, start VM vdisk changes with CUCM: Increase vdisk Size: shutdown VM, make changes, start VM (COP file needed for 9.1 or earlier) Otherwise (different number of disks, smaller disks ): fresh install and DRS backup-restore. 86

87 UC Application Upgrade UC App release X UC App release Y OVA for release X OVA for release Y Verify if VM configuration (OVA specifications) is changing. Resize/Modify VM Configuration Perform the software upgrade. If applicable, change OS version in VM settings. Upgrade VMware Tools. 87

88 VMware ESXi Upgrade Example ESXi 4.1 ESXi Verify Collab apps and Platform compatibility with new ESXi release 2. Perform ESXi upgrade DAS: shutdown VM SAN: shutdown VM or temporarily relocate VM using vmotion Perform ESXi Upgrade Power on VM (or relocate back VM) 3. No Cisco UC requirement to upgrade VM hardware version (VMV) (see VMware for Best Practices) 4. Upgrade VMware Tools for each VM 88

89 Server Hardware Upgrade 1. Prepare new server (RAID configuration, install ESXi, ) 2. Relocate VMs No need to to a fresh install / DRS Shutdown VMs and copy VMs files to new server. If SAN, instead, can directly relocate VMs using vmotion during maintenance window (verify specific UC application supports vmotion) 89

90 Agenda Overview Platforms VMWare Selection Sizing and VM placement Storage Design Network Design Deployment Design 90

91 Call to Action Recommended Reading: for reading material and further resources for this session, please visit

92 Complete Your Online Session Evaluation Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt. All surveys can be completed via the Cisco Live Mobile App or the Communication Stations 92

93 93

94

95 Appendix

96 Link Summary Solution Reference Network Design (SRND): Cisco Preferred Architecture: Cisco Validated Design CVD: Docwiki: Cisco Collaboration Sizing Tool: