Planning and Designing a Cisco Unified Customer Voice Portal Deployment

Planning and Designing a Cisco Unified Customer Voice Portal Deployment BRKCCT-2020

Session Overview Fundamentals of planning and designing a Cisco Unified Customer Voice Portal (CVP) deployment Details of various functional deployment models with their call flows, design details and a migration case study This is an intermediate to advanced level session Some familiarity with the Contact Center Enterprise products is required

Agenda Architecture & Overview CVP Overview CVP Components Functional Models High Availability Designs Ingress and VXML Gateways SIP including CUPS and CUSP CVP Call Servers Planning and Design Platform and Virtualization Sizing Bandwidth, QoS and Latency Geographic Models Edge Queuing Techniques Case Study Better Late than Never Trust but Verify Load Balancers (ACE and CSS) Survivability

About The Presenters Rue Green Technical Leader Customer Collaboration Service Line Cisco Systems Advanced Services Rahul Maniktala Consulting Systems Engineer Connected Architectures Partner Organization Cisco Systems

Architecture & Overview CVP Overview

CVP Main Features Service Creation Queuing

CVP Service Creation Call Studio Easy to use, yet flexible Vibrant user community <vxml version="2.0 > <form id="form-record"> <record name="greeting" beep="true > <prompt> Please record your greeting. </prompt>

CVP Queuing SIP Call Control Queuing at the edge saves money Cisco network is the platform P$TN

CVP Overview Big Picture Customers Four Pillars of CVP Solution Agents

Architecture & Overview CVP Components CVP requires a solution level mindset

CVP Components High Level Caller Agent Conversation VXML 6 IVR 4 V 1 5 3 2 1 - New call from TDM GW to CVP 2 - New call to UCCE from CVP 3 - Play Hello World prompt 4 - CVP sends call to VXML Gateway Caller hears HW. Subsequent IVR 5 - Agent is now available 6 - CVP sends call to an agent Caller is disconnected from IVR Caller is connected to an agent

CVP Component Details Caller Agent Conversation ASR/TTS HTTP Media Server (Media Files) 6 V VXML VXML 4 1 VXML Server IVR SS SIP SS I C M SS 5 2 3 JTAPI PG RTP IP RPT SIP Proxy can be optionally inserted at 1, 4, 6 CUIC 1 - New call from TDM GW to CVP 2 - New call to UCCE from CVP 3 - Play Hello World prompt 4 - CVP sends call to VXML Gateway Caller hears IVR 5 - Agent is now available 6 - CVP sends call to an agent

Architecture & Overview Functional Models

Functional Models CVP Standalone (optionally with ICM Lookup) Automated Self Service IVR, No queuing, Limited call control, No formal agent Call director IP-enabled or TDM ACDs, TDM migration to IP, On-Net routing & transfer. Save $$ VRU-only IVR and queuing. Call control via SS7. Comprehensive Pure IP-based contact center, IVR, call control, queuing Basic service video Audio-only IVR, call control, queuing, video agent

CVP Standalone CVP Studio Voice Gateway / VXML Browser Caller (TDM or IP) 1. Incoming Call Leg (SIP or TDM) 2. HTTP / VoiceXML Documents CVP VoiceXML Server SIP HTTP

CVP Standalone Terminates calls independently of UC Manager and ICM Remotely distributed call delivery and IVR served by centralized applications avoids media across WAN For advanced speech applications allowing customization and element reusability Back-end integration flexibility (Database or web services) Use with TDM or SIP Basic Transfers (no queuing)

CVP Standalone with ICM Lookup Voice Gateway / VXML Browser PG CVP VoiceXML Server Caller (TDM or IP) 1. Incoming Call Leg (SIP or TDM) 2. HTTP / VoiceXML Documents 3. NEW CALL Route Request SIP GED-125 HTTP

CVP Standalone with ICM Lookup Typical application is to query the ICM only when the caller opts out to an agent ICM Lookup can return a Translation Route label which CVP standalone could SIP REFER back into a CVP Comprehensive farm CVP Standalone with ICM Lookup cannot process an *incoming* Translation Route label

SIP CVP Comprehensive GED-125 RTP CUCM INAP HTTP VXML Gateway 5. SIP INVITE 8. SIP INVITE SS7 4. Connect to VRU 6. HTTP / VoiceXML Documents 7. Connect to Agent PSTN Ingress Gateway CVP Call Server /VXML Server 1. Incoming Call Leg (SIP or TDM) 2. SIP INVITE 3. GED-125 New Call

CVP Comprehensive IVR, call control, queuing ICM keeps track of agents VXML Session: VXML-GW to Call Server (IVR Service) or VXML Server Components: Call Server VXML Server (optional) VXML Gateway SIP Proxy (optional) ICM and VRU PG CUCM or TDM ACD Reporting Server (optional) ASR/TTS (optional) CSS/ACE (optional)

CVP Basic Video

CVP Basic Video Audio-only IVR, call control, queuing, video agent Same call flow as comprehensive SIP Supports both audio and video calls Video supported with SIP only Video Endpoints: Cisco Unified Video Advantage TelePresence 7985G Cisco H324M 3G Gateway

Planning and Design Platform and Virtualization

HW Platform and OS Support Call Server, VXML Server, Reporting Server must use MCS-7845 or higher server Windows 2003 Standard supported for all components VXML Server Tomcat or WebSphere* Operations Console MCS-7825 or higher server Unified Call Studio Windows XP SP2/Windows Vista. Windows OS compatible HW Windows 7 in Call Studio 8.5 * Several caveats

Hardware Components in CVP 8.5 UCS B-Series Blade Server (B200M2) CVP Server Virtualization UCS with VMware ESXi 4.0 Cisco Unified SIP Proxy (CUSP) ISR based Proxy Server Application Content Engine (ACE) Next generation content engine used for load balancing ISR G2 (2900 & 3900 Series) Greater Performance ISR G2 & 15.0(1)M - supported with CVP 8.0(1) as well 15.0(1)M IOS release required for ISR G2 UCS C-Series Rack Mount Server (C210M2)

Virtual CVP on UCS Solution Highlights CVP in a Virtual (VMware) Environment Supported with VMware ESXi 4.0 SIP deployments only UCS hardware must be used + + + Specific VM configurations/templates

Virtual Machine Planning/Designing No oversubscription of RAM or CPU! Mix of Cisco UC applications on same blade OK Mixed environments of physical and virtual servers Use published OVA templates (cores, memory and disk) HA: Distribute Application nodes across UCS blades, chassis and sites 3rd party applications (Email, SQL, Web Server etc.) on the same blade with Cisco UC apps - not supported except Domain Controller and Media Server 3 rd part app servers

Supported VM Templates for CVP Component & Scale * vcpu vram (GB) vdisk (GB) Call+VXML+Media (900 Calls, 10 cps) 4 4 1 x 80 Reporting (Med) (420 Msg/sec) 4 4 1 x 364 Reporting (Large) (420 Msg/sec) 4 4 1 x 510 OAMP Server 2 2 1 x18 Supported VM Templates for UCCE - Example Component & Scale * vcpu vram (GB) vdisk (GB) ** Router 8K agents 4 4 1 x 50 ** Logger 8K agents 4 4 1 x100 ** Agent PG 2000 agents 2 4 1 x 50 ** VRU PG 9600 ports 2 4 1 x 50 * CPU referenced is 2.53GHz Xeon E5540, RAM 1066MHz ** Requires 2 vnics. All others only require single vnic.

Virtualized CVP Design Basic Rules SRND application-layer guidelines for UCS are *not* same as when on MCS Determine quantity or role of CVP nodes. Re-size for VM. Private network requirement for VRU PG Mixed clusters of bare-metal HP or IBM and UCS are supported Subject to common sense rules e.g. don t make Primary CVP Call Server less powerful than Secondary CVP Call Server or vice-versa

Planning and Design Sizing

CVP Component Sizing ( Non UCS/VM ) Component Scalability per Server Redundancy HW Platform Call Server (SIP) 1200 ports, 15 cps N+1 to N*2 MCS-7845 Call Server (H.323) 500 ports 7 cps N+1 to N*2 MCS-7845 VXML Server 1200 ports, 15 cps N+1 to N*2 MCS-7845 Operations Console NA NA MCS-7825 or higher Co-Res (SIP, VXML, Media Server) Co-Res (H323, VXML, Media Server) 1200 SIP ports + 1200 VXML ports 15 cps 500 H323 ports + 500 VXML ports N+1 to N*2 MCS-7845 N+1 to N*2 MCS-7845 VRU PG 9600 ports N*2 MCS-7845

Supported VM Templates for CVP Component & Scale * vcpu vram (GB) vdisk (GB) Call+VXML+Media (900 Calls, 10 cps) 4 4 1 x 80 Reporting (Med) (420 Msg/sec) 4 4 1 x 364 Reporting (Large) (420 Msg/sec) 4 4 1 x 510 OAMP Server 2 2 1 x18 * CPU referenced is 2.53GHz Xeon E5540, RAM 1066MHz

VXML Gateway Sizing Maximum VXML Sessions: 12.4.15T The numbers assume the only activities running on the GW are VXML with basic routing and IP connectivity. These figures apply to NTE 75% CPU, VAD off, Cisco IOS 12.4.15T5. The numbers represent performance with Call Studio generated scripts running on CVP VoiceXML Application Servers. VoiceXML 2.0 and MRCPv2 were tested. The 1861 requires 12.4.20T as a minimum release. *NTE 80% CPU. Voice Gateway Platform Dedicated VXML GW All calls VXML-Controlled; no PSTN interfaces present Voice Gateway + VoiceXML All calls are PSTN calls and all calls are VXML- Controlled DTMF ASR/TTS DTMF ASR/TTS 1861 5 4 4 2 2801 7 6 6 4 2811 30 24 25 20 2821 48 36 36 30 2851 60 56 56 48 3825 180 140 210 130 3845 200 155 230 145 AS5350XM* 240 192 240 160 AS5400XM* 240 192 240 160

VXML Gateway Sizing Maximum VXML Sessions: 15.0.1M Voice Gateway Platform Dedicated VXML GW All calls VXML-Controlled; no PSTN interfaces present Voice Gateway + VoiceXML All calls are PSTN calls and all calls are VXML- Controlled DTMF ASR/TTS DTMF ASR/TTS 1861 5 3 4 2 2801 7 4 5 3 2811 30 20 23 15 2821 48 32 36 25 2851 60 40 45 30 3825 130 85 102 68 3845 160 105 125 83 5400XM 200 135 155 104 2901 12 8 9 6 2911 60 40 47 31 2921 90 60 71 48 2951 120 80 95 64 3925 240 160 190 127 3945 340 228 270 180 29xx and 39xx are ISR G2 15.0.1M, G.711, basic calls, Ethernet egress, CPU NTE 75%, VAD is ON

Voice Gateway Platform Flow-Through with VAD OFF Dedicated CUBE GW Flow-Through with VAD ON Flow-Around CUBE + VoiceXML All SIP Calls & VXML controlled with Flow-Through & VAD ON DTMF ASR/TTS 2801 55 75 200 5 3 2811 110 150 400 23 15 2821 200 300 600 36 25 2851 225 325 750 45 30 3825 400 500 750 102 68 3845 500 600 750 125 83 AS5000XM 600 850 3000 155 104 2901 100 130 400 9 6 2911 200 260 800 47 31 2921 400 520 1500 71 48 2951 600 780 2500 95 64 3925-800 1000 3000 190 127 SPE100 3945- SPE150 1000 1250 4500* 270 180 15.0.1M, G.711, basic calls, Ethernet egress, CPU NTE 75%, VAD is ON

CUSP Sizing with CVP CVP 8.5 SRE License Standard Mode (record route on) Requests per second Lite mode (record route off) Requests per second FL-CUSP-2 2 5 FL-CUSP-10 10 25 FL-CUSP-30 30 75 FL-CUSP-100 100 450 FL-CUSP-200 200 750 NME License Standard Mode (record route on) Requests per second Lite mode (record route off) Requests per second FL-CUSP-10 10 10 FL-CUSP-30 30 30 FL-CUSP-100 100 450

CUSP Sizing with CVP (Continued) CVP 8.5 Performance based on 1000 routes / no normalization For CUSP sizing purposes, assume each CVP Call uses on average 4 SIP CUSP requests: GW to CVP Call Server CVP Call Server to VXML gateway CVP Call server to Ringtone gateway CVP Call server to CUCM

Planning and Design Bandwidth, QoS and Latency

Call Signaling and Voice Bearer Traffic Assumes typical H.323 call flow uses about 7000 bytes per call Every call into a branch gateway requires 6000 bytes plus 1000 bytes for each transferred call to an agent. (7,000 bytes / call) * (8 bits/byte) = 56,000 bits per call (cps) * (56,000 bits / call) = Average kbps per branch Assumes typical SIP call flow uses about 17,000 bytes per call (17,000 bytes / call) * (8 bits/byte) = 136,000 bits per call (cps) * (136,000 bits / call) = Average kbps per branch Remember, there are 4 SIP call legs per average UCCE call For Voice Bearer Traffic calculations depend on the codec used G.711 or G.729

VXML Traffic 1 IVR prompt = 1 VXML document Average VXML document = 7,000 bytes 7,000 bytes * 8 bits = 56,000 bits per prompt (cps) * (56,000 bits / prompt) * (# of prompts / call) = kbps/branch VXML Document Type VXML Document Size (Approximate) Root document (one required at beginning of call) Subdialog_start (at least one per call at beginning of call) Query gateway for Call-ID and GUID (one required per call) Menu (Increases in size with number of menu choices) 19,000 bytes 700 bytes 1,300 bytes 1,000 bytes + 2,000 bytes per menu choice Play announcement (simple.wav file) 1,100 bytes Cleanup (one required at end of call) 4,000 bytes

Media File Retrieval Traffic Assumes HTTP Traffic (# prompts) * (size (bytes)/prompt) * (8 bits/byte) = Total bits (Total bits) / (cache refresh interval (secs)) = Average kbps per branch Example: 25 prompts, average size 50kB each, refresh int. 15 mins. (25 prompts) * (50,000 bytes/prompt) * (8 bits/byte) = 10,000,000 bits (10,000,000 bits) / (900 secs) = 11.1 Average kbps per branch Not caching prompts at the VXML gateway causes significant Cisco IOS performance degradation as much as 35% to 40%, directly impacted the overall bandwidth budget for the solution.

Sample Bandwidth Usage CPS BHCA Agents Queue Time VXML Ports H.323 SIP VXML.03 100 6 45 sec 9 2 kbps 4 kbps 40 kbps.14 500 21 27 sec 18 8 kbps 19 kbps 106 kbps.28 1000 38 24 sec 27 16 kbps 38 kbps 168 kbps 1.39 5000 172 17 sec 84 78 kbps 189 kbps 600 kbps Based on the following assumptions: IVR prompt and collect = 30 seconds, 1 Prompt every 6 seconds Agent talk time = 120 seconds CPS above assumes even distribution of BHCA, worst case CPS should be used *Equations Used to Find These Values Can Be Found in the SRND

CVP QoS Class Model

CVP Port Usage and QoS Setting Component Port Queue PHB DSCP Maximum Latency (Round Trip) Media Server TCP 80 CVP-Data AF11 10 200 ms Unified CVP Call Server Unified CVP IVR Service Unified CVP VXML Server Ingress Gateway VoiceVXML Gateway TCP 1720 (h.323) TCP/UDP 5060 (SIP) TCP 8000 (HTTP) TCP 8443 (HTTPS) TCP 7000 (HTTP) TCP 7443 (HTTPS) TCP 1720 (H.323) TCP/UDP 5060 (SIP) TCP 1720 (H.323) TCP/UDP 5060 (SIP) Call Signaling CS3 24 200 ms CVP-Data AF11 10 200 ms CVP-Data AF11 10 200 ms Call Signaling CS3 24 200 ms Call Signaling CS3 24 200 ms H.323 Gatekeeper UDP 1719 Call Signaling CS3 24 200 ms SIP Proxy Server TCP/UDP 5060 Call Signaling CS3 24 200 ms MRCP TCP 554 Call Signaling CS3 24 200 ms

Network Latency Latency < 200 ms round trip is recommended Solution will tolerate latency as high as 400ms round trip Since HTTP is primary protocol used to fetch VXML instructions and media files, HTTP should treated with higher priority than normal HTTP HTTP traffic should be treated as CVP call signaling traffic Lets take a look closer look at a WAN with ~250 ms round-trip delay.

Network Latency Example

VXML Call Prompt Setup Times ICM Micro Apps WAN RTT Latency SIP Call Setup Micro App Download (Initial VoiceXML) Time Until Prompt Plays 50 ms 0.5 seconds 1.0 seconds 1.5 seconds 100 ms 0.7 seconds 1.5 seconds 2.2 seconds 150 ms 0.9 seconds 2.0 seconds 2.9 seconds 200 ms 1.1 seconds 2.6 seconds 3.7 seconds 250 ms 1.3 seconds 3.2 seconds 4.5 seconds 300 ms 1.5 seconds 3.8 seconds 5.3 seconds 350 ms 1.7 seconds 4.3 seconds 6.0 seconds 400 ms 1.9 seconds 4.8 seconds 6.7 seconds

VXML Call Prompt Setup Times Call Studio Apps WAN RTT Latency SIP Call Setup Studio App Download (Initial VoiceXML) Time Until Prompt Plays 50 ms 0.5 seconds 2.0 seconds 2.5 seconds 100 ms 0.7 seconds 3.3 seconds 4.0 seconds 150 ms 0.9 seconds 4.7 seconds 5.6 seconds 200 ms 1.1 seconds 5.9 seconds 7.0 seconds 250 ms 1.3 seconds 7.3 seconds 8.6 seconds 300 ms 1.5 seconds 8.6 seconds 10.1 seconds 350 ms 1.7 seconds 10.0 seconds 11.7 seconds 400 ms 1.9 seconds 11.2 seconds 13.1 seconds

Understanding Source of Latency Latency Sources SIP Signaling Ingress->SIP Proxy->CVP->ICM->VRU Leg = SIP call setup time Unified CVP Call Server "ping Follows VRU Leg, checks to see if CVP is alive Unified CVP Call root doc fetch Part of the ping includes root doc fetch Unified CVP VXML Server root doc fetch (Cisco Unified Call Studio only) Similar to Micro-App however involves Micro-App and Call Studio App Relatively large transfer and represents the largest amount of delay for the five phases Cisco Unified Call Studio-based VXML fetch (Cisco Unified Call Studio only) Follows root doc fetch, represents first interaction with caller Relatively small fetch resulting in minimal delay Mitigation Techniques Reduce WAN RTT latency (Obvious choice) Inject Audio to mask the silence observed by high WAN RTT Latency Move CVP to a distributed model, placing Call, VXML and Media Servers closer to edge

Planning and Design Geographic Models

Geographic Models Customer Voice Portal (CVP) Geographic Models (1 of 4) Single Site Advantages Simplest Centralized ingress and call control Ingress and VXML gateways (should) be deployed separately Dial plan and troubleshooting simplification. Avoids the need for edge queuing techniques Disadvantages Centralized point of failure. Non-flexible recovery options Higher equipment costs

Geographic Models Customer Voice Portal (CVP) Geographic Models (2 of 4) Multisite w/ Centralized Call Processing Advantages Similar to Single Site advantages Terminate calls to local agent branches Ingress and VXML on one gateway at branch locations Autonomous capacity flex Disadvantages Similar to Single Site disadvantages WAN failure: = device deregistration, loss of call control, queuing, agent state updates, etc. Possible transcoding and DSP requirements

Geographic Models Customer Voice Portal (CVP) Geographic Models (3 of 4) Multisite w/ Distributed Call Processing Advantages Similar to Multisite with Central Call Processing Single site failure redundancy Disadvantages Significant solution costs due to infrastructure replication Physically diverse networks (private and public) primarily to meet UCCE fault tolerance requirements Multiple dial plans needed for CUCM WAN + codec requirements similar to Multisite Central Call Processing

Geographic Models Customer Voice Portal (CVP) Geographic Models (4 of 4) Clustering Over WAN Advantages Most common deployment. Combination of centralized and distributed call processing. Fault tolerant and survivability flexible More efficient use of HW resources Designed for large enterprises Disadvantages Similar to Multisite Distributed Heaviest dependence on WAN (private and public) availability

Branch Design Design for Queuing at the Edge If call comes into remote site, it is desirable for the call to be queued at that site CVP normally load-balances among all VXML gateways not optimal for branch designs Send To Originator sends the call back to same GW for VXML in SIP settransferlabel causes CVP to always transfer back to originating gateway in H.323 case More Edge Queuing Techniques in a moment

Call Admission Control Why Is Call Admission Control (CAC) Needed? Circuit-Switched Networks Packet-Switched Networks PSTN IP WAN IP WAN links LLQ Is Provisioned for 2 VoIP Calls (Equivalent to 2 Virtual Trunks) Physical Trunks PBX Third Call Rejected STOP IP WAN Link Router/ Gateway Call Manager No Physical Limitation on IP Links. 3 rd Call Can Go Through, but Voice Quality of All Calls Degrades. Call Admission Control Blocks 3 rd Call

Branch Design Call Admission Control: The Problem Plenty of BW PSTN IP WAN Branch A Headquarters From the perspective of CM, the gateways are behind the CVP server CM treats every call as if they came from the same device (SIP Proxy/CVP/GK) based on source IP Branch B Affects: locations CAC

Branch Design Queue at Edge : CAC Not an Issue Till This Point 3 PSTN 1 IP WAN 2 Branch A Headquarters Call received from PSTN, sent to CVP CVP receives label for VRU, CVP does GK /SIP Proxy lookup to find destination gateway One VRU label is used, destination VXML-GW could be any For SIP Send To Originator causes CVP to transfer back to originating gateway With these commands, CAC is not an issue for VXML leg Branch B

Location Awareness and Location Based Routing Without Location Awareness, You Will Have Bandwidth miscalculations in CAC with IP originated callers, as well as with any post transfers from agents Inability to deterministically select a local VXML GW for VRU treatment at the branch office during warm transfers from an agent due to no correlation between the two calls at consult More on how to solve this, stay tuned

Planning and Design Edge Queuing Techniques

Edge Queuing Set Transfer Label (H.323) Send to Originator (SIP) Technology Prefix Stripping (H.323) Significant Digits (SIP) Location Based Call Admission Control (LBCAC)

Send To Originator (SIP) Simple to configure and troubleshoot using OPs Console configuration Requires that the ingress gateway also provide VXML functionality SIP Header is checked by CVP SIP service to determine how warm transfer was initiated, avoiding sending VRU label to CUCM. Best Practice: Always build a centralized VXML router Farm for all transfers when using this technique

Significant Digits (SIP) More difficult to configure, requiring pre-pending at ingress gateway Site Code for VRU leg is pre-pended Leg Label at Ingress: 228005551212 22 Site Code for VRU Handling Rest is DNIS for ICM Site Code must also be embedded for any warm transfers initiated by CUCM, increasing complexity for CUCM dial plan SIP Proxy dial plan must accommodate switch and VRU labels including site codes Best Practice: Always build a centralized VXML router Farm for all transfers when using this technique.

Location Based CAC CVP gets the location information from CUCM, associate it to a particular branch gateway and assign a siteid. CVP has information about all the gateway locations now For SIP: CVP passes the location information in the Call-Info Header

Location Based CAC Solution (Deterministic Selection of VXML Gateway) CVP gets the location information from CUCM, associate it to a particular branch gateway and assign a siteid. CVP has information about all the gateway locations now CVP can now select the correct and closest VXML gateway (near to the caller) to queue the call including warm transfers. Saves WAN bandwidth Proper SIP proxy configuration is required to route label based on the site-id information Valid for IP originated calls as well Best Practice: Preferred method for SIP edge queuing addressing CAC and deterministic edge queuing for warm transfers. (No need for centralized VXML farm for queuing transfers!)

High Availability Designs Ingress and VXML Gateways

Ingress Gateway Configure SIP Server definition via User Agent resolving to DNS hostname Configure multiple SIP Proxy dial-peers with same preference for local Data Center proxies. Configure multiple SIP Proxy dial-peers with higher preference for remote data center. (Yes, higher preference is actually less preferred) If no SIP proxy server is deployed, setup same dial-peers pointing to Unified CVP Servers. Consider using DNS SRV values resolved via DNS Server Consider using DNS SRV values resolved via local static SRV records. (Doesn t scale and must exist on ALL ingress gateways)

High Availability Designs SIP

SIP Service SIP is the preferred protocol for CVP Only SIP (no H323) allowed for new deployments from CVP 8.0 H323 to be totally deprecated in CVP 9.0 SIP provides improved scalability and performance to CVP SIP and H.323 can co-exist at the same time, however protocol used to initiate the switch leg must remain the same for the entire length of the call! (valuable during H323->SIP migration) SIP Proxy, DNS SRV or static routes replace Gatekeeper function SIP packets tagged with proper QoS markings vs. ACL for H.323 CVP 8 supports RFC 2833 (in-band RTP NTE) as well as KPML (OOB SIP-Notify) for DTMF MTP will be allocated if IP phones and SIP trunk on CUCM are not configured for, or do not support RFC2833

SIP Routing SIP Routing is the method in which a DN is resolved to an IP Also used for load-balancing and redundancy Several options available: SIP Proxy DNS SRV Local SRV Static routes Combination of above SIP Proxy desirable for significant number of SIP user agents or for complex dial plan. Static routes/local SRV desirable for small deployments with simple dial plan.

Algorithm and Routing Priority When a Label Is Received By CVP from UCCE Location SiteId (if present) is inserted for Comprehensive callflows before selecting a destination SigDigits will be prepended if defined before routing to a destination If SendToOriginator is matched for label, the callers source IP/host is used ONLY if caller is a Cisco IOS gateway If (use outbound proxy) is set, then use the host of the SIP proxy If local static route is found, then use the destination as SIP URI If local SRV and SIP server group is defined, then destination is determined dynamically based on SIP element status and priority/weight

SIP High Availability SIP HA can be achieved in multiple ways Redundant SIP Proxies DNS SRV records Local SRV records Static routes Combination of above As of CVP 8.0 TCP is preferred transport for SIP with CVP. Was UDP prior to CVP 8.0 CVP running on VMware has issues with UDP

Cisco Unified SIP Proxy (CUSP) CVP 8.5 Network Module (NME) in ISR Routers 3800, 2951, 3900, 3900E routers only Service Module (SRE) in ISR Routers 2900(any), 3900, 3900E Prior to 8.5 CUSP requires dedicated chassis and can not coreside with VXML/TDM GW. 8.5 and after allows co-residency. Redundancy: Two ISR GWs geographically separated Double Capacity with the above by using 2 CUSP/ISR SIP SIP V SIP ISR SIP Ingress GW CUSP CUCM 73

SIP Proxy Design IOS-Based SIP Proxy, Design Option 1 Single module, very fault tolerant Cisco Unified SIP Proxy (CUSP) 1. Qualified starting in CVP v7.0(2) 2. CUCM is no longer required 3. SIP module hosted on dedicated Cisco Integrated Service Router (ISR) Redundancy Static routes not replicated between sites SIP option pings can be enabled for CVP network for dynamic CVP state awareness Option pings at the ingress not yet supported Site A Secondary DialPeer No Static Route Replication ISR ISR Primary SIP Trunk Secondary SIP Trunk Primary SIP Trunk CTI Custer Aware CUCM Publisher CUCM Subscriber Site B 74

SIP Proxy Design IOS-Based SIP Proxy, Design Option 2 Site A Site B Dual module, very fault tolerant Cisco Unified SIP Proxy (CUSP) Tertiary DialPeer 1. Qualified starting in CVP v7.0(2) 2. CUCM is no longer required 3. SIP module hosted on dedicated Cisco Integrated Service Router (ISR) ISR No Static Route Replication ISR Redundancy Tertiary SIP Trunk CTI Custer Aware Static routes not replicated between sites or between SIP local SIP Proxies CUCM Publisher CUCM Subscriber SIP option pings can be enabled for CVP network for dynamic CVP state awareness 75

High Availability Designs CVP Call Servers

Call and VXML Servers Deploy redundant call servers per data center with sufficient redundant ports Note: Redundant CVP port licenses are purchased licenses and live Note: Gateway/VXML licenses are paper only but must be purchased! Configure SIP Server groups for outbound proxy redundancy Configure SIP Server groups for dynamic SIP routing and availability Configure primary and backup VXML Servers on VXML gateway Consider using content load balancers to load balance VXML and Media Server requests.

High Availability Design CVP Hardware Redundancy Options V Server Crash Process Crash/Hang Performance Bottleneck Network Outage N + 1 V V V V V V One additional CVP Call Server Port redundancy for single server only Capacity compromised if more than one server is impacted N + N Site redundancy without compromising capacity Supports maximum hardware, geodistributed redundancy V

SIP Server Group and Dynamic Routing CVP 8.0 OAMP Server Groups replace local SRV manual file creation. srv.xml file still used under the covers. CVP knows the status of the destination before sending SIP INVITE using the SIP OPTIONS heartbeat feature

SIP Server Group

High Availability Design CVP SRV (Visual) Site A Site B SRV Priority 2 SRV Priority 2 V V SRV Priority 3 SRV Priority 3 CTI Custer Aware CUCM Publisher CUCM Subscriber

High Availability Designs Load Balancers

Load Balancers Use to Load Balance HTTP, Media Resource Control Protocol (MRCP), and RTSP traffic Cannot load balance call control traffic such as H.323 and SIP ACE uses a dedicated Fault Tolerant VLAN, CSS uses a VRRP With ACE, the IP address and MAC address of the FT VLAN do not change during failover With ACE to achieve active-active redundancy, a minimum of two contexts and two FT groups are required on each ACE. Best Practice: For larger, geographically separated deployments, deploy pairs of load balancers per data center for localized redundancy. Caveat: ACE and CSS are the only supported Load Balancer for CVP (F5, etc. are not officially supported)

ACE Load Balancing Caveat: One arm implementations will break deployments using MRCPv2 since its based on SIP messages and not RTSP/RTP as with MRCPv1 (RFC 4463).

High Availability Designs Media Servers

Media Servers Configure redundant host entries in VXML gateway ip host mediaserver 10.1.1.10 ip host mediaserver-backup 10.1.1.11 Keep in mind that -backup is only available in Comprehensive Mode (Anyone know why? Bueller?) In Standalone mode, use error handling and scripts to pick secondary media server if first fails at VXML gateway As a fail safe, implement the new default media server configuration in Unified CVP. Best Practice: Always implement a supported load balancer, like ACE or CSS for large, geographically separated media server farms.

Media Server The CVP OAMP Media Server device was enhanced to populate the media server device list to all Call Servers The Media Server device list can be updated to all the CallServers by pressing the new "Deploy" button from the OAMP Media Server device list A default media server device may be specified in OAMP. If specified in OAMP, micro-applications will use that default media server if the ECC variable for media server is not defined in the UCCE ICM script. The default media server is specified in the OAMP Media Server list page. Supports primary and backup media server via hostnames Supports ACE if there are multiple media servers Supports FTP on media server for agent greetings

Default Media Server (Cont.) Due to default media server, ICM scripting is now simplified Media server is defined by ECC variable Default media server is configured in CVP OAMP ( Allowed in Release 8.5(1) )

High Availability Designs Survivability

Survivability Allows for a call to reroute during or after system outage Redirect the caller to alternate destination in the event of either start-of-call or mid-call failures (e.g. ICM Dialogue failure events, failure of IVR leg, media inactivity) Control recovery behavior based on incoming DNIS and/or time-of-day Various methods of call recovery: TDM-only: DTMF*8, Hookflash, TBCT SIP or TDM: Hairpin Transfer to CVP VXML server application Retry to same DN optionally pre-pended or appended with digit string. Allows finer control over ICM script recovery behavior. For example: Original DNIS = 8005551212 Recovery DNIS = 999<retry> = 9998005551212 Works in CVP Standalone or Comprehensive Models Requires ASR1000, CUBE or TDM gateway Cannot recover call context from the original call

CVP Survivability VXML Gateway X SIP GED-125 RTP Survivability CVP Call Server Caller (TDM or IP) Voice Gateway Alternately, if the gateway detects a loss of media to an IP phone, it can invoke survivability if media inactivity timers are set. X X Upon receiving failure notification from ICM or VXML gateway, Call Server sends BYE with abnormal cause code which causes gateway to invoke survivability.

Case Studies What happens if you fail to plan?

Better Late than Never The Situation: Customer designed and deployed a centralized Unified CCE solution using the Clustering over the WAN deployment model with SIP and edge queuing. Branch locations were located in UK, Germany, Hong Kong, China, etc. An international carrier was picked to provide for local and international dialing services as well as TFN ingress services at the branch. Unified Call Studio was deployed to build self service applications as well as Collect and Prompt services on servers in the US. Progression: US Based branches were deployed without any issues UK and Germany based branches were deployed with minor issues and complaints about prompt delays. This was determined to be ~2-3 seconds longer than previous system. APAC deployment suffered from massive delays from the time the call was placed to when the caller heard a prompt.

Better Late than Never The Issue: US Based branches had low latency WAN connections to their branches <50 ms UK and Germany based branches had higher than normal latency than the US, placing them around 100 to 150 ms, still acceptable from the business perspective. APAC branches suffered from massive latency ranging anywhere from 300 ms to 400 ms RTT (10-13 second addition to overall prompt delivery). It was discovered that APAC branches were also suffering from a carrier connect delay of 5-6 seconds. Resolution and Lesson: Customer eventually had to move CVP to a distributed model, placing call, VXML servers, SIP Proxy, etc. into a regional hub where latency to APAC branch offices was greatly reduced. Know your latency numbers and design accordingly! Distributing CVP after the fact is expensive and a painful process.

Trust but Verify The Situation: Customer designed and deployed a centralized Unified CCE solution using the Clustering over the WAN deployment model with SIP and edge queuing. Branch locations were located throughout the US with CVP located in two US based data centers. Progression: US Based branches deployed initially without any serious issues Initial deployment of live agents was minor in size. Within 2 months, almost 2K agents were placed on the solution Slowly the quality of call treatment transactions and agent calls degraded over time to a point of failure. Customer s voice and network teams defended their network, their QoS policies and implementations with vigor

Trust but Verify The Issue: It was discovered that why the customer had a QoS policy and template, its implementation from the LAN and WAN perspective were not handled by the same group nor were they consistent and correct. Traffic was being marked close to the source, however, not trusted as it crossed into the WAN realm, essentially unmarking it. Furthermore, HTTP traffic, critical to CVP VXML instructions, was being marked as scavenger traffic and drop due to contention of bandwidth. Customer also didn t have accurate bandwidth utilization numbers for WAN connections to the branches prior to implementation. Resolution and Lesson: Trust that your customer has QoS implemented, Verify end to end that it is implemented and honored correctly. Know the bandwidth consumption of a branch, especially if you are adding agents and voice services.

New Unified CVP Book Published December 2011 by Cisco Press Title: Cisco Unified Customer Voice Portal: Building Unified Contact Centers ISBN-10: 1-58714-290-2 and ISBN-13: 978-1-58714-290-1 Authored by: Rue Green Grab a copy at the Cisco Book Store 97

Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Passport points for each session evaluation you complete. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center. Don t forget to activate your Cisco Live Virtual account for access to all session material, communities, and on-demand and live activities throughout the year. Activate your account at the Cisco booth in the World of Solutions or visit www.ciscolive.com. 98

Connect with Your Peers and Cisco Discuss business, IT, architecture, adoption and product topics with peers conferencing, customer care, enterprise social software, IP communications, messaging, mobile applications, telepresence Interact with Cisco Product Managers, Technical Marketing Engineers and Services Consultants Learn about new product announcements Join the Collaboration User Group Influence product direction Access to early product releases and info Get VIP perks and exclusive sessions at Cisco Live Visit the Cisco Collaboration Booth (#1289) at Cisco Live: Learn more about the community. Sign up for the user group. Pick up your polo shirt and badge ribbon (user group members only). Cisco Collaboration Community and User Group communities.cisco.com

Final Thoughts Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042 Come see demos of many key solutions and products in the main Cisco booth 2924 Visit www.ciscolive365.com after the event for updated PDFs, ondemand session videos, networking, and more! Follow Cisco Live! using social media: Facebook: https://www.facebook.com/ciscoliveus Twitter: https://twitter.com/#!/ciscolive LinkedIn Group: http://linkd.in/ciscoli 100