OpenStack - based NFV Cloud at Swisscom Challenges and Experiences
Agenda Swisscom IPTV A story of Innovation and Success E2E Signal Chain E2E Service Orchestration Vision Lessons Learned Swisscom-HPE virtual headend project Legacy vs virtual headend Maturity levels of applications Enterprise Cloud vs. Media processing platform Virtual Network performance and Container Networking Multicast on OpenStack AVI Networks: Elastic Application Services A distributed architecture Application Services Elastic scale Demo Ranga Rajagopalan AVI Networks Inc. CTO rangar@avinetworks.com Mihajlo Zivkovic Swisscom (Schweiz) AG Product Owner mihajlo.zivkovic@swisscom.com Marco Loetscher Hewlett Packard Enterprise Solution Architect marco.loetscher@hpe.com
Swisscom TV & Entertainment 3
Swisscom is a Marketleader since December 2015 1.5 Mio. Customers 2 Mio. STB s in the field 3.4 Mio. Households 8 Mio. Population 2Mio. on Swisscom DSL 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 4
Swisscom TV Some Facts and Figures Customer Satisfaction > 8..more then 1.5 Mio Customer >90 Pbyte of Storage 25 Pbyte Storage / Year > 2.8 Tbit/s Unicast Streaming peak from Datacenter..more then 2 Mio STB s > 650 Channels (SD, HD, UHD) > 750K unique user per month on Web & Mobile 5
The Journey of Swisscom TV It s a Story of Innovation and Success 2006 1. November Launch Bluewin TV 2009 25. February Start HD-Sender Launch TV1.0 2010 23. March Swisscom TV Air 2011 30. June Marketleader Digital TV 2013 28. Decembar 1 Million Customers 2015 April HBB TV 2017 November Mediaroom Phase-out Entertainment 0S3 2012 16. January Launch Replay TV 2014 Decembar Launch Teleclub Play (Subscription VOD) 2014 April Launch TV 2.0 2015 Decembar Marketleader Switzerland 2016 April Voice Search with Swiss Dialects UHD 6
Next Generation Entertainment & TV Services Swisscom TV E2E Signal Chain 7
Next Generation Entertainment & TV Services TV Platform Virtualization 1 st step virtualization 3rd step virtualization 2nd step virtualization 8
Monitoring and Service Monitoring MediaCloud: The Big Picture Goal to have all Entertainment Service on the MediaCloud MediaCloud Channel/Service Definition Bussiness process workflow Service Orchestration vhe Radio AVI AppBE npvr LiveTV LivingDocs xyz VNF Mgr VNF Mgr VNF Mgr VNF Mgr VNF Mgr VNF Mgr VNF Mgr VNF Mgr Virtual CPU Virtual Memory Virtual Storage Virtual Network Virtualization Layer / RedHat OpenStack / KVM CPU Hardware Memory Hardware Storage Hardware Network Hardware 9
Media Cloud - Virtual HeadEnd Lessons learned Organizational challenges: Vendor management (working mode with Vendors). Mindset of the project staff. Transforming engineers into the new "cloud" world. Operation of two environments (legacy and cloud). Knowledge and the experience (combined application with infrastructure and network) Organization and processes. Technological challenges: Working on the bleeding edge of the technology. Technology unknowns. Infrastructure : very large high performance media cloud solution. Zero packet loss in cloud environment with the multicast. 10
Media Function Virtualization 11
Current headend operations challenges Video Signal Transcoders A Manager Transcoders B Manager Transcoders C Manager Multiplexers X Manager Multiplexers Y Manager Transcoder A Transcoder A Transcoder A Transcoder B Transcoder B Transcoder C Active headend Ch. 1 Ch. 2 Ch. 3 Ch. 4 Ch. 5 Ch. N Multiplexer X Multiplexer Y Multiplexer Z Current situation Too many managers to manage Disaster Recovery expensive Costly Equipment Upgrades/Exchanges Operational complexity Missing Flexibility to add new channels Switch STB Multiplexers Z Manager Backup headend Transcoder A Transcoder A Transcoder A Ch. 1 Ch. 2 Ch. 3 Multiplexer X CDN Dedicated Management Transcoder B Transcoder B Ch. 4 Ch. 5 Multiplexer Y Transcoder C Ch. N Multiplexer Z 12
Virtualized headend with HPE VHM Virtualized Headend Video Signal One interface to manage it all HPE Virtual Headend Manager Transcoder A Transcoder A Transcoder A Transcoder A Transcoder B Transcoder B Transcoder B Ch. 1 Ch. 2 Ch. 3 Backup Ch. 4 Ch. 5 Backup Multiplexer X Multiplexer X Multiplexer Y Multiplexer Y 1 click channel deployment Switch STB Transcoder C Ch. N Multiplexer Z Transcoder D Channels on the fly Multiplexer W CDN Fast time to launch new service One click channel deployment Non-proprietary infrastructure Quick and automated lifecycles Simplicity: One interface for all CAPEX and OPEX reduction 13
NFVI Virtual Headend ETSI Reference Architecture OSS/BSS Service Orchestration Service, VNF and Infrastructure Description NS and VNF Catalogs NFV Orchestrator EMS 1 VNF 1 EMS 2 VNF 2 EMS 3 Transcoder Muxer Probe VNF 3 VNF Manager(s) VNFM Virtualization Layer (HyperV, KVM, ESX) Virtual Computing Computing Hardware Virtual Storage Virtualization Layer Standard Infrastructure Storage Hardware Hardware Resources Virtual Network Network Hardware Infrastructure Orchestration Virtualized Infrastructure Manager(s) NFV MANO 14
Maturity Levels of virtualized applications Market View R&D View Decoupled Functions separated from underlying hardware Bare Metal x86 Virtualized Functions deployed on hypervisor-driven, virtualized infrastructure resources Virtualize Cloudified Operated as part of the cloud, lifecycle fully orchestratable Automate / Orchestrate Decomposed Monolithic functions are decomposed into micro-services Microservices 15
Enterprise cloud vs. virtual Headend workloads IT Cloud Virtual Headend 1. CPU, RAM or Storage bound performance 1. CPU & I/O bound performance (DPDK, SR-IOV, etc.) 2. Aggregated view of resources (CPU, Memory, resources overcommitted) 2. Enhanced platform awareness (Internal Architecture relevant for guests) 3. Endpoints (Applications need the OS) 3. Middlepoints (Data-plane network bypass the OS) 4. Many and small virtual machines 4. Fewer and larger VMs 16
Key network performance and quality features PCI-passtrhough https://docs.openstack.org/admin-guide/compute-pci-passthrough.html SR-IOV (Single Root I/O Virtualization) http://en.wikipedia.org/wiki/x86_virtualization#pci-sig_single_root_i.2fo_virtualization_.28sr-iov.29 https://wiki.openstack.org/wiki/sr-iov-passthrough-for-networking NUMA (Non Uniform Memory Access) http://en.wikipedia.org/wiki/non-uniform_memory_access DPDK (Data Plane Development Kit) http://en.wikipedia.org/wiki/data_plane_development_kit 17
Containers in VMs vs. Baremetal The transcoding applications from Ateme and Harmonic are provided in Docker containers We decided to put the containers into a virtual machine Why? Docker Networking Hardware independence VM as bridge between application and physical environment Management and Monitoring App 1 App 1 App 1 App 1 App 1 App 1 Docker Engine Hypervisor Virtualization Overhead: 5-10 % Operating System Operating System Additional Component (Guest OS) to be managed Cost of Hypervisor Physical Server Physical Server 18
Docker Networking net=bridge: Container is connected with the Network Bridge Same IP addresses per Host, 1 NIC only Standard Mode net=none: No network connection of the container (i.e. for number crunchers, batch jobs, etc.) net=container:<container ID>: Container is using the NW connection of a neighbor Allow reachability of multiple segregated processes that under the same IP net=host: Container has direct access to physical NICs of the hos 19
Multicast in OpenStack No multicast available out-of-the-box Virtual layer-2 switches support IGMP snooping: Open vswitch 2.5 supports IGMP snooping Open vswitch 2.7 contains bugfixes to enable multicast with multiple provider VLANs http://openvswitch.org/features/ https://github.com/openvswitch/ovs/blob/master/faq.md Linux Bridge 2.4 supports IGMP snopping http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#snooping 20
Multicast in OpenStack recommendations Neutron provider network: Don t use Neutron-L3-Agent Neutron does not route any multicast traffic Use Neutron provider network Use routable public VLAN configured on top-of-rack switches Use router outside OpenStack Use gateway outside OpenStack Attach Nova instances directly to provider network Configure multicast routing / PIM on the physical router 21
AVI Networks 22
Monolithic Appliances to Modern Distributed Architecture Management Plane: UI/CLI Monolithic Appliance Software Data Plane: LB Service Engines APPLICATIONS Controller DATA CONTROL Separate Control & Data Plane Manage as one, not many devices 23
Avi Platform Modern Distributed Architecture Visibility Actionable insights key to automation Public Cloud Service Engines Bare Metal Virtualized Containers Controller REST API Automation Highly programmable, Plug-n- Play On Premises DATA CONTROL 10x Performance 4 Tbps 12M SSL TPS Universal Solution Both traditional and modern use cases Separate Control & Data Plane Manage as one, not many devices Elasticity On-demand scalability up / down 24
Comprehensive Services For All Major Environments Application Services Load Balancing WAF & Security Analytics Platform L7 (HTTP) LB L4 (TCP/UDP) LB Global Load Balancing Content Switching Caching/Compression Auto-Scaling Web app firewall (WAF) SSL Termination DDoS Protection L3-4 ACLs L7 Rules/Policies Micro-Segmentation Application map Service Health Score Network performance App Performance Request Logging Security Insights Central Management 100% REST API / SDK Self-Service Multi-Tenancy Service Discovery IPAM/DNS Out-of-Box Automation CONTAINERS SDN OPENSTACK AUTOMATION ON-PREM or OFF Bare Metal MESOS 25
Record Performance On Commodity Servers Single Fabric Central Management and Monitoring Avi Fabric Performance 4 Tbps 12M SSL TPS Performance scales with cores (Moore s Law) 32 cores 80,000 SSL TPS VIPRION 4800, ~1M TPS Centralized API, Mgmt, Monitoring ECC- 48K, RSA 80K 2 core 5,000 SSL TPS 1 core 2,500 SSL TPS 2x 1-core SEs 5,000 SSL TPS Fabric performance scales with Engines 26
We are happy to answer questions 27