Network Mul,tenancy in Xen- based Clouds. Chiradeep Vi;al CloudStack Commi;er Citrix Sep

Network Mul,tenancy in Xen- based Clouds Chiradeep Vi;al CloudStack Commi;er Citrix Systems @chiradeep Sep 18 2013

Agenda Introduc,on to CloudStack Mul,- tenant IAAS Network Virtualiza,on / SDN L3 isola,on CloudStack s Network Model CloudStack s na,ve SDN approach

Apache CloudStack! Product from Cloud.com / Citrix (thru acquisition)! Open Source since May 2010! Donated by Citrix to the ASF (Apr 2012)! Graduated as Top-level Project in March 2013! In production since 2009! Tons of deployments, including large-scale commercial ones!

How did Amazon build its cloud? Amazon ecommerce Platform AWS API (EC2, S3, ) Amazon Orchestration Software Open Source Xen Hypervisor Networking Commodity Servers Commodity Storage

How can YOU build a Xen- based cloud? Amazon Optional ecommerce Portal Platform CloudStack AWS API (EC2, or AWS S3, API ) CloudStack Amazon Orchestration Software Open Hypervisor Source (XenServer/XCP) Hypervisor Networking Servers Storage

Zone Architecture Admin/User API End users CloudStack MySQL DC Edge L3/L2 core Access Sw Hypervisor (Xen /VMWare/KVM) Snapshot Image Image Secondary Storage VM VM Primary Storage NFS/ISCSI/FC Disk Disk Snapshot Pod Pod Pod Pod Pod

Mul,- tenancy Internet L3/L2 core Hypervisor A C A A A B A C

Mul,-,er virtual networking Internet! Loadbalancer (virtual or HW)!! Virtual appliance/! Hardware Devices! IPSec or SSL site-to-site VPN! MPLS VLAN! Customer! Premises! Network Services! IPAM! DNS! LB [intra]! S-2-S VPN! Static Routes! ACLs! NAT, PF! FW [ingress & egress]! Web VM 1! Web VM 2! Web VM 3! App VM 1! App VM 2! DB VM 1! Web VM 4! Web subnet! 10.1.1.0/24! App subnet 10.1.2.0/24! DB Subnet! 10.1.3.0/24!

Network Isola,on Op,ons L2 Isola,on Each network /,er is a separate subnet Overlapping IP addresses (between networks) allowed L2 adjacency between VMs in same network Mul,cast / broadcast may be allowed.

Network Isola,on Op,ons L3 Isola,on Mul,ple tenants / applica,on,ers on the same physical subnet Isolated at IP (L3). No L2 adjacency in the same,er / tenant No Mul,cast / Broadcast

Network Isola,on Op,ons PVLAN Mul,ple tenants are placed on the same L2 domain. Only allowed to communicate via upstream router No mul,cast or broadcast (except ARP) Limited use cases

L2 Isola,on Op,ons Network Virtualiza,on The illusion of isolated networks on top of shared physical infrastructure VLAN Old, reliable technology, use OVS or bridge 4k limit (12 bit VLAN id) All usable VLANs need to be trunked down to all hypervisors Overlays ( SDN ) E.g., GRE, STT, VxLAN Currently only GRE available in Xen (with OVS) GRE tunnels are established between hypervisors to carry Ethernet frames between VMs on the same network Requires orchestrator / SDN controller to manage overlays

Network Virtualiza,on in IAAS 1 Virtual Network 10.1.1.0/24 Internet! Public Network Public IP address 65.37.141.11! 65.37.141.36 1! Edge Services Appliance(s)! NAT! DHCP! FW Gateway address 10.1.1.1 1 VM 1! 1 VM 2! 1 VM 3! 10.1.1.2 10.1.1.3 10.1.1.4 1 VM 4! 10.1.1.5

Network Virtualiza,on in IAAS 1 Virtual Network 10.1.1.0/24 Internet! Public Network Public IP address 65.37.141.11! 65.37.141.36 1! Edge 1! Services Edge Appliance(s)! Services NAT! Appliance(s)! DHCP! FW Load Balancing! VPN Gateway address 10.1.1.1 1 VM 1! 1 VM 2! 1 VM 3! 1 VM 4! 10.1.1.2 10.1.1.3 10.1.1.4 10.1.1.5

Network Virtualiza,on in IAAS 1 Virtual Network 10.1.1.0/24 Internet! Public Network Public IP address 65.37.141.11! 65.37.141.36 1! Edge 1! Services Edge Appliance(s)! Service(s)! NAT! DHCP! FW Load Balancing! Gateway address 10.1.1.1 1 VM 1! 1 VM 2! 1 VM 3! 1 VM 4! 10.1.1.2 10.1.1.3 10.1.1.4 10.1.1.5 Public IP address 65.37.141.24! 65.37.141.80 2! Edge Services! VPN! NAT! DHCP Gateway address 10.1.1.1 2 Virtual Network 10.1.1.0/24 2 VM 1! 2 VM 2! 2 VM 3! 10.1.1.2 10.1.1.3 10.1.1.4

CloudStack s Network Virtualiza,on 1 Virtual Network 10.1.1.0/24 Internet! Access Sw Public Network Public IP address 65.37.141.11! 65.37.141.36 Public IP address 65.37.141.24! 65.37.141.80 1! Edge Services 1! Appliance(s)! Edge Service(s)! NAT! DHCP! FW Load Balancing! Gateway address 10.1.1.1 Gateway address 10.1.1.1 2! Edge Services! VPN! NAT! DHCP DC Edge L3/L2 core 1 VM 1! 1 VM 2! 1 VM 3! 1 VM 4! 2 VM 1! 2 VM 2! 2 VM 3! 10.1.1.2 10.1.1.3 10.1.1.4 10.1.1.5 2 Virtual Network 10.1.1.0/24 10.1.1.2 10.1.1.3 10.1.1.4 Pod Pod Pod Pod Pod

VLAN example! VM A1! VM A2! VM B1! VM C1! Virtual Nics! untagged (usually)! vswitch! vswitch! vswitch! Physical! Nics! 192.168.1.0/24! 192.168.1.0/24! 10.1.1.0/24! VLAN TRUNK! VLAN 10! VLAN 20! VLAN 30!

GRE tunnel example! GRE Key 1 GRE Key 2 OVS User 1 OVS User 1 OVS User 1 User 2 OVS User 1 OVS User 2

CloudStack + SDN Technologies Nicira NVP Midokura MidoNet Nuage BigSwitch Stratosphere Coming soon Open Daylight Juniper

L3 isola,on with distributed firewalls Public Internet Public IP address 65.37.141.11 65.37.141.24 65.37.141.36 65.37.141.80 Pod 1 L2 Switch 10.1.0.1 1 VM 1 2 VM 1 10.1.0.2 10.1.0.3 L3 Core Pod 2 L2 Switch 10.1.8.1 1 VM 2 10.1.0.4 Load Balancer Pod 3 L2 Switch 10.1.16.1 2 VM 2 10.1.16.12 2 VM 3 10.1.16.21 1 VM 3 10.1.16.47 1 VM 4 10.1.16.85

L3 Isola,on in CloudStack + Xen CloudStack orchestrates dom0 firewall (iptables) Requires iptables across bridge and ipset package Does not work with OVS Scales to tens of thousands of vms and tenants

CloudStack Network Model: Network Services Network Services Service Providers! Network Isola?on L2 connec,vity IPAM DNS Rou,ng ACL Firewall NAT VPN LB IDS IPS ü Virtual appliances! ü Hardware firewalls! ü LB appliances! ü SDN controllers! ü IDS /IPS appliances! ü VRF! ü Hypervisor! No isola,on VLAN isola,on Overlays L3 isola,on

Service Catalog Cloud users are not exposed to the nature of the service provider Cloud operator designs a service catalog and offers them to end users. Gold = {LB + FW, using virtual appliances} Platinum = {LB + FW + VPN, using hardware appliances} Silver = {FW using virtual appliances, 10Mbps}

Service Catalog examples L2 network with software appliances! 10.1.1.0/24! VLAN 100 65.37.141.1 11! 65.37.141.1 12 CS! Virtual Router! DHCP, DNS! NAT! Load Balancing! VPN 10.1.1.1 10.1.1. 2 10.1.1. 3 VM 1! VM 2! 10.1.1.4 VM 3! 10.1.1.5 VM 4!

Service Catalog examples L2 network with software appliances! L2 network with hardware appliances! 10.1.1.0/24! VLAN 100 10.1.1.0/24! VLAN 100 65.37.141.1 11! 65.37.141.1 12 CS! Virtual Router! DHCP, DNS! NAT! Load Balancing! VPN 10.1.1.1 10.1.1. 2 10.1.1. 3 VM 1! VM 2! 10.1.1.4 VM 3! 65.37.141.11 1 Juniper SRX! Firewall! 65.37.141.11 2 Netscaler! Load Balancer! 10.1.1.1 NAT, VPN! 10.1.1.112 10.1.1.2 VM 1! 10.1.1.3 VM 2! 10.1.1.4 VM 3! 10.1.1.5 VM 4! 10.1.1. 5 VM 4! Upgrade DHCP, DNS! CS! Virtual Router!

More Info CloudStack Wiki h;ps://cwiki.apache.org/confluence/x/fwdfaq CloudStack Docs h;p://cloudstack.apache.org/docs/en- US/ index.html Mailing Lists h;p://cloudstack.apache.org/mailing- lists.html IRC Freenode #cloudstack- dev, #cloudstack