Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Transcription

1 NET1522BU Kubernetes Networking with NSX-T Deep Dive #VMworld #NET1522BU

2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. CONFIDENTIAL 2

3 NET1522BU Kubernetes Networking with NSX-T Deep Dive Yves Fauser / Yasen Simeonov #VMworld #NET1522BU

4 Agenda 1 NSX-T Overview 2 Kubernetes Overview 3 NSX-T & Kubernetes Integration 4 Demo #NET1522BU CONFIDENTIAL 4

5 NSX-T Overview

6 NSX Vision: Driving NSX Everywhere Managing security and connectivity for many heterogeneous end points Branch offices/edge computing/iot vcloud Air Network Cloud VMworld 2017 Content: Not for publication New app frameworks On-premises data center End users Automation IT at the speed of business Security Inherently secure infrastructure Application Continuity Data center anywhere #NET1522BU CONFIDENTIAL 6

7 NSX-T Architecture NSX Architecture and Components Cloud Consumption Management Plane Control Plane Data Plane NSX Manager Management Plane (MP) Node VM form factor Central Control Plane (CCP) Nodes- VM form factor ESXi (+ kernel modules) NSX Controllers Transport Nodes VMworld 2017 Content: Not for KVM (+ kernel modules) Self Service Portal OpenStack, K8s, Custom Concurrent configuration portal REST API entry-point UI publication Talks to Dataplane over a Control-Plane Protocol Separation of Control and Data Plane High Performance Data Plane Scale-out Distributed Forwarding Model Hypervisors NSX Edge (L3 + Adv Services) L2 Bridge (L2 Overlay- VLAN) Physical Infrastructure #NET1522BU CONFIDENTIAL 7

8 NSX-T Architecture Operations Workflow Configuration is persisted Configuration is pushed to CCP MP Node CCP Node CCP Node CCP Node VMworld 2017 Content: Not for X User makes a configuration publication Configuration is realized MPA LCP Transport Node MPA LCP Transport Node MPA LCP Transport Node #NET1522BU CONFIDENTIAL 8

9 Data Plane Improved performance and resiliency Designed for multi-tenancy and scale Tenants/CMP GENEVE Tunnel TEP: Overlay Tunnel End Point (with its own IP address) Admin p1 TEP vswitch 1 Overlay Transport Zone p2 HV TN1 New distributed edge architecture with increased performance with DPDK p1 HV TN1 Edge Node p2 TEP vswitch 2 Edge Node Edge Node Edge Node Edge Cluster Next gen overlay maintaining performance with increased flexibility #NET1522BU CONFIDENTIAL 9

10 NSX-T VMworld Session & Lab NSX-T Breakout Session Introduction to NSX-T Architecture NET1510BU (US) NET1510BE (Europe) VMworld 2017 NSX-T Hands On Lab VMware NSX-T - Getting Started SPL182601U (US) SPL182601E (Europe) Content: Not for publication #NET1522BU CONFIDENTIAL 10

11 Kubernetes Overview

12 What Is Kubernetes? Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure. #NET1522BU CONFIDENTIAL 12

13 Kubernetes Components > _ Kubectl CLI K8s master K8s master dashboard Controller Manager Scheduler K8s Master(s) K8s Nodes K8s API Server K8s Master Key-Value Store K8s node K8s node K8s node K8s node kubelet Kube-proxy c runtime K8s Cluster Consists of Master(s) and Nodes K8s Master Components API Server Scheduler Controller Manager Dashbord K8s Node Components Kubelet Kube-Proxy Containers Runtime (Docker or Rocket) #NET1522BU CONFIDENTIAL 13

14 Kubernetes Pod Pod /16 nginx tcp/ mgmt tcp/22 pause container ( owns the IP stack) logging udp/514 IPC External IP Traffic A Pod is a group of one or more containers that shares an IP address and a Data Volume #NET1522BU CONFIDENTIAL 14

15 Kubernetes Namespace Namespace: foo Base URI: /api/v1/namespaces/foo redis-master Pod: /api/v1/namespaces/foo/pods/redis-master redis service: /api/v1/namespaces/foo/services/redis-master Namespace: bar Base URI: /api/v1/namespaces/bar redis-master Pod: /api/v1/namespaces/bar/pods/redis-master VMworld 2017 redis service: /api/v1/namespaces/bar/services/redis-master Namespaces are a way to divide cluster resources between multiple uses They can be considered as Tenants They are a way to provide Resources Quotas, RBAC, Networking Multitenancy, and Names Overlapping Content: Not for publication #NET1522BU CONFIDENTIAL 15

16 K8s Load Balancing /16 East-West Load Balancing Web Front-End Pods redis-slave svc Redis Slave Pods Nginx HAProxy etc. LB Pods Web Front-End (e.g. Apache) Pods North-South Load Balancing Web Front-End Ingress East-West Load Balancing is provided through K8s Service using ClusterIP & IPTables Can be achieved through K8s Ingress or External third Party Load Balancer using NodePort #NET1522BU CONFIDENTIAL 16

17 Kubernetes Networking Topologies Flat routed topology ip route / ip route / int eth0 int cbr net.ipv4.ip_forward= / Node int eth0 int cbr net.ipv4.ip_forward= / Every Node is an IP Router and responsible for its Pod Subnet Subnets are associated with Nodes, not Tenants Physical Network Configuration is required Node #NET1522BU CONFIDENTIAL 17

18 Kubernetes Networking Topologies Node-to-Node overlay topology int eth0 int cbr net.ipv4.ip_forward= / Node Key-Value Store int eth0 int cbr net.ipv4.ip_forward= / Node Overlays are typically used to avoid Physical Network Configuration Overlay #NET1522BU CONFIDENTIAL 18

19 NSX-T and Kubernetes Integration

20 NSX-T K8s Integration Namespaces & Pods kubectl create namespace foo namespace foo" created kubectl create namespace bar namespace bar" created kubectl run nginx-foo --image=nginx -n foo deployment "nginx-foo" created kubectl run nginx-bar --image=nginx -n bar deployment "nginx-bar" created VMworld 2017 NSX / K8s topology Namespace: foo NAT boundary NAT boundary / / /24 Namespace: bar Content: Not for publication K8s Masters K8s nodes #NET1522BU CONFIDENTIAL 20

21 NSX-T K8s Integration Routed Namespaces vim no-nat-namespace.yaml apiversion: v1 kind: Namespace metadata: name: no-nat-namespace annotations: ncp/no_snat: "true kubectl create f no-nat-namespace.yaml namespace no-nat-namespace" created VMworld 2017 admin@k8s-master:~$ kubectl run nginx-no-nat --image=nginx n no-nat-namespace deployment "nginx-k8s" created NSX / K8s topology Namespace: no-nat-namespace /26 Direct Routing Content: Not for publication /26 K8s Masters K8s nodes #NET1522BU CONFIDENTIAL 21

22 NSX-T K8s Integration Pods Micro-Segmentations Option1: Predefined Label Based Rules kubectl label pods nginx-foo nltrf secgroup=web -n foo Pod "nginx-nsx nltrf" labeled kubectl label pods nginx-bar z09x2 secgroup=db -n bar pod "nginx-k8s z09x2" labeled kubectl get pods --all-namespaces -Lsecgroup NAMESPACE NAME READY STATUS RESTARTS AGE SECGROUP k8s nginx-foo z09x2 1/1 Running 0 58m web nsx nginx-bar nltrf 1/1 Running 0 1h db Security Groups are defined in NSX with ingress and egress policy Each Security Group could be microsegmented to protect Pods from each other VMworld 2017 NSX / K8s topology Content: Not for publication Namespace: foo NAT boundary NAT boundary Namespace: bar / / /26 DB Web #NET1522BU CONFIDENTIAL 22

23 NSX-T K8s Integration Pods Micro-Segmentations Option 2: K8s Network Policy vim nsx-demo-policy.yaml apiversion: extensions/v1beta1 kind: NetworkPolicy metadata: name: nsx-demo-policy spec: podselector: matchlabels: app: web ingress: - from: - namespaceselector: matchlabels: ncp/project: db ports: - port: 80 protocol: TCP admin@k8s-master:~$ kubectl create -f nsx-demo-policy.yaml VMworld 2017 State: released on K8s 1.7 (Beta on 1.6) Capability: Using Network Policy, users can define firewall rules to allow traffic into and out of a Namespace, and between Pods. The network policy is a Namespace property. The default is drop. NSX / K8s topology Namespace: foo NAT boundary Routed / / /26 Web Label: app=web Namespace: bar Content: Not for publication DB Label: app=db #NET1522BU CONFIDENTIAL 23

24 NSX-T K8s Integration Pods Micro-Segmentations Option 2: K8s Network Policy $ kubectl create -f nsx-demo-policy.yaml Once the Network Policy is applied, NSX will dynamically create source & destination Security Groups and apply the right policy Dynamic Creation of Security Groups Dynamic Creation of Security Policy based on k8s Network Policy #NET1522BU CONFIDENTIAL 24

25 NSX-T K8s Integration Pods Micro-Segmentations Firewalling in Kubernetes Micro-Segmentation in K8s: The data model to describe segmentation policies between Namespaces, and within namespaces is called Network Policies and is released on Kubernetes 1.7 (Beta on 1.6) K8s Network Policy NSX could utilize K8s Network Policies to define Dynamic Security Groups & Policies Capabilities are limited to K8s Network Policy capabilities VMworld 2017 Firewalling in NSX / K8s Pre-Defined Label based rules Security Groups & Policies could be predefined on NSX. Labels are used to specify Pods Membership Content: Not for publication Mapping of IP based groups, egress rules, VM based matching could be available to be used in the policy definition The NSX / K8s integration intends to support both the pre-defined label based rules and K8s network policy #NET1522BU CONFIDENTIAL 25

26 East-West Load Balancing K8s master K8s master dashboard Controller Manager Scheduler NSX CNI Plugin K8s API Server K8s Master OVS Node VM NSX Kube Proxy K8s Services are delivered through NSX Kube-Proxy Delivered as a container image, so that it can be run as a Kubernetes Daemon-Set on the Nodes NSX Kube-Proxy would replace the native distributed east-west load balancer in Kubernetes called Kube-Proxy OpenVSwitch (OVS) load-balancing is used Pods #NET1522BU CONFIDENTIAL 26

27 North-South Load Balancing Once an Ingress Controller is added, NSX will define SNAT & DNAT rules Nginx Ingress LB Pod Web Front-End Ingress VMworld 2017 Content: Not for publication / / #NET1522BU CONFIDENTIAL 27

28 K8s / NSX Components NSX Container Plugin (NCP) VMworld 2017 Content: Not for NCP is a software component provided by VMware in form of a container image, e.g. to be run as a K8s Pod publication NCP is build in a modular way, so that individual adapters can be added for different CaaS and PaaS systems #NET1522BU CONFIDENTIAL 28

29 K8s / NSX Workflows Namespace / Topology creation K8s master etcd API- Server Scheduler NSX/ K8s topology NS: foo 2) 1) 3) NSX Container Plugin K8s Adapter NSX Manager NS: bar NCP Infra NSX Manager API Client 4) Namespace creation workflow 1. NCP creates a watch on K8s API for any Namespace events 2. A user creates a new K8s Namespace 3. The K8s API Server notifies NCP of the change (addition) of Namespaces 4. NCP creates the network topology for the Namespace: a) Requests a new subnet from the preconfigured IP block in NSX b) Creates a logical switch c) Creates a T1 router and attaches it to the pre-configured global T0 router d) Creates a router port on the T1 router, attaches it to the LS, and assigns an IP from the new subnet #NET1522BU CONFIDENTIAL 29

30 NSX-T Container Interface (CIF) mgmt network eth0 Minion Mgmt. IP Stack NSX CNI Plugin cif eth2 Vlan 10 vlan 11 cif OVS DFW Node VM DFW cif eth2 vlan 10 vlan 11 cif OVS eth0 mgmt network Minion Mgmt. IP Stack Hypervisor (ESXi & KVM) Node VM Management Interface is Separated from the interface used for Pods traffic CIF is used per K8s Pod CIFs are differentiated through locally significant VLAN tags NSX CNI Plugin is responsible for tagging the traffic with the right VLAN NCP will map the VLAN tags to a specific CIF Pods Pods NSX CNI Plugin #NET1522BU CONFIDENTIAL 30

31 NSX-T Operational Tools for K8s VMworld 2017 Content: Not for NSX-T Operational Tools Traceflow Port Mirroring Port Connection Tool Spoofguard Syslog Port Counters IPFIX publication NSX-T Traceflow #NET1522BU CONFIDENTIAL 31

32 Demo

33 NSX-T Values for K8s Enterprise-class Networking Unified VM-to- Pod Networking N S X - T Va l u e s f o r K 8 s Advanced Security Pods Micro- Segmentation Full Network Visibility Enhanced Operations Enterprise Support F e a t u r e s #NET1522BU CONFIDENTIAL 33

34 Hands On Lab Self-Paced Lab VMware NSX-T with Kubernetes SPL182602U(US) SPL182602E(Europe) Kubernetes and VMware NSX Blog kubecon-2017.html/ #NET1522BU CONFIDENTIAL 34

35 Where to Get Started Engage and Learn Join VMUG for exclusive access to NSX vmug.com/vmug-join/vmug-advantage Connect with your peers communities.vmware.com Find NSX Resources vmware.com/products/nsx Network Virtualization Blog blogs.vmware.com/networkvirtualization Try VMworld 2017 Experience Dozens of Unique NSX Sessions Spotlights, breakouts, quick talks & group discussions Visit the VMware Booth Product overview, use-case demos Visit Technical Partner Booths Integration demos Infrastructure, security, operations, visibility, and more Meet the Experts Join our Experts in an intimate roundtable discussion Content: Not for publication Take Free Hands-on Labs Test drive NSX yourself with expert-led or self-paces hands-on labs labs.hol.vmware.com Training and Certification Several paths to professional certifications. Learn more at the Education & Certification Lounge. vmware.com/go/nsxtraining #NET1522BU CONFIDENTIAL 35

36

37