Project Kuryr. Here comes advanced services for containers networking. Antoni Segura

Transcription

1 Project Kuryr Here comes advanced services for containers networking Mohammad Gal Antoni Segura

2 Outline Motivation and Introduction Current state Kubernetes Getting involved Demo

3

4 Networking as the starting point... Reinventing networking abstractions Changing and vendor specific solutions Hard to connect VMs, bare metal and nested containers No unified networking infrastructure 2 Overlay for VM nested containers Performance, latency, SLA, management penalties Lack of isolation and policy level constructs Security 3-Tier Applications

5 Similar Concepts

6 Nested Containers Overlay

7 Mission The Glue Between Containers Ecosystem and OpenStack

8 Current state

9 Kuryr Project Overview Part of OpenStack Big-Tent Connecting Containers with OpenStack Services: Networking (Neutron), Authentication (Keystone) and Storage (Cinder) Aims to support: Different container Runtimes: docker, rkt, etc Multi-host/Clustered environments: Kubernetes, Mesos, Docker Swarm Working together with OpenStack community Magnum, Kolla, Neutron, Keystone, Cinder

10 Kuryr Newton contribution

11 Current Supported Features Keystone v2 and v3 support with multiple pluggins Supports Docker networking (CNM): libnetwork Partial Support for Kubernetes networking (CNI) Allows use of Security Groups (exposed ports and pod annotation) Supports use of existing Neutron resources Remote driver IPAM driver Networks Subnets Load balancers Supports Docker Swarm*

12 Components Multiple Repos openstack/kuryr (kuryr-lib) Configuration Binding Llibrary openstack/kuryr-libnetwork Libnetwork remote driver Libnetwork ipam driver openstack/kuryr-kubernetes K8s Watcher CNI plugin

13 Kuryr VIF Binding drivers Part of Kuryr-lib Virtual device drivers Veths Ipvlan Macvlan vlan* Supports ovs, lb, midonet and others Used by both Docker libnetwork and CNI To be enhanced to use Oslo versioned objects and os-vif

14 How to Use: Docker Native API $ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet= /24 \ --gateway= mynet 08192d75a75a490163ac33434fb2c8a74a7841ad42abb985a1fdc7cde3c92f17 $ docker run -it --net=mynet alpine sh

15 Behind the Scenes: Neutron Network $ neutron net-list id name subnets a747d101-eefb-4aa c7ad181f8e0 kuryr-net-08192d75 42a8d0c5-687d-4284-ac3e-2db86f5579f /24 $ neutron net-show kuryr-net-08192d Field Value id a747d101-eefb-4aa c7ad181f8e0 name kuryr-net-08192d75 subnets 42a8d0c5-687d-4284-ac3e-2db86f5579f6 tags kuryr.net.uuid.lh:08192d75a75a490163ac33434fb2c8a7 kuryr.net.uuid.uh:4a7841ad42abb985a1fdc7cde3c92f17 network tags: associate Neutron and Docker networks

16 Existing Neutron Networks $ neutron net-create mynet $ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet= /24 \ --gateway= \ -o neutron.net.name=mynet \ mynet_d $ ID=$(neutron net-list grep mynet \ awk '{print $2}') $ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet= /24 \ --gateway= \ -o neutron.net.uuid=$id mynet_d

17 Existing Neutron Networks (Cont d) $ neutron net-show mynet Field Value tags kuryr.net.uuid.uh:4ca3f3fc3fc48a8c9cd902ed7508f1cd kuryr.net.uuid.lh:12f769bd2697f2200f27f60753bd5dad kuryr.net.existing Neutron tags added to the network Subnet if existing is used, otherwise gets created Neutron network is not deleted upon deletion of Docker network If using an old release (i.e., Liberty or earlier): Neutron network name is modified Deleting Docker network deletion of Neutron network if no ports

18 Exposed ports and Security Groups $ docker run --net=mynet --expose=1234/udp -it alpine sh $ neutron port-list id fixed_ips f0fbec-67ac b7044c99 {"subnet_id": "2b3c390c-a8b bb a6d71d4c", bb a6d71d4c", "ip_address": " "} $ neutron port-show 53f0fbec-67ac b7044c Field Value security_groups 4062dc c-a577-f4aef58ef99d e50a0ae3-0d79-4c20-93a7-51dafd9b2311 Docker expose supported by Neutron security groups

19 Exposed ports (Cont d) $ neutron security-group-list id name security_group_rules dc c-a577-f4 53f0fbec-67ac egress, IPv4 aef58ef99d 11b7044c99-exposed_ports egress, IPv6 ingress, IPv4, 1234/udp e50a0ae3-0d79-4c20-93a7-51 default egress, IPv4 dafd9b2311 egress, IPv6 ingress, IPv4, remote_group_id: e50a0ae3-0d79-4c20-93a7-51dafd9b2311 ingress, IPv6, remote_group_id: e50a0ae3-0d79-4c20-93a7-51dafd9b

20 Kubernetes

21 Kubernetes Container orchestration engine Implemented in Go Thriving multi vendor community Main components API server Scheduler Kubelet Controller manager Etcd datastore Pluggable networking with Container Networking Interface

22 Kubernetes Integration Deployment options Kuryr CNI plugin only communicates with K8s API Kuryr watcher on the project instances Kuryr watcher on admin/tenant machine Kubelet already has a channel to the API Uses kuryr-lib for binding to the correct Neutron port Kuryr watcher updates API servers Neutron to create the resources for the Kubernetes deployment Kubernetes resource annotations to place Neutron resource info

23 Kubernetes components Kubelet configured with Kuryr CNI The watcher reads k8s resource event streams Events can have multiple handlers Multiple Watchers for HA*

24 Kubernetes Services

25 Kubernetes Services

26 VM Nested Containers and Magnum Will support net modes: Ipvlan Macvlan Trunk / subport If Neutron is routable from controller VM, watcher can go there CNI only needs to talk to k8s API Depending on vendor, lbaas will need a node with Octavia

27 Getting involved

28 Packaging Automated container builds at libnetwork watcher kubelet Kolla Integration Distribution packaging with systemd unit files Debian RDO

29 Kuryr Roadmap Ocata cycle Kubernetes integration HA Multitenancy Policy support using security groups Nested containers and Magnum integration Oslo Versioned Objects binding interface Kuryr-Kubernetes release Kuryr-lib Kuryr-libnetwork 1.0.0

30 Kuryr and Storage Kuryr as incubator for bringing native OpenStack services to containers Cinder Manila Swift Freezer

31 Kuryr work sessions Thursday 27th Friday 28th 9:00-9:40 K8s services 9:50-10:30 K8s HA 11:00-11:40 Kolla and containers 11:50-12:30 Storage - Cinder 13:50-14:30 Kuryr - Magnum 16:40-17:30 Container-in-VM 17:30-18:10 Multitenancy

32 Join us! Be the Kuryr! Project Launchpad Project Git Repository IRC Weekly IRC Meeting Freenode Mailing List openstack-dev@lists.openstack.org ([Kuryr])

33 Join us! Be the Kuryr! Documentation Getting Started Blog posts

34 Demo