Docker Networking Deep Dive online meetup

Transcription

1 Docker Networking Deep Dive online meetup

2 Agenda What is libnetwork CNM 1.12 Features Multihost networking Secured Control plane & Data plane Service Discovery Native Loadbalacing Routing Mesh Demo

3 Overview

4 What is libnetwork? It is not just a driver interface Docker networking fabric Defines Container Networking Model Provides builtin IP address management Provides native multi-host networking Provides native Service Discovery and Load Balancing Allows for extensions by the ecosystem via plugins

5 Design Philosophy Users First: Application Developers IT/Network Ops Plugin API Design Batteries Included but Swappable

6 Docker Networking - Libnetwork - CNM - Migrated Bridge, host, none drivers to CNM - Multihost Networking - Network Plugins - IPAM Plugins - Network UX/API - Aliases - DNS Round Robin LB Service Discovery (using /etc/hosts) Distributed DNS - Load Balancing - Encrypted Control and data plane - Routing Mesh - Built-in Swarm-mode networking

7 Container Networking Model Endpoint Network Sandbox Drivers & Plugins

8 Network driver overview

9 Use-case1 Default Bridge Network (docker0)

10 C1 C2 C3 C1 C2 C3 C1 C2 C3 docker0 docker0 docker0 iptables : NAT / port-mapping iptables : NAT / port-mapping iptables : NAT / port-mapping ToR switch / Hypervisor switch /

11 Use-case2 User-Defined Bridge Network

12 C1 C2 C3 C4 C5 C6 C7 C8 C9 brnet brnet brnet iptables : NAT / port-mapping iptables : NAT / port-mapping iptables : NAT / port-mapping Host1 Host2 Host3 ToR switch / Hypervisor switch / Host1 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig Host2 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig Host3 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig

13 Use-case3 Bridge Network plumbed to underlay with built-in IPAM (no NAT / Port-mapping)

14 C1 C2 C3 C4 C5 C6 C7 C8 C9 brnet brnet brnet Host1 eth Host2 eth Host3 eth ToR switch / Hypervisor switch / Virtual-box host-only / (Gateway : ) Host1 : $ docker network create -d bridge --subnet= /24 --ip-range= /28 --gateway= aux-address DefaultGatewayIPv4= o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2 $ docker run --net=brnet -it busybox ifconfig Host2 : $ docker network create -d bridge --subnet= /24 --ip-range= /28 --gateway= aux-address DefaultGatewayIPv4= o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2 $ docker run --net=brnet -it busybox ifconfig Host3 : $ docker network create -d bridge --subnet= /24 --ip-range= /28 --gateway= aux-address DefaultGatewayIPv4= o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2 $ docker run --net=brnet -it busybox ifconfig

15 Use-case4 Docker Overlay Network

16 Docker overlay networking VXLAN-VNI 100 VXLAN-VNI 100 VXLAN-VNI 100 ov-net1 ov-net1 ov-net1 C1 C2 C3 C4 C5 C6 C7 C8 C9 eth1 eth1 eth1 eth1 eth1 eth1 eth1 eth1 eth1 docker_gwbridge docker0 iptables : NAT / port-mapping docker_gwbridge docker0 iptables : NAT / port-mapping docker_gwbridge docker0 iptables : NAT / port-mapping ToR switch / Hypervisor switch /

17 Use-case5 Plumbed to underlay vlan with built-in IPAM macvlan driver (& experimental ipvlan)

18 # vlan 10 (.10) $ docker network create -d macvlan subnet= /24 gateway= o parent=.10 mcvlan10 $ docker run --net=mcvlan10 -it --rm alpine /bin/sh # vlan 20 (.20) $ docker network create -d macvlan subnet= /24 gateway= o parent=.20 mcvlan20 $ docker run --net=mcvlan20 -it --rm alpine /bin/sh # vlan 30 (.30) $ docker network create -d macvlan subnet= /24 gateway= o parent=.30 mcvlan30 $ docker run --net=mcvlan30 -it --rm alpine /bin/sh

19 Docker 1.12 Networking

20 New features in 1.12 swarm mode Multi-host Networking without external k/v store Cluster aware De-centralized control plane Highly scalable Secure Control-Plane Load Balancing CNM Secure Data-Plane Service Discovery Routing Mesh

21 Swarm-mode Multi-host networking VXLAN based data path No external key-value store Central resource allocation Improved performance Highly scalable Task Create Service Create Task Dispatch Manager Orchestrator Allocator Scheduler Dispatcher Task Dispatch Network Create Engine Libnetwork Worker1 Gossip Engine Libnetwork Worker2

22 Secured network control plane Cluster Scope Gossip Gossip based protocol Network scoped Fast convergence Secure by default periodic key rotations swarm native key-exchange Gossips control messages Routing-states Service-discovery Plugin-data Highly scalable W1 W3 W2 Network Scope Gossip Network Scope Gossip W1 W4 W5

23 Secure dataplane Available as an option during overlay network creation Uses kernel IPSec modules On-demand tunnel setup Swarm native key-exchange Periodic key rotations Worker1 secure network secure network IPSec Tunnel IPSec Tunnel IPSec Tunnel Worker2 secure network secure network nonsecure network Open UDP traffic nonsecure network Worker3

24 Service Discovery Provided by embedded DNS Highly available Uses Network Control Plane to learn state Can be used to discover both tasks and services DNS Resolver DNS requests DNS Resolver DNS Server engine

25 Load balancer Internal & Ingress load-balancing Supports VIP & DNS-RR Highly available Uses Network Control Plane to learn state Minimal Overhead Task1 ServiceA Task2 ServiceA Task3 ServiceA VIP LB VIP LB Client1 Client2

26 Routing mesh External Loadbalancer (optional) Builtin routing mesh for edge routing Worker nodes themselves participate in ingress routing mesh All worker nodes accept connection requests on PublishedPort Port translation happens at the worker node Same internal load balancing mechanism used to load balance external requests Worker1 Task1 ServiceA Ingress Network Task1 ServiceA Worker > >80 VIP LB VIP LB Task1 ServiceA

27 Routing Mesh :8080 Manager :8080 :8080 Worker 1 Worker 2 frontend frontend frontend User accesses myapp.com:8080 :8080 Worker 3 Operator reserves a swarmwide ingress port (8080) for myapp Every node listens on 8080 Container-aware routing mesh can transparently reroute traffic from Worker3 to a node that is running container Built in load balancing into the Engine DNS-based service discovery $ docker service create --replicas 3 --name frontend --network mynet --publish 8080:80/tcp frontend_image:latest

28 Routing Mesh: Published Ports Worker 1 frontend :8080 :8080 Manager Worker 2 frontend :8080 frontend User accesses myapp.com:8080 :8080 Worker 3 Operator reserves a swarmwide ingress port (8080) for myapp Every node listens on 8080 Container-aware routing mesh can transparently reroute traffic from Worker3 to a node that is running container Built in load balancing into the Engine DNS-based service discovery $ docker service create --replicas 3 --name frontend --network mynet --publish 8080:80/tcp frontend_image:latest

29 Deep Dive

30 Service, Port-Publish & Network docker service create name=test network=mynet -p 8080:80 replicas=2 xxx Host1: 8080 iptables default_gwbridge Host1 daemon embedded DNS server service -> VIP mynet eth1 ingress-sbox iptables eth1 Container-sbox DNS Resolver iptables eth2 mynet-br vxlan tunnel to host2 - vni-101 ipvs ipvs Ingress- Network vxlan tunnel to host3 - vni-100 ingress-overlay-bridge vxlan tunnel to host2 - vni-100

31 Day in life of a packet - IPTables & IPVS

32 Day in life of a packet - Routing Mesh & Ingress LB iptables NAT table DOCKER-INGRESS DNAT : Published-Port -> ingress-sbox Host1 iptables NAT table DOCKER-INGRESS DNAT : Published-Port -> ingress-sbox Host2 default_gwbridge default_gwbridge eth1 iptables MANGLE table PREROUTING MARK : Published-Port -> <fw-mark-id> ingress-sbox eth1 ingress-sbox eth1 Container-sbox (backs a task/ service) iptables NAT table PREROUTING Redirect -> service-port IPVS Match <fw-mark-id> -> Masq {RR across container-ips) Ingress- Network Ingress- Network ingress-overlay-bridge vxlan tunnel with vni ingress-overlay-bridge

33 Day in life of a packet - Internal LB Host1 daemon embedded DNS server service2 -> VIP2 Host2 eth1 Application looks up service2 (using ) container-sbox (service1) DNS Resolver iptables MANGLE table OUTPUT MARK : VIP -> <fw-mark-id> Container-sbox (service2) IPVS Match <fw-mark-id> -> Masq {RR across container-ips) eth2 eth2 mynet mynet vxlan tunnel with vni mynet-overlay-bridge vxlan tunnel with vni mynet-overlay-bridge

34 Thank you!