2016 Mesosphere, Inc. All Rights Reserved.

Transcription

1 MesosCon Qian Zhang (IBM China), Avinash Sridharan, Jie Yu (Mesosphere) Container Network Interface (CNI) for Mesos: The `network/cni` isolator. 1

2 Qian Zhang Software Engineer Avinash Sridharan Jie Yu Software Engineer Software Engineer 2

3 Current state of networking in `MesosContainerizer` Agent and Mesos containers share the host network namespace Host Network Namespace Container X Container Y Agent Frameworks launching containers have to manage TCP/UDP ports. eth0

4 What we would like? IP-per-container!! Its own network namespace and IP address. Containers communicate over Layer2 / Layer 3 network. Host Network Namespace Network Namespace Network Namespace Container X Container Y veth1 Agent veth2 Container Network eth0 4

5 Why not `DockerContainerizer`? Reduce dependency on the Docker daemon. Unified Containerizer is the future!! Supports docker and appc images.

6 The challenge Democratize Networking Give operator the freedom to choose the underlying network technology.

7 The curse of choice VLAN VxLAN IPvlan MACvlan

8 Solution: A driver-based model Separation of concerns : Mesos takes care of creating network namespace. Network driver encapsulates business intelligence.

9 Container Network Interface (CNI) Proposed by CoreOS : Simple contract between container runtime and CNI plugin defined in the form of a JSON schema.

10 Why not CNM (a.k.a libnetwork )? Docker centric. An API driven model. Traction barrier to entry lower for CNI (Kubernetes is helping!!).

11 How does it work? Framework ContainerInfo { type = MESOS;... NetworkInfo{ name = cni-mesoscon } } CNI Configuration { "name": "cni-mesoscon", "type": "bridge", "bridge": "mesos-cni0",... Container X Ipam : { Type : dhcp... } veth0 } Agent Bridge plugin mesos-cni0 IPAM

12 Some internal details A container gets MNT NET UTS namespace. The isolator mounts /etc/hosts, /etc/hostname and /etc/resolv.conf into MNT namespace. Checkpoints the network namespace, IP address and CNI network name for recovery.

13 Current Limitations CNI spec does not support port mapping. Need to implement port mapping in the `network/cni` isolator. (MESOS-4823) Behavior of container operation when CNI configuration is modified, or deleted, is undefined. (MESOS-5310)

14 Demo Time!!

15 Setup /24 Nginx HAProxy Nginx /24 Bridge-green Bridge-blue Agent /24 Client Master Framework (nginx-blue) Framework (HAProxy) Framework (nginx-green)

16 Questions?

17 THANK YOU! 17

18 Setup /24 Nginx HAProxy Nginx /24 Flannel-green Flannel-blue Agent /24 Client Master Framework (nginx-blue) Framework (HAProxy) Framework (nginx-green)

19 Container Network Interface (CNI) for Mesos: The `network/cni` isolator. Qian Zhang (IBM China), Avinash Sridharan and Jie Yu

20 The curse of choice Containers and Hosts are on the same network. Containers have their own network, but Host network has reachability information about the container network. Host network is completely agnostic of containers. Examples. Examples. VLAN MacVLAN IPVLAN Very simple. Fits into existing networks. Examples. Calico Provides a flat address space. Uses BGP to make container address space routeable. Flannel Weave Encapsulates all container traffic into an overlay transport.

21 What we would like? IP-per-container Each container has its own network namespace and hence its own IP address. Containers talk to each and the Agent through a Layer 2/ Layer 3 network. Containers behave like end hosts. Host Network Namespace Network Namespace Network Namespace Container X Container Y veth1 Agent veth2 Container Network eth0

22 Why not `DockerContainerizer`? Want to reduce dependency on the Docker daemon. Intractable to maintain two container runtimes. Unified Containerizer makes `MesosContainerizer` a full-fledged container runtime at par with docker run-time (and soon Rkt runtime). Makes sense to invest effort to bring richer networking capabilities to `MesosContainerizer`.

23 The challenge Democratize Networking We should allow the operator to choose the underlying technology used to network the containers.

24 The curse of choice Containers and Hosts are on the same network. Containers have their own network, but Host network has reachability information about the container network. Host network is completely agnostic of containers. Examples. Examples. VLAN MacVLAN IPVLAN Fits into existing networks. Examples. Calico Uses BGP to make container address space routeable. Flannel Weave Encapsulates all container traffic into an overlay transport.

25 Solution: A driver-based model Have a separation of concerns between container runtime and network drivers. Mesos takes care of creating network namespace. Network driver encapsulates business intelligence to attach containers to network.

26 Container Network Interface (CNI) Proposed by CoreOS : Has gained a lot of traction with Kubernetes, and some other projects. Container runtime is responsible for creating the network namespace and invoking a CNI plugin at some stage of the container life cycle. CNI plugin responsible for configuring resources in the namespace and invoking IPAM. Contract between container runtime and CNI plugin defined in the form of a

27 Why not CNM (a.k.a libnetwork )? Very Docker centric. An API driver model. Has this notion of local and remote drivers. Local drivers reside in the container runtimes process, remote drivers are daemons. Traction barrier to entry lower for CNI (Kubernetes is helping!!).

28 How does it work? Framework ContainerInfo { type = MESOS;... NetworkInfo{ name = cni-mesoscon } CNI Configuration { "name": "cni-mesoscon", "type": "bridge", "bridge": "mesos-cni0",.... Container X veth0 } } Agent Bridge plugin mesos-cni0

29 Some internal details When joining a CNI network a container gets MNT NET UTS namespace. The `network/cni` isolator mounts /etc/hosts, /etc/hostname and /etc/resolv.conf into MNT namespace. For restartability the `network/cni` isolator checkpoints the network namespace, IP address and CNI network name for the container.

30 VISIT OUR BOOTH Located at D1, near the main entrance Learn more by visiting dcos.io and mesosphere.com 30

31 Limitations CNI spec does not support port mapping. Need to implement port mapping in the `network/cni` isolator. (MESOS-4823) Behavior of container operation when CNI configuration is modified, or deleted, is undefined. (MESOS-5310)