PLACE IMAGE OVER THIS SPACE Docker IP Routing Having your first-hop load-balancer on Docker 1
Who are you? thorvald@medallia.com Medallia: Software to improve the customer experience Aggregating 1B documents in 1s or less This talk: Infrastructure to run the crunching application 2
Problem to solve Want a reliable, flexible data-center Docker for everything! Docker for applications! Docker for load-balancers! Docker for zookeeper! And DNS! 3
Problem to Solve Today Moving non-movable services DataCenter Firewall Host: 10.1.2.3:80 172.17.0.3:80 nginx Host: 10.1.2.4:2181 172.17.1.0:2181 zookeeper Host: 10.1.2.5:80 172.17.1.2:80 application 4
Design for Failure It will fail sooner or later; choose sooner Buy the cheapest possible servers Optimize for performance Sacrifice redundancy Service Contract: Unrack and ship by mail Solve the redundancy at a higher level Run multiple instances of everything Rapidly restore status quo 5
Design Principles No special snowflakes Commodity Components & Supported Open Standards Fully automated provisioning and reinstall Cheap Scalable Simple Every component must be able to run anywhere 6
Two Problems Storage and Network Going to talk about Network today. (Storage next time) 7
Things we tried That didn t work for us VLAN Can t do leaf/spine; need monstrous core switches VXLAN Network Island: How do you exit it? Vendor interoperability for forwarding DB (VXLAN-to-IP) Time to leave layer 2 behind? 8
Docker Bridged Networking Model host1 eth0 10.1.2.3/24 docker0 container 1 veth0 veth1 172.16.1.2 172.16.1.1 9
Docker* new strategy Default (Bridged) Strategy Creates a pair of veth. Moves one to the container namespace. Renames the container veth to eth0 Attaches the host veth to the docker0 bridge Configure port forwarding in iptables Routed Strategy Creates a pair of veth. Moves one to the container namespace. Renames the container veth to eth0. Add route to 0.0.0.0/0 via eth0 in container. Add route to container IP via veth0 in the host. 10
Docker* Routed Networking Model host2 % ip route 10.4.5.6 dev veth0... eth0 host3 host1 10.1.2.3/24 container-a hostn veth0 eth0 % ip route default eth0 10.4.5.6/32 OSPF Area 11
Infrastructure Route to 10.1.2.3/32 Spine Leaf Server 10.1.2.3/32 10.1.2.3/32 12
Infrastructure Servers, Network, OSPF Spine Leaf Server 13
Old and boring is the new sexy OSPF: 1998 Open Shortest Path First Propagated Link State Database Supported by every vendor OSPF is computationally expensive On a 1998-style embedded controller: Yes On a 2015-style Intel Atom 64-bit: No Everything is point-to-point L3 links Switches and Servers run OSPF (Quagga) Cumulus! OSPF unnumbered 14
Running a Container (technically, create and start a container) % docker run -it --name=foo --net=routed --ip-address=10.2.3.4/32 ubuntu /bin/bash (Will likely change to use labels) 15
demo! 16
Pros and Cons Awesomeness IP Mobility Quick failover No special snowflakes Everything in docker Sucksies T2 routing limit: 128K entries Future: CRIU? 17
How difficult is the network config? Leaf /etc/network/interfaces auto lo iface lo inet loopback address 10.225.10.245/32 %for v in range(1,17): auto swp${v} iface swp${v} mtu 9000 address 10.225.10.${v*8+1}/30 %endfor %for v in range(17,33): auto swp${v} iface swp${v} mtu 9000 address 10.225.10.245/32 %endfor Server /etc/network/interfaces auto lo iface lo inet loopback To Servers auto data0 iface data0 inet static mtu 9000 address 10.225.10.10 netmask 255.255.255.252 gateway 10.225.10.9 To Leaf To Spines 18
How difficult is the network config? Leaf/Spine Switch ospfd.conf router ospf ospf router-id 10.225.10.245 network 10.224.0.0/12 area 0.0.0.0! interface swp1 ip ospf network point-to-point! interface swp2 ip ospf network point-to-point!. Server ospfd.conf! Bootstrap Config router ospf ospf router-id 10.225.10.10 redistribute kernel passive-interface default no passive-interface data0 network 10.224.0.0/12 area 0.0.0.0! log syslog! interface data0 ip ospf network point-to-point! 19
Performance Good enough 24-39 Gbit/s (core affinity) 13us ICMP ping 20
Boot2Docker Local Development With Style IP Mobility on Local Laptop Allow for easy and rapid development 21
What day is today as 2 services Back-End (On 10.10.2.2) while true; do date nc -l 9999 done Front-End (On 10.10.2.1) while true; do echo Today is $(nc 10.10.2.2 9999) nc -l 8080 done 22
Boot2Docker on OSX My MacBook Boot2Docker VM 10.10.0.0/16 10.10.0.0/16 10.10.2.1/32 frontend on 8080 10.10.2.2/32 backed (date) on 9999 lo0: 10.10.2.2/32 backend My Shell 23
demo! 24
Next Steps for us In the next sprint... Share with the world! Everything you ve seen today is (or will be) open source Storage with CEPH Already works in medallia-container (think systemd-nspawn) Porting to docker SSH Hot Redirect SSH to container => SSH to host with automatic docker exec Docker Labels in Aurora/Mesos 25
Checkout and have fun www.github.com/medallia/docker www.github.com/medallia/boot2docker-iso docker@medallia.com 26
Questions? 27