Wolfram Richter Red Hat. OpenShift Container Netzwerk aus Sicht der Workload

Transcription

1 Wolfram Richter Red Hat OpenShift Container Netzwerk aus Sicht der Workload

2 Why this session? OpenShift is great for web applications, but we want to do X will this work? X { Analytics, Non-HTTP, High- Performance Computing, Big Data, Object storage, NAS, Replicated Databases, } Let s take a look from a networking PoV!

3 Agenda What is OpenShift? How plain Docker Networking works and what OpenShift does differently Container Networking across nodes Kubernetes Services Ingress: OpenShift Router Egress Pods and Network Policy

4 OpenShift Platform & Container as a Service Built for both traditional and cloud-native applications An integrated hybrid cloud application platform for application development and deployment Develop, build, and manage container based applications Easily turn source code into running applications with source-to-image capabilities

5 OpenShift High-Level Architecture

6 Container Networking Problem Statements As an X, I want my containerized applications to be able to connect to other services, so that they can perform meaningful work. As an X, I want my containerized applications to be accessible externally, so that a wide range of users can use them.

7 RFC1918 IP assigned by docker daemon

8 Host NIC Docker bridge Docker host

9 So we can use the ip command Container

10 Indicates which IF the veth device is connected to Container

11 Endpoint of the container s veth device Docker host

12

13 Source IP appears to be Node IP

14 Outbound traffic is masqueraded Inbound traffic is forwarded to container Docker host

15 Container Networking Problem Statement As an X, I want to use network attached storage from within my container, so I can provide stateful services (*). (*) and storage traffic shouldn t share application network bandwith

16

17

18 Container Networking Problem Statement As a X, I want name resolution to work inside the container like they would on a dedicated machine, so that I don t have to care about them.

19 /etc/hosts /etc/hostname /etc/resolv.conf

20 Files in the container fs are overwritten Container

21 OpenShift Networking Problem Statement As a X, I want container networking to work seamlessly across multiple nodes, so that I don t have to worry where which containers run (*). (*) while still maintaining compatibility with plain docker containers

22

23 Host NICs Docker bridge docker bridge <-> ovs bridge ovs bridge ovs bridge <-> host NICs OpenShift node

24 Look, there s no container connected OpenShift node

25 Container veth endpoints on ovs bridge OpenShift node

26

27

28

29 Pod IP address OpenShift node

30 Ping works from container on same host Container

31 Ping fails from container on different host Container

32 Each node has specific IP range

33 IP Range node 1 IP Range node 2 OpenShift node

34 IP packets destined for pod on other node is encapsulated via VXLAN

35 ... and sent out via the node s IP stack (MTU impact!)

36 Port 1 is VXLAN OpenShift node

37 Flow rules that trigger VXLAN encapsulation Destination node IP address OpenShift node

38 Two pods in the same namespace on different nodes OpenShift node

39 can communicate with each other OpenShift pod

40 OpenShift Networking Problem Statement As a X, I want to ensure that a rogue pod cannot access pods in another project, so that I have a base level of security.

41 Project-specific VXLAN ID

42 Pod in a different namespace cannot be reached OpenShift pod

43 OpenShift Networking Problem Statement As a X, I want to be able to connect to other containerized services using a stable endpoint, so I don t have to reconfigure my application when other containers come and go.

44

45 Service IP Address OpenShift node

46 Namespace in search suffix list Name resolution via OpenShift dnsmasq on node OpenShift pod

47 Service name is resolved into IP adress OpenShift pod

48 Communication via service IP OpenShift pod

49

50 Kubernetes Service Modes User-space mode IPTables rules forward packages destined to the service IP address to the kube-proxy Kube-proxy will in turn initiate connections to the actual destination IP and proxy between the two endpoints Key advantage: can detect non-responding pods and retry connection to other pods IPTables mode kube-proxy continuously updates the node s IPTables rules forward packets directly to one of the target pod s IP Key advantage: increased throughput

51 OpenShift Networking Problem Statement As an X, I want my containerized applications to be accessible externally, so that a wide range of users can use them (*). (*) without having to care on which node a container/pod runs

52 Ingress router pod bound to host port

53 Host Port = port exposed on the node (containerized) haproxy

54

55 OpenShift Routing Layer 7 Routing: HTTP(S), TLS-SNI To properly route other protocols, deploy dedicated customized routers Alternatively instrument external load balancers such as F5, etc.

56 OpenShift Networking Problem Statement As an operator, I want to be able to fall back to the known working version of a service when deploying a new version so I have a safety net (blue/green deployments)

57 Router reconfiguration allows blue/green deployments oc patch route/api-gateway -p '{ "spec": { "to": { "name": "api-gateway-green" }}} oc patch route/api-gateway -p '{ "spec": { "to": { "name": "api-gateway-blue" }}}'

58 OpenShift Networking Problem Statement As an operator, I want my containerized applications to use specific source IP addresses to access external services, so I can restrict service access via (external) firewall rules.

59

60

61 Egress source IP Egress target IP (points to external service IPA ) Egress default GW

62 Points to egress-1 pod

63 Retrieving from egress-1 service works OpenShift pod

64 Egress source IP Egress target IP External service

65 if2: node s eth0 OpenShift egress pod

66 OpenShift Networking Problem Statement As an operator, I want to control which services my containerized applications can access, so I can limit access via internal means.

67 Egress Network Policy { }, } "kind": "EgressNetworkPolicy", "apiversion": "v1", "metadata": { "name": "default }, "spec": { } "egress": [ { "type": "Allow", "to": { "cidrselector": " /24 } { "type": "Deny", "to": { "cidrselector": " /32 } } ]

68 Summary If the question is OpenShift is great for web applications, but we want to do X will this work?, the answer is most likely yes (from a networking point of view) (*). (*) keep in mind that there is an MTU impact, multiple processing hops which impact latency, etc