Life of a Packet. KubeCon Europe Michael Rubin TL/TLM in GKE/Kubernetes github.com/matchstick. logo. Google Cloud Platform

Transcription

1 logo Life of a Packet KubeCon Europe 2017 Michael Rubin <mrubin@google.com> TL/TLM in GKE/Kubernetes github.com/matchstick Google Cloud Platform

2 Kubernetes is about clusters Because of that, networking is pretty important Most of Kubernetes centers on network concepts Our job is to make sure your applications can communicate: With each other With the world outside your cluster Only where you want

3 The IP-per-pod model

4 Every pod has a real IP address This is different from the out-of-the-box model Docker offers No machine-private IPs No port-mapping Pod IPs are accessible from other pods, regardless of which VM they are on Linux network namespaces (aka netns ) and virtual interfaces

5 Network namespaces Node

6 Network namespaces root netns Node

7 Network namespaces root netns pod1 netns Node

8 Network namespaces root netns vethxx vethxy pod1 netns Node

9 Network namespaces root netns vethxx pod1 netns Node

10 Network namespaces root netns vethxx vethyy pod1 netns Node pod2 netns

11 Network namespaces cbr0 root netns vethxx vethyy pod1 netns Node pod2 netns

12 Life of a packet: pod-to-pod, same node root netns vethxx cbr0 vethyy pod1 netns Node pod2 netns

13 Life of a packet: pod-to-pod, same node src: pod1 dst: pod2 root netns cbr0 vethxx vethyy pod1 netns Node pod2 netns

14 Life of a packet: pod-to-pod, same node src: pod1 dst: pod2 root netns cbr0 vethxx vethyy pod1 ctr1 netns netns Node pod2 ctr2 netns

15 Life of a packet: pod-to-pod, same node src: pod1 dst: pod2 root netns cbr0 vethxx vethyy pod1 ctr1 netns netns Node pod2 ctr2 netns

16 Life of a packet: pod-to-pod, same node src: pod1 dst: pod2 root netns cbr0 vethxx vethyy pod1 ctr1 netns netns Node pod2 ctr2 netns

17 Flat network space Pods must be reachable across Nodes, too Kubernetes doesn t care HOW, but this is a requirement L2, L3, or overlay Assign a CIDR (IP block) to each Node GCP: Teach the network how to route packets

18 Life of a packet: pod-to-pod, across nodes Node1 pod1 pod2 vethxx vethyy root cbr0 root vethxx cbr0 vethyy Node2 pod3 pod4

19 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

20 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

21 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

22 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

23 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

24 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

25 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

26 Life of a packet: pod-to-pod, across nodes src: pod1 dst: pod4 Node1 root pod1 ctr1 vethxx cbr0 vethyy ctr2 pod2 root vethxx cbr0 vethyy Node2 pod3 ctr3 pod4 ctr4

27 Overlays

28 Overlay networks Why? Can t get enough IP space Network can t handle extra routes Want management features Encapsulate packet-in-packet Traverse the native network between Nodes

29 Overlay networks Why Not? Latency overhead in some cloud providers Complexity overhead Often not required Use it when you know you need it

30 Overlay example: Flannel (vxlan) VXLAN interface acts like any other vnic Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

31 Overlay example: Flannel src: pod1 dst: pod4 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

32 Overlay example: Flannel src: pod1 dst: pod4 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

33 Overlay example: Flannel src: pod1 dst: pod4 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

34 Overlay example: Flannel src: pod1 dst: pod4 root netns Node1 vethxx cbr0 flannel0

35 Overlay example: Flannel src: pod1 dst: pod4 root netns Node1 vethxx cbr0 flannel0

36 Overlay example: Flannel Encapsulates the packet MAC Flannel device implementation: Simple VXLAN, developed by CoreOS for containers and kubernetes Uses Linux native VXLAN devices A userspace agent for address resolution Data path is in-kernel (fast) src: pod1 dst: pod4 Payload flannel0 src: node1 dst: node2 UDP src: pod1 dst: pod4 Payload

37 Overlay example: Flannel src: pod1 src: node1 dst: pod4 dst: node2 encapsulated root netns flannel0 Node1 vethxx cbr0

38 Overlay example: Flannel src: node1 dst: node2 root netns Node1 vethxx cbr0 flannel0

39 Overlay example: Flannel src: node1 dst: node2 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

40 Overlay example: Flannel src: node1 dst: node2 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

41 Overlay example: Flannel src: node1 dst: node2 flannel0 cbr0 root netns vethxx Node2

42 Overlay example: Flannel src: node1 src: pod1 dst: node2 dst: pod4 decapsulated flannel0 root netns cbr0 vethxx Node2

43 Overlay example: Flannel src: pod1 dst: pod4 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

44 Overlay example: Flannel src: pod1 dst: pod4 Node1 root pod1 vethxx cbr0 pod2 vethyy flannel0 root vethxx cbr0 flannel0 vethyy Node2 pod3 pod4

45 Dealing with change A real cluster changes over time: Rolling updates Scale-up and scale-down events Pods crash or hang Nodes reboot The pod addresses you need to reach can change without warning You need something more durable than a pod IP

46 Services

47 The service abstraction A service is a group of endpoints (usually pods) Services provide a stable VIP VIP automatically routes to backend pods Implementations can vary We will examine the default implementation The set of pods behind a service can change Clients only need the VIP, which doesn t change

48 Service What you submit is simple Other fields will be defaulted or assigned kind: Service apiversion: v1 metadata: name: store-be spec: selector: app: store role: be ports: - name: http port: 80

49 Service What you submit is simple Other fields will be defaulted or assigned The selector field chooses which pods to balance across kind: Service apiversion: v1 metadata: name: store-be spec: selector: app: store role: be ports: - name: http port: 80

50 Service What you get back has more information Automatically creates a distributed load balancer kind: Service apiversion: v1 metadata: name: store-be namespace: default creationtimestamp: T19:16:56Z resourceversion: "7" selflink: /api/v1/namespaces/default/services/store-be uid: 196d bf-11e a800fe3 Spec: type: ClusterIP selector: app: store role: be clusterip: ports: - name: http protocol: TCP port: 80 targetport: 80 sessionaffinity: None

51 Service What you get back has more information Automatically creates a distributed load balancer The default is to allocate an in-cluster IP kind: Service apiversion: v1 metadata: name: store-be namespace: default creationtimestamp: T19:16:56Z resourceversion: "7" selflink: /api/v1/namespaces/default/services/store-be uid: 196d bf-11e a800fe3 Spec: type: ClusterIP selector: app: store role: be clusterip: ports: - name: http protocol: TCP port: 80 targetport: 80 sessionaffinity: None

52 Endpoints selector: app: store role: be app: store role: be app: store role: fe app: db role: be app: store role: be app: db role: be app: store role: be

53 Endpoints selector: app: store role: be app: store role: be app: store role: fe app: db role: be app: store role: be app: db role: be app: store role: be

54 Endpoints When you create a service, a controller wakes up kind: Endpoints apiversion: v1 metadata: name: store-be namespace: default subsets: - addresses: - ip: ip: ip: ports: - name: http port: 80 protocol: TCP

55 Endpoints When you create a service, a controller wakes up Holds the IPs of the pod backends kind: Endpoints apiversion: v1 metadata: name: store-be namespace: default subsets: - addresses: - ip: ip: ip: ports: - name: http port: 80 protocol: TCP

56 Life of a packet: pod-to-service root netns vethxx cbr0 pod1 netns

57 Life of a packet: pod-to-service src: pod1 dst: svc1 root netns vethxx cbr0 pod1 ctr1 netns

58 Life of a packet: pod-to-service src: pod1 dst: svc1 root netns vethxx cbr0 pod1 ctr1 netns

59 Life of a packet: pod-to-service src: pod1 dst: svc1 root netns iptables cbr0 vethxx pod1 ctr1 netns

60 Life of a packet: pod-to-service src: pod1 dst: svc1 dst: pod99 DNAT, conntrack root netns iptables cbr0 vethxx pod1 ctr1 netns

61 Conntrack Linux kernel connection-tracking Remembers address translations Based on the 5-tuple Does a lot more, but not very relevant here Reversed on the return path { protocol = TCP src_ip = pod1 src_port = 1234 dst_ip = svc1 dst_port = 80 } => { protocol = TCP src_ip = pod1 src_port = 1234 dst_ip = pod99 dst_port = 80 }

62 Life of a packet: pod-to-service src: pod1 dst: pod99 root netns iptables cbr0 vethxx pod1 ctr1 netns

63 Life of a packet: pod-to-service src: pod99 dst: pod1 root netns iptables cbr0 vethxx pod1 ctr1 netns

64 Life of a packet: pod-to-service src: pod99 src: svc1 dst: pod1 un-dnat root netns iptables cbr0 vethxx pod1 ctr1 netns

65 Life of a packet: pod-to-service src: svc1 dst: pod1 root netns iptables cbr0 vethxx pod1 ctr1 netns

66 Life of a packet: pod-to-service src: svc1 dst: pod1 root netns iptables cbr0 vethxx pod1 ctr1 netns

67 A bit more on iptables The iptables rules look scary, but are actually simple: if dest.ip == svc1.ip && dest.port == svc1.port { pick one of the backends at random rewrite destination IP } Configured by kube-proxy - a pod running on each Node Not actually a proxy Not in the data path Kube-proxy is a controller - it watches the API for services

68 DNS Even easier: services are added to an in-cluster DNS server You would never hardcode an IP, but you might hardcode a hostname and port Serves A and SRV records DNS itself runs as pods and a service

69 DNS Service Requests a particular cluster IP Pods are auto-scaled with the cluster size Service VIP is stable kind: Service apiversion: v1 metadata: name: kube-dns namespace: kube-system spec: clusterip: selector: k8s-app: kube-dns ports: - name: dns port: 53 protocol: UDP - name: dns-tcp port: 53 protocol: TCP

70 DNS Service Requests a particular cluster IP Pods are auto-scaled with the cluster size Service VIP is stable kind: Service apiversion: v1 metadata: name: kube-dns namespace: kube-system spec: clusterip: selector: k8s-app: kube-dns ports: - name: dns port: 53 protocol: UDP - name: dns-tcp port: 53 protocol: TCP

71 Simple and powerful Can use any port you want, no conflicts Can request a particular clusterip Can remap ports

72 That s all there is to it Services are an abstraction - the API is a VIP No running process or intercepting the data-path All a client needs to do is hit the service IP:port

73 Sending external traffic Services are within a cluster What happens if you want your pod to reach google.com?

74 Egress

75 Leaving the GCP project Nodes get private IPs (in /8) Nodes can have public IPs, too GCP: Public IPs are provided by 1-to-1 NAT

76 Life of a packet: Node-to-internet 1:1 NAT Cluster root netns Node

77 Life of a packet: Node-to-internet src: internal-ip dst: Cluster 1:1 NAT root netns Node

78 Life of a packet: Node-to-internet src: internal-ip src: external-ip dst: Cluster 1:1 NAT root netns Node

79 Life of a packet: Node-to-internet src: external-ip dst: Cluster 1:1 NAT root netns Node

80 Life of a packet: Node-to-internet src: dst: external-ip Cluster 1:1 NAT root netns Node

81 Life of a packet: Node-to-internet src: dst: external-ip dst: internal-ip Cluster 1:1 NAT root netns Node

82 Life of a packet: Node-to-internet src: dst: internal-ip Cluster 1:1 NAT root netns Node

83 Life of a packet: pod-to-internet 1:1 NAT Cluster root netns vethxx cbr0 Node pod1 netns

84 Life of a packet: pod-to-internet src: pod1 dst: Cluster 1:1 NAT root netns vethxx cbr0 Node pod1 netns

85 Life of a packet: pod-to-internet src: pod1 dst: Cluster 1:1 NAT root netns vethxx cbr0 Node pod1 netns

86 Life of a packet: pod-to-internet src: pod1 dst: Cluster 1:1 NAT root netns vethxx cbr0 Node pod1 netns

87 Life of a packet: pod-to-internet dropped! Cluster 1:1 NAT root netns vethxx cbr0 Node pod1 netns

88 What went wrong? The 1:1 NAT only understands Node IPs Anything else gets dropped Pod IPs!= Node IPs When in doubt, add some more iptables MASQUERADE, aka SNAT Applies to any packet with a destination *outside* of /8

89 Life of a packet: pod-to-internet src: pod1 src: internal-ip dst: MASQUERADE Cluster root netns 1:1 NAT iptables cbr0 vethxx Node pod1 netns

90 Life of a packet: pod-to-internet src: internal-ip dst: Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

91 Life of a packet: pod-to-internet src: internal-ip src: external-ip dst: Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

92 Life of a packet: pod-to-internet src: external-ip dst: Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

93 Life of a packet: pod-to-internet src: dst: external-ip Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

94 Life of a packet: pod-to-internet src: dst: external-ip dst: internal-ip Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

95 Life of a packet: pod-to-internet src: dst: internal-ip Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

96 Life of a packet: pod-to-internet src: dst: internal-ip dst: pod1 Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

97 Life of a packet: pod-to-internet src: dst: pod1 Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

98 Life of a packet: pod-to-internet src: dst: pod1 Cluster 1:1 NAT iptables root netns vethxx cbr0 Node pod1 netns

99 Receiving external traffic Kubernetes builds on two: Network Load Balancer (L4) HTTP/S Load balancer (L7) These map to Kubernetes APIs: Service type=loadbalancer Ingress

100 L4: Service + LoadBalancer

101 Service Change the type of your service Implemented by the cloud provider controller kind: Service apiversion: v1 metadata: name: store-be spec: type: LoadBalancer selector: app: store role: be ports: - name: https port: 443

102 Service The LB info is populated when ready kind: Service apiversion: v1 metadata: name: store-be #... spec: type: LoadBalancer selector: app: store role: be clusterip: ports: #... sessionaffinity: None status: loadbalancer: ingress: - ip:

103 Life of a packet: external-to-service Cluster pod1 pod2 pod3 Node1 Node2 Node3

104 Life of a packet: external-to-service Net LB Cluster pod1 pod2 pod3 Node1 Node2 Node3

105 Life of a packet: external-to-service Net LB Cluster pod1 pod2 pod3 Node1 Node2 Node3

106 Life of a packet: external-to-service src: client dst: LB Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

107 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Choose a Node pod1 pod2 pod3 VM1 Node1 VM1 Node2 VM1 Node3

108 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Choose a Node VM1 Node1 pod1 pod2 pod3 Node2 Node3

109 Life of a packet: external-to-service src: client dst: LB Cluster Net LB VM1 Node1 pod1 pod2 pod3 Node2 Node3

110 Life of a packet: external-to-service src: client dst: LB Cluster Net LB VM1 Node1 pod1 pod2 pod3 Node2 Node3

111 Balancing to Nodes Most LB only knows about Nodes Nodes do not map 1:1 with pods Node1 Node2 Node3

112 The imbalance problem Assume the LB only hits Nodes with backend pods on them The LB only knows about Nodes Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

113 The imbalance problem Net LB Cluster 50% 50% pod1 pod2 pod3 Node1 Node2 Node3

114 The imbalance problem Net LB Cluster 50% 50% 50% 25% 25% pod1 pod2 pod3 Node1 Node2 Node3

115 Balancing to Nodes Most cloud LB only knows about Nodes Nodes do not map 1:1 with pods How do we avoid imbalance? iptables, of course Node1 Node2 Node3

116 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Choose a pod iptables pod1 pod2 pod3 Node1 Node2 Node3

117 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Choose a pod iptables pod1 pod2 pod3 Node1 Node2 Node3

118 Life of a packet: external-to-service src: client dst: LB dst: pod2 Cluster iptables Net LB NAT pod1 pod2 pod3 Node1 Node2 Node3

119 Life of a packet: external-to-service src: client dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

120 Life of a packet: external-to-service src: client dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

121 Life of a packet: external-to-service src: client dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

122 Life of a packet: external-to-service src: client dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

123 Life of a packet: external-to-service src: pod2 dst: client Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

124 Life of a packet: external-to-service src: pod2 dst: client Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

125 Life of a packet: external-to-service src: pod2 dst: client Cluster Net LB INVALID iptables pod1 pod2 pod3 Node1 Node2 Node3

126 Life of a packet: external-to-service src: client src: Node1 dst: LB dst: pod2 NAT Cluster iptables Net LB pod1 pod2 pod3 Node1 Node2 Node3

127 Life of a packet: external-to-service src: Node1 dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

128 Life of a packet: external-to-service src: Node1 dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

129 Life of a packet: external-to-service src: Node1 dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

130 Life of a packet: external-to-service src: Node1 dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

131 Life of a packet: external-to-service src: pod2 dst: Node1 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

132 Life of a packet: external-to-service src: pod2 dst: Node1 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

133 Life of a packet: external-to-service src: pod2 dst: Node1 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

134 Life of a packet: external-to-service src: pod2 src: LB dst: Node1 dst: client Cluster iptables Net LB pod1 pod2 pod3 Node1 Node2 Node3

135 Life of a packet: external-to-service src: LB dst: client Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

136 Life of a packet: external-to-service src: LB dst: client Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

137 Explain the complexity To avoid imbalance, we re-balance inside Kubernetes A backend is chosen randomly from all pods Good: Well balanced, in practice Bad: Can cause an extra network hop Hides the client IP from the user s backend Users wanted to make the trade-off themselves

138 OnlyLocal Specify an external-traffic policy iptables will always choose a pod on the same node Preserves client IP Risks imbalance kind: Service apiversion: v1 metadata: name: store-be annotations: service.beta.kubernetes.io/external-traffic: OnlyLocal spec: type: LoadBalancer selector: app: store role: be ports: - name: https port: 443

139 Opt-in to the imbalance problem In practice Kubernetes spreads pods across nodes If pods >> nodes: OK If nodes >> pods: OK If pods ~= nodes: risk Cluster Net LB 50% 50% iptables iptables 50% 25% 25% pod1 pod2 pod3 Node1 Node2 Node3

140 Life of a packet: external-to-service Not considered Health-check fails if no backends Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

141 Life of a packet: external-to-service src: client dst: LB Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

142 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Choose a Node pod1 pod2 pod3 VM1 Node1 Node2 VM1 Node3

143 Life of a packet: external-to-service src: client dst: LB Cluster Net LB Node1 pod1 pod2 pod3 Node2 VM1 Node3

144 Life of a packet: external-to-service src: client dst: LB Cluster Net LB pod1 pod2 pod3 Node1 Node2 VM1 Node3

145 Life of a packet: external-to-service src: client dst: LB Cluster Net LB iptables Choose a pod pod1 pod2 pod3 Node1 Node2 Node3

146 Life of a packet: external-to-service src: client dst: LB dst: pod2 Cluster Net LB iptables DNAT pod1 pod2 pod3 Node1 Node2 Node3

147 Life of a packet: external-to-service src: client dst: pod2 Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

148 Life of a packet: external-to-service src: pod2 src: LB dst: client Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

149 Life of a packet: external-to-service src: LB dst: client Cluster Net LB iptables pod1 pod2 pod3 Node1 Node2 Node3

150 Life of a packet: external-to-service src: LB dst: client Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

151 Life of a packet: external-to-service src: LB dst: client Cluster Net LB pod1 pod2 pod3 Node1 Node2 Node3

152 Network Policy

153 Network Policy A common pattern for applications is to organize into micro-services or tiers Example: The classic three-tier app Users want to lock down the network. app: store role: fe app: store role: be Allow some tiers to communicate with others, but not a free-for-all. app: store role: db

154 Network Policy Networks can disallow communication between tiers that are not allowed An FE should never reach around to the DB An FE should never talk to another FE Labels are dynamic so these rules must be so also app: store role: fe app: store role: be X X app: store role: db

155 Namespaces Private scope for creating and managing objects (Pods, Services, NetworkPolicies...) Namespaced objects are always namespaced If you don t specify a namespace in YAML, use kubectl command-line or the current context kubectl -n my-namespace create -f file.yaml

156 Network Policy When first created, the namespace allows all pods to reach each other. Remember we said all pods can reach each other. app: store role: fe app: store role: be app: store role: db

157 Network Policy Describe the allowed links Example: Pods labelled role: be can receive traffic from role: fe Pods labelled role: db can receive traffic from role: be app: store role: fe app: store role: be app: store role: db

158 Network Policy Install policies Per-namespace API Specify: Pods subject to this policy Pods allowed to connect to the subjects Port(s) allowed apiversion: extensions/v1beta1 kind: NetworkPolicy metadata: name: store-net-policy spec: podselector: matchlabels: role: db ingress: - from: - podselector: matchlabels: role: be ports: - protocol: tcp port: 6379

159 Network Policy Switch the network isolation mode to DefaultDeny Absent policies, no network traffic can flow app: store role: fe X app: store role: be app: store role: db X X X X

160 Network Policy kind: Namespace apiversion: v1 metadata: name: store-namespace annotations: net.beta.kubernetes.io/network-policy: { "ingress": { "isolation": "DefaultDeny" } }

161 Network Policy Connections allowed by NetworkPolicies are OK Ordering is very important for these steps app: store role: fe X app: store role: be X app: store role: db

162 Network Policy There may be multiple NetworkPolicies in a Namespace Purely additive There is no way to specify deny in NetworkPolicy Implemented at L3/L4, no L7 support app: store role: fe app: store role: be app: store role: db

163 Open implementation Beta in v1.6 Expected GA in v1.7 Kubernetes allows the user to select the best method to implement Network policy. Today there is a wide range of choices: Calico Contiv Openshift Romana Trireme WeaveNet And more...

164 Watch this space Kubernetes Networking is a moving target The efforts of Open Source developers continue to improve and simplify the system Hopefully the next KubeCon we will have the opportunity to present more.

165 Kubernetes is Open open community open design open source open to ideas Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io Google Cloud Platform