Ray Tsang Developer Advocate Google Cloud Platform

Transcription

1 Ray Tsang Developer Advocate Google Cloud +RayTsang

2 Ray Tsang Developer Architect Traveler Photographer flickr.com/saturnism

3 Writing a Kubernetes Autoscaler Kubernetes API - In Depth

4 Keep in Mind I ve never written an autoscaler before!

5 What we are scaling today? Highly scalable distributed in-memory cache

6 Infinispan Distributed in-memory cache Key-value NoSQL store Wonderful Elastic Property

7 Node 2 Cache Node 1 B Cache Node 3 A Cache C

8 Node 2 Cache Node 1 B C Cache Node 3 A Cache B A C N-Copies for Redundancy

9 Node 2 Cache Node 1 B Cache C Node 3 A Cache B A C

10 Node 1 Cache Node 3 A Cache B C A B C

11 Node 1 Node 2 Node 3 Node 4 Cache Cache Cache Cache A C B C B A More Capacity!

12 Node 1 Node 2 Node 3 Node 4 Cache Cache Cache Cache A B C B A Rebalanced! C

13 Wouldn t it be nice to Autoscale this?

14 Keep in Mind I ve never written an autoscaler before!

15 Autoscaling 101 What I learned

16 Horizontal Scaling Vertical Scaling

17 Easy Stuff Scale Out System Metrics Stateless Workload

18 Difficult Stuff Custom Metrics Stateful Workload Scale In

19 Today - Difficult Stuff! Autoscale Infinispan, with target # of entries per node, out and in!

20 Autoscaling Pattern Metrics Server Node Node Your App Your App JMX Metrics JMX Metrics Metrics Collector Metrics Collector I AP Aggregated Metrics Metrics Store Autoscaler Scaling Config Ins tan tiat ea new Desired # of Nodes ins tan ce Actuator Read Image / Template Your App Metrics Collector

21 Autoscaler Configuration Metric CPU, Memory, QPS, etc. Autoscaler Target utilization 80% or 100 QPS Scaling Config Image / Template To start a new instance Min # of instances No less than this many Max # of instances No more than this many

22 Push Metrics Metric Metric Name CPU, Memory, QPS, etc. Utilization / Value 80% or 100 QPS Instance Name node-1431 Cluster Name Infinispan Cluster Timestamp 12:00:34.123

23 Keep in Mind I ve never written an autoscaler before!

24 Kubernetes API My Actuator

25 Autoscaling Pattern Metrics Server Node Node Your App Your App JMX Metrics JMX Metrics Metrics Collector Metrics Collector I AP Aggregated Metrics Metrics Store Autoscaler Scaling Config Ins tan tiat ea new Desired # of Nodes ins tan ce Actuator Read Image / Template Your App Metrics Collector

26 Enter Kubernetes Greek for Helmsman ; also the root of the word Governor Container orchestrator Runs containers Supports multiple cloud and baremetal environments Inspired and informed by Google s experiences and internal systems Open source, written in Go Manage applications, not machines

27 A quick recap Node / Kubelet Pod Master Container Container Container Node / Kubelet Pod Pod Container Container Container Container Container Container Replication Controller Replication Controller Pod Node / Kubelet Pod Container Container Node / Kubelet Pod Pod Container Container Container Container

28 Kubernetes Exposes REST API

29 Let s See It!

30 Talking to Kubernetes API

31 Certificate Authority Ugh! Let s see the code

32 Using the API for Node Discovery! A customized JGroups plugin that discovers Infinispan nodes with the API

33 Using headless service A stable DNS name that maps to pod IP addresses Great for the client

34 Collecting Metrics JMX!

35 Autoscaling Pattern Metrics Server Node Node Your App Your App JMX Metrics JMX Metrics Metrics Collector Metrics Collector I AP Aggregated Metrics Metrics Store Autoscaler Scaling Config Ins tan tiat ea new Desired # of Nodes ins tan ce Actuator Read Image / Template Your App Metrics Collector

36 JMXTrans to the rescue!

37 JMX Output Writer { "servers" : [ { "url": "service:jmx:http-remoting-jmx://${infinispan.host}:${infinispan.port}", "queries" : [ { "obj": "jboss.infinispan:type=cache,component=statistics,name=\"namedcache(dist_sync)\",*", "attr": [""], "outputwriters": [ { "@class": "com.googlecode.jmxtrans.model.output.a8rwriter", "metricname": "custom.cloudmonitoring.googleapis.com/infinispan/namedcache/" } ] } ] } ] }

38 Configuration & Storing Metrics

39 Autoscaling Pattern Metrics Server Node Node Your App Your App JMX Metrics JMX Metrics Metrics Collector Metrics Collector I AP Aggregated Metrics Metrics Store Autoscaler Scaling Config Ins tan tiat ea new Desired # of Nodes ins tan ce Actuator Read Image / Template Your App Metrics Collector

40 Example Metrics Instance Name Metric Name Metric Value Timestamp node :01:00 node :01:00 node :01:10 node :01:10 node :01:20 node :01:20 node :01:30 node :01:30

41 Example Metrics Instance Name Metric Name Metric Value Timestamp node :01:00 node :01:00 node :01:10 node :01:10 node :01:20 node :01:20 node :01:30 node :01:30

42 Example Metrics Instance Name Metric Name Metric Value Timestamp node :01:00 node :01:00 node :01:10 node-2 Instance Name node-1 node-1 node-2 node-2 node-1 Sum node Average Metric Value :01:10 23:01:20 23:01:20 23:01:30 23:01:30

43 Autoscaler Algorithm target # instances = ceil( sum(utilization) target utilization )

44 Autoscaler Algorithm target utilization = 100 target # instances = ceil( sum(300) 100 )

45 Autoscaler Algorithm min < target # instances < max

46 It s expensive to scale out Delay the scaling in Watch out for noises

47 Wait if an instance goes way We need to remove the metrics!

48 Example Metrics Instance Name Metric Name Metric Value Timestamp node :01:00 node :01:00 node :01:10 node :01:10 node :01:20 node :01:20 node :01:30 node :01:30

49 Example Metrics Instance Name Metric Name Metric Value Timestamp node :01:00 node :01:00 node :01:10 node-2 Instance Name node-1 node-1 node-2 node-2 node-1 Sum node Average Metric Value :01:10 23:01:20 23:01:20 23:01:30 23:01:30

50 Let s write an autoscaler! The real stuff

51 Let s see some code

52 I used Spring Boot @EnableScheduling class Application { static void main(string[] args) { SpringApplication.run(Application, args) } }

53 Metrics Store - Used a Tree Cache For the real thing, use something like InfluxDB

54 Microservices @Slf4j class AutoscalerService { }

55 class AutoscalerDefintion String String Double @Slf4j class AutoscalerService {... } Integer minreplicas = Integer maxreplicas }

56 class AutoscalerService { log.info "Updated autoscaler for $fqn, {}", definition }

57 Wake up = "\${autoscaler.wakeupinterval}") void autoscale() { autoscalercache.root.children.each { scale(it.data as AutoscalerDefintion) } void scale(autoscalerdefintion definition) { }

58 SSLSocketFactory with CA SSLSocketFactory String cacertfile) throws Exception {... }

59 RestTemplate with Headers and SSLSocketFactory resttemplate.setrequestfactory(new SimpleClientHttpRequestFactory() { protected void prepareconnection(httpurlconnection conn, String httpmethod) throws IOException { if (headers["authorization"]) { conn.setrequestproperty("authorization", headers.getfirst("authorization")) } if (sslsocketfactory && conn instanceof HttpsURLConnection) { ((HttpsURLConnection) conn).setsslsocketfactory(sslsocketfactory) } super.prepareconnection(conn, httpmethod); };

60 Autoscaler Algorithm

61 Live Demo! Demo time

62 Try Kubernetes Today!

63 In the Roadmap - Native Autoscaling in Kubernetes Nodes Pods Horizontal # of nodes # of pods Vertical resources for a node resources for a pod

64 Horizontal Pod Autoscaling Autoscaler is First Class Citizen Autoscaler Controller and Resource Using Pod CPU / Memory Utilization

65 Vertical Pod Autoscaling vertical auto-sizer sets the compute resource limits and request for pods which do not have them set, and periodically adjust them based on demand signals

66 Horizontal Node Autoscaling Pretty much done on Google Compute Engine! Using Node CPU / Memory Utilization

67 Try out Google Container Engine

68 Thanks! Images by Connie Zhou