What is Spring Cloud 1
What is Spring Cloud Service Discovery (Eureka) API Gateway (Zuul) Config Server (Spring Config) Circuit Breaker (Hystrix, Turbine) Client Side Load-balancing (Ribbon) Distributed Tracing (Zipkin) Distributed Messaging (Spring Bus) Data Pipelines (Spring Cloud Data Flow) Many more 2
Spring Cloud Config 1 / 29
Config in a Spring Context Configuration in a Spring context can usually be described as values that wire up Spring beans. Spring has provided several approaches to setting config, including externalizing (via Command Line argumments, Env Variables, etc.) Still, gaps exist: Changes to config require restarts No audit trail Config is de-centralized No support for sensitive information (no encryption capabilities) 3 / 29
Config in a 12 Factor Context 12 Factor application design states that configuration should be kept in OS environment variables. In Cloud Foundry, this is accomplished in the following ways: Using the cf set-env command Using a manifest with env: sections This works well for applications with less demanding configuration needs. 4 / 29
Challenges When Using Env. Variables Often times we want more capabilities because of the following: Managing many env variables can be a challenge Cloud Foundry uses immutable containers. Any configuration change requires restarting the app. If you want zero downtime then blue/green deployments are necessary. This may be too much overhead. 5 / 29
More Demanding Config Use Cases Changing logging levels of a running application in order to debug a production issue Change the number of threads receiving messages from a message broker Report all configuration changes made to a production system to support regulatory audits Toggle features on/off in a running application Protect secrets (such as passwords) embedded in configuration 6 / 29
Externalized, Versioned, Distributed, Configuration Q: How can these new use-cases be handled? A: Externalize configuration to a service. Configuration Mgmt approach should support: Versioning Auditability Encryption Refresh without restart 7 / 29
Spring Cloud Config Review Config Mgmt Server Client Spring Cloud Services 8 / 29
Configuration Flow 9 / 29
Spring Cloud Config Server The Server provides an HTTP, resource-based API for external configuration (name-value pairs, or equivalent YAML content). Bind to the Config Server and initialize Spring Environment with remote property sources Embeddable easily in a Spring Boot application using @EnableConfigServer Encrypt and decrypt property values (symmetric or asymmetric) 10 / 29
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-config-server</artifactid> </dependency> 11 / 29
Embedded Server The server is easily embeddable in a Spring Boot application using the @EnableConfigServer annotation. This app is a config server: @SpringBootApplication @EnableConfigServer public class ConfigServer { public static void main(string[] args) { SpringApplication.run(ConfigServer.class, args); } } 12 / 29
Server Config Application configuration data is stored in a backend Git, Subversion and File System backends are supported Git is the default backend. It's great for auditing changes and managing upgrades Setting the git backend is done via the spring.cloud.config.server.git.uri configuration property Sample application.yml: spring: cloud: config: server: git: uri: https://github.com/spring-cloud-samples/config-repo.git 13 / 29
Server Endpoints Config server exposes config on the following endpoints: /{application}/{profile}/[{label}] /{application}-{profile}.yml /{label}/{application}-{profile}.yml /{application}-{profile}.properties /{label}/{application}-{profile}.properties {application} maps to "spring.application.name" on the client side; {profile} maps to "spring.active.profiles" on the client (comma separated list); and {label} which is a server side feature labelling a "versioned" set of config files. 14 / 29
Configuration Files What files do I put my configuration in? Config files are stored in the git repository. Example Input: spring.application.name=foo spring.active.profiles=dev Union of all sources with overriding behavior. Repo Files: application.yml - shared between all clients application-dev.yml - shared between all clients but profile specific (takes precedence over application.yml) foo.yml - app specific (takes precedence over application.yml files) foo-dev.yml - app and profile specific (takes precedence over foo.yml) 15 / 29
Spring Cloud Config Review Config Mgmt Server Client Spring Cloud Services 16 / 29
Client Application Typical Spring Configuration without Spring Cloud Config: @SpringBootApplication @RestController public class ClientConfigApplication { } public static void main(string[] args) { SpringApplication.run(ClientConfigApplication.class, args); } @Value("${greeting}") //<-- configuration injected from environment private String greeting; @RequestMapping("/greeting") String greeting() { return String.format("%s World", greeting); } 17 / 29
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-config</artifactid> </dependency> 18 / 29
Client Config For Spring clients to use the configuration server, specify the spring.cloud.config.uri configuration property: Sample bootstrap.yml spring: cloud: config: uri: http://my-config-server.io/ spring.cloud.config.uri defaults to http://localhost:8888 19 / 29
Refreshable Application Components What can be refreshed in a Spring Application? Loggers logging.level.* @ConfigurationProperties beans Beans with @RefreshScope annotation 20 / 29
@RefreshScope Annotate bean with @RefreshScope: @SpringBootApplication @RestController @RefreshScope // <-- Add RefreshScope annotation public class ClientApplication { } public static void main(string[] args) { SpringApplication.run(ClientApplication.class, args); } @Value("${greeting}") private String greeting; @RequestMapping("/greeting") String greeting() { return String.format("%s World", greeting); } 21 / 29
Refreshing Configuration Two step process for applications getting new configuration: 1) Update Repository 2) Send a request to the application(s) to pick up new config values Send a POST request to the refresh endpoint in the client app to fetch updated config values: http://127.0.0.1:8080/refresh Requires the Actuator dependency on the classpath (pom.xml). <dependency> <groupid>org.springframework.boot</groupid> <artifactid>spring-boot-starter-actuator</artifactid> </dependency> 22 / 29
Spring Cloud Bus When running many applications, refreshing each one can be cumbersome. Instead, leverage Spring Cloud Bus pub/sub notification with RabbitMQ. Send a POST request to the refresh endpoint to fetch updated config values: http://127.0.0.1:8080/bus/refresh Requires the Cloud Bus AMQP dependency on the classpath (pom.xml). <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-bus-amqp</artifactid> </dependency> 23 / 29
Cloud Bus Diagram 24 / 29
Spring Cloud Config Review Config Mgmt Server Client Spring Cloud Services 25 / 29
Spring Cloud Services Brings Spring Cloud to Cloud Foundry Includes: Config Server, Service Registry & Circuit Breaker Dashboard services 26 / 29
Spring Cloud Services: Config Server 1) Include dependency: pom.xml <dependency> <groupid>io.pivotal.spring.cloud</groupid> <artifactid>spring-cloud-services-starter-config-client</artifactid> </dependency> 2) Create a Config Server service instance 3) Configure the service instance with a Git Repository 4) Bind the service to the app 27 / 29
Spring Cloud Services: Config Server 28 / 29
Cloud Bus in Cloud Foundry 1) Include dependency: pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-bus-amqp</artifactid> </dependency> 2) Create a RabbitMQ service instance 3) Bind the service to the app 29 / 29
Spring Cloud Netflix: Service Discovery 1 / 19
Spring Cloud Netflix: Service Discovery Review Service Discovery Eureka Server Eureka Client Spring Cloud Services 2 / 19
Challenge Service Discovery is one of the key tenets of a microservice based architecture. In distributed systems, application dependencies cease to be a method call away. Trying to hand configure each client or use some form of convention can be very difficult to do and can be very brittle. 3 / 19
Where We Have Been How have we discovered services in the past? Service Locators Dependency Injection Service Registries 4 / 19
Spring Cloud Netflix: Service Discovery Review Service Discovery Eureka Server Eureka Client Spring Cloud Services 5 / 19
Service Discovery with Spring Cloud 6 / 19
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-eureka-server</artifactid> </dependency> 7 / 19
Eureka Server An example Eureka server. @EnableEurekaServer annotation. @SpringBootApplication @EnableEurekaServer public class ServiceRegistryApplication { } public static void main(string[] args) { SpringApplication.run(ServiceRegistryApplication.class, args); } 8 / 19
Eureka Resiliency Eureka does not have a backend data store. All data is kept in memory. Service instances in the registry all have to send heartbeats to keep their registrations up to date Clients also have an in-memory cache of eureka registrations (so they don t have to go to the registry for every single request to a service) 9 / 19
Spring Cloud Netflix: Service Discovery Review Service Discovery Eureka Server Eureka Client Spring Cloud Services 10 / 19
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-eureka</artifactid> </dependency> 11 / 19
Client Application An example client application. @EnableDiscoveryClient annotation. @SpringBootApplication @EnableDiscoveryClient public class GreetingServiceApplication { } public static void main(string[] args) { SpringApplication.run(GreetingServiceApplication.class, args); } 12 / 19
Client Config Configuration required to locate the Eureka. application.yml spring: application: name: fortune-service eureka: client: serviceurl: defaultzone: http://localhost:8761/eureka/ defaultzone - the default serviceurl to reach Eureka spring.application.name is the name the application will use to register its service 13 / 19
Registering with Eureka When a client registers with Eureka, it provides meta-data about itself such as host and port, health indicator URL, home page etc. Eureka receives heartbeat messages from each instance belonging to a service. If the heartbeat fails over a configurable timetable, the instance is normally removed from the registry. 14 / 19
Discovery Client The DiscoveryClient is used to locate the stores service. @Autowired private DiscoveryClient discoveryclient; public String serviceurl() { InstanceInfo instance = discoveryclient.getnextserverfromeureka("stores", false); return instance.gethomepageurl(); } 15 / 19
Spring Cloud Netflix: Service Discovery Review Service Discovery Eureka Server Eureka Client Spring Cloud Services 16 / 19
Spring Cloud Services Brings Spring Cloud to Cloud Foundry Includes: Config Server, Service Registry & Circuit Breaker Dashboard services 17 / 19
Spring Cloud Services: Service Registry 18 / 19
Spring Cloud Services: Service Registry 1) Add dependency <dependency> <groupid>io.pivotal.spring.cloud</groupid> <artifactid>spring-cloud-services-starter-service-registry</artifactid> </dependency> 2) Create a Service Registry service instance 3) Bind the service to the app 19 / 19
Spring Cloud Netflix: Circuit Breakers with Hystrix 1 / 22
Spring Cloud Netflix: Circuit Breakers with Hystrix Review Fault Tolerance Circuit Breakers Hystrix Dashboard 2 / 22
Fault Tolerance Fault Tolerance Patterns like the Circuit Breaker stop cascading failures. As described by, Michael T. Nygard in Release It! 3 / 22
Fault Tolerance One failure must not cause a cascading failure across the entire system. For example, for an application that depends on 30 services where each service has 99.99% uptime, here is what you can expect: 99.99^30 = 99.7% uptime 0.3% of 1 billion requests = 3,000,000 failures 2+ hours downtime/month even if all dependencies have excellent uptime. Reality is generally worse. Source: https://github.com/netflix/hystrix/wiki 4 / 22
Healthy Request Flow Source: https://github.com/netflix/hystrix/wiki 5 / 22
Dependency Down 6 / 22
Cascading Failure 7 / 22
Spring Cloud Netflix: Circuit Breakers with Hystrix Review Fault Tolerance Circuit Breakers Hystrix Dashboard 8 / 22
Hystrix - Defend Your App Hystrix is designed to do the following: Give protection from and control over latency and failure from dependencies accessed (typically over the network) via third-party client libraries. Stop cascading failures in a complex distributed system. Fail fast and rapidly recover. Fallback and gracefully degrade when possible. Enable near real-time monitoring, alerting, and operational control. 9 / 22
Circuit Breaker Diagram 10 / 22
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-hystrix</artifactid> </dependency> 11 / 22
@EnableCircuitBreaker Annotation @SpringBootApplication @EnableDiscoveryClient @EnableCircuitBreaker public class GreetingHystrixApplication { } public static void main(string[] args) { SpringApplication.run(GreetingHystrixApplication.class, args); } 12 / 22
@HystrixCommand annotation @Service public class FortuneService {... @HystrixCommand(fallbackMethod = "defaultfortune") public String getfortune() { } return resttemplate.getforobject ("http://fortune-service", String.class); } public String defaultfortune(){ logger.debug("default fortune used."); return "This fortune is no good. Try another."; } 13 / 22
Customize Behavior with @HystrixProperty @Service public class FortuneService {... @HystrixCommand (fallbackmethod = "defaultfortune", commandproperties = {@HystrixProperty (name="execution.isolation.thread.timeoutinmilliseconds", value="500")}) public String getfortune() { } } return resttemplate.getforobject ("http://fortune-service", String.class); 14 / 22
Wrapping Calls With Hystrix 15 / 22
Metrics Hystrix publishes real-time metrics for each @HystrixCommand: Informational and Status (iscircuitopen) Cumulative and Rolling Event Counts (countexceptionsthrown & rollingcountexceptionsthrown) Latency Percentiles (latencyexecute_percentile_995) Latency Percentiles: End-to-End Execution ( latencytotal_percentile_5) Property Values (propertyvalue_circuitbreakerrequestvolumethreshold) ** Thread Pool metrics are also available 16 / 22
Event stream To expose the /hystrix.stream as a management endpoint include the actuator dependency. <dependency> <groupid>org.springframework.boot</groupid> <artifactid>spring-boot-starter-actuator</artifactid> </dependency> 17 / 22
Spring Cloud Netflix: Circuit Breakers with Hystrix Review Fault Tolerance Circuit Breakers Hystrix Dashboard 18 / 22
Include Dependency pom.xml <dependency> <groupid>org.springframework.cloud</groupid> <artifactid>spring-cloud-starter-hystrix-dashboard</artifactid> </dependency> 19 / 22
@EnableHystrixDashboard Annotation @SpringBootApplication @EnableHystrixDashboard public class HystrixDashboardApplication { } public static void main(string[] args) { SpringApplication.run(HystrixDashboardApplication.class, args); } 20 / 22
Configure the Dashboard Enter the the /hystrix.stream endpoint. 21 / 22
Hystrix Dashboard When Netflix began to use this dashboard, their operations improved by reducing the time needed to discover and recover from operational events. The duration of most production incidents (already less frequent due to Hystrix) became far shorter, with diminished impact, due to the real-time insights into system behavior provided by the Hystrix Dashboard. Source: https://github.com/netflix/hystrix/wiki/dashboard 22 / 22
Spring Cloud Netflix: Client Side Load Balancing with Ribbon 1 / 15
Spring Cloud Netflix: Client Side Load Balancing Review Load Balancing Ribbon Ribbon with Spring 2 / 15
Some Context for This Discussion This is load balancing with regard to middle-tier applications This is NOT about load balancing for public facing applications 3 / 15
Server Side Load Balancing 4 / 15
Server Side Load Balancing Challenges One size does not fit all Multiple setup/configurations Extra hops Not cloud/app aware (zones, circuit-breakers, etc.) Limited algorithms 5 / 15
Client Side Load Balancing 6 / 15
Client Side Load Balancing Benefits Pick the right algorithm for your app No additional setup, just deploy apps 1 less hop Cloud/app aware (zones, circuit-breakers, etc.) Extensible algorithms 7 / 15
Spring Cloud Netflix: Client Side Load Balancing Review Load Balancing Ribbon Ribbon with Spring 8 / 15
Ribbon Ribbon is a client side IPC (Inter Process Communication) library that is battle-tested in cloud. The primary usage model involves REST calls with various serialization scheme support. Features: Multiple and pluggable load balancing rules Integration with service registry (Eureka) Built-in failure resiliency Cloud enabled Clients integrated with load balancers Archaius configuration driven client factory 9 / 15
Ribbon Load Balancing Algorithms Simple Round Robin LB Weighted Response Time LB Zone Aware Round Robin LB Random LB 10 / 15
Zone Aware Load Balancer 1. The LB calculates/examines zones stats in real time. If average active requests have reached a configured threshold the zone will be dropped. 2. Once the the worst zone is dropped, a zone will be chosen among the rest with the probability proportional to its number of instances. 3. A server will be returned from the chosen zone with a given Rule. Source: http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html 11 / 15
Spring Cloud Netflix: Client Side Load Balancing Review Load Balancing Ribbon Ribbon with Spring 12 / 15
Using the API Directly Use the LoadBalancerClient. Example: public class MyClass { @Autowired private LoadBalancerClient loadbalancer; public void dostuff() { ServiceInstance instance = loadbalancer.choose("stores"); URI storesuri = URI.create(String.format("http://%s:%s", instance.gethost(), instance.getport())); } } //... do something with the URI 13 / 15
Spring RestTemplate as a Load Balancer Client @LoadBalanced is a qualifier to make certain the load balanced resttemplate is injected public class MyClass { @Autowired @LoadBalanced private RestTemplate resttemplate; } public String dootherstuff() { String results = resttemplate.getforobject("http://stores/stores", String.class); return results; } 14 / 15
Customizing the Ribbon Client Spring Cloud Netflix provides the following beans by default for ribbon (BeanType beanname: ClassName): IRule ribbonrule: ZoneAvoidanceRule IPing ribbonping: NoOpPing ILoadBalancer ribbonloadbalancer: ZoneAwareLoadBalancer... Creating a bean of one of those types and placing it in a @RibbonClient configuration allows you to override each one of the beans described. Example: @Configuration @RibbonClient(name = "foo", configuration = FooConfiguration.class) public class FooConfiguration { @Bean public IPing ribbonping(iclientconfig config) { return new PingUrl(); } } This replaces the NoOpPing with PingUrl. 15 / 15