Microservices < Resilient and Fault Tolerant >
In a Monolith application, a single error has the potential of bringing down the entire application. This can be avoided in a Microservice architecture since it contains smaller independently deployable units, which wont effect the entire system. But does that mean a Microservice architecture is resilient to failures? No, not at all. Converting your monolith into Microservices does not resolve the issues automatically. In fact working with a distributed system has its own challenges
While architecting distributed cloud applications, you should assume that failures will happen and design your applications for resiliency. A Microservice ecosystem is going to fail at some point or the other and hence you need to learn embracing failures. Don’t design systems with the assumption that its going to be sunny throughout the year. Be realistic and account for the chances of having rain, snow, thunderstorms and other adverse conditions. In short, design your microservices with failure in mind. Things don’t go as per plan always and you need to be prepared for the worst case scenario.
If Service A calls Service B which in turn calls Service C, what happens when Service B is down? What is your fallback plan in such a scenario?
- Can you return a pre-decided error message to the user?
- Can you call another service to fetch the information?
- Can you return values from cache instead?
- Can you return a default value?
What can go wrong in a Microservice architecture?
There are a number of moving components in a Microservice architecture, hence it has more points of failures. Failures can be caused by a variety of reasons – errors and exceptions in code, release of new code, bad deployments, hardware failures, datacenter failure, poor architecture, lack of unit tests, communication over the unreliable network, dependent services etc.
Why do you need to make service resilient?
A problem with Distributed applications is that they communicate over network – which is unreliable. Hence you need to design your microservices in such a way that they are fault tolerant and handle failures gracefully. In your microservice architecture, there might be a dozen of services talking with each other. You need to ensure that one failed service does not bring down the entire architecture.
Avoid cascading failures
When you have service dependencies built inside your architecture, you need to ensure that one failed service does not cause ripple effect among the entire chain. By avoiding cascading failures, you will be able to save network resources, make optimal use of threads and also allow the failed service to recover!
Do not hammer a service with additional requests which is already down. Please give the service time to recover.
Handle failures gracefully – Allow fast degradation
You should design your microservices such that it is fault tolerant – if there are errors/exceptions, the service should handle it gracefully by providing an error message or a default value. Your microservice needs to be fault tolerant and handle failures gracefully.
Design for Failures
By following some commonly used design patterns you can make your service self-healing. Let us discuss about these design patterns in detail now.
What are the Design Patterns to ensure Service Resiliency?
Circuit Breaker Pattern
If there are failures in your Microservice ecosystem, then you need to fail fast by opening the circuit. This ensures that no additional calls are made to the failing service, once the circuit breaker is open. So we return an exception immediately. This pattern also monitor the system for failures and once things are back to normal, the circuit is closed to allow normal functionality.
This is a very common pattern to avoid cascading failure in your microservice ecosystem.
You can use some popular 3rd party libraries to implement circuit breaking in your application — Pollyand Hystrix
Hystrix
Retry Design Pattern
This pattern states that you can retry a connection automatically which has failed earlier due to an exception. This is very handy in case of temporary issues with one of your services.
Lot of times a simple retry might fix the issue. The load balancer might point you to a different healthy server on the retry, and your call might be a success.
Timeout Design Pattern
This pattern states that you should not wait for a service response for an indefinite amount of time — throw an exception instead of waiting for too long.
This will ensure that you are not stuck in the limbo state – continuing to consume application resources. Once the timeout period is met, the thread is freed up.
Bulkheads
Bulkheads are used to avoid faults in one part of a system taking the entire system down by limiting the number of concurrent calls to a component. It is mainly used to segregate resources.
Say there is an application which has two components. If the requests to components 1 start hanging, it will result in all threads hanging. To avoid this, bulkheads will separate the number of threads and only threads allocated to component 1 will hang; others will be there to process component 2's requests.
Building Fault-Tolerant and Resilient MicroServices using SpringBoot And Netflix’s Hystrix .
First, we need to add the Spring Cloud Starter Hystrix dependency in our build configuration file.
Maven users can add the following dependency in the pom.xml file ?
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix</artifactId>
</dependency>
Now, add the @EnableHystrix and @EnableCircuitBreaker
annotation into your main Spring Boot application class file. The @EnableHystrix annotation is used to enable the Hystrix functionalities into your Spring Boot application.
For the Circuit Breaker to work, Hystix will scan @Component or @Service annotated classes for @HystixCommand annotated methods, implement a proxy for it and monitor its calls.
We’re going to create a @Service class first, which will be injected to a @Controller. Since we’re building a web-application using Thymeleaf, we also need an HTML template to serve as a view.
This will be our injectable @Service implementing a @HystrixCommand with an associated fallback method. This fallback has to use the same signature as the ‘original’:
@Service
public class GreetingService {
@HystrixCommand(fallbackMethod = "defaultGreeting")
public String getGreeting(String username) {
return new RestTemplate()
.getForObject("https://localhost:9090/greeting/{username}",
String.class, username);
}
private String defaultGreeting(String username) {
return "Hello User!";
}
}
We’ll set up the controller using our GreetingService:
@Controller
public class GreetingController {
@Autowired
private GreetingService greetingService;
@GetMapping("/get-greeting/{username}")
public String getGreeting(Model model, @PathVariable("username") String username) {
model.addAttribute("greeting", greetingService.getGreeting(username));
return "greeting-view";
}
}
<!DOCTYPE html>
<html xmlns:th="https://www.thymeleaf.org">
<head>
<title>Greetings from Hystrix</title>
</head>
<body>
<h2 th:text="${greeting}"/>
</body>
</html>
To ensure that the application is listening on a defined port, we put the following in an application.properties file:
server.port=8080
To see a Hystix circuit breaker in action, we’re starting our ‘REST Consumer’ and pointing our browser to https://localhost:8080/get-greeting/Cid. Under normal circumstances, the following will be shown:
Hello Cid!
To simulate a failure of our ‘REST Producer’, we’ll simply stop it, and after we finished refreshing the browser we should see a generic message, returned from the fallback method in our @Service:
Hello User!
The Hystrix Dashboard
A nice optional feature of Hystrix is the ability to monitor its status on a dashboard.
To enable it, we’ll put spring-cloud-starter-hystrix-dashboard and spring-boot-starter-actuator in the pom.xml of our ‘REST Consumer’:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
<version>1.1.5.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>1.4.0.RELEASE</version>
</dependency>
The former needs to be enabled via annotating a @Configuration with @EnableHystrixDashboard and the latter automatically enables the required metrics within our web-application.
After we’ve done restarting the application, we’ll point a browser at https://localhost:8080/hystrix, input the metrics URL of a ‘hystrix.stream’ and begin monitoring.
Finally, we should see something like this:
another Example With Microservice Will be in the last article
Github source code :