Circuit-Breaker, a nice pattern for microservices

Patterns provide general solution, well documented and often proven to be helpful for making a piece of code robust and maintainable. Recently, I discovered a new one that I was not aware of, the circuit-breaker. A circuit breaker is a kind of protection for operations that are likely to fail. You avoid waiting on timeouts for the client, and a broken circuit avoids putting load on a struggling server.

Theory

I have implemented a small and simple version on my github but I will try to explain in details how it works in this article. I like code but adding some words on top of it might help you to get the idea behind this patterns. If I should represent how a circuit breaker work, I will show you this image.

Let's image, you have two APIs communicating with each other, API 1 and API 2. A client can only communicate with the first one. So for communicating to the API 2, the client need to ask the API 1 to make the request for him.

It will work well in 99% of the case but what about the 1% remaining. Let's say the server where the API 2 is located got burn in your cloud. It can happen, let's not forget what happen with OVH. So in this case, if a client make a request to API 2 through API 1, it will certainly ended with a timeout request. Depending of the server and API configuration, the next clients might also wait multiple seconds before they got the same result too. This is where a circuit-breaker shines.

If a circuit-breaker has been implemented, after few failed requests to API 2, the circuit breaker will change his status to CLOSED. In this state, no request will be made to the API 2. A result will be returned instantly by the API 1. This way, the clients dont have to wait the timeout response.

From time to time, the circuit-breaker will change status to HALF-CLOSED. In this state, few attempt calls will be sent to the API 2. In case of the success, the circuit-breaker change his state to OPEN and will authorize again all request to API 2. In case of fail, the circuit-breaker will go back to his CLOSED state.

A bit of coding

Let's look at how you could possibly implement this.

let counter = 0;
let date_last_fail = null;

const callApi2 = async (isOpen = true) => {
  try {
    const rsl = await axios.get(SERVER_API2_URL);
    const { data } = rsl;
    const { message } = data;
    counter = isOpen ? Math.max(counter - 1, 0) : 0;
    return message;
  } catch {
    counter++;
    date_last_fail = new Date();
    return isOpen
      ? `CIRCUIT OPEN (fail: ${counter})`
      : `CIRCUIT HALF-OPEN (fail: ${counter})`;
  }
};        

This part of the code is responsible for managing the call to API 2. A try using axios is made. If the call succeed, we simply decrement the number of fail or in case the state was OPEN, we set the number of fail to 0. If the call fails, we increment the number of fail and keep the date of this attempt.

app.get('/', async (req, res) => {
  if (counter < CLOSED_THRESHOLD) {
    const message = await callApi2();
    return res.send({ message });
  } else {
    const now = new Date();
    const diff = now.getTime() - date_last_fail.getTime();
    if (diff > HALF_OPEN_THRESHOLD_IN_SECOND * 1000) {
      const message = await callApi2(false);
      return res.send({ message });
    }


    return res.send({ message: 'CIRCUIT CLOSED' });
  }
});        

This is where the magic happen. This is the endpoint in API 1. In the first condition of the if, I check that the number of fail is under a certain threshold to let the circuit OPEN. If the number of fail is too high, the else condition takes place. In this case, I calculate the difference between the last attempt and now, if it's under a certain threshold, the status is HALF-OPEN. If the threshold has not been reached, the circuit is CLOSED.

And Voila!

Last words

It always fun to work on some codes and out of nowhere ending up discovering a new pattern we were not familiar with. This is why I love coding, you are always discovering new things. My bucket list of technology and pattern to learn dont decrease. I never heard about circuit breaker but now that I know about it, it will certainly be something I will implement in the future. And you, do you few patterns under the hood ?