Need of messaging queue in Microservices Architecture

In my last article, i wrote about “microservices vs monolithic architecture” which emphasis on advantage of using microservices architecture, and also focused on disadvantage of using monolithic architecture. So now you know the advantages of microservices architecture, and how it works, how it can help you in scaling up your application, deployment, faster delivery of new features, save you from single point failure.

You are excited to write your new application using microservices architecture. There are certain standards or practices we need to follow to get most out of microservices architecture. And messaging queue is one of them(ex. rabbit mq, kafka etc.) I would keep writing about remaining practices in next series of articles. But why would i use messaging queue? what happens if don’t make use of it? We will see how messaging queue makes our life easy for certain scenarios in microservices architecture.

Now lets say you have ecommerce application where users come and buy various items. And they get certain loyalty points, or coupon based on their activities like ex. placing an order, giving review on purchased item etc. Every 6 months, and you want to checks past activity stream of users and update their membership level, loyalty points or coupon. Now you need to proposed a solution which can send these details to users. There are two services here, first which calculate these attributes, and second which basically send notification to the end users via mail, sms, push notification. So you get huge number of requests to your first services, which might be processing it with many number of threads. And after that you would want to touch messaging service which takes care of sending notification to end users.

There are two ways you can handle it. First rest synchronous model, where you make external call to messaging service and wait for the response. Second, you go for messaging based asynchronous model.

In first approach, there are considerable problems like you might not able to respond in very short time as there could be huge amount of request, and for each request you are making external call synchronously, it blocks thread too until it get response from external service. There might be cases where external service is down. You are not sure on external dependency of messaging service. It might cause delay in response. At the same time if we use second approach, which basically push these changes/event to queue/topic and respond back to users. And it also allow multiple receivers to process these messages asynchronously, which can help you out in terms of good performance. If you run it in multiple nodes, could give you result in terms of high availability. So once you push message to queue, your job is done. Messaging queue will take care of rest. And your not blocking anything here. So consumers/receivers can read messages from the queue and, process it asynchronously. There could be more examples of using messaging based model.

We will see few more use cases of using queue based asynchronous model, where basically you are pushing some events to other microservices to maintain consistency. Let say you are working with catalog team, which takes care of products uploaded by merchant side, and mapping it to the clean schema of products in your application. When you done with mapping, you need to send out this mapping event to other services ex. search service which gives you mapped product based on this. Once search receive this, it can update its products so that when next time user search it should give correct result specific to mapped product. Imagine you got some changes in attribute of your product in product service. You might want to push these changes to other microservices which basically make use of some logic which is based on products. And if you look at these two examples, you would not want to block your current thread to process this. Lets consider one more use cases where you would want to get response from second services too. As messaging queue provides one way communication, so you would need to have one more queue in opposite direction.

So thats how messaging based asynchronous model can make your life easy in certain scenarios ex. when external service is down, high latency, availability, sending change events to multiple microservices which can seriously affect your application performance. It also provides loose coupling between microservices. I would be writing about about kafka, microservices, various design patterns available in microservices architecture, event driven architecture in next series of articles.Thank you so much for reading this.