Build loosely coupled microservices in .NET

Loosely coupled microservices mean that when one microservice fails, it will not impact another running microservice. Besides, you get to scale each microservice accordingly, as per your business requirements. Building loosely coupled microservices is quite easy in .NET as there are service buses, message brokers, etc. Let's see an example.

Let's assume we have a very popular application like Facebook and thousands of users are posting to their news feeds. If we have a single service dealing with the traffic and requests surge, the service might crash. We don't want that..!! Instead, we want to store the requests in a queue and process them sequentially. Or we can also adapt to the outbox pattern.

Our web service will write the messages in a database and the worker will read from a collection and mark the instance as read. Let's code the first system.

<PackageReference Include="MiniApp.Api" Version="1.0.7-alpha01" />        

Add this nuget to the WebApi project. Then start configuring the service. We have to configure the web service while bootstrapping.

Here I have created a simple minimal API. When you run the application and /GET the resource it prints "Running...". Let's plug in the message broker now.

With rabbitMQ

{
  "RabbitMqServer": "amqp://guest:guest@localhost:5672",
}        

Create appsettings.json file and add this configuration to it. I have my rabbitMq server running locally at this port. You can do the same by using docker and running the following command.

docker run -it -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.12-management        

Now that we have an active and running message broker and the connection string is in the appsettings.json file, nothing else is left from our side. The core package will configure everything by itself. Now, we just use the mediator service and use the SendAsync method to publish the message to a specified queue. But hold on, we don't have any consumers yet! Let's create one.

Add this package reference to your worker service.

<PackageReference Include="MiniApp.Host" Version="1.0.4" />        

In your worker console application add the above code snippet. Add the same appsettings.json here as well. You may notice we are listening on that specific queue here with a prefetch count of 10. So, the worker will parallel process 10 messages. Notice, that apart from the above code snippet we don't need any further configurations for consuming via rabbitMq. This is possible because everything that happens behind the scenes is taken care of by the core package. Now, the message that we published will appear on this handler.

Where we can do the necessary processing and further ops. That's it. We have decoupled our requestAPI and requestProcessor. Where we can scale each of them individually. Let's say we are getting a lot of posts, we can just scale out the API service. The worker can work out the messages gradually.

With outbox pattern

This is a straightforward way of decoupling microservices. Here we don't need any message brokers or service buses. We will solely depend on the database to decouple the microservices.

Let's create another minimal API to handle messages for the outbox pattern.

If you take a closer look you may notice that we are not sending the message to any queue here, rather we are mediating the message to a requestHandler. In the request handler class, we will save the message in a collection/table for processing later hence the name Outbox. Let's take a look at the code and how can we do that.

This is the outbox mediator handler where the message will be saved to the outbox for later processing.

Now, we will create a hosted service and execute the process with a 10-second interval and poll from the database in a batch of posts. After getting the batch of posts we will process them in a similar fashion that we already discussed in step 1. Let's take a look at the codebase.

So, we have achieved our desired loosely coupled microservice where we can scale out each of the services similarly. One benefit of the outbox pattern is that for relational databases we can ensure atomicity. Where each outbox might hold an atomic transaction. This ensures "at least once" processing of the message. This will lead the system to be less prone to faults and more data integrity.

While working on this example I organized my files maintaining clean architecture practice. Finally, it looks like this.

This is the GitHub repo: https://github.com/ittahad/rabbitMqAndOutbox