Leader’s Daily Digest: Microservices Unpacked

Let’s take a trip back to the 2000s. Back then, coding felt simpler. Small development teams could quickly build a monolithic application—a single package containing everything from the database to the user interface. Even complex applications came bundled neatly, whether shipped on a CD or downloaded online. So what’s changed in the last 15 to 20 years that has us rethinking how we build software?

As technology advanced, the need to handle more data and provide richer user experiences skyrocketed. The rise of SaaS (Software as a Service) opened up a new world of web-based software. Instead of users running separate copies of software with their own databases, centralized systems began handling massive amounts of data for thousands of users at once. This shift gave us the ability to provide real-time logging, detailed reporting, and better user experiences—but it also forced us to rethink how we design and build our systems.

As software became more complex and user demands grew, the monolithic architecture struggled to keep up. Maintaining one large, tightly-coupled application became a headache, especially as development teams grew larger and more specialized. That’s when microservices started to gain traction.

However, let's not rush to label microservices as the ultimate solution. They’re not always the best choice, and for many projects, microservices can introduce more complexity than necessary.

My Start with Microservices

Looking back, my journey into microservices began with my work at Prime Clinical Systems from 2004 to 2013. As the lead developer on the Patient Chart Manager (PCM), an EMR product for healthcare, I dealt with a variety of system integrations. We used standards like HL7 for lab systems and DICOM for medical imaging. At the time, we weren’t using microservices, but we were connecting different systems to work together. This gave me my first exposure to thinking about software as a collection of independent components that could still exchange data.

Then, in 2016, everything clicked when I discovered AWS Lambda. It was my "aha" moment. Lambda functions, with their focus on small, event-driven, stateless processes, showed me how decoupled services could work together seamlessly. While Lambda is serverless, it introduced me to the principles that define microservices: small, independent services that communicate through simple interfaces.

Since then, my understanding of microservices has evolved, and it continues to grow. It’s a fascinating architecture with endless possibilities and challenges that keep me learning every day.

What Are Microservices?

At its core, microservices is an approach to building software where the application is split into small, independent parts, called services. Each service focuses on doing one specific job, like processing payments or handling user logins, and it can work on its own. These services communicate with each other through APIs or by sending messages through queues. The use of queues allows services to interact asynchronously, meaning they don’t need to wait for each other to respond immediately. This makes the system easier to scale, update, and maintain, as changes to one service won’t affect the others.

Shifting the Mindset

When I first started with microservices, it was tough to shift away from the monolithic mindset. In a monolithic system, everything is connected—one big database stores all your data. For example, placing an order would involve locking inventory, charging the customer, and generating receipts in one smooth transaction. It’s all synchronous, everything happens in order, and in one place.

But in microservices, things are different. Each part of the process—inventory, billing, customer info—is handled by separate services that work independently. The billing service, for example, only handles payment processing and doesn't care about inventory. They communicate asynchronously, meaning they don’t need to wait for each other to complete their tasks.

This shift in thinking can take time to get used to, especially if you’re coming from a background of tightly-coupled, synchronous systems.

The Key Concepts in Microservices

There’s a lot to microservices, but I’ll touch on a few key concepts that I believe are essential when using this architecture. These insights are based on my personal experience, and I’d love to hear your thoughts if you have more to add.

Independent Services

The heart of microservices lies in independence. Each service should be self-contained and perform one specific function. At first, it can be tricky to logically separate services, but a good rule of thumb is to ensure that each service focuses on one task and doesn’t try to do too much.

Here are some examples based on different types of applications:

• E-commerce: User Service, Product Catalog Service, Order Service, Payment Service, Inventory Service.
• Social Media: User Profile Service, Post Service, Comment Service, Notification Service, Friend/Follow Service.
• Streaming Platform: User Subscription Service, Content Delivery Service, Recommendation Service, Media Library Service.
• Healthcare: Patient Management Service, Appointment Service, Billing Service, Medical Records Service.

Decentralized Data Management—Keeping Data Separate

One of the most common mistakes I’ve seen is teams using the same database for multiple services, thinking they’re working with microservices. In reality, this tightly couples the services and defeats the purpose of microservices.

When services share a database, it can lead to issues like difficulty tracing bugs or bottlenecks during database schema changes. Decoupling data, on the other hand, allows each service to maintain its own database, making the architecture more flexible and scalable.

Some developers worry that separating data makes reporting more difficult. But in a microservices architecture, you can get creative. For example, a separate reporting service can hold its own copy of data, and it doesn’t need to be real-time. Many systems take minutes or even hours to reflect changes in reports, which is perfectly fine for most use cases.

Asynchronous Communication via Queues (Including CQRS and Event-Driven Architecture)

One of the most exciting parts of microservices is how they communicate. Instead of directly calling another service, messages can be sent to queues, which introduce a layer of decoupling between services.

For instance, if Service A sends a message to Service B, but Service B is down, the message can wait in the queue until Service B is available again. This approach keeps the services independent and prevents system failures when a service is temporarily unavailable.

Event-Driven Architecture also ties in here. A service can emit events when something significant happens (like an order being placed), and other services can subscribe to these events and react accordingly. This allows the system to be even more flexible and responsive without directly linking services.

CQRS (Command Query Responsibility Segregation)

CQRS is another key concept in microservices. It separates the write operations (commands) from the read operations (queries). For example, when an order is placed, the Order Service processes the data and stores it. Instead of other services reading the data directly from the Order Service, they can pick it up from a queue when the system is ready, improving performance and scalability.

Service Discovery

Microservices often live in dynamic environments, and services can move or change over time. Service Discovery ensures that services can find and communicate with each other, even when things shift. This is critical for managing microservices at scale, ensuring services route correctly to the right version or instance.

Eventual Consistency

Because each service maintains its own database, achieving immediate consistency across the system can be difficult. Many microservices systems adopt eventual consistency, meaning data will eventually sync up across services, even if it’s not updated everywhere at the same time.

Security

Security in microservices can be challenging, especially since you have many independent services interacting with each other. Each service must be secured individually, but the communication between them also needs protection.

One commonly used practice is Mutual TLS (mTLS), which ensures that both the client and the server authenticate each other before exchanging data. This helps establish secure communication between services.

For APIs, OAuth 2.0 and JWT (JSON Web Tokens) are popular authentication mechanisms. OAuth 2.0 allows services to verify the identity of users or other services securely, while JWT tokens are often used to carry the necessary identity and authorization information between services.

Role-Based Access Control (RBAC) is another essential layer of security. By controlling which users or services have access to which parts of your system, RBAC helps prevent unauthorized actions, ensuring that only specific services or individuals can perform particular tasks.

Additionally, running your microservices inside a Virtual Private Cloud (VPC) is a best practice. This ensures that service-to-service communication happens internally, reducing the risk of external attacks. All external access to the system can be tightly controlled, adding an extra layer of security.

To further protect data, it's important to encrypt data both in transit and at rest. This means securing data as it moves between services and when it's stored in databases or file systems, safeguarding sensitive information from unauthorized access.

Conclusion

Microservices are not a one-size-fits-all solution, but for the right applications, they provide flexibility and scalability that monolithic systems can’t match. The transition to microservices may involve a mindset shift, but once you adapt, the possibilities are vast and exciting.

If you’ve had experience working with microservices, I’d love to hear your thoughts. Share your stories, insights, and lessons learned—we can all learn from each other!

Cheers!