De-mystifying MICROSERVICES architecture for Technology and Business Leaders

For many years now we have been building systems and getting better at it. Several technologies, architectural patterns, and best practices have emerged over those years

For CIO’s and CTO’s today the architecture choices are many, but Microservices based architecture has been the most prevalent and favorite due to various reasons, ranging from modularity, resilience, scalability (ability to scale up to increased workloads in terms of number of users, features and underlying data), faster development, improved data security, cross-functional to develop simultaneously and most importantly be language and technology agnostics

In this article, I will try to simplify the various concepts (and the tools) with examples on this architecture pattern and discipline called “Microservices” for product and technology leaders and enthusiasts

Consider an e-Commerce platform, the various microservices can be “Add to Cart”, “Wishlist”, “Order Taking”, “Inventory Management” and “User Management”

As you see in the diagram (photo credit goes to @Rocky Bhatia) there are various elements to Microservices (herein referred from now on as MS)

Container - A container is a standard unit of software that packages up code and all its dependencies (like runtime parameters, libraries, environment variables, files and configuration settings) so the application runs quickly and reliably across computing environments

MS is one such use case of the containerization framework/technology that enables rapid, lightweight and distributed application development and deployment

One of the world’s largest video and content streaming platform (Netflix) with its complex ecosystem (security, content encoding, studio technologies, machine learning, big data and all its internal engineering tools) have developed its own container management tool “Titus” that adds up to 200,000 clusters and half a million containers a day!

Container images becomes containers during run-time and is available for both Linux and Windows environments

DOCKER is the de-factor containerization framework for application containers

?????????????????? ?????????????????????????? automates containers' deployment, management, scaling, and networking. Enterprises that need to deploy and manage hundreds or thousands of Linux? containers and hosts can benefit from container orchestration.

Kubernetes is the de facto standard for container orchestration

???????????????????? is a portable, extensible, open-source platform for managing containerized workloads and services, It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available (e.g AKS or Microsoft Azure Kubernetes Services or OpenShift)

The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes combines over 15 years of Google's experience running production workloads at scale with best-of-breed ideas and practices from the community.

Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior gets automatically handled by a system?

That's how Kubernetes comes to the rescue! Kubernetes provides us with a framework to run distributed systems resiliently. It takes care of scaling and failover for application and ensures a stable CI/CD pipeline

So back to our example of the e-Commerce architecture, all the MS written (say for example in Java) can be containerized and orchestrated to run using DOCKERS

Next comes the Cloud Services that are cloud-based platform, infrastructure, application, or storage services. Much like a homeowner would pay for a utility such as electricity or gas, companies typically have to pay only for the amount of cloud services they use, as business demands require.

Besides the pay-per-use model, cloud service providers also give companies a wide range of benefits. Businesses can take advantage of scalability and flexibility by not being limited to physical constraints of on-premises servers, the reliability of multiple data centers with multiple redundancies, customization by configuring servers to your preferences, and responsive load balancing that can easily respond to changing demands.

Though businesses should also evaluate security considerations of storing information in the cloud to ensure industry-recommended access and compliance management configurations with secured IAM (identity access management) and practices are enacted and met.

An important factor to consider during this process is, what cloud technologies will you be able to handle within your enterprise, and which should be delegated to a cloud service provider?

Having infrastructure, platforms, or software managed for you can free your business to serve your clients, be more efficient in overall operations, and improve or expand your development operations (DevOps) strategy.

Organizations needs to select between public, private or hybrid cloud architecture all of which have their own benefits and drawbacks. For example, public clouds (where computing is delivered as a SaaS via internet but the underlying infrastructure is shared across organizations) provides scalability, low maintenance overheads and agility to innovate, but have minimal control and challenges on security and availability. On the other hand, private clouds (where the VM’s are dedicated for the organization) are more customizable, regulation compliant and more efficient to manager, but highly expensive (TCO) and might lack mobility. A good balance can be a hybrid cloud (which is a combination of public and private cloud) which offers the benefits of both, but also shares the same drawback of both public and private cloud. It all depends on the CIO’s agenda to manage and control costs vs flexibility vs security and the complexity and criticality of the applications

Cloud service providers are companies that establish public clouds, manage private clouds, or offer on-demand cloud computing components (also known as cloud computing services) like Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS). Cloud services can reduce business process costs when compared to on-premise IT.?

GCP (Google Cloud Platform), AWS (Amazon Web Services), Microsoft Azure, OCI (Oracle Cloud) are all available cloud providers

Using a cloud service provider is a helpful way to access computing services that you would otherwise have to provide on your own, such as

Infrastructure: The foundation of every computing environment. This infrastructure could include networks, database services, data management, data storage (known in this context as cloud storage), servers (cloud is the basis for serverless computing), and virtualization.

Platforms: The tools needed to create and deploy applications. These platforms could include operating systems like Linux?, middleware, and runtime environments.

Software: Ready-to-use applications. This software could be custom or standard applications provided by independent service providers.

An important factor to consider during this process is, what cloud technologies will you be able to handle within your enterprise, and which should be delegated to a cloud service provider?

Having infrastructure, platforms, or software managed for you can free your business to serve your clients, be more efficient in overall operations, and improve or expand your development operations (DevOps) strategy.

Many public cloud service providers have a set of standard support contracts that include validating active software subscriptions, resolving issues, maintaining security, and deploying patches. Managed cloud service providers' support could be relegated to simple cloud administration or it can serve the needs of an entire IT department.

SaaS (complete software been run in the cloud) requires significant investment in change management and process re-engineering to adopt the vanilla product, whereas PaaS (platform as a service) provides more flexibility for customizations, much like a PaaS extension to a SaaS offering. “Lift and Shift” are more IaaS offerings where the entire on-prem application is migrated to the cloud, the organization manages the application and the cloud provider only manages the underlying VM’s (along with the compute and storage)

???????? ?????????????????? refers to efficiently distributing incoming network traffic across a group of backend servers, also known as a server farm or server pool.

Modern high?traffic websites must serve hundreds of thousands, if not millions, of concurrent requests from users or clients and return the correct text, images, video, or application data, all in a fast and reliable manner. To cost?effectively scale to meet these high volumes, modern computing best practice generally requires adding more servers.

A load balancer acts as the “traffic cop” sitting in front of your servers and routing client requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance. If a single server goes down, the load balancer redirects traffic to the remaining online servers. When a new server is added to the server group, the load balancer automatically starts to send requests to it.

NGINX is one of the topmost providers of load balancers

???????????????????? ?????? ????????????????: Understanding the state of your infrastructure and systems is essential for ensuring the reliability and stability of your services. Information about the health and performance of your deployments not only helps your team react to issues, it also gives them the security to make changes with confidence. One of the best ways to gain this insight is with a robust monitoring system that gathers metrics, visualizes data, and alerts operation and control teams when things appear to be broken, be it security, application or infrastructure monitoring

In a microservice architecture, if you want to have a reliable application or service, you have to monitor the functionality, performance, communication, and any other aspect of your application in order to achieve a responsible application

#Prometheus is widely popular monitoring tool

?????????????????????? ?????????????? - Moving your applications from a monolithic design to a microservices-oriented design introduces several advantages during development and in operations. However, that move has a price. New challenges are introduced, as traditional metrics and log information tend to be captured and recorded in a component and machine-centric way. When your components are spread across machines and physical locations and are subject to dynamic horizontal and vertical scaling over transient compute units, traditional tools to capture and analyze information become powerless.

Distributed tracing is a technique that addresses logging information in microservice-based applications. A unique transaction ID is passed through the call chain of each transaction in a distributed topology.

Coming back to the example of user interaction with a website. A unique ID is generated at the entry point of the transaction. This ID is then passed to each service that is used to finish the job and written as part of the services log information. It's equally important to include timestamps in the log messages along with the ID. The ID and timestamp are combined with the action that a service is doing and the state of that action.

Unique identifiers, such a transaction IDs/session ID’s and user IDs, are helpful when you gather analytics and debug. Unique IDs can point you to the exact transaction that failed. Without them, you must look at all the information that the entire application logged in the time frame when your problem occurred.

After you implement the generation and usage of the unique ID in your logs, you can use the unique ID in several ways and when it comes to microservice architecture, and given that any request passes through multiple services across distributed applications and infrastructure, this is where distributed tracing is useful (like Zipkin and Open Telemetry)

?????????????? ???????????? - A message broker is software that enables applications, systems, and services to communicate with each other and exchange information. The message broker does this by translating messages between formal messaging protocols. This allows interdependent services to “talk” with one another directly, even if they were written in different languages or implemented on different platforms.

Message brokers can validate, store, route, and deliver messages to the appropriate destinations. They serve as intermediaries between other applications, allowing senders to issue messages without knowing where the receivers are, whether or not they are active, or how many of them there are. This facilitates decoupling of processes and services within systems.

In order to provide reliable message storage and guaranteed delivery, message brokers often rely on a component called a message queue that stores and orders the messages until the consuming applications can process them. In a message queue, messages are stored in the exact order in which they were transmitted and remain in the queue until receipt is confirmed.

Kafka is such an example of a message broker

?????????????????? - in most systems, we need to persist data, because we would need the data for further processing and reporting

Choosing which database to use is one of the most important decisions we can make when starting working on a new app or website which are built around microservices and backend storage

If you realize down the line that you’ve made the wrong choice, migrating to another database is very costly and sometimes more complex to do with zero downtime.

Taking time to make an informed choice of database technology upfront can be a valuable early decision for your application.

Data can be structured (like a classic RDBMS – relational database model), semi-structured (json, xml formats, which can be graphical or columnar in nature), and completely unstructured (blob, image, videos, social feeds)

RDBMS scale vertically. We need to upgrade hardware (more powerful CPU, higher storage capacity) to handle the increasing load.

NoSQL datastores scale horizontally. NoSQL is better at handling partitioned data, so you can scale by adding more virtual machines. Mondo DB, Cassandra are good examples of NoSQL databases that specializes in structured, semi-structured as well as un-structured data

Typically in an e-Commerce architecture, data like personalized recommendations, Wishlist, carts, product content, inventory are stored in underlying NoSQL databases (like #Hadoop, #Mongo DB or #Cassandra clusters)

?????????????? ???????????????????????? -A service registry is a database used to keep track of the available instances of each microservice in an application. The service registry needs to be updated each time a new service comes online and whenever a service is taken offline or becomes unavailable

A microservice needs to know the location (IP address and port) of every service it communicates with. If we don’t employ a Service Discovery mechanism, service locations become coupled, leading to a system that’s difficult to maintain. We could wire the locations or inject them via configuration in a traditional application, but it isn’t recommended in a modern cloud-based application, which requires dynamic switching on/off the services

Dynamically determining the location of an application service isn’t a trivial matter. Things become more complicated when we consider an environment where we’re constantly destroying and distributing new instances of services. This may well be the case for a cloud-based application that’s continuously changing due to horizontal autoscaling to meet peak loads, or the release of a new version. Hence, the need for a Service Discovery mechanism

Netflix Eureka and Apache Zookeepers are good examples of service registration and discovery tools

?????????????? - Caches reduce latency and service-to-service communication of microservice architectures. A cache is a high-speed data storage layer that stores a subset of data. When data is requested from a cache, it is delivered faster than if you accessed the data’s primary storage location.

We should cache only data that is frequently accessed and/or relatively stale or does not change that fast in the source systems/services. Typically, you should cache objects and application-wide settings. The objects can be business entities or objects that hold frequently accessed and relatively stale data. You can also cache application-specific settings

Redis is a good example of industry renowned caching tools

In our example of the e-Commerce platform, application settings like currency, elastic search patters and configurations, home page banners and facets can be cached

?????? ???????????????????? is the process of designing, publishing, documenting and analyzing APIs (application programmable interface) in a secure environment. Simply put an API is a contract that is setup for applications/services to communicate with each other (using a request/response mechanism). Application programming have evolved over the years from simple SOAP based API’s (service oriented access protocol) which are simple client-server messaging using XML’s, to moder day REST API’s (representative state transfer protocol) which are 2-way, more flexible and stateless. Stateless means server do not store the client sent data, like a client sending a web URL and the server returning plain text instead of graphically rendered images

A Web API or Web Service API is an application processing interface between a web server and web browser. All web services are APIs but not all APIs are web services. REST API is a special type of Web API that uses the standard architectural style explained above

Through an API management solution, an organization can guarantee that both the public and internal APIs they create are consumable and secure.

API management solutions in the market can offer a variety of features, however, the majority of API management solutions allow users to perform the following tasks

API design - API management solutions provide users – from developers to partners – the ability to design, publish and deploy APIs as well as record documentation, security policies, descriptions, usage limits, runtime capabilities and other relevant information.

API gateway - API management solutions also serve as an API gateway, which acts as a gatekeeper for all APIs by enforcing relevant API security policies and requests and also guarantees authorization and security.

API store - API management solutions provide users with the ability to keep their APIs in a store or catalog where they can expose them to internal and/or external stakeholders. This API “store” then serves as a marketplace for APIs, where users can subscribe to APIs, obtain support from users and the community and so on.

API analytics - API management allow users to monitor API usage, load, transaction logs, historical data and other metrics that better inform the status as well as the success of the APIs available.

?????????????????????? ?????????????? -An application gateway or application-level gateway (ALG) is a firewall proxy that provides network security. It filters incoming node traffic to certain specifications, meaning only transmitted network application data is filtered.

Application Gateway can make routing decisions based on additional attributes of an HTTP request, for example the URL path or host headers.

For example, you can route traffic based on the incoming URL. So, if there are images in the incoming URL, you can route traffic to a specific set of servers (known as a pool) configured for images. If video is in the URL, that traffic is routed to another pool that's optimized for videos

So now that we have de-mystified and simplified a lot on MS based architecture principles, tools, technologies and the framework around it, how about using these concepts to read the below MS based e-Commerce platform architecture?

?

?