Devising a Cloud Load Balancing Solution

My product manager friend Ram Singh was engaged in producing a modern load balance solution development for one of the cloud initiatives.

Ram Singh does not prefer to adopt the standard load balancer that shows up with the cloud services.

• His solution should be vendors agnostic and should be strong in resiliency
• He chose to develop a resilience solution for cloud applications to mitigate present blackouts by providing resiliency at both intra- and inter-cloud levels.
• He wishes to have the probability for pieces of applications to migrate from one server to another.
• The solution should be able to defend applications from several issues, stretching from cyber attacks to hardware breakdowns
• If the cloud server goes down, the solution itself should be alive and kicking, as resources will be directed to the provider that is still up
• He wants to design and implement load balancing at both the intra- and inter-cloud levels so that all requests are serviced, as well as duplicated data across distinct cloud providers; in order to secure that solution users invariably have access to their prevailing data.
• The solution should be able to run on multiple cloud providers and be stable when various cloud instances are switched off.

The solution should satisfy the following

• Request-level load balancing and queuing between hosts across clouds
• Distributed writes to all database solutions
• Intelligence to read from any healthy database server
• If the database layer of one cloud provider goes down, the application should still function.
• If the compute layer of one cloud provider goes down, the application should still function.
• DNS?fail-over so that front-end can consistently be served
• Protection of & restoration from database blackouts

The project launched with a web application development. The solution was designed for AWS?and?GCP

On?the GCP side solution was for the full stack, from cloud CDNs, to compute VMs and databases, and hooked up them collectively.

On AWS, the solution utilized EC2 instances for the application and data golang servers, and while using Compute Engine in the Google cloud.

The application layer, front-end, and load balancers all use docker containers.

For databases, the solution used AWS RDS?and?Google Cloud SQL.

The prototype phase was established

• a primary web server running on GCP that can hook up to CloudSQL
• a primary web server running on AWS that can hook up to RDS
• the web app can handle static content & dynamic content

The solution was developed at each layer

Data layer:

To handle various database services across many clouds, the solution is constructed to have a distributed database access layer that ensures consistency across all DBs.

The solution establishes consistency and availability, so writes are distributed across all DBs.

There installs a cluster of data access servers (DAS) in all clouds that park in between the web server and the database service.

If the AWS side of the system receives a request to read from the DB, and the AWS RDS is up, it will directly read from RDS.

If RDS is down, however, the AWS DAS will call for a read from the GCP DAS and information will be fetched from CloudSQL and transmitted to the AWS.

The solution was to investigate the state of the databases at the point of every transaction.

If a database goes down, or if a database is down, moved back to live, on the afterward transaction, the solution data servers will notice this and take action.

Application Layer:

The application layer in each cloud consists of a cluster of request servers.

These servers pick up incoming requests, execute stateless business logic, and ultimately forward the request to the load balancer interfacing with the data layer.

The application layer is stateless and does not require consensus; it can be scaled up and down as much and as fast as necessary. As long as one application server is up, the system will be alive and running.

Front-End Layer:

The solution has CDNs set up in both clouds serving the front-end code to clients

By having the CDN endpoints map to numerous IPs, each for a different cloud, the solution can load balance client requests to either cloud.

This IPs served by the CDN map instantly to the application layer.

The overall solution should cover all these use-case scenarios

• will handle DB reads, writes, and desirable concurrency issues as a base to address resiliency strategies to
• Offer load balancing between clouds
• Host level load balancing within clouds
• Turning off host instances should not crash the application for an extensive period
• DNS fail-over if the main server goes down
• Turning off all compute instances of a Cloud should not kill the application
• Distributed DB writes
• Distributed reads from any healthy database
• Turning off database instances should not break the application
• Numerous Request servers per Cloud

Test the integrated system by touching the front end and following requests crawled through to the data layer

Hook up all simultaneously in both clouds

As a product coach, I was trying to learn from Ram Singh. He has shared the whole story about the cloud implementation project.