Echo Connect (CI-CD with Google Cloud Platform)

The aim of building this project is to, deploy a simple social media application which is based on event driven microservice based architecture, via CI-CD pipeline on to the Google Cloud (i.e. into production).

GitHub link to the project: https://github.com/Sayed-Imran/EchoConnect

Architecture of the Project

The SCM (Source Code Management) tool used here is GitHub and CI-CD pipeline is setup over the Google Cloud Platform, with the following tools:

• Cloud Build - To setup the CI-CD pipeline
• Container Registry - To store Container Images
• Google Kubernetes Engine(GKE) - To deploy the application
• Compute Engine - Indirectly use by GKE to run the worker nodes.

Tools and Technologies used in development of the application

The tools and technologies used to develop the application are as follow:

• FastAPI (Python framework to develop the backend microservices)
• Angular (JavaScript framework by Google to develop the frontend of application)
• Docker (To containerize the application, for microservice)
• Pub/Sub (an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages)
• MongoDB (NoSQL database to store the information)
• Cloud Storage (Object file storage to store and retrieve media files)

Application Architecture

The social media application named as Echo Connect has the following event driven architecture with the help of Pub/Sub messaging service provided by Google Cloud.

The social media application allows users to post media files such as images, videos, GIFs or simple texts even. The complete application is broken down into microservices and are event driven with the help of Pub/Sub messaging service provided by Google Cloud.

Backend Microservices

There are 3 main microservices i.e. Post, Like and React which are directly accessed by the frontend, and there are two internal microservices which are subscribers to the Pub/Sub and are triggered as and when required.

User Microservice

This microservice is responsible for the creation, deletion and updating of a new user or existing user. All the operations related to user is handled by User Microservice.

Post Microservice

This microservice is responsible for the creation, deletion, updating of a any post by the user. Users have the option to upload any media file of any type like jpg, gif, mp4, etc. All these are know as objects in terms of Cloud Technology, which here in this application will be stored in Cloud Storage

Once a post is created by the user, all the details of the same are stored in one collection of database and a message is published for the same in Pub/Sub, then there's a internal subscriber microservice which will create a record with the Post ID generated along with an empty list in the Post React Details collection of the database, which will store the details of who all liked the post.

Like Microservice

This microservice is responsible for storing the data of who likes or unlike posts and also publishes a message to increment/decrement the like count to a Pub/Sub topic, and the internal subscriber receives the message and does the same as per the message received.

Frontend

The frontend of the application is developed with Angular JS which is a JavaScript Framework designed by Google.

Login/Register Page

Home Page (Phone View)

Profile Page (Side-bar)

CI-CD Pipeline

CI/CD is a method to frequently deliver?apps?to customers by introducing?automation?into the stages of?continuous delivery, and continuous deployment. CI/CD is a solution to the problems?integrating?new code can cause for development and operations teams (AKA "integration hell").

In this project, Cloud Build is the tool used to deploy the pipeline which builds the container images and deploys the microservices to Google Kubernetes Engine.

Multi branch pipeline has been configured for all the microservices.

As soon as there is a push in the desired branch of the microservice, the triggers for the respective microservice gets triggered and the desired microservice is rolled out on to the server, without affecting the other services. And even if the rolled out microservice fails, other services are safe hence avoiding the problem of SPOF (single point of failure).

Conclusion

The objective of building this project was to show the real-world scenario of how an microservice based application is developed, build and deployed onto the production servers with utmost automation into the different stages of delivering the application to the end user.

#googlecloud #cicd #cloud #microservices #eventdriven #socialmedia #devops #kubernetes #docker #python #angular