GCP Services - The Basics

The cloud platform is the most important thing in the modern days of application development and deployment. Shifting towards the cloud environment allows the organizations of all sizes to run faster, be more agile, innovate their businesses in better way, and this move has completely transformed the way of working, communication, and collaboration and becoming an essential part to stay competitive in the digital world.

Google Cloud Platform (GCP) is one of the biggest cloud platforms in the market in addition to Microsoft Azure and Amazon AWS. This article would be good for the GCP beginners to cover the breadth with high-level description without going into their deeper details. The pricing of the services / free-tier details would not the the focus of the article.

GCP Services Hierarchy

At high-level, the GCP services can be divided into 4 main categories / sections and most of Google Cloud Platform services would be classified in any of them.

We will cover 18 most commonly used services here (along with 4 from supporting services) in their respective sections with the brief details below.

1: Compute

The Compute is the most commonly used section for development and deployment of applications of all sizes ranging from custom business application to large scale solutions. We will discuss few services from the Compute section.

1.1: Compute Engine

The Computer Engine can be considered "Infrastructure as a Service". This is actually a computing and hosting service which offers a platform to run virtual machines (VMs) on the Google Cloud. The Compute Engine services can be used to deploy the application with auto-scaling. The Compute Engine offers Windows & Linux as Operating System on it. This service can be considered similar to the Microsoft Azure VM or Amazon AWS EC2 service in comparison with other cloud services providers.

1.2: Kubernetes Engine

Google has a built-in GKE which offers the support of Kubernetes technologies for all the applications being deployed on it. The Kubernetes engine provided by the Google is capable of running the containerized application on secure, modern & managed way on the Cloud with more flexibility. This service can be considered similar to Microsoft Azure AKS or Amazon EKS service.

1.3: App Engine

The App Engine is most commonly used service for web application deployments. This service offers the large-scale deployment of Web Application, Web APIs, or even websites. This a managed service and supports auto-scaling capabilities. This service supports many technologies e.g. Node.js, Python, PHP, Java, .NET etc. This service can be much similar to the Microsoft Azure AppService or the Amazon AWS Elastic Beanstalk.

1.4: Cloud Functions

The Cloud Functions are very much similar to App Engine but having a slight difference with it. Usually, this service is used for single purpose function or light weight applications e.g. a small utility to import an excel file or one time job for data processing etc. This service would be good for small backend jobs etc. ETLs. The Cloud Functions may not be recommended for the large-scale integrated solutions. This service can be similar to the Microsoft Azure Function App or Amazon AWS Lambda.

2: Storage

The Storage is the key section for deployment of relational / No-SQL databases, permission driven file system structure and data warehouse solutions etc. We will discuss few services from the Storage section.

2.1: Cloud BigTable

The Cloud BigTable can be considered to a No-SQL database service. This is very large petabyte scale service having Big Data support inside along with many features. One of the important features is high performance gain and fast data retrievals. This service is similar to the Microsoft Azure Cosmos DB or Amazon AWS DynamoDB.

2.2: Cloud Storage

The Cloud Storage is simply a storage drive on the cloud i.e. file / objects can be stored in the cloud in customized folder structure which are in permission driven style. The public folders / objects can access directly while the private would be access after the authentication & authorization process. This service supports many file types directly as a built-in capability. The Cloud Storage can be considered very much similar to the Azure Blob Containers or Amazon AWS S3.

2.3: Cloud SQL

The Cloud SQL is the RDBMS service on the Google Cloud Platform. This is fully managed relational database service having super fast maintenance & administration capabilities. The Cloud SQL would be an option for storing the relational & transactional data on the GCP. Usually it offers the MySQL by default as free tier but other RDBMS systems would also be available. The Cloud SQL service can be considered similar to the Azure SQL or Amazon AWS RDS service.

2.4: Cloud Spanner

The Cloud Spanner can be like a big brother of Cloud SQL with big data capability as well. If the customer needs storing the huge relational & transactional data, massive data loads with big data support & high performance too, then the Cloud Spanner service can be the best fit. So in simple words the Cloud Spanner = Cloud BigTable + Cloud SQL.

2.5: Cloud Datastore

The Cloud Datastore is highly scalable No-SQL database, with light SQL support as well. This service is derived from the Cloud BigTable service with additional capabilities. It supports ACID transactions in the database, and also has the sql-like syntax queries for data operations. The Cloud Datastore also supports database Indexing structure in No-SQL for performance gain in data retrieval.

3: Big Data

The Big Data is one of the important sections which contains big data storage services offered by Google Cloud Platform. We will discuss few services from this section below.

3.1: BigQuery

The BigQuery is a data warehouse solution for highly scalable systems. This service is capable of storing huge amount of data coming from different sources with different formats, and fulfills the data lake requirements as well. The BigQuery is capable of running the massive data loads and running the data analytics at high performance.

3.2: Pub/Sub

The Pub/Sub is usually used for ingesting the streaming data & real-time reliable messaging into the Google Cloud. For example, a server is sending logs after every few seconds. The logs must be captured, analysed and stored in the cloud in a predefined customized structure / format. The Pub/Sub service can be a best fit middleware to sit between for fetching data from datastore, saves it into any of the compatible storage service in the Google Cloud etc. BigTable through the publisher subscriber model i.e. enterprise service bus ESB in async can be a good example.

3.3: Data Flow

The Data Flow is an ETL product from the Google Cloud Platform, just like Informatica or DataStage from IBM. The first step is data extraction i.e. reading data from different sources through available data adapters. The data transformation would be done at the next steps based upon the business needs/rules and then data would be loaded into the Google Cloud for further analysis or generating reports. Another example of this service could be SSIS packages from the Microsoft.

3.4: Data Catalog

The Data Catalog is a metadata management service. This service would be required to discover the patterns present in the data coming from different cloud services and understand what the data is in actual. It becomes very important to understand at some stages what data is actually floating around at the cloud platform, otherwise data lake can become data swamp if we don't control it. Usually big application / large enterprises use this service, and it may not be required for the small apps or companies.

4: Artificial Intelligence

The Artificial Intelligence is an interesting section which contains many services running on Google Cloud Platform based upon & developed on AutoML model. We will discuss following services from this section with details below.

4.1: Natural Language API

This service is developed on Machine Learning model of GCP. This drives the insights from unstructured data / unstructured text using google machine learning and extracts the meaningful data out of it. The best use of this service can be a sentiment analysis i.e. text analysis of twitter in election days. Tweets are upload, train the model & keep training, and analyse sentiments of the tweets data. The input to this service would be tweets uploading, and this service can help analysing & generating a report in desired format.

4.2: Vision API

Machine learning also helps reading the images for analysis. Image would be uploaded under Vision API. The API would read the image content and help identify data points within that image i.e. type & objects inside e.g. sports venue 60%, building 64%, city 30%, stadium 40% and so on. This is an efficient data processing technique within the image for analysis & generating statistics. If the source data is mostly present in images, the Vision API can be top notch to identify & classify the objects within the data.

4.3: Video Intelligent API

This AutoML API service is very much similar to the Vision API in functionality, except this service has the capability to identify and classify the content from the video source. The pretrained ML models are used to recognize the objects, places and action in the stored & streaming video content. This service API would be much beneficial when most the source content is in video format and generating report out of it becomes difficult at times.

4.4: Speech API

This is simply a speech to text API using machine learning services offered by the Google Cloud Platform. Someone would be speaking and that speech would be converted into the text using the Speech API interactively. The generated text from the API can further be used for analysis and generating statistics for different decisions. Currently the API supports around 120 languages. This would be much helpful when we have the source data in the speech format and the API would work as another option to support generating meaningful patterns from the given source.

4.5: Translate API

The Translate API is used for text translation from one language to anther desired language e.g. Chinese to English language. The translated text can be used for further analysis or building reports / dashboards for executives e.g. sentiments analysis etc. This can be the best option to produce content coming from multiple sources in different languages into one language.

A new update of "Video Translation API" is added into it to accommodate translation from the video content as well. Language pairs are being uploaded at first step, and the model would be trained on that. The translation API would then generate the content in the chosen language. This API is smart enough to detect the source language as well, if not even provided.

GCP Supporting Services

This is not all about the Google Cloud Platform services, there are many other services as well in the GCP which have not been discussed yet. We will touch few more of them which are considered under "Supporting Services" section.

Billing:

The Cloud Billing is a tools collection?that can help to track and understand the spendings on Google Cloud, pay the bill, and optimize the costs as maximum as possible. It can also be configured to generate alerts on reaching to the different threshold values as well.

IAM & Admin:

Provides tools to manage resource permissions with minimum hassle and high automation. Job functions within the company can be mapped to groups and roles. Users get access only to what they really need to get their job done, and admins can easily grant permissions to entire groups of users.

VPC Networks:

A Virtual Private Cloud (VPC) network is a virtual version of a physical network that is implemented inside of Google Cloud. It provides global, scalable & flexible networking functionality to cloud based resources i.e. Compute Engine VM, Google Kubernetes Engine (GKE) clusters, and the App Engine instances in the GCP.

Google Partners:

Google Partners services allows the users to use the components from the 3rd party vendors through the integrations & connectivity between the on-premises network and the VPC network through a supported service provider.