A Case Study on how Netflix uses AWS Services.

Netflix is a subscription-based streaming service which offers online streaming of a library of films and television series, including those produced in-house. Now-a-days it provides seamless streaming by using two things open connect which is a CDN(Content Delivery Network) and AWS. Here, we are going to discuss how Netflix uses AWS to stream.

->What is Cloud Computing?

Cloud Computing is the practice of using a network of remote servers hosted on the Internet to store, manage, and process data, rather than a local server or a personal computer and AWS is one of the leading cloud computing service providers.

->What are the Problems that Netflix faced and Why it Shifted to Cloud?

--Before Netflix used cloud for all its services, Netflix xompletely dependent on IT,they thaught that IT can make every system perfect,they had high end hardware and devices, but in 2008,they faced a two-day outage when one of the data center went down.

--Netflix become a very popular streaming service,due to which new problems arrived such as the data might exceed the data center capacity so it needed vast increase in data center capacity, increase in traffic, unpredictable errors while streaming and huge workload.

--So, Netflix decided that instead of creating a new data center and utilizing all its resources to create a new data center to solve all these problems, they have decided to go with low cost cloud computing services. It was one of the first companies who had streaming services to do something like this.

--Netflix uses AWS for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more—hundreds of functions that in total use more than 100,000 server instances on AWS.

--> How Netflix uses AWS services?

--Netflix first stores its content on a object storage, a static object storage called Amazon S# which can be accessed globally by everyone, which is than monitored and cached through AWS Cloud Front services, due to which it can distribute the Video content all around the world with low latency speed.

->How Netflix handles users across the globe having different internet speeds and devices with different resolution?

In different parts of the world, different users have different internet speed, so there comes different format for video content some may need high resolution and higher storage,some may need high resolution and low storage, etc. Apart from these we must handle different sizes and formats of video needed for different devices, like some might need mp4, some want to watch HD videos, some want to watch at 720P , due to this different formats and different resolution for videos, there exists different kinds of videos by combining both format and resolution and we cannot just upload a single video and hope it works for everyone. So, Netflix uses AWS Elastic Transcoder for video formatting to format the video according to what users need.

Image credits: AWS

-->Application Monitoring on a Massive Scale

Netflix uses Amazon Web Services (AWS) for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more—hundreds of functions that in total use more than 100,000 server instances on AWS.

This results in an extremely complex and dynamic networking environment where applications are constantly communicating inside AWS and across the Internet. Monitoring and optimizing its network is critical for Netflix to continue improving customer experience, increasing efficiency, and reducing costs. In particular, Netflix needed a solution for ingesting, augmenting, and analyzing the multiple terabytes of data its network generates daily in the form of virtual private cloud (VPC) flow logs. This would enable Netflix to identify performance-improvement opportunities, such as identifying apps that are communicating across regions and collocating them. The company would also be able to increase uptime by quickly detecting and mitigating application downtime.

Each log record carries information about the communications between two IP addresses. However, in a dynamic environment like the one at Netflix, where an IP address can float between applications from day to day or even minute to minute, IP addresses alone don’t have much meaning. “The data sources we had before we took on this initiative were one sided,” says John Bennett, senior software engineer at Netflix. “We’d know an application was connecting to others, but we didn’t know both sides of the conversation and how to optimize those communications or the placement of the applications on the network.”

Netflix set out to establish a new data source that could give it more insight into communication among applications and regions by combining VPC flow logs with application metadata.

-->Centralizing Flow Logs Using Amazon Kinesis Data Streams

From the outset, AWS enabled Netflix to experiment with different approaches to analyzing its network data. The solution Netflix ultimately deployed—known internally as Dredge—centralizes flow logs using Amazon kinesis Data Streams. The application reads the data from Amazon Kinesis Data Streams in real time and enriches IP addresses with application metadata to provide a full picture of the networking environment. 

-->Improving Customer Experience with Real-Time Network Monitoring

Netflix’s Amazon Kinesis Data Streams-based solution has proven to be highly scalable, each day processing billions of traffic flows. Typically, about 1,000 Amazon Kinesis shards work in parallel to process the data stream. Netflix is now able to identify new ways to optimize its applications, whether that means moving an application from one region to another or changing to a more appropriate network protocol for a specific type of traffic.

-->How Every user of Netflix gets its own personalized home page, each person having different video recommendation, how does that happen?

To get different recommendation for different users Netflix actually uses Different Machine Learning Algorithms for recommendations, but for that we need data, for the ML model to produce some result. So, whenever we watch a video on Netflix, it records the videos we watch, the genre of video, and uses big data services to store like Hadoop, data streaming services like Kafka and No SQL databases like AWS DynamoDB Database, to store these data.

-->How Netflix provides different content based on the region that it provides services?

Netflix locks the contents region wise, due to which we have different videos available in different countries by employing digital rights management solutions in the cloud, it also uses AWS IAM Services to lock permission for different users, for security.

-->How Netflix handles traffic and scalability in AWS and Fault tolerance in AWS:

--To stream videos, when the traffic or number of users watching videos on Netflix increases, AWS automatically scales up its resources, and when the no. of users decreases, AWS automatically Scales downs, which lowers the cost of management for Netflix as AWS has pay as we go feature which means we only pay for resources that we Use, which makes Netflix highly efficient.

--Apart, from this when a data center goes down for some reason which was streaming videos, it is automatically transferred to another nearest data center in AWS, so users can stream videos daily without being interrupted, thus it is very reliable for a big streaming platform like Netflix

---------------------------------------------------------------------

Although, Netflix to provide videos, depends more on its own CDN i,e, Open Connect rather than AWS, but to manage its resources and provide efficient and seamless streaming services it uses many services of AWS which makes it one of the highest user if AWS.

Hope, I was able to capture everything through my resource and show you how Netflix uses AWS and how Cloud Computing in this modern era is an essential for companies, and what makes AWS a leading company in this cloud computing field.

Thanks For Reading this article, I hope you liked it and enjoyed it.