Foundations Of Highly Available System Design - Data Replication And Replication Strategies

What is data replication?

Database replication is the process of copying data from a source(Leader) database server to one or more target(Follower) database servers. It involves the frequent copying or streaming of data from a database server to another database server so all users have access to synced data.

Let's take an example of single leader - followers data replication model -

• In this model Write request is performed on Leader and Read request can be performed on leader/followers.
• The data is synced with followers to make the data consistent b/w Leader and Followers on the basis of different consistency models discussed ahead.

Why do we need Data Replication?

1. High availability, fault tolerance and disaster recovery - Suppose there is no replication followed in a system, and the database instance outage happens, the system will be left inaccessible. Now multiple copies will provide fault tolerance so that if one fails, then other can take its place ensuring high availability.
2. Even Load Distribution and increased read throughput - Generally the systems are read heavy, which means there will be more read requests than write requests. Here data replication ensures even load distribution to plural follower instances and hence increases the read throughput of the system.
3. Reduced Latency - Lets take an example to understand this advantage.

Suppose your main database server is running in USA and the user wants to read data all the way from India which will take a lot of time here. So what you can do is you can keep follower replica of data in India i.e. keeping data close to geographical users which will provide results fast hence reducing latency.

4. Support for real-time analytics - data replication is a continuous real-time process, it allows businesses to get immediate insights from their data. A simple example is populating a dashboard. A more advanced example is using data replication to pull user behavioral data from various data sources to analytical data stores, then running a predictive model to provide real-time personalized recommendations to improve the customer experience.

What is the most challenging part of Data replication?

Maintaining Consistency of data(Same copy) among Leader and Followers is the challenging problem here. For this we have various consistency models each having its own trade-offs.

• Strong Consistency/Synchronous Replication - With strong consistency, changes/writes made to the data on leader are immediately reflected on followers in the system hence guarantees the up-to date data in followers. Here the request is marked complete only if write in all the followers is complete until then the client has to wait for OK response.

What is the trade off here?

Here the system will offer high consistency, if there is read request then the data returned as response will be the latest one but availability will go down as it increases the waiting time.

Example - Banking applications where consistency of transactions is really important.

• Eventual Consistency/Asynchronous Replication - weakest form of consistency. With eventual consistency, changes made to the data on one leader may not be immediately reflected on followers in the system.

What is the trade off here?

Here the write request will be performed on leader and OK response will be immediately returned hence offering high availability. Suppose a read request is performed on a follower, it may return stale data hence decreasing the data consistency.

Note - Given enough time, the data will eventually become consistent across all followers in the system, so after one point the data b/w leader-followers will be synced.

Example - Like and view count in applications such as YouTube Twitter Instagram , where availability is more important than immediately returning exact count of views and likes.

• Hybrid Consistency/Synchronous - Asynchronous Replication - hybrid of above two consistency models in which some of the followers are synced with leader writes(preferably those which are located in same data center) and other are left to be synced eventually.

This hybrid model will provide fair availability as well as consistency(strong consistency > immediate wait time and data consistency > eventual consistency).

If you find the article useful and want more such articles than subscribe the newsletter and follow?Kartik Sapra

Cheers

Kartik Sapra

#systemdesign #systemarchitecture #techinterview #faang #maang #google #microsoft #amazon #instagram #socialemedia