Scaling from zero to millions of users - Databases

In the previous chapter, we built the initial structure of our system. Now, let’s move our focus to databases - the foundation for storing and managing data.

In this chapter, we’ll explore how to choose, organize, and optimize databases to ensure reliability and scalability. Ready to dive in? Let’s go!

--

From where we stopped, this is the current state of our application (Image 1).

It contains a visual interface (that could be browser or mobile app), connecting with DNS to get the IP address, just so we can send HTTP requests to the expected backend server. This server should connect with a database server.

But why?

Because we want database and backend servers to scale independently.

Which databases to use?

Choosing the right database goes beyond just SQL versus NoSQL—it’s also about understanding the constraints and trade-offs your system can handle. This is where the CAP theorem joins our room. The CAP theorem specifies that, in a distributed system, you can only guarantee two of the following three properties:

• Consistency (C): All nodes see the same data at the same time. Every read reflects the most recent write.
• Availability (A): Every request gets a response, even for partial failures.
• Partition Tolerance (P): The system keeps working even when there are issues related to network or communication between nodes.

Understanding how databases handle these trade-offs can guide your choice:

Relational Databases (SQL):

• Trade-offs: Typically prioritize Consistency and Availability in scenarios where partitioning is not a problem.
• Use Cases: Financial systems, inventory management, and applications requiring strict transactional integrity.
• Examples: PostgreSQL, MySQL, Oracle.

Non-Relational Databases (NoSQL):

• Trade-offs: Most commonly designed for Partition Tolerance and either Consistency (e.g., MongoDB in its strong consistency mode) or Availability (e.g., Cassandra with eventual consistency).
• Use Cases: Real-time analytics, distributed applications and unstructured data.
• Examples: MongoDB, Cassandra, DynamoDB.

Summarizing:

Making the Choice

Your choice should reflect the priorities of your system:

• For strict consistency (bank transactions), SQL databases or strongly consistent NoSQL options like MongoDB are preferred.
• For high availability and scalability (social networks), NoSQL databases like Cassandra or DynamoDB are great options.
• When partition tolerance is a priority, choose distributed databases that can maintain operation even during network disruptions.

Each database comes with trade-offs, and no single solution fits all scenarios. As we progress, we’ll discuss how to design your system to take advantage of these strengths while mitigating limitations.

Vertical scaling vs horizontal scaling

When designing a system, scaling strategies play a key role in handling bigger demands. The two main approaches are vertical scaling and horizontal scaling:

Vertical Scaling (Scale Up): This involves adding more power (CPU, RAM, etc.) to a single server. It’s simple and effective when traffic is low, but it got some limitations:

• There’s a limit to how much hardware can be added to a single machine.
• If the server goes down, the entire system goes offline since there’s no redundancy.

Horizontal Scaling (Scale Out): This approach adds more servers (nodes) to distribute the workload. It offers:

• Redundancy: If one server fails, others keep the system running.
• Scalability: Easily handle growth by adding more servers.
• Challenges: Requires a distributed system architecture and careful handling of data consistency.

Due to the limitations of vertical scaling, horizontal scaling is often the better choice for large-scale applications. But it doesn't mean that you are supposed to choose just one. Many systems use a combination of both to maximize performance and resilience.

Why is it so important?

In our previous design, all users connected directly to a single web server. This creates two problems:

1. If the server goes offline, the application becomes completely unavailable.
2. When traffic exceeds the server’s capacity, users experience slower responses or can’t connect at all.

A load balancer solves these issues by distributing traffic across multiple servers, ensuring better performance and reliability. But let’s not get ahead of ourselves—load balancers will be the focus of the next chapter.

--

To close this part, let's introduce a case for studies. We'll be building the structure for a payment system. Some of the concepts that we'll run through would not be applicable for payments context, but it doesn't matter, we'll cover as much as possible.

This system needs to handle financial transactions, manage user balances, and process transfers securely and efficiently.

Based on what we’ve covered so far:

Database choice: For strict consistency in critical operations like balance updates, MySQL is ideal due to its transactional guarantees. If you’re not familiar with the concept of ACID (Atomicity, Consistency, Isolation, Durability), it’s worth researching to understand why MySQL is a strong choice for financial operations. To handle non-critical, high-volume data like logs, DynamoDB provides scalability and speed.

Initially, given that we don't have that many users, we can keep with a node, scaling vertically, but as the user base grows, horizontal scaling will ensure the system can handle increased demand.

For now, we’ve covered the basics of scaling strategies and why they’re crucial for growing systems. These concepts set the stage for building applications that can handle millions of users reliably. See you in the next chapter!

Previous chapter: