Scalable Sytems - An intro

As software engineers, we often start with the fundamentals—writing efficient, correct code that runs on a single machine. However, in today’s world of complex software services, being able to scale is just as crucial. Scalability defines whether our system can handle growing loads and increasing numbers of users without compromising performance. But how do we achieve this?

The Gap in Traditional CS Education

Many traditional Computer Science curriculums are great at teaching the basics—writing programs for correctness and efficiency. However, they often fall short in preparing students for the practicalities of building large, distributed systems. As a result, many graduates enter the workforce unprepared for the challenges of scaling real-world applications.

At the core of this gap is the difference between building programs that run on a single machine and services that handle requests from multiple users, potentially distributed globally. While single-machine programs focus on complexity (like O(n log n) time for sorting), scalable systems need to consider many other factors: concurrent users, global data storage, and performance under load.

Types of Scaling

Vertical Scaling: This method involves upgrading your existing machine to a faster, more powerful one. Imagine running your service on a single machine, and as demand increases, you simply buy a bigger server. Vertical scaling can be effective for smaller loads and is relatively simple to implement because it often doesn’t require code changes.

Pros:

• Easy to implement.
• Most off-the-shelf software supports vertical scaling.

Cons:

• There's a limit—no matter the machine's power, there's always a ceiling.
• Expensive! Top-tier servers can have a poor price-to-performance ratio.
• Single point of failure.

Horizontal Scaling: When vertical scaling can no longer meet the demand, it’s time to scale horizontally—by adding more machines. This is the approach used by global-scale services. Instead of relying on one super-powerful machine, horizontal scaling distributes the load across multiple machines.

Pros:

• No hard limits to growth.
• Better fault tolerance—if one machine fails, others can take over.

Cons:

• Complex! It requires coordination between machines, which brings challenges in data sharing, synchronization, and handling failures.

Vertical Scaling in Detail

Vertical scaling—also known as “scaling up”—is about increasing the capacity of a single machine. Whether it's adding more CPU cores, increasing memory, or moving to faster storage solutions, vertical scaling improves the ability of a machine to handle a larger load.

Modern cloud platforms like AWS make vertical scaling easy. You can change the type of virtual machine instance you're using with just a reboot. However, this simplicity comes with a cost. High-performance machines, especially those with advanced CPUs or massive RAM, become increasingly expensive, and there’s always an upper limit on how much you can scale vertically.

Horizontal Scaling in Detail

Horizontal scaling—also known as “scaling out”—is the cornerstone of systems designed to handle millions of users. In this approach, instead of making a single machine more powerful, we add more machines to the network. Cloud computing services like AWS, Google Cloud, and Azure provide this flexibility, allowing you to spin up new virtual machines (VMs) on demand.

However horizontal scaling is not without its challenges. Coordinating data between machines can be tricky. How do you ensure that two different machines processing requests for the same data don’t conflict? What happens when one machine fails? These are the challenges of distributed systems that software architects must solve.

Choosing between vertical and horizontal scaling depends on your application’s requirements. If you're working with a small, contained system, vertical scaling might be sufficient and easier to implement. But for global applications that demand constant uptime and can’t afford downtime, horizontal scaling is essential.

In the next article of this series, we’ll dive deeper into microservices architecture—a key enabler of horizontal scaling. Microservices allow you to break down complex applications into smaller, independently scalable services.

This newsletter and articles inside are inspired by Stephen P. Tarzia who is the Director of Engineering at MongoDB