Peering into Data Management - 4/5

Part - 3 is here.

Since the DBMS architecture is mature, there are two ways to create a new DBMS:

1. Making changes to internal components of an existing solution.
2. Creating a custom solution by investigating the problem at hand.?

When technology is mature enough, creating new variations by amending the internal subsystems is a commonly taken approach.

Let's consider a practical example: When you request the deletion of an entry, one DBMS can remove that entry immediately. Another DBMS could update the entry's deletion markers (also called tombstones) and collect the entry later whenever the garbage collector kicks in.

Both DBMS are deleting entries just as the user requested. But the low-level implementation of the deletion is different.

Another example is DBMSs that do not update existing entries. They work in an append-only manner, and the updates are stored as a new entry referenced by the stale entry. This is a common approach in DBMS that specialize in storing time-series data. Another popular append-only system is Blockchain.

Similarly, the other components can be tweaked, thereby changing the performance and guarantees of the offering, giving birth to a different product.

The deletion operation is a straightforward example. In most cases, the variation is not just in how an operation is implemented but in how a subsystem is implemented. For example, the Storage Engines for a DBMS are implemented differently.

The most common real-world example of that is MySQL. MySQL offers several storage engine options, including InnoDB, MyISAM, and RocksDB. MongoDB allows switching between WiredTiger (default) and In-Memory storage engines. Earlier, it offered the MMAPv1 storage engine, which has been deprecated and removed.

Overall, making changes to subsystems is one approach to creating new products. The second approach is to investigate the problem we have on our hands and either use an existing solution (like an existing DBMS that is better performant in our case) or create a custom product (like how some companies have a homegrown proprietary DBMS).

Conclusions

While DBMSs seem like a simple plug 'n play component, many complexities and variations underline their functionality.

DBMS's taxonomy outlines this field's vast diversity. The choice between different classes of DBMS can get complex for a new application. Some applications are niche enough to warrant their custom solution.

In a world increasingly driven by data, acknowledging and embracing the constraints and possibilities offered by various DBMSs can make the difference between an efficient, scalable system and one that struggles to meet its objectives.

Part - 5 is here.