What makes a good database engineer

The art of truly understanding database systems boils down to the following principles.

• You cannot do much with data on disk until you read it to memory.
• You cannot do much with data on memory until it is read into the CPU L caches.
• You cannot do much with data on the CPU cache until it is read into the CPU registers.
• Reading from disk (SSD) is 2000~ times slower than reading from memory.
• Reading from memory is 200~ times slower than reading from the CPU cache (L1d)
• Reading from L caches is 20 times slower than reading from the CPU registers
• Disk is abundant.
• Memory is limited.
• CPU cache is a scarce resource.
• And CPU registers are extremely limited.
• When reading a block of bytes from disk to memory, plan to use it fully, over and over again.
• The block of bytes will be cached in the CPU L cached available to you hot. If you don’t use it something else will take its place.
• When your block is kicked out, you incur the cost of bringing it back on

With these principles in mind, a good database engineer will appreciate the colossal task of a simple index scan and rewrite your SQL queries, configuration, data modeling and perhaps the storage engine itself to achieve optimum efficiency.

Finally these are only guiding principles and meant to be broken.

—

For those interested in databases you can grab my fundamentals of database engineering course

https://databases.win