How To Avoid Premature Optimization

Premature optimization can make or break a company. So, leaders on the business side of tech companies rightfully extol the virtues of focusing on minimum viable products. And they do it instead of making perfect the enemy of good and only shipping what’s necessary as quickly as possible. These goals certainly resonate with me as an operational founder. That’s why I emphasize them with my teams every day.

At my core, I’m a software engineer. Thus, I skewed my instincts towards a mentality with an opposite set of goals. I strive for comprehensive solutions, build things that last, and mitigate problems before they arise. In other words, I’m guilty of premature optimization.

How Premature Optimization Found Footing

Years ago, I added the Plugin Framework to Concourse, helping developers extend it with modular enhancements. The framework allowed each plugin to run separately and communicate with the database server over dedicated inter-process communication (IPC) channels.

While there are a lot of promising native approaches to low-level IPC (i.e., domain sockets or named pipes), Concourse sought to be as cross-platform as possible. But I was limited to higher-level constructs like anonymous network sockets or rolling my solution from scratch. So, I chose the latter.

The technical benefits were clear. Building my own IPC over shared memory would minimize latency and facilitate future improvements. Therefore, plugins could communicate directly using the same shared memory segment. Meanwhile, anonymous sockets require binding to a specific port for each unique plugin pair that exchanges messages.

I soon discovered that this was a foolhardy approach. And before the feature was working, I was trying to make it fast. Even worse, I accommodated development for possible future improvements before completing the core functionality. Long story short, I wasted a lot of time going down this path.

How I Avoid Premature Optimization

I learned to avoid premature optimization as quickly as I could. First, I had to implement shared memory using memory-mapped files and a combination of spin-locks and file system notifications. That way, I could facilitate efficient message consumption by a neighboring process.

Eventually, I ran into a problem where the underlying memory-mapped files grew too large. Then, the off-heap memory usage exploded. So, I had to add compaction logic to the shared memory implementation. And this implementation worked for Concourse instances with only one plugin!

But chaos ensued with multiple plugins installed because the system lacked concurrency controls to regulate the different communication streams. Therefore, I had to build synchronization logic from scratch. The job is hard enough on its own, but the compaction functionality compounded my problem even more.

I ultimately developed an intricate system that uses a combination of both JVM and file locks to coordinate readers, writers, and compaction. I was certainly proud of myself and impressed at the feat. Yet, my work proved unusable due to a MAJOR limitation inherent in most file systems. Broadcasts about file changes only occur once per second. So, it’s impossible to detect changes that happen in 999ms or less.

In total, I spent a month developing this system. Then I spent another two weeks trying to debug intermittent issues. Eventually, I got fed up and decided to scrap it in favor of anonymous network sockets. Two years later, Concourse’s plugin framework is very reliable. Plus, bugs related strictly to the communication infrastructure haven’t plagued it yet.

?

Conclusion

The advantages of using shared memory still stand. And the use of anonymous sockets causes Concourse plugins to require more operating system resources. But the feature works. And it provides value without premature optimization, which is the most important thing.

As a developer, I understand the urge for premature optimization. It’s a habit for some of the world’s best software engineers and architects. However, it’s imperative to keep that inclination in check to ensure rapidly delivered value. Then, get feedback from users while making the right trade-offs as your software evolves.

Originally Published on JeffNelson.com