Turn Boring Projects into Opportunities

This edition of the newsletter contains

• one quick write-up that will help you grow faster in your career
• a video I posted
• a paper I read

I have also shared 3 super-interesting articles I read over the weekend. Thank you once again for reading this edition of my Newsletter. Now without further ado, let’s jump right in …

By the way,

Being hands-on is the best way for you to learn. Practice interesting programming challenges like building your own BitTorrent client, Redis, DNS server, and even SQLite from scratch on CodeCrafters.

Sign up, and become a better engineer.

Turn Boring Projects into Opportunities

No company can give you the most exciting, challenging, and impactful project around the year, so the ups and downs in the kind of work you'd get are expected.

On average, you would get 4 months of high-impact work, 4 months of moderate work, and 4 months of mundane work in a year. Instead of feeling bad about it, leverage the time and energy to advance your career and secure more impactful projects in the future.

Remember, the most important and impactful project is not given to the smartest engineer, but it is given to the one holding the track record of getting things done well and on time. The trust you instill by doing grunt work or maintenance would and should put you in a position to get the most impactful project.

Given that the maintenance work does not eat up a lot of your mental bandwidth, use this breathing room to identify more significant problems within your organization. Spend time digging deeper into systemic issues, find a solution, create a project plan, and present it to the leadership. This showcases your initiative and capacity for impactful contributions.

When I was given some mundane work, I used the additional time to figure out the root cause of recurring outages and proposed an architecture change to solve it once and for all. Because I was thorough with my homework, it became a no-brainer for leadership to approve.

You can also use this time to enhance your visibility and influence within your organization. The easiest of all is to use this time to hold tech talks and mentor early engineers. These interactions will make it more likely that you'll be top of mind when exciting projects are conceived.

To be honest, this is what I did at all of my stints across all the companies I have worked at. I gave talks on some of the lessons learned from past projects, new industry trends, or introductions to new technologies. This helped me establish a good reputation within the org.

Remember, every phase cannot be exciting, but it is important to know how to leverage it well. Impactful projects will not be served to you on a silver platter, you need to earn it.

Here's the video I posted

I published a video - How Atlassian reduced latency by 70% by using Sidecar pattern and when should you use it

Sidecar is a pretty underrated pattern in system design, but Atlassian used it to standardize all the clients and apply practices that improved their latency and stability.

To understand this better, I dissected their engineering blog and compiled my learnings in a video. In the video, I also added a bunch of my experiences and learnings while using this pattern.

Paper I read and would highly recommend

I spent some time reading Scalable blocking for very large databases

Join is considered to be a super expensive operation, but imagine what it would take to make joins efficient at a massive scale.

Sometime back, I read a paper by AWS AI Labs on scalable blocking for large databases. The work Blocking may be misleading, but it is all about finding potential matches between records in large databases. These match operations form the core of several other database operations like shuffle, joins, etc.

Traditional methods (like loading hashmaps and sets) do not work well for large datasets, so this paper proposes Hashed Dynamic Blocking (HDB) to improve efficiency and accuracy.

HDB leverages Locality Sensitive Hashing and Count-Min Sketch for efficient block computing keeping the entire process lean and scalable, even on datasets that exceed hundreds of millions of records.

A good weekend read if you have time :)

You can download this and other papers I recommend from my papershelf.

Three interesting articles I read

I read a few engineering blogs almost every day, and here are the three articles I would recommend you to read.

• It’s a knowledge problem! Or is it?
• Why Does Integer Addition Approximate Float Multiplication?
• Boring tech is mature, not old

Thank you so much for reading this edition of the newsletter ?? If you found it interesting, you will also love my courses

1. System Design Course for Beginners
2. System Design Course for SDE-2, SDE-3, and above
3. Redis Internals Course

I keep sharing no fluff stuff across my socials, so, if you resonate do give me a follow on Twitter, LinkedIn, YouTube, and GitHub.