The Consolidated Developer Index: Measuring an Engineering Organization's Power

(Author's Note: This is the first in a series I'll be doing around the theme of expanding engineering teams. More articles in this series will follow)

Introduction

“It's all happening!”? This is one of my favorite lines from Cameron Crow's 2000 masterpiece Almost Famous.? (Author's note, if you haven't seen this movie, please stop reading this article right now and go watch it!)

In my case, what's "happening" is this small development team I've been leading now needs to grow into an engineering behemoth.? I need to grow from five engineers to 15-20 as quickly as possible.

Now, naturally one of the questions I get from my CEO (or VP or Executive Director or whoever, insert boss’s title here) is "So, you should have your first new engineer on the team by April 1st.? That means the team should be able to deliver 20 percent more starting that day right?”? Ahhhh, no.? I make an attempt at that point to try to explain to her exactly why this math is faulty and I end my day in frustration with her for not being able to get it.? Later I realize my frustration should NOT really be with her but it should really be with myself for not having some way of explaining to her why the engineering team ramp up is not that simple.

Enter the Consolidated Developer Index or CDI.? The CDI uses a number of factors to gauge a developers power on a team:

• Developer Experience or DE
• Platform Experience or PE
• Deduction 1 or D1, Deduction 2 or D2…Deduction X.

In a nutshell, the CDI for each developer is calculated as:

(((DE * 2) + PE)/3) - (D1 + D2 + DX)

Like many ideas a leader has, it should be noted that I did not develop this in a vacuum.? I came up with the initial version, presented it to my direct reports and, of course, they made great suggestions on how I could improve it.? It's very rare that a single person owns an idea.

CDI Factor One: Developer Experience

The first part of the CDI is straight developer experience.? It is common in software engineering titling to categorize developers as Junior, Mid-Level and Senior so I'll use that to keep this example simple.? For straight developer experience, I will index a Senior developer as a 1.? Any developer lower than that would be some fraction of this number.? For example, Justin, who is teetering somewhere between a junior and a mid level engineer gets an index of .3.? Ella, who was an intern until January 1st when she was brought on full time as a junior developer is indexed at a .1.? And yes, in case you were wondering, it is possible for a senior to be worth ten times more than a junior developer.

So after all is said and done we have the start of our CDI (Link here):

CDI Factor Two: Platform Experience

So, that's it right?? The strength of our team is that simple.? Unfortunately, no.? It doesn't matter how good the absolute most rockstarish your new hire super engineer is, she does not come into an established organization with an established code base and starts producing at 100% capacity.? She needs to ramp up.??

How long does that take?? Well, that depends:

• How large/complex is your platform?
• How many unusual/niche technologies does it use?? For example, I may have hired a great senior engineer, but he’s never worked with the Spring Framework, which my codebase is completely built around.
• How good are your on boarding processes?? For example, do you have an in house training team that can train all new hires on your application?

???????????????????????????????????????????????????????????????

We now have a new number we need to add to our CDI computation; platform experience.? We will also index this from 0 to 1.? One thing you might consider here is to weigh the Developer Experience above Platform Experience.? We can do that by using a simple formula to factor the two together:

CDI = ((DE * 2) + PE)/3

This is included in our spreadsheet calculation(Link here):

CDI Deductions

So now we've got what we need right?? Well,? not quite.? The problem is when a development team starts to grow, like any organization, it becomes less efficient.? This is not something that can be completely avoided.? Here are some areas that are sure to be deductive:

Lead Responsibilities

You are going to need to promote some of your developers to Team Leads.? A good ratio might be 1 lead to every 4 developers, but YMMV.? The responsibilities they take on as leads is going to eat into their productivity as individual contributors.? Mentorship, decision making, lead level meetings, performance review authorship are some examples of these additional responsibilities.? A good rule of thumb is 10 percent productivity subtracted for each direct report engineer.

Hiring

Who does your hiring?? Well, as the head of engineering, you do.? But you don't do it alone.? You will need your senior engineers and leads to help.? Developer interviews must be technical affairs. You will also need your technical leaders to buy in since they will take on some responsibility for new team members' work.? For this reason, while you're hiring, each of your senior engineers and leads will need some time subtracted for hiring.? How much?? Well, if you are asking an engineer to come to 2 interviews a week, assume that is about six hours (1.5 hours per interview, doubled for prep and post analysis).? 6/40 hours a week ~ 15 percent.

Tier 3 Support

In an engineering organization, you hire people specifically to do Tier 1 and 2 Support.? Your Tier 3 Support, however, should be handled by developers.? My preference is that this falls to the newest people on the team.? This forces them to learn your platform from a user's perspective.? I will elaborate on matters like this in a future article. If a developer takes an average of 1 support shift per week and is pulled away about a fourth of their time on that day, that is 2 hours = 2/40 ~ 5 percent.

Other Responsibilities

These are wildcards and will vary by organization and what bigger things are currently going on.? Dushyant might be on temporary loan for 25% of his time to the AWS team .? His time doing that work should also be deducted? from the CDI.

With the above deductions included, we now have the following (Link here):

Our CDI now indicates that this five developer team really has the power of about 2.5 senior engineers.

Expanding the team!

Expansion One

Let's imagine we started the CDI in April 2024.? Suddenly in May, your CEO comes to you and says we are going to put all of our extra money into expanding the engineering organization.? After putting out the fire that somehow started spontaneously in your hair, you dust off your hiring function and get to work.

By July 1st, you’ve somehow managed to bring on five more engineers.? On their start date, your CDI might look like this (Link here):

We’ve managed to increase our CDI by 1.73.? This is probably a bit of an overreach on day one.? A team on its first day is never immediately powerful.? This also shows one drawback of making the DE have more weight than the PE.? Even the most senior developer doesn’t start being productive for weeks.? But it’s good enough for illustration purposes.

Our initial team’s CDI stayed fairly close to what it was.? They lost some power because Kurian now has two team members.? But they also experienced an increase in platform experience, offsetting Kurian’s lead deduction.? Getting back Dushyant from the AWS team? helped as well.

By the way, we would not have actually kept the team composition as illustrated.? I did this in the example to make differences between groups of engineers more obvious.? We would have actually scrambled the teams to mix the newbie with the veterans.

Expansion 2

By October, we had managed to triple the size of the team.? In a nod to the current AI buzz, we decide the third team will all be robots.? Our new calculation looks like this (Link here):

The second team (Lions) has now taken over Tier 3 Support and it has had some real benefits to their platform experience level.? Each lead now has three team members to manage and, therefore, takes a hit in their individual CDIs.? Also, Andrew, Jason and Gopi are now helping with the hiring so their team took a hit.? In reality though, we probably don’t need as much from each senior/lead for hiring because we have more of them.

RIFs and/or Departures (Ahhh!)

It’s unavoidable.? Some day you will experience circumstances that result in a downsizing of the engineering staff. This, of course, will have an immediate and detrimental affect on your CDI.??

In our example, we lost Meg and Rosey, who both accepted jobs elsewhere (Link Here):

Our CDI dropped by > 1.? This is a greater than fifteen percent drop in team power. This was offset by gains in experience but these offsets would have been experienced had we not lost the resources. This illustrates how devastating the loss of skilled and experienced engineers can be.

Source the CDI

We still have one burning question. How do we derive these numbers?? You're certainly not coming up with them yourself, especially with the team growing to fifteen developers.? If you are a CTO/Head of Engineering, you have most likely also hired QA Engineers, Product owners, Tier 2 Support Engineers, DevOps and hopefully a dedicated security team.? Your team has likely ballooned to 40-45 people.? It won’t be possible for you to know firsthand? the skill level of every software engineer.??

There are a number of ways to do this.? One option that worked for me in the past is holding a weekly meeting with all leads.? In this meeting we cycled through each individual contributor and updated each of their entries.? Is Horatio still a .2 DE?? Has his PE increased after 3 months (I would hope so).? Has he taken on additional responsibilities??

With four 1-hour meetings a month, we were able to get through everyone at least once a month and, therefore, have a CDI calculated at that frequency.

Conclusion

We needed a way to measure a team's power.? Objective measures, while providing some use, fall short of telling the real story.

A subjective measure, the CDI allows us to track team productivity over time by taking into account all variables that affect this.? Tracking these numbers provides several advantages:

• We can placate stakeholders who are concerned when team productivity does not grow linearly or as quickly as they would like.
• We can track our own progress at getting new engineers plugged into the platform and producing at 100 percent as quickly as possible.
• We can provide some predictability?

I hope you will find benefit in this CDI methodology as well.

Acknowledgments

As I mentioned, I did not come up with this in a vacuum.

Eric Eldard provided countless inputs into the CDI as we refined it. Eric also had the idea of holding a weekly meeting to discuss developer performance.

Andre Onuki had the brilliant idea of weighting the DE and PE so as to count overall experience more than platform experience.

Eunju Namkung also provided a number of suggestions, including the use of a pivot table to compare team CDIs.

Eugene Medynskiy inspired me to document this approach and reviewed this article.

Maria Canfora provided much needed proof reading.