The Impact Case Studies and Technical Assessments are Having on the Software Engineering Interview Process

The labor market is on fire right now and if you're a software engineer, everyone wants to speak with you. But I wanted to offer a little bit of caution should you undertake an active job search. I'm starting to see the interview process for software engineers (and Product Managers for that matter too) get extended a good deal beyond what it used to be. And it's causing some real fatigue that's impacting the interview performance. I remember a time not long ago when the interview process for a software engineer used to be the following:

1. Introductory call.

2. Technical Deep dive (Historically has been a 60-90 minute phone screen)

3. Final onsite.

But what I've seen develop in the past year has been the emergence of take home assignments, technical projects and case studies. A software engineer will do the introductory call and then even have a phone screen with someone inside the engineering department but from there, instead of moving to an on-site, they do a project. Language is important and the verbiage of this next step has changed as well. For years, it was universally known as a code test but now it's often referred to as a technical assessment or a case study. And when I speak with engineers about their status on a project, they referenced having worked multiple hours on it. I remember a time when the average duration for a code test was approximately an hour. 

The result of this change in process is that the amount of time now invested by software engineers has increased significantly. Let me paste an email that I received from a software engineer who had recently been on a job search. He was actively looking for a new position and he decided to engage with five companies at one time. True, that is a pretty heavy load to begin with but in the original model I describe above, with some good time management, it can be pulled off. But look what he had to say about his job search and where he stood after four weeks of interviewing, 

Quick update for you, Mark. I've come to understand that while I am an experienced developer, I am an inexperienced job hunter. Most importantly, it would have made quite a difference had I known how much time these projects would have taken before jumping in. I realize now that I essentially needed to be training for interviewing. 

?

My general strategy of going through the rounds with multiple companies simultaneously was exhausting, to the point where I have significant interview fatigue. If none of these initial companies are going to work for me, I will approach again much more deliberately, selecting single companies that I know I'd want to work for and going one at a time. I have never experienced something like this before. 

This note was from a very strong Sr. full-stack engineer and you can tell in his words that he got caught off guard with all of these simultaneous interviews and the projects required for each of them. I even feel the fatigue in his email to me. And it's not just engineering either. I know of a software company that is looking for a Sr. Product Manager and their process is as follows, 

1. Introductory call with VP of Product - one hour
2. 2nd round phone call with VP of Engineering - one hour
3. Product Manager case study - Time investment is 3-4 hours. 
4. Case study debrief with Product management team - three separate one hour individual phone calls. 
5. Final call with Head of Product Design - 1 hour. 
6. Roundtable to consider yes or no on an offer. 

If you add this up, we're talking approximately ten hours spread out over 1-2 weeks. We had a candidate for the opportunity who was really interested and he did the first two rounds. However, when the case study was presented to him, he had this realization that he was looking at another 8-10 hours investment of his time and he backed out. It was disappointing to lose him but what also stung was that the client was upset with us because they felt we were presenting candidates that weren't sincerely interested in the opportunity. On my end, I have to admit that if we present an interview process to a candidate that requires a time investment of 10-12 hours, my sympathies lie with the candidate for sure. 

So for you, the job seeker, what can you do to keep yourself from getting caught up in an interview process that takes such a big investment of time and thought on your end? When you apply for a position and the company reaches out to request an interview, send some availability and do this first call with no questions asked. Since this first call will be introductory in nature, you will have the opportunity to ask some questions and please make sure this is one of them,  

I've really enjoyed this conversation and I am looking forward to exploring this opportunity at your company. If you don't mind, I was curious about the process from here. My work schedule is demanding at the moment so in addition to the schedule of virtual interviews, do you have any details on projects or case studies that might be assigned to me? For instance, do you have a rough estimate on how much time is required of them? If you don't have that information, perhaps the hiring manager would? I ask only because I would like schedule appropriately on my end to make sure I put my best foot forward. 

The company should be willing to disclose details on how much time the projects take as well as how many followup calls will be asked of you. This information will give you a good idea of what's expected of you from this company. We have a lot of "new normals" out here right now and these case studies/take home projects seem like they might be around for a while. The net result will be a smaller pool of companies you can simultaneously interview with and I have to admit that is a bit disappointing. You'll just have to do more front-end research on the companies and target the ones you find most interesting. 

Addendum: A quick addition to this post are the words from Sankalp Acharya. He is recognized as an elite engineering leader here in Austin and here was his counter. I liked it so much that I wanted to put it in here. Please see belowThanks, 

Mark Cunningham

Technical Recruiter

512-699-5719

[email protected]

https://thebiddingnetwork.com

https://markcunningham91.blogspot.com

https://www.dhirubhai.net/in/markhc

While I understand the imposition this makes from the candidate POV, the flip side, from the hiring manager POV, is that it is a huge opportunity cost of company time to make a hiring mistake.

Maybe you are excited after the phone screen and bring someone in for an onsite and realize in the first 10 minutes that it’s not going to work out. Your interviewers have now lost precious heads-down time as they plod on through the interview, seeking to preserve candidate experience.

It’s even more expensive to get to the end of the process and realize there’s no clear answer — and so maybe you take a chance (because you needed people 2 quarters ago), and then 3-6mo later you start kicking yourself because the now-hired candidate is clearly not raising your bar and is at best struggling.

Ultimately, there are a lot of software engineering positions out there, and our hiring processes across the industry generally are quite inefficient at separating the wheat from the chaff.

I’m not suggesting that a lengthier process is “better” — I’m just saying, from either the candidate or the company POV, it’s hard.

And if companies are choosing to take their time with candidates, candidates should do the same thing for themselves (after all — this is a business decision) when vetting prospective companies.