Evolving the Technical Interview

(Author's Note: This is the second in my series about expanding engineering teams. See my previous article The Consolidated Developer Index: Measuring an Engineering Organization's Power for more help in this area)

Why a technical interview?

In my 30+ year engineering/engineering management career, I've hired 100+ developers and interviewed hundreds more. It would be an understatement to say I have some experience when it comes to hiring technical talent. But that experience came with plenty of mistakes. I still remember my first hire back in 2000, seemingly days after I'd been hired myself into my first management role. At the time, there were few established interviewing methodologies I could look to for how to hire technical talent.? All I had to go on were my own interviews as a candidate. And in my experience at the time, there were two types of interviews:

1. Interviews conducted by a non-technical manager.? These focused on discussing my education, what I’d listed on my resume as past experience and other non-technical topics.
2. Interviews conducted by a technical lead.? These tended to be very focused on the? specific technologies I’d worked with, as illustrated on my resume.

As an budding (and naive) engineering manager, my blueprint was really number two above.? What I discovered is that this did very little to predict how good a hired developer would be at their day to day activities.? This led to some unfortunate situations where I had to let engineers go after six months who frankly had no business in the coding profession in the first place.? These people where good at memorizing the? technical buzzwords and their meanings very well, but when the time came for them to actually perform, they fell short.

Over the years after that I developed the technical interview I’m going to outline here.? I did not do this in a vacuum.? Much of what I’ve outlined here was refined by me along with whatever peers and team members were part of the collective hiring team at the time.?

Many of the engineers I’ve hired over the years are now very successful Principal Software engineers or Technical managers/executives in their own right.

I hope what I’ve outlined will give budding technical managers a starting point for interviewing technical resources.

Pre Interview Considerations

The panel interview is best

I recently had coffee with a former employee who was scheduled to interview with one of the "big" technical companies.? He was concerned because he’d been told that the technical interview would be 4+ hours long!? I've been there.? One of the worst technical interview experiences I've had ended up being five hours long.? By the middle of the third hour I was frazzled.? Having to perform technically to a group of strangers is much harder work than doing a regular coding job.? In fact, I think 1 hour of interviewing is worth at least 3 hours of coding, architecting, etc.? So imagine how frazzled a candidate is by the end of the fourth (translated to twelfth) hour of an interview.

The reason my interview was so long is that I ended up interviewing with three sets of two engineers, each for 1.5 hours.? I did end up getting that job and I launched a crusade in that group to find a way to shorten future interviews.

One way we did this was to structure the technical portion of the interview to be a full panel.? The ideal panel size is 3-5 with a tech lead or technical manager leading.? The ideal length for the technical portion is about 1.5 hours.? I believe that anything beyond this has diminishing returns because candidates, who are already stressed out from the experience, are going to deteriorate quickly after this.

Before the face to face: The take home problem.

Before a candidate makes it to the technical interview, they need to complete a take home problem.? The take home problem is designed to weed out candidates who’s coding skills have not progressed to the desired level.

The key to ensuring the candidate doesn’t cheat in this phase is to have a group of engineers on the team responsible for reviewing take home submissions.? I like to use the pool of mid-level engineers on the team to do this, under the supervision of engineering leads.? This is a two step process:

1. The problem should be reviewed for completeness, functional merit, style, structure and effort.
2. A thirty minute zoom call should be scheduled with the candidate.? The primary objective here is to have the candidate walk the interviewer through the problem.? The interviewer should be able to determine if the work presented was actually done by the candidate.

Technical Portion: Part One

Step One: Start off easy.

After no more than ten minutes of introductions and light discussion, we start the technical portion of the interview.? I like to start with something really easy.? Fizzbuzz, string reversal or some other quick solution problem to get the juices flowing. I ask almost every candidate to complete this portion, regardless of their experience level, because we'll build on it in Step Two (and in all honesty I've weeded out some "seniors" at this step).? Here’s the prompt I like for fizzbuzz:

• Iterate from 0 to 99.
• If the number is divisible by 3, print "fizz"
• If the number is divisible by 5, print "buzz"
• If the number is divisible by both 3 and 5, print "fizzbuzz"
• Wrap this code in a method/function and if the target language requires, class.?

Here is one way this problem might be solved:

package org.nomansys.interview;

public class FizzBuzzer
{
    public void printFizzBuzz()
    {
        for(int i = 0; i < 100; i++)
        {
            System.out.println("i: " + i);
            if(i % 3 == 0 && i % 5 == 0)
            {
                System.out.println("FizzBuzz");
            }
            else if(i % 3 == 0)
            {
                System.out.println("Fizz");
            }
            else if(i % 5 == 0)
            {
                System.out.println("Buzz");
            }
            System.out.println("");
        }
    }
}        

public class FizzBuzzRunner
{
    public static void main(String[] args)
    {
        // Part 1
        System.out.println("Running Part 1");
        new FizzBuzzer().printFizzBuzz();
    }
}        

Step Two: Let's add a challenge.

If your candidate is anything but a coding novice, they are probably thinking this interview is going to be easy.? Let’s add a bit of a challenge though to what we’ve done so far.

Interviewer: So where is your FizzBuzz method printing to?

Candidate: Uhh, the console?

Interviewer: Correct.? Now I'd like you to modify your program to allow the location we are printing to to be controlled by the caller.? We may, for example, want to print to a file or a database or Twitter/X or some other location that is invented in the future.

At this point, about half of the candidates I’ve given this problem to have modified the method signature to pass in a variable that Fizzbuzz would then check to decide where to print to:

public void printFizzBuzz(WhereToPrintToEnum whereToPrintTo)        

I’m now forced to crush their hopes somewhat when I point out that this solution will not be a very good separation of concerns.? It will force us to have to change our method every time we want to add a new place to print to.? What if we want to release our FizzBuzzing method/class in a library, to be used by someone without access to the source code?

Here’s one that I’ll accept and would have worked even before Java added Lambdas in JDK 8:

package org.nomansys.interview;
/**
  * Implement this interface to provide a concrete Sting printing method.
 */
public interface StringPrinter
{
    public void print(String target);
}        

public class FizzBuzzerWithInterface
{
    public void printFizzBuzz(StringPrinter printer)
    {
        for(int i = 0; i < 100; i++)
        {
            System.out.println("i: " + i);
            if(i % 3 == 0 && i % 5 == 0)
            {
                printer.print("FizzBuzz");
            }
            else if(i % 3 == 0)
            {
                printer.print("Fizz");
            }
            else if(i % 5 == 0)
            {
                printer.print("Buzz");
            }
            System.out.println("");
        }
    }
}        

package org.nomansys.interview

public class ConsoleStringPrinter implements StringPrinter
{
   @Override
   public void print(String target)
   {
       System.out.println(target);
   }
}        

package org.nomansys.interview;

public class TwitterStringPrinter implements StringPrinter
{
   @Override
   public void print(String target)
   {
       //Send String to Twitter
   }
}        

Of course, a shorter approach a candidate might provide takes advantage of OutputStreams or Lambdas:

import java.util.function.Consumer;

public class FizzBuzzerWithConsumer
{
    public void printFizzBuzz(Consumer<String> printer)
    {
        for (int i = 0; i < 100; i++)
        {
            printer.accept("i: " + i);
            if (i % 3 == 0 && i % 5 == 0)
            {
                printer.accept("FizzBuzz");
            }
            else if (i % 3 == 0)
            {
                printer.accept("Fizz");
            }
            else if (i % 5 == 0)
            {
                printer.accept("Buzz");
            }
            printer.accept("");
        }
    }
}        

package org.nomansys.interview;

public class FizzBuzzRunner
{
    public static void main(String[] args)
    {
        // Part 1
        System.out.println("Running Part 1");
        new FizzBuzzer().printFizzBuzz();

        // Part 2 Solution 1
        System.out.println("Running Part 2, Solution 1");
        ConsoleStringPrinter consoleStringPrinter = new ConsoleStringPrinter();
        new FizzBuzzerWithInterface().printFizzBuzz(consoleStringPrinter);

        // Part 2 Solution 2
        System.out.println("Running Part 2, Solution 2");
        new FizzBuzzerWithFunction().printFizzBuzz(System.out::println);
    }
}        

A successful solution to part two of our problem demonstrates object oriented problem solving skills, which are much harder to learn by rote and to reapply than if I were to ask syntax or configuration questions about specific technologies.? I’m not trying to hire engineers who know a specific technology.? I’m trying to hire engineers who can solve a problem with any technology!

I would expect the following depending on developer level:

Junior Developer: Should do fairly well on step 1.? Will struggle coming up with step 2. In this instance, I like to provide solution 1 above to them and see if they understand it.

Mid-level Developer: Should have no problem with step 1.? May need some coaching to get through step 2.

Senior Developer: No problem with Step 1.? Should hit on one of the two solutions to step 2.

Principal Developer/Technical Lead: No problem with Step 1.? Should choose one of the two solutions to Step 2 and ideally suggest the other as a possibility.

Next

In Part 2, I’ll conclude with the big problem.? This problem will have the candidate build a small subsystem to be reused extensively by the functional modules in our codebase.? Candidates will need to have knowledge of data structures, concurrency, error handling and other slightly more advanced topics to impress us in this part of the interview.

Acknowledgements

I'd like to thank Eric Eldard , Eugene Medynskiy , Yinchong Han , Joe Matar and Rosanne Callens, Career and Success Coach for suggestions, corrections and general help in producing this. I'd also like to thank the countless individuals who gave their time to sit through one of these interviews!