A Coding Test Won’t Tell You if Someone Can Write Software; Here’s Why

“To Avoid Criticism, Say Nothing, Do Nothing, Be Nothing”

--Elbert Hubbard

"One half-court swish throw does not an NBA all-star make and vice-versa."

--Jackson Wayfare

I wrote here that I don't like HackerRank, leetCode, testGorilla or any online programming tests tools to make hiring decisions. Someone mentioned that they read my post--right before I took the "max-increase-to-keep-city-skyline" test below. One of the reasons I mentioned it is that the tools for testing are almost cripple ware. But, that's not a great reason not to use a coding test and not a sufficient reason at least for many people.

A little background about me. I was on the ACM programming team at Michigan State University. This was the team the year after MSU won second in the world beating UofM, Stanford, Harvard, MIT, and everyone else except the University of Melbourne. My team, my year, didn't medal as it were. (This is like making it to the majors without a trophy.)

?So, I know I am capable of occasionally spitting out puzzle code very fast, but sometimes its a slow mushy meh! This is also not a reason to not like puzzles as coding tests. So, I thought about it and tried to put a pin in what I don't like about puzzle code.

First, and foremost, timed programming tests might get you a fast and cheap answer, hacky code. Is that desirable in a programmer? Most likely such code is a big ball of technical debt. People who are good at puzzles may not be good at writing software at all. Puzzles are not software.

Below is leetCode's City Skyline puzzle. In this instance the puzzle is poorly written and misleading at about page of instructions. What its is really asking among all of the gobbledegook is "how much can the number in each cell in a grid be increased so as not to exceed the smaller of the maximum vertical and maximum horizontal cell number"? So, if across the horizontal the largest number is 8 and the vertical is 9 then the cells in that row and column can all be increased to 8. So if a cell is 3 and the max is 8 then the increase is 5 for that cell. The difference. The problem is the sum of all differences to get each cell to its maximum allowable height.

The puzzle seems to have been written by an end user with no BA filtering in between. Its an over complication of metaphors like "skyline", "building", and "blocks" and such. Its not a user story or use case; its an explanation in dire need of process re-engineering. The example is how a user thinks about a problem, which is sometimes what you get. Sometimes you get gobbledegook even from BAs. When building systems something written that convolutedly should be kicked back to a BA.

Assume you have muddled specifications like driving directions in Tennessee. Further assume you figured out the directions in "southern", including the right turn past "my cousins house near the Walmart but not the one in Smyrna; if you get to Smyrna you've gone too far". Now, you have a 20 minutes or more to figure out how to code the problem. If you haven't run out of time then you might hack together a solution. What you don't get is software.

A reasoned solution that you could build software on follows in Listing 1. (I will include the hack-tastic solution in another post.)

Listing 1: Solution to the CityPlanner

public static class CityPlanner
{
    public static int[,] IncreaseSkyline(this int[,] city)
    {
        for (int i = 0; i < city.Length; i++)
        {
            int j = city.GetJ(i);
            int k = city.GetK(i);
            int height = city.GetHeight(i);
            int minmax = city.GetMinMax(i);
            city[j, k] = minmax;
        }
        
        return city;
        
    }

    public static int Adjustment(this int[,] city)
    {
        int adjusted = 0;
        
        for(int i = 0; i < city.Length; i++)
        {
           adjusted += city.GetMinMax(i) - city.GetHeight(i);
        }

        return adjusted;
    }
    
    public static int BoundCol(this int[,] city)
    {
        return city.GetUpperBound(1) + 1;
    }

    public static int BoundRow(this int[,] city)
    {
        return city.GetUpperBound(0) + 1;
    }

    public static int GetMinMax(this int[,] city, int i)
    {
        return city.GetMinMax(city.GetJ(i), city.GetK(i));
    }
    
    public static int GetJ(this int[,] city, int i)
    {
        return i / city.BoundRow();
    }

    public static int GetK(this int[,] city, int i)
    {
        return i % city.BoundCol();
    }

    public static int GetHeight(this int[,] city, int i)
    {
        return city[i/city.BoundCol(), i%city.BoundCol()];
    }
    
    public static int GetMinMax(this int[,] city, int row, int col)
    {
        return Math.Min(city.GetCol(col).Max(), city.GetRow(row).Max());
    }
    
    public static IEnumerable<int> GetRow(this int[,] city, int row = 0)
    {
        for(int i=0; i<city.GetLength(1); i++)
            yield return city[row, i];
    }

    public static IEnumerable<int> GetCol(this int[,] city, int col = 0)
    {
        for (int i = 0; i < city.GetLength(0); i++)
            yield return city[i, col];
    }
}

#region private::Tests

private static readonly int[,] cityScape = { { 3, 0, 8, 4 }, { 2, 4, 5, 7 }, { 9, 2, 6, 3 }, { 0, 3, 1, 0 } };
private static readonly int[,] cityScapeRectangle = {{3, 0, 8},
                                                     {2, 4, 5},
                                                     {9, 2, 6},
                                                     {0, 3, 1}};
    
[Fact]
void Test_Xunit() => Assert.True(
    CityPlanner.Adjustment(cityScape) == 35);
       
[Fact]
void Test_GetRow() => Assert.True(CityPlanner.GetRow(cityScape, 2).SequenceEqual(new int[] { 9, 2, 6, 3 }));

[Fact]
void Test_RowMax() => Assert.True(cityScape.GetRow(2).Max() == 9);

[Fact]
void Test_GetCol() => Assert.True(CityPlanner.GetCol(cityScape, 0).SequenceEqual(new int[] { 3, 2, 9, 0 }));

[Fact]
void Test_GetMinMaxInversion_Test() 
{
    Assert.True(CityPlanner.GetMinMax(cityScape, 0, 1) == 4); 
}


[Fact]
void Test_GetMinMax()
{
    Assert.True(CityPlanner.GetMinMax(cityScape, 0, 1)==4);
    
}

[Fact]
void Test_GetBoundColumn()
{
    Assert.True(CityPlanner.BoundCol(cityScapeRectangle) == 3);
}

[Fact]
void Test_GetBoundRow()
{
    Assert.True(CityPlanner.BoundRow(cityScapeRectangle) == 4);
}


[Fact]
void Test_GetRowLength()
{
    Console.WriteLine(cityScapeRectangle.GetLength(0));
    Assert.True(cityScapeRectangle.GetLength(0) == 4);
}

[Fact]
void Test_GetColLength()
{
    Console.WriteLine(cityScapeRectangle.GetLength(1));
    Assert.True(cityScapeRectangle.GetLength(1) == 3);
}

#endregion        

This code took more than 30 minutes to get to. The final result has very low technical debt. The code is highly refactored. Each algorithm has very few lines of code, excellent cyclomatic complexity, excellent maintenance heuristics, high composability, a good O(n) performance characteristics, doesn’t require useless comments, and is substantively tested.

One could argue that “it took too long”. I have never seen anyone running a stop watch on a single class. Most of the cost of code is in ownership and maintenance. Code like that above has a very low ownership cost (TCO). Further, if there is a bug people don’t have to mentally refactor or read comments to figure it out. The code is already refactored, and its hard to hide bugs in one or two liners.

Based on the Refactored code it looks like one could solve the main problem with a single function loop, LINQ most likely, and even a clever Regular Expression probably. The problem with coding monolithic algorithms is you wouldn't be able to do anything else with the code. The composability quotient would be zero.

In short, the code in Listing 1 will not keep you up nights and weekends and won’t result in your going to court. The code in Listing 1 is the code you want to see in software. Quick solutions to puzzles are parlor tricks.

[Final thought: The code in Listing 1 is about one factor away from a general solution to grow a matrix. Refactored general solutions produce cohesive, highly composable bullet proof code. Design a coding test for cohesion, composability, and testability and you’ve got a test worth knowing the results.]

[Disclaimer: Anyone can use these programming tests. I don't use them for hire decisions just insight into future training areas. Sometimes the puzzle results come slowly and can be frustrating. A capability is defined by a persons record not a moment in time. One time I did a HackerRank for Rocket Mortgage so fast--two in six minutes--their "AI" suggested to the RocketMortgage recruiter that I had cheated.]