The Secretary Problem: Mathematically Optimal Hiring Process

Do you know that "37 %" is the mathematically optimal solution for how many candidates you should interview before hiring someone, to maximize your chances of finding the best one in a fixed time or number of interviews? It can also be applied to how many interviews to give before accepting an offer or even in scenarios like, when to stop your search for finding a spouse or finalizing clothes to buy during rush in a sale.

This is a type of optimal stopping problem in algorithm design. It comes from statistics and operations research and is called the Secretary Problem, due to using example of hiring a secretary. Note that it assumes that once a candidate is rejected or not chosen for next round of interviews, they won't be available anymore. The optimal strategy is in the form of a stopping rule. You have to do a Look phase of conducting 37 % of the interviews and not choosing anyone, and then you have to Leap and choose the first one who is better than all the previous ones.?

37 % is actually 1/e, which is called the law of best choice. The main idea is that you need a sample of candidates to get information and set a benchmark for selecting one, and by rejecting candidates, your chances of finding the best among next ones decay quickly. Around 37% candidates is where the probability converges, and it's interesting that the success probability of finding best candidate, itself peaks at and is around 37 %. For smaller number of candidates between 3-7, the stopping percentage varies between 25-40 % and success probability can be up to 50 %, e.g. if there are 3 candidates then you can reject first one (33 %) and then choose the next one better than him/her and have 50 % success rate for finding best. There are some assumptions and limitations, and more information could be useful in making good decisions too. So there are variations of the Secretary Problem and alternate solutions.?

A better solution exists if you are not just comparing candidates with each other and ranking them, but you have a quantifiable measure of how good someone is in general, and an idea about the candidates distribution, assuming it to be a normal distribution. The problem will then become, what is know as a perfect-information game in Game Theory.? The measure can be anything like number of correct answers in interviews or test scores of a standard test that you conducted, based on your past interviews and hiring for same roles.?

You'll use a Threshold Rule in this case and set a threshold percentile considering the number of remaining candidates and select the candidate as soon as you find one above the threshold percentile e.g. for 5 candidates you can select immediately if they score above 90 percentile score and for 10 candidates you can select if above 95 percentile. The percentile required should drop at high rate when you are left with very few candidates. You can also look at the standard graph for optimal stopping thresholds for full-information secretary problem. The probability of success in selecting the best candidates comes to around 58 % in that case, which is significantly better than 37 % in the case where we had a no-information game. Still there are some limitations and assumptions, but I think these strategies can be useful in many cases.

If we assume that hiring is to be done based on test score only then conducting simultaneous online test and choosing the best candidate will easily give you a very high probability of choosing the best candidate. However in some cases, further interviews might be needed to judge and candidates may refuse your offers. Still, you are likely to have large number of candidates in such tests and you can contact the next best one with less fear of exhausting options. So those tests can be a pretty effective way, especially for hiring fresh graduates, if the tests are well designed.

I was introduced to the main idea of this problem by the book named "Algorithms to live by" by Brian Christian and Tom Griffiths.