The AI/Human Balance

We continue to build Vina - our AI interview assistant for hiring managers.

If you have not read about it, here is the first write-up on Vina.

Asking AI to review if an interviewee's answer is not a matter of getting a 'Good/Bad' verdict – which itself is a hard problem. With Vina, we are trying to assess if a candidate is the 95th percentile or 50th percentile. We are also asking Vina to come up with scenario-based questions that even experienced interviewees have to deliberate a lot before answering.

All of these are not problems LLMs are meant to solve. But the silver-lining is that humans are not good at interviewing either :).

90% of us don't have a set of values and scenarios we would evaluate a person against. Only few interviewers adapt their questions based on the caliber of the answers. By the time we wrap up the interviews, we have a 'summary feeling' with all our biases and baggages. The easy rationalization for bad hires is that an interview is not enough to judge a candidate's fit.

This is why, with all the limitations of LLMs it still makes sense to lean on them for evaluation facets for a role, the questions as they relate to seniority levels, and relying on the LLM's verdict as one more reference to check against our own evaluation.

Vina is still being built but we are summing up this week's lessons here:

Leave your questions and comments.