Safeguarding Interviews from ChatGPT Misuse

Shannon Ogborn here, your host of Offer Accepted. In this episode of Offer Accepted, I sit down with Aline Lerner , Founder & CEO of interviewing.io, and Mike Mroczka , Software Engineer (ex- 谷歌 ) and Dedicated Coaching Interviewer, to tackle the growing challenges AI tools like ChatGPT are creating in technical hiring. We talk about their study on cheating in interviews, how these tools can disrupt hiring processes, and what recruiters and hiring teams can do to adapt.

From rethinking interview questions to designing fairer, more transparent processes, Aline and Michael share practical insights to help teams navigate these new challenges while maintaining hiring integrity. Here are my top takeaways:

Takeaway 1: Rethink Generic Questions to Reflect Real-World Challenges ??

In a study evaluating how difficult it is to cheat in technical interviews using ChatGPT, Interviewing.io found that no interviewee was caught cheating, but verbatim questions had a 73% pass rate compared to just 25% for custom questions. This highlights the risk of relying on predictable problems and the opportunity to craft challenges inspired by your team’s workflows. By using the “algorithmic kernel” of the team's recent challenges, you can design more accurate evaluations that align with the job and reduce the impact of cheating with AI tools like ChatGPT.

Why It Matters:

Generic, widely available questions inflate pass rates and fail to assess candidates’ ability to handle real-world tasks. Aline and Michael emphasized how custom challenges lead to more accurate hiring decisions while improving the candidate experience by showcasing the meaningful problems the team is solving. This improves the experience for interviewers as well, as they are more likely to maintain engagement when interview questions feel relevant, unique, and indicative of their day-to-day work.

Quick Tips:

• Use Real-World Kernels: Extract algorithmic concepts from recent, real-world challenges your team has tackled and incorporate them into interview questions.
• Avoid Predictable Questions: Steer clear of verbatim questions from sources like Leetcode to reduce the likelihood of inflated pass rates.
• Refresh Questions Every 6 Months: Following Aline and Michael’s advice, update questions biannually to stay ahead of leaks and adapt to team evolution.

Takeaway 2: Shift Focus to Problem-Solving Over Polished Answers ??

Interviewing.io’s study revealed that while AI tools like ChatGPT can produce highly polished answers, they often lack the depth and nuance of genuine problem-solving. This creates an opportunity to adapt interview questions that emphasize thought process and adaptability, helping teams distinguish authentic skills from AI reliance.

Why It Matters:

Customizing questions to highlight problem-solving reveals how candidates think through challenges, even when AI-generated responses are involved. Aline and Michael emphasized the importance of evaluating how candidates approach complex problems and adapt their reasoning when requirements evolve. This not only identifies strong problem-solvers but also creates a more engaging and relevant candidate experience.

Quick Tips:

• Use Process-Focused Questions: Ask candidates to explain how they arrived at their solutions, step by step, rather than just presenting the final answer.
• Introduce Evolving Scenarios: Design challenges that change midway, requiring candidates to adjust their approach dynamically.
• Spot Check AI Reliance: Include layered questions that go beyond surface-level answers, ensuring candidates demonstrate understanding rather than repetition.

Takeaway 3: Demystify the “Why” Behind Your Technical Interviews ??

Candidates often feel frustrated and disillusioned by hiring processes that seem opaque or arbitrary, especially for technical roles. Aline and Michael highlighted how creating a transparent process — where candidates understand the purpose of each stage and how they’ll be evaluated — fosters trust and reduces anxiety.

Why It Matters:

Transparency helps candidates focus on showcasing their skills and knowledge rather than deciphering unclear processes. A more open approach also reduces negative sentiment, strengthens employer branding, and encourages meaningful engagement during interviews.

Quick Tips:

• Clarify the Intent: Clearly explain the purpose of each technical interview stage and what skills the questions aim to evaluate.
• Provide Evaluation Criteria: Share a high-level overview of how candidates will be assessed, ensuring fairness and consistency.
• Deliver Actionable Feedback: Where possible, offer specific feedback post-interview to help candidates improve while leaving a positive impression of your company.

What Hiring Excellence Means to Aline + Michael

For Aline, hiring excellence isn’t just about finding candidates with the right skills—it’s about identifying those who’ve overcome challenges and grown in meaningful ways. She focuses on "distance traveled," which reflects how far someone has come in their career relative to their starting point. This approach reduces bias and opens doors for candidates from non-traditional backgrounds.

Michael added that hiring excellence requires empathetic communication at every stage, from interview prep to rejection feedback. He emphasized that even when candidates aren’t selected, they should leave the process feeling respected and valued, which strengthens your employer brand and leaves doors open for future opportunities.

Michael's Recruiting Hot Take ??

Resumes, according to Michael, aren’t a true measure of a candidate’s potential. He argues that too much weight is placed on what’s written on paper, when the real insights come from having direct, meaningful conversations with candidates. Michael advocates for reducing reliance on resumes and instead prioritizing skills assessments, real-world problem-solving, and structured interviews to get a full picture of a candidate’s abilities.

You can check out Aline + Michael's full episode on Apple, Spotify, YouTube, and our blog.

Up next, a refresh on one of our most insightful and actionable episodes. Subscribe to get the latest directly to your inbox.

Until next time,

Shannon Ogborn

Community @ Ashby