AI IN ASSESSMENT (It's not fair)!
Organisations have short attention spans. 3 months ago the worry was that AI would replace all of our jobs and now, predictably, attention turns to the poor human being. These pesky humans might be using AI to help them get jobs!! Well, well, well! Who knew that humans might turn technology to their own advantage. How dare they use AI to help with Interviews or assessments... and land a job.
There are so many things to unpack here, and I will try to summarise and always welcome the comment and debate.
Firstly, history suggests that this is nothing new. For all the years I have been in Resourcing candidates have always attempted to gain an advantage. The advent of Glassdoor and other social media meant that candidates have shared interview and assessment experiences, rehearsed answers to questions to help them prepare... Is this cheating? or unfair?
In addition, we know that for years middle class parents have helped their offspring write personal statements for University applications and cover letters for job applications. We even might have used our networks to leverage work experience opportunities for our precious darlings. - Cheating?
I do know of parents who have part written assignments for their children while at college or university. Will these candidates get a better job because of that help? Maybe - does the parent consider it cheating?
An academic friend of mine suggests that if you can't tell whether AI has written the essay, you are asking the wrong question. Perhaps this is true for assessment and selection. Why are we worried about candidates using tools for applications when the expectation is they will use them in the workplace.
Let's have a reality check of where we are.... Whilst many organisations are using psychometric tools for assessment, most organisations still really on largely unstructured interviews (especially at mid and senior level) to hire those critical positions. Occupational Psychologists will tell you that the predictive validity of this is 0.5 or in layman's terms no better than flipping a coin. Yet we persist with this approach because a) The more senior we get, the better we think we are at judgement, b) It cheap (on the surface) c) A lack of humility to think we may be fallible in some way.
Of those organisations who do use assessments - most use them for volume hiring (Early careers, front line roles) and generally speaking there has been success in terms of efficiency, improvements in experience and outcomes. I think if we were honest the main driver of this is generally efficiency/cost - eg the business case is made because it will mitigate or save cost primarily. This is not to say that the outcomes generally reported are much better on every metric (Quality, diverse hires, retention) but the value of this approach is generally difficult to communicate across organisations, HR included.
领英推荐
I am no technologist or Occupational Psychologist, but is clear to see that AI poses challenges to those who provide these tools in terms of access and security. Naturally, we would expect them to develop tools that are robust but the challenge isn't all for the vendor.
Before organisations cry foul, there are a number of challenges that need to be addressed that, in reality, the majority haven't addressed - a few are listed below.
Work Design - in the world of super machines, are we being mindful of what we think humans are good at and can be doing versus the machines. Even before AI most organisations I worked in never gave this any thought.
Defining What Does Good Look Like - Even before this current debate has raged most organisations don't define what good looks like. Still stuck in the I'll know it when I see it approach to hiring - which we know leads to recency bias, poor candidate experience and a lack of diversity regardless of the tools you are using.
Assessment Design - Following on from the previous points. If we know some of the work is going to be augmented by machines ensure we are assessing the right things - the human attributes that count. In a past life, I was always struck by a manager's insistence of using numerical reasoning tests in roles where there was no need to make any calculations at all....
Equity and Fairness - The question, for me, that the current debate raises is about fairness and access. Access to AI tools, currently, is not equal. However, over all years I have been in HR - this has always been the case. Assessment isn't the only lever we need to pull here but it is an important one.
Before organisations externalise their fears and ire on the tools or vendors, it is important to consider current approaches. Does the advent of AI make it any different to the current state of affairs? Maybe AI just begins to shine sharper light into practices we would rather not show to the outside world.
Co-Founder & CEO of Clu ?? | #SkillsLiteracy? | AI Activist | Use Clu to build a people strategy around the skills AI won't replace
1 年The way we assess is not to select in. It’s to select out. Ethically, and specially from an inclusion stand point, this is conflicted, and AI only exasserbates this: so new approaches are needed if the end goal is equity of opportunity and a access to a broader talent pool.
Director
1 年Thanks, Jon. That was an interesting read.
LinkedIn Top Voice ????♀? What's Neurodiversity? ??♀? Invite me: speak & consult ?? Listen 2 my Podcast 'Neurodiversity - with Theo Smith'?? Order my award winning book: Neurodiversity @ Work ??
1 年An issue for candidates; a lack of Equity, the free chatgpt 3.5 is easily outperformed by the paid version of chat gpt4, and improved again with additional well-defined prompts, many candidates and indeed employees, wont realise the significant difference this makes. On the flip side Chat GPT will force innovation is assessment which in my view is much needed, moving to skills-based assessment, as they are far more difficult to utilise generative ai tech and tools. And by the way skills-based assessment will ultimately be better for everyone, but specifically, those who've previously seriously struggled with traditional assessment methodology. I'm fascinated by the opportunity ai is already providing but like anything new, it can be used and utilised in many ways, both good and bad, so let's get inclusion and equity right in at the start of the design and thinking!
Podcaster at This Isn't Working
1 年We hosted this discussion for our members last week - fascinating stuff! I'll post a short summary on here tomorrow. The premise of the session was: "AI has turned assessment on its head... but, let's be honest, was it really working anyway?" You can probably guess the answer... ??
Proud dad / Helping organisations navigate, HR Tech, GenAI and AI Compliance #AMS Verified
1 年AMS Claudia Nuttgens are excited to be speaking at the event with you Jon ?? ??