Artificial Intelligence: The Next Frontier of Language Assessment

“As a technologist, I see how AI and the fourth industrial revolution will impact every aspect of people’s lives.” – Fei-Fei Li

Artificial intelligence (AI) is no longer limited to science fiction novels and the imagination. As we take steps closer to a full integration of AI and processing of information it can seem unsettling not knowing how it will affect the education realm itself. How will we approach something as complex and “human” as language testing using AI? Will the emergence of AI be a positive or negative influence on the way we approach it, and will it allow us to refine the testing process itself? Pearson’s very own Director of Academic Standards and Measurement Dr. Rose Clesham discussed these questions and more recently at events in Singapore.

Dr Chlesham’s presentation entitled ‘Artificial Intelligence: Changing the Face of Formative and Summative Assessment’ outlines how computer-based tests such as the PTE Academic are at the forefront of harnessing the transformational power of AI in language assessments.

The event, organised by PTE Academic at the Hilton Hotel, was well attended by clients and stakeholders from universities, colleges, language schools and education agents. Presenting as a keynote speaker, Dr Clesham explores new insights and revelations on the power and potential of AI to positively influence the way language assessment is conducted. She believes in the potential for AI to improve and refine the way language assessment is conducted globally.

When discussing AI and its integration with education, many would be unsure of its suitability  with language testing due to its complexity. Dr. Clesham, whose area of research is learning assessment, revealed to the conference that she too was initially in this line of thought. After all, AI is still not considered to be a genuine replacement for human intellect, how would a computer be able to gauge and asses the nuances and rhythm of human language?

Dr Clesham says that her views shifted as she engaged with AI and studied the applications. It was found that it was the complexity of language testing that made it a perfect fit for an AI to work with. For high stakes language testing on a global scale, there is a need for efficient, secure, and fair testing conditions which also adhere to a golden standard. Computer based tests that are facilitated through AI technology allow for these strict standards to be met, and provide every test taker to undergo the same experience.

Dr Clesham proffered several complications faced by human markers. Those issues include:

• consistent application of rubrics by many raters,
• use of extreme categories of the marking scale i.e. perfect or zero.
• cross-contamination among different traits (i.e. fluency or vocabulary contaminating grammar, etc.),
• potential rater bias due to handwriting, spelling, disagreement with ideas (W&S), gender, culture, ethnicity, appearance, accents, tone of voice, personality, interactional style etc.

These issues can be avoided utilising AI. We rely on computers to perform routine tasks as they don’t get bored, they make fewer mistakes, and they are unbiased and unswayed by emotion or prejudice. By allowing AI to filter out the potential for human error we can provide more accurate test results, and in turn we can monitor the AI’s ability to give fair assessments.

Dr Clesham’s presentation was met with ripples of recognition and relatability throughout the presentation. One of these powerful ‘Ah-Ha’ moments came with the presentation of a side-by-side comparison of a proficient vs. non proficient speaker. The visualisation of sound wave measurements, and the explanation of the algorithm measuring fluency, accent, errors, and WPM, elicited nods of recognition and understanding from the crowd.

Using PTE as an example, she stresses the importance of validating the AI marking engines by correlating and training them with massive inputs from expert markers. PTE Academic uses human markers as a safety net in the process; when the AI is presented with unrecognisable speaking or writing then the material referred to this safety net. This ensures that the test taker’s results are fair and balanced, and also helps to educate, validate and improve the AI marking system. In other words, if the AI is unable to process the information then expert markers step in to educate the AI.

For those amongst the audience concerned with their roles being replaced by AI, Dr Clesham offered this advice:

“Teachers will learn to embrace technology that supports them and enables them to better fulfill their primary job, educating learners. AI is not here to replace humans, there is nothing artificial about it, it is made by humans and thus ultimately designed to positively impact human lives and society.”

In other words, we must embrace AI and view it as a tool that will enable educators and testing to reach their full potential.

After the talk, there was an interactive session where attendees were able to express their perspective on what they saw as either the strengths or drawbacks in using AI for language testing. The Q&A session during this time produced some useful insights, and some of the preconceptions and enduring notions that will need to be overcome in educating the market moving forward.

These sessions by Dr Rose Clesham gave the opportunity to better understand and explore this new era of education and the developments in the future for language testing.