DET Research & Development: November '24

Our team has been busy this fall, pushing the boundaries of language testing, AI-supported education, and assessment security. From new research on writing task length and detecting LLM-assisted cheating, to accepting applications for our competitive research grants, we’ve got a lot going on at the moment!

?? Research publications ??

How long should a writing task be?

DET researchers Ben Naismith , Yigal Attali , and Geoff LaFlair examined the impact of writing task duration on test scores, reliability, and validity. The results might surprise you!

→ Learn more about task length

Detecting LLM-assisted cheating

DET security experts Chenhao Niu , Kevin Yancey, Ruidong Liu , Mirza Basim Baig , André Kenji Horie , and James Sharpnack presented their paper, Detecting LLM-Assisted Cheating on Open-Ended Writing Tasks on Language Proficiency Tests, at the recent Empirical Methods in Natural Language Processing (EMNLP) ‘24.

→ Read the paper

Understanding our test takers

Allison Michalowski , Ramsey Cardwell , Steven Nydick , and Ben Naismith conducted an in-depth demographic analysis of DET test-takers, exploring why they take the test and how scores vary across groups.

→ Read the synopsis

?? Funding opportunities & calls for proposals ??

iRAISE workshop

This workshop Innovation and Responsibility in AI-Supported Education (iRAISE) workshop, organized by Jill Burstein and James Sharpnack as part of the Association for the Advancement of Artificial Intelligence (AAAI) conference, will explore novel research, methods, and tools in AI-supported education. Proposals are due Friday, November 22, 2024.

→ Submit a proposal

Competitive Research Grants

Duolingo is inviting research grant proposals that promote better language assessment related to the Duolingo English Test; Winners will receive up to $120,000 based on project scope. Proposals are due by Sunday, January 12, 2025.

→ Apply here

?? Conference highlights ??

European Association of Educational Assessment (EAEA) ‘24

Alina von Davier co-led a pre-conference workshop with Marie Wiberg and Jorge I Gonzaleo Gonzalez on Generalized Kernel Equating, based on her recent book, Generalized Kernel Equating with Applications in R.

→ Check out the book

European Association of Test Publishers (E-ATP) ‘24

Rose Hastings from our security team participated in a panel discussing operational considerations for assessment companies in light of the EU AI Act.

→ Learn more

Language Testing Forum (LTF) ‘24

The theme for this year's Language Testing Forum is “What do we want the future of language assessment to look like?: Taking stock of what we know now.” Ben Naismith and Sarah Goodwin will present their paper, Speech rate in L2 listening assessment: past challenges, future opportunities; Ramsey Cardwell and William Belzak will present their work, Examining the Equity Implications of Traveling for a Test.

→ Explore the program

?? Team updates & insights ??

DET's Research Governance Committee

Last month, We held our inaugural meeting of the Research Governance Committee, which will oversee proposals, reports, and publications from external researchers involved in our grants and commissioned projects.

→ Learn more about the RGC

The Art & Science of Learning podcast

DET’s Chief of Assessment Alina von Davier joined Kinga Petrovai, PhD ’s podcast to discuss her keynote speech at The Learning Ideas Conference 2024. Alina shared insights on creating digital-first assessments and the role of AI in adaptive and personalized testing.

→ Listen to the episode

You can find more DET publications on our research page, and learn about the impact we're having on the assessment industry on Test Center, the official DET blog!