What's my AI brain thinking about today?

I thought in this next article to focus not on an established female leader in the field of AI, but a promising future one. Still with the same prompt “What’s my AI brain thinking today?,” I asked MIT student, Elise Harvey her thoughts- knowing they likely aren’t about corporate culture, pondering where in the organization AI should sit, but perhaps something more tech-centric, tangible, and personal. Slight disclosure here- I’ve known Elise since she was younger- playing AAU basketball with my daughter:) Now an MIT Senior and Women’s basketball captain, Elise is the epitome of a hard worker, and a wonderful human being. I was so pleased when she was able to carve out a few minutes between her studies and internship at Amazon to respond to the prompt. Love her focus here on ethics and bias in AI!

So, Elise, what's your AI brain thinking about today?

Elise Harvey: In one of my current classes, we discussed the Language Agency Bias Evaluation (LABE) benchmark, developed by Yixin Wan and Kai-Wei Chang from UCLA. LABE defines a framework for measuring gender, racial, and intersectional language agency biases in Large Language Models (LLMs). Through their research, Wan and Chang discovered that LLMs tend to exhibit more language agency bias than people, and that prompt-based methods intended to reduce these biases can sometimes worsen them.

I find the LABE framework interesting because of its potential impact on the future of LLMs and testing practices.

While most people are aware that LLMs can be biased, the idea of automating tests and quantifying specific biases in these models introduces a new approach to fairness.

This approach is more rigorous across intersecting demographics and has the potential to change industry standards. I believe LABE is one of several scalable benchmarks that could transform the way we evaluate and address biases in LLMs. By revealing biases LLMs learn from training, these types of benchmarking tools help us advocate for higher standards in fairness and accountability in AI systems.

Bio: As a senior at MIT, Elise is pursuing a degree in Artificial Intelligence and Decision Making with a concentration in Science, Technology, and Society. Her journey into AI began during an internship at The Coding School, where she helped develop TRAIN, an AI/ML initiative designed to make technical education accessible. Since then, she has built her expertise through coursework and two internships at Amazon's Artificial General Intelligence organization as a software engineer.

Beyond academics, Elise is also a leader on the court as the captain of MIT’s women’s basketball team, demonstrating her ability to balance technical rigor with teamwork and leadership.