Is AI in Higher Ed Admissions Good or Bad?

Admissions teams are faced with sifting through mountains of applications while navigating ever-changing policies and fighting with outdated technology. And sometimes, this results in relying on grades and test scores as means of making important admission decisions.

But what if AI could help? Not to replace humans, but as a supplemental tool that can help admissions teams in very specific ways.

Before we dive in, here’s what we are and are not saying in this article:

We are not saying that AI should:

• Replace admissions staff
• Make admission decisions
• Dictate policies and procedures
• Be the only means of applicant consideration

We are saying about AI in higher ed admissions:

• It should be used with caution and optimism
• It’s a supplemental tool—not a replacement
• Certain AI can help admissions teams with specific tasks
• AI can help standardize certain aspects, like interviews, and make them more equitable and accessible

Large Language Models & Chatbots

Large Language Models (LLMs) understand, interpret, and generate human language by analyzing and learning from extensive datasets, enabling them to measure personal qualities in college admissions essays both equitably and accurately.

UPenn study used Large Language Models (LLM) to assess personal qualities in college admissions essays?

Research published in October 2023 by the University of Pennsylvania indicates that certain LLMs, if trained properly and thoroughly, can review applicants’ essays for personal qualities that predict college graduation on par with human admissions staff.

Researchers analyzed & scored over 300k essays for specific traits

UPenn researchers analyzed over 300,000 college essays and scored them on the absence or presence of seven traits: prosocial purpose (helping others), leadership, learning, goal pursuit, intrinsic motivation, teamwork, and perseverance.?

Training RoBERTa

Then they trained an LLM, RoBERTa, to recognize and evaluate similar qualities and characteristics in essay submissions without showing biases toward race, gender, or socioeconomic status.?

Why use RoBERTa instead of GPT?

RoBERTa was used because it excels at understanding the context and meaning of language, which makes it an effective tool for understanding emotions, text classification, and translations.

What’s the difference between between RoBERTa and GPT

Both are LLMs that share the same architecture but excel in specific tasks, similar to cars with the same frame but different tires and suspension systems for certain terrains.?

• RoBERTa drives better in the city (understanding language nuances, emotions, and context), but it can still make it on certain off-road trails (creating content).
• GPT can navigate some city streets, but it’s best for driving on any off-road trails.

Key research findings

1. RoBERTa recognized personal qualities without bias: It recognized qualities like teamwork and intrinsic motivation in applicants from diverse backgrounds, without showing bias towards race, gender, or socioeconomic status.
2. RoBERTa’s predictions were accurate: Its predictions of the likelihood of students graduating were slightly more accurate than humans, but not by much.

Limitations

While this research has nuances, like the length of the essays (only about 150 words each), it demonstrates that admissions teams can potentially use specific LLMs as a supplemental tool to review written submissions equitably and accurately during the admissions process.

Words of advice from the researchers: “An AI approach to measuring personal qualities warrants both optimism and caution… We recommend AI be used to augment, not replace, human judgment. No algorithm can decide what the goals of a university’s admissions process should be or what personal qualities matter most for that community.”

Large language models are basically the engines that power chatbots like ChatGPT, Claude, and Google Gemini. These LLMs allow chatbots to understand and respond to language in a way that can be helpful for reviewing essays.

Using chatbots to review written submissions

Chatbots, which can write about pretty much anything, can also be given prompts (instructions) to review written submissions for inclusive language, grammar, punctuation, syntax, etc.?

Chatbot limitations

Remember, AI can be biased and chatbots, while helpful for specific tasks, still have limitations, like inaccuracies and understanding writing nuances that human experts may better understand.

So, think of chatbots as a supplemental tool, not a sole source of truth. Use them to review essays for very specific characteristics, like in the prompt examples we’ll show you in this section.

You can customize the prompts in this section to get responses in specific formats, such as bulleted lists of errors or short explanations.?

For example: “Review this paragraph and provide a bulleted list of any slang or idioms that may be culturally insensitive.”

Prompts to help assess the use of inclusive language

• Check for the respectful and accurate use of terms related to individuals’ identities, such as disability, race, ethnicity, sexual orientation, sex, and gender identity.
• Identify any slang or idioms that may be culturally insensitive.
• Review the essay for any language that could be perceived as discriminatory or biased.

Prompts to review grammar, punctuation, and syntax

• Identify and list any grammatical and punctuational errors in this essay.
• Analyze this essay for syntax issues.
• Review the essay for subject-verb agreement.
• Assess the use of tense throughout the essay and list inconsistencies.

Prompts to detect plagiarism

Google Bard is better for plagiarism detection because it has live access to the internet, unlike ChatGPT.

• Highlight any text that matches text from other published sources.
• Identify any text that closely paraphrases text from the original source.
• Check this essay for proper citations and attribution of quotes.

Again, it’s crucial to verify any findings for yourself—especially when it comes to plagiarism detection.

Automated interview platforms

AI-powered interview platforms can standardize admissions interviews with structured interviews that remove common challenges like travel costs and exam scheduling, and they can help reduce biases.

How do automated interviews work?

In general, the interviewee responds to pre-recorded questions, and their responses are recorded and transcribed for easy review.

Are automated interviews a good idea?

Is an automated interview—a candidate talking to their screen—a good idea??

It depends on who you ask. Some people may love the idea of not having all the pressure of talking to a stranger, but others may hate the idea of not having the opportunity to truly engage with the interviewer. Regardless, here are some of the? benefits and some drawbacks:

Benefits of automated interviews:

• Standardizes interview questions: automated platforms ask everyone the same questions, so there’s no chance of getting thrown a curveball that others might not face.
• Reduces biases: interview responses can be transcribed and anonymized to remove biases such as accents (which decrease levels of trust), sex or gender, etc.?
• Increases flexibility: automated platforms offer the ability to complete the interview at the convenience of the candidate. The reviewer also has the flexibility to watch the interview recording and listen to specific responses again to better understand the message.
• Less pressure on the candidate: interviews can be nerve-wracking for many, but automated interviews could help reduce some candidates’ anxiety because they’re likely in a familiar setting and they’re only talking to their computer.

Drawbacks of automated interviews:

• No personal connection: automated interviews don’t provide the two-way communication needed for personal connections.
• Requires internet access and specific hardware and software: candidates don’t always have access to a device with a working camera or microphone, which presents a significant barrier if they can’t travel to a location? that offers the necessary requirements.
• Questions can still have biases: AI platforms ask the questions you provide, which may unintentionally include biases.

Use AI responsibly and with purpose

Whether it’s just using AI writing assistants, like Grammarly, to check admissions essays for typos or whether it’s feeding AI platforms big data to help predict future admissions trends, all AI tools need to be used purposefully and with full transparency.?

This requires a deep understanding of how the AI works, its training processes, the data it collects, how that data is protected, and open communication with learners.?

Most importantly, the value of human oversight cannot be overstated. It ensures that the nuances and complexities of individuals and situations, which AI might overlook, are taken into consideration along with other contextual factors.

Download the 3-part ebook:

Using AI to improve DEI in online learning