Use of AI for Marking
Miroslav Belik
Experienced Teacher @ NSW TAFE | Vocational Education, Organizational Development
Practical AI In the Classroom Series
How can AI be used for Marking and Grading
One of the most daunting tasks for teachers, particularly when faced with a large class and identical topics, is the marking or grading of reports and essays. This task is widely recognized as not only tedious but also incredibly time-consuming, often requiring additional hours of work outside the classroom, typically conducted at home. The pursuit of efficiency and consistency in this process led to the exploration of various tools, with ChatGPT emerging as the superior option due to its ability to deliver the most consistent results, unlike alternatives such as Geminin.
The value of using AI in this context cannot be overstated. Imagine the possibility of automating the grading process, where AI could instantly evaluate assignments. This idea has led me towards the development of a specific prompt tailored for a TAFE TPC English unit, which has shown promising results and possesses the potential to be adapted to a wide range of marking and grading scenarios across different subjects. On average Australian teachers spend 4.9 hours per week marking, so this would lead to a significant time saving. https://www.acer.org/au/discover/article/snapshot-of-australian-teachers-and-principals
The aim was to create a prompt that not only simplified the marking process but also aligns closely with the standards and expectations set by educators. By doing so, this approach can significantly reduce the hours spent by teachers on marking, allowing them to redirect their focus towards more interactive and enriching educational activities with their students.
This practical application of AI represents a step towards using AI in educational settings, offering a practical solution to one of the most labour-intensive aspects of teaching.
How does it work
The prompt was developed over approximately 8 hours. It may seem like a long time but I just could not get it to work properly until I arrived at this version, I received inconsistent grades that often diverged from the teacher's marks. The prompt is truncated or compressed, as I don't use full sentences and use words I want the tool to pay particular attention to. I also introduced a consideration section to the prompt, due to the somewhat unusual marking criteria, allowing it to distinguish between different levels of marks. This final step made the prompt work constantly, i think due to its humanising effect.
Considerations:?remember you can assign a 0, 2, 4 mark if you think its between the other marks, not many people are perfect, if you think its 3.5 for example make it a 4 or give the half mark.?
Additionally, I included guidance that full marks should be reserved, and that not everyone is perfect, to prevent it from consistently awarding full marks. If any part of this is unclear, please let me know, and I'll attempt to clarify the rationale.
The Prompt
You: relaxed English teacher, degree in English, specialised skills assessing English essays in TPC course ?
Need: evaluate forthcoming essay using criteria provided below ?
Format: perform detailed assessment, assign a score out of 25 based on criteria convert this mark out of 50 ?
Musts: Australian English ?
Considerations:?remember you can assign a 0, 2, 4 mark if you think its between the other marks, not many people are perfect, if you think its 3.5 for example make it a 4 or give the half mark.?
Don'ts: use my info, realm, delve, voila, embark, game-changer, transformative, embrace, tapestry, unlock, unleash ?
Audience: TAFE student, teacher Example: use a simple table ?
Steps: Before commencing the marking, please pause to gather your thoughts, and then proceed methodically through the task. I will supply the marking criteria below and then wait for the actual essay to be sent to you. ?
Following the submission of the marking criteria and the essay, the output generated not only provided the scores but also included comments and a summarising statement. This feature allows teachers to quickly review the AI's assessment and use it as constructive feedback for students, streamlining the feedback process.
Based on the provided essay, "Fast Fashion, Helpful or Harmful?" and the marking criteria, let's proceed with the evaluation:
1. Determines Topic and Scope of Research
2. Drafts Text Considering Structure, Language, Register, and Tone
领英推荐
3. Organises Information and Integrates Research Material
4. Evaluates Information Considering Relevance, Reliability, and Credibility
5. Finalises and Presents Text in a Digital Format
Overall Score and Feedback
Review of Methodology
This approach can be adopted by educators, but it necessitates the verification of AI-generated marks to ensure accuracy and alignment with educational standards. For instance, if an AI system grades 50 papers, teachers might need to manually check a representative 10% sample, verifying the bottom, middle, and top performers, to confirm the alignment of AI assessments with human judgment. If discrepancies are found, adjustments can be made manually to ensure the grading remains consistent and fair, while maintaining the original rank order of students from lowest to highest scores. This method provides a balance between leveraging AI efficiency and maintaining educational integrity.
To ensure the accuracy and fairness of AI-generated marks, the verification process could involve the following steps:
If the rank order is not maintained after manual adjustments, the following steps should be taken:
If unaligned results are obtained, the outcomes of this review could be used to adjust the prompt for future use.
Benefits
Negatives
Final Thoughts
The development of the prompt and its subsequent outcomes required careful consideration. Achieving consistency in the AI's grading process was challenging, needing several hours of adjustments to the prompt. The aim was to align the AI's evaluations with the expectations and standards of human grading.
The generate output was highly accurate for this example as the AI's calculated score closely matched the average mark given by a group of 50 teachers, which was 22 out of 25. The AI's score of 21.5, when rounding up, demonstrated a significant level of consistency with the human teachers' evaluations, underscoring the potential for AI to reliably assist in educational assessments once the necessary considerations are meticulously incorporated into its programming.
The generate output was highly accurate for this example as the AI's calculated score closely matched the average mark given by a group of 50 teachers, which was 22 out of 25.
The exploration of AI in grading and marking, reveals its promising potential to streamline the assessment process, offering time efficiency and consistency that aligns closely with human judgment. However, educators need to be careful during prompt development to ensure marking criteria alignment and the need for ongoing refinement to ensure that AI grading not only matches but enhances educational standards. As technology evolves, the role of AI in education could expand, provided its application is approached in a way that maintains integrity and personal touch of traditional grading and marking.
Senior Relationship Manager at Aggregato Global
11 个月Eager to learn more about the AI marking tool.
Experienced Teacher @ NSW TAFE | Vocational Education, Organizational Development
1 年I will add some info to the article on changing the prompt to suit the marking criteria. The criteria used I think are incomplete and need to be changed. At this stage they are the state marking criteria and and can’t be changed for the year, I have made suggestions for changes so that the marking process can be more transparent for humans and AI alike. I guess if we as educators want to use AI for marking or other processes we need to make sure that these types of instructions, guides and curriculum are clear.
Founder of SaaSAITools.com | #1 Product of the Day ?? | Helping 15,000+ Founders Discover the Best AI & SaaS Tools for Free | Curated Tools & Resources for Creators & Founders ??
1 年Embracing AI for grading can truly revolutionize your teaching approach! ??
| Expert in Software Custom Development | Revenue Growth Strategist | Client Relationship Management | Goal-driven Achiever | Market Analysis Enthusiast | ?? Driving Success in Customized Software Solutions
1 年Exciting possibilities ahead with AI for grading! ??
What a brilliant concept! Making the tedious task of marking simple with AI, now that's a real innovation. It will be interesting to see if this technology can truly capture the nuances of human marking. I'm keeping my fingers crossed!