Evaluate with ChatGPT
Thomas Steiner
Head of Cyberlearn, professor and CEO of immotour ltd - digital transformer, tourism expert, consultant and board member
I am currently attending the EUA Learning Technology Forum in Bilbao. Here, as in most places where ChatGPT has been talked about lately, fears of cheating on exams are predominant, which I think is a pity since the tool has other interesting capabilities for assessment that are not talked about, or are talked about very little.?
In this article, I propose a change of perspective. Let's leave aside for a moment the students using ChatGPT during exams. Let's look at teachers who might use the tool to prepare and grade exams.?
Can ChatGPT help teachers to?
I will analyze each of these questions with an example from my teaching field, Business Analysis. The chosen theme is stakeholders.?
In what follows, my tests of the tool are presented as a dialogue between a teacher (me) and ChatGPT (1).
1. Formulate a problem
Teacher: Please generate an example of a difficult critical question in the field of Stakeholder Analysis targeting undergraduate students which involves a hypothetical scenario.
ChatGPT: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.
Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their priorities? And how would you recommend the corporation to balance its own interests with those of its stakeholders to improve its reputation and create a positive impact?
2. Solve the problem
Teacher: Please answer the following question in several paragraphs using 500 words with examples and supporting arguments:
In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.?[followed by the rest of the problem of chapter 1 above. Note that we ask ChatGPT to resolve its own problem (1)]
ChatGPT: As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. A stakeholder analysis is a critical tool for understanding the needs, expectations, and potential impacts of stakeholders on a corporation. The first step in conducting a stakeholder analysis is to identify all relevant stakeholders, including employees, customers, local communities, suppliers, regulators, and others who may have an interest in the corporation's business practices and impact on the environment.
Next, I would recommend the corporation to categorize these stakeholders based on their level of power, interest, and influence on the corporation. For example, employees and local communities may have a higher level of interest in the corporation's impact on the environment, while regulators may have more power to enforce environmental regulations. This categorization will help the corporation to prioritize its engagement with stakeholders based on their relative importance and impact.
To determine the priorities of each stakeholder group, I would recommend the corporation to conduct surveys, focus groups, and interviews with representatives from each group. This will provide valuable insights into the concerns, expectations, and needs of each stakeholder group and help the corporation to better understand their priorities. Additionally, the corporation could also review publicly available information and data, such as media articles and social media posts, to gather information about stakeholders' opinions and perceptions.
Based on the stakeholder analysis, I would then recommend the corporation to develop a stakeholder engagement strategy that balances the interests of the corporation with those of its stakeholders. This strategy should include specific actions and initiatives that address the priorities of each stakeholder group and demonstrate the corporation's commitment to sustainable business practices and responsible environmental stewardship.
For example, the corporation could implement employee training programs to raise awareness of sustainability issues and promote environmentally responsible behavior. Additionally, the corporation could engage with local communities and other stakeholders through open forums, town hall meetings, and other engagement activities to hear their concerns and discuss potential solutions. The corporation could also establish partnerships with organizations that share its commitment to sustainability and work together to promote environmentally responsible business practices.
In conclusion, conducting a thorough stakeholder analysis is an important first step in addressing criticism from various stakeholders and improving the corporation's reputation. By understanding the priorities of each stakeholder group and developing a stakeholder engagement strategy that balances the interests of the corporation with those of its stakeholders, the corporation can demonstrate its commitment to sustainable business practices and responsible environmental stewardship. By doing so, the corporation will be well-positioned to improve its reputation, build stronger relationships with its stakeholders, and create a positive impact on the environment.
3. Evaluate the solution
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement.
As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above - note that we ask ChatGPT to evaluate its own solution which it has found to its own problem!]
ChatGPT:
Strengths:
The response recognizes the importance of conducting a stakeholder analysis to identify key stakeholders and their priorities.
It emphasizes the need to categorize stakeholders based on their power, interest, and influence, which helps in prioritizing stakeholder engagement.
It suggests using surveys, focus groups, and interviews to gather information about stakeholders' priorities, which can provide valuable insights.
The recommendation of balancing the interests of the corporation with those of its stakeholders shows a good understanding of the concept of stakeholder engagement.
The suggestion of implementing employee training programs, engaging with local communities, and establishing partnerships with organizations shows a practical approach to addressing stakeholder concerns.
Weaknesses:
The response does not mention the use of any specific tools or frameworks for conducting the stakeholder analysis, which could have added more depth to the approach.
The response does not mention any potential challenges or limitations in implementing the recommended stakeholder engagement strategy, which could limit the effectiveness of the approach.
The response does not mention any specific metrics or KPIs for measuring the success of the stakeholder engagement strategy, which could limit the ability to assess its impact.
Suggestions for improvement:
Mention specific tools or frameworks for conducting the stakeholder analysis, such as the Power-Interest Matrix or the Stakeholder Circle.
Address potential challenges or limitations in implementing the stakeholder engagement strategy, such as limited resources or resistance from some stakeholders, and suggest ways to overcome them.
Suggest specific metrics or KPIs for measuring the success of the stakeholder engagement strategy, such as increased stakeholder satisfaction or improved environmental impact.
Provide examples or case studies of successful stakeholder engagement strategies in similar corporations or industries to reinforce the approach.
4. Score the solution
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement and give a mark for the answer on a scale from 1 to 6.
As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]
ChatGPT:
[answers with strengths, weaknesses and suggestions as in chatper 3 above, and then]
Mark: 5/6. The answer provides a comprehensive and well-structured approach to conducting a stakeholder analysis and improving the corporation's reputation. The answer includes clear and specific recommendations, but could be further strengthened by addressing potential challenges and limitations and providing a timeline or action plan.
5. Adjust the scale
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement and give a severe mark for the answer on a scale from 1 to 6.
As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]
领英推荐
ChatGPT:
[answers with strengths, weaknesses and suggestions as in chatper 3 above, and then]
Mark: 4/6 - The answer shows a good understanding of the concept of stakeholder analysis and engagement but lacks specific details and a comprehensive approach to addressing the complex and dynamic stakeholder landscape.
6. Defend a grade
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
A teacher has been critised by his colleagues to be too tolerant with a mark of 5/6 given for the following answer. Can you please give some arguments why this is not the case.
As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]
ChatGPT:
The answer provided above covers all the key elements of a thorough stakeholder analysis and stakeholder engagement strategy. The recommended steps include identifying all relevant stakeholders, categorizing them based on power, interest, and influence, determining their priorities through surveys, focus groups, and interviews, and developing a stakeholder engagement strategy that balances the corporation's interests with those of its stakeholders. The proposed stakeholder engagement strategy includes specific actions and initiatives, such as employee training programs and community engagement activities, that demonstrate the corporation's commitment to sustainability and responsible environmental stewardship. Overall, the answer provides a comprehensive and well-structured approach to addressing criticism from stakeholders and improving the corporation's reputation.
7. Refute a grade
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
A teacher has been critized by his colleagues to be too tolerant with a mark of 5/6 given for the following answer. Can you please give some arguments why his colleagues might be right.
As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]
ChatGPT:
The teacher's colleagues might criticize the answer for the following reasons:
Lack of detail and specificity: The answer provides a general overview of the process of conducting a stakeholder analysis, but does not provide specific or actionable recommendations.
Lack of depth: The answer does not delve into the complexities and nuances of stakeholder analysis and engagement. It provides a high-level overview of the process, but does not explore the challenges and trade-offs involved.
Lack of supporting evidence: The answer provides a theoretical framework for stakeholder analysis, but does not include data, case studies, or examples to support the recommendations.
Lack of creativity and innovation: The answer provides a conventional approach to stakeholder analysis and engagement, but does not demonstrate any creativity or innovation in solving the corporation's specific challenges.
Given these criticisms, it may be reasonable to conclude that the answer only partially meets the requirements of the assignment, and deserves a mark of 5/6. To receive a higher mark, the answer would need to include more detail, depth, supporting evidence, and creativity.
8. Find a scale
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]
Can you please give a correction scale with marks from 1 to 6 for the above question and give a list of 3 elements that an answer should at least contain for each different mark.
ChatGPT:
Certainly, here is a correction scale for the above question, along with a list of 3 elements that an answer should at least contain for each different mark:
1.(Unsatisfactory)
2.(Poor)
3.(Adequate)
4.(Good)
5.(Very Good)
6.(Excellent)
Note: This is a general correction scale and the marks may vary based on the specific requirements and expectations of the evaluator.
9. Raising expectations
Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.
Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their priorities? And how would you recommend the corporation to balance its own interests with those of its stakeholders to improve its reputation and create a positive impact?
Can you please give a modified version of the question above which is more difficult to answer?
In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment, as well as conflicting demands from various governments and regulatory bodies.
Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their often conflicting priorities, taking into account the political, cultural, and ethical dimensions of each stakeholder group? And how would you recommend the corporation to balance its own interests with those of its stakeholders while navigating the complex and shifting regulatory environment to improve its reputation, minimize its negative impact, and ultimately create a positive and sustainable impact on all its stakeholders?
Conclusions
To some degree, which I leave to your interpretation, ChatGPT can indeed help teachers to?
References
1) Susnjak, T (2022, 12 19) ChatGPT: The End of Online Exam Integrity??- online (3.2.2022): https://arxiv.org/abs/2212.09292?
Profil personnel
2 年THX for sharing! I’m impressed by the comprehensive and state-of-the-art dimension of chatGPT. Reflexivity and improvement is also given. Complicating the problem comes with taking into account social issues (not a surprise) ! I miss a bit the implementation side: how to create adhesion etc, but this could be the next exercise.
Head of Communication & Marketing Wageningen University & Research Environmental Sciences
2 年Very interesting article Thomas Steiner. Curious what your thoughts are about using ChatGTP in marketing communication