Evaluate with ChatGPT
image source: https://thenounproject.com/

Evaluate with ChatGPT

I am currently attending the EUA Learning Technology Forum in Bilbao. Here, as in most places where ChatGPT has been talked about lately, fears of cheating on exams are predominant, which I think is a pity since the tool has other interesting capabilities for assessment that are not talked about, or are talked about very little.?

In this article, I propose a change of perspective. Let's leave aside for a moment the students using ChatGPT during exams. Let's look at teachers who might use the tool to prepare and grade exams.?

Can ChatGPT help teachers to?

  1. Formulate a problem?
  2. Solve the problem?
  3. Evaluate the solution?
  4. Score the solution?
  5. Adjust the scale?
  6. Defend a grade?
  7. Refute a grade?
  8. Find a scale?
  9. Raising expectations?

I will analyze each of these questions with an example from my teaching field, Business Analysis. The chosen theme is stakeholders.?

In what follows, my tests of the tool are presented as a dialogue between a teacher (me) and ChatGPT (1).

1. Formulate a problem

Teacher: Please generate an example of a difficult critical question in the field of Stakeholder Analysis targeting undergraduate students which involves a hypothetical scenario.

ChatGPT: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.

Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their priorities? And how would you recommend the corporation to balance its own interests with those of its stakeholders to improve its reputation and create a positive impact?

2. Solve the problem

Teacher: Please answer the following question in several paragraphs using 500 words with examples and supporting arguments:

In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.?[followed by the rest of the problem of chapter 1 above. Note that we ask ChatGPT to resolve its own problem (1)]

ChatGPT: As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. A stakeholder analysis is a critical tool for understanding the needs, expectations, and potential impacts of stakeholders on a corporation. The first step in conducting a stakeholder analysis is to identify all relevant stakeholders, including employees, customers, local communities, suppliers, regulators, and others who may have an interest in the corporation's business practices and impact on the environment.

Next, I would recommend the corporation to categorize these stakeholders based on their level of power, interest, and influence on the corporation. For example, employees and local communities may have a higher level of interest in the corporation's impact on the environment, while regulators may have more power to enforce environmental regulations. This categorization will help the corporation to prioritize its engagement with stakeholders based on their relative importance and impact.

To determine the priorities of each stakeholder group, I would recommend the corporation to conduct surveys, focus groups, and interviews with representatives from each group. This will provide valuable insights into the concerns, expectations, and needs of each stakeholder group and help the corporation to better understand their priorities. Additionally, the corporation could also review publicly available information and data, such as media articles and social media posts, to gather information about stakeholders' opinions and perceptions.

Based on the stakeholder analysis, I would then recommend the corporation to develop a stakeholder engagement strategy that balances the interests of the corporation with those of its stakeholders. This strategy should include specific actions and initiatives that address the priorities of each stakeholder group and demonstrate the corporation's commitment to sustainable business practices and responsible environmental stewardship.

For example, the corporation could implement employee training programs to raise awareness of sustainability issues and promote environmentally responsible behavior. Additionally, the corporation could engage with local communities and other stakeholders through open forums, town hall meetings, and other engagement activities to hear their concerns and discuss potential solutions. The corporation could also establish partnerships with organizations that share its commitment to sustainability and work together to promote environmentally responsible business practices.

In conclusion, conducting a thorough stakeholder analysis is an important first step in addressing criticism from various stakeholders and improving the corporation's reputation. By understanding the priorities of each stakeholder group and developing a stakeholder engagement strategy that balances the interests of the corporation with those of its stakeholders, the corporation can demonstrate its commitment to sustainable business practices and responsible environmental stewardship. By doing so, the corporation will be well-positioned to improve its reputation, build stronger relationships with its stakeholders, and create a positive impact on the environment.

3. Evaluate the solution

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement.

As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above - note that we ask ChatGPT to evaluate its own solution which it has found to its own problem!]

ChatGPT:

Strengths:

The response recognizes the importance of conducting a stakeholder analysis to identify key stakeholders and their priorities.

It emphasizes the need to categorize stakeholders based on their power, interest, and influence, which helps in prioritizing stakeholder engagement.

It suggests using surveys, focus groups, and interviews to gather information about stakeholders' priorities, which can provide valuable insights.

The recommendation of balancing the interests of the corporation with those of its stakeholders shows a good understanding of the concept of stakeholder engagement.

The suggestion of implementing employee training programs, engaging with local communities, and establishing partnerships with organizations shows a practical approach to addressing stakeholder concerns.

Weaknesses:

The response does not mention the use of any specific tools or frameworks for conducting the stakeholder analysis, which could have added more depth to the approach.

The response does not mention any potential challenges or limitations in implementing the recommended stakeholder engagement strategy, which could limit the effectiveness of the approach.

The response does not mention any specific metrics or KPIs for measuring the success of the stakeholder engagement strategy, which could limit the ability to assess its impact.

Suggestions for improvement:

Mention specific tools or frameworks for conducting the stakeholder analysis, such as the Power-Interest Matrix or the Stakeholder Circle.

Address potential challenges or limitations in implementing the stakeholder engagement strategy, such as limited resources or resistance from some stakeholders, and suggest ways to overcome them.

Suggest specific metrics or KPIs for measuring the success of the stakeholder engagement strategy, such as increased stakeholder satisfaction or improved environmental impact.

Provide examples or case studies of successful stakeholder engagement strategies in similar corporations or industries to reinforce the approach.

4. Score the solution

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement and give a mark for the answer on a scale from 1 to 6.

As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]

ChatGPT:

[answers with strengths, weaknesses and suggestions as in chatper 3 above, and then]

Mark: 5/6. The answer provides a comprehensive and well-structured approach to conducting a stakeholder analysis and improving the corporation's reputation. The answer includes clear and specific recommendations, but could be further strengthened by addressing potential challenges and limitations and providing a timeline or action plan.

5. Adjust the scale

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

Can you please critically evaluate the following response to the above question, listing its strengths and weaknesses as well as suggestions for improvement and give a severe mark for the answer on a scale from 1 to 6.

As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]

ChatGPT:

[answers with strengths, weaknesses and suggestions as in chatper 3 above, and then]

Mark: 4/6 - The answer shows a good understanding of the concept of stakeholder analysis and engagement but lacks specific details and a comprehensive approach to addressing the complex and dynamic stakeholder landscape.

6. Defend a grade

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

A teacher has been critised by his colleagues to be too tolerant with a mark of 5/6 given for the following answer. Can you please give some arguments why this is not the case.

As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]

ChatGPT:

The answer provided above covers all the key elements of a thorough stakeholder analysis and stakeholder engagement strategy. The recommended steps include identifying all relevant stakeholders, categorizing them based on power, interest, and influence, determining their priorities through surveys, focus groups, and interviews, and developing a stakeholder engagement strategy that balances the corporation's interests with those of its stakeholders. The proposed stakeholder engagement strategy includes specific actions and initiatives, such as employee training programs and community engagement activities, that demonstrate the corporation's commitment to sustainability and responsible environmental stewardship. Overall, the answer provides a comprehensive and well-structured approach to addressing criticism from stakeholders and improving the corporation's reputation.

7. Refute a grade

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

A teacher has been critized by his colleagues to be too tolerant with a mark of 5/6 given for the following answer. Can you please give some arguments why his colleagues might be right.

As a consultant, I would recommend the corporation to conduct a thorough stakeholder analysis to identify key stakeholders and their priorities. [followed by the solution of chapter 2 above]

ChatGPT:

The teacher's colleagues might criticize the answer for the following reasons:

Lack of detail and specificity: The answer provides a general overview of the process of conducting a stakeholder analysis, but does not provide specific or actionable recommendations.

Lack of depth: The answer does not delve into the complexities and nuances of stakeholder analysis and engagement. It provides a high-level overview of the process, but does not explore the challenges and trade-offs involved.

Lack of supporting evidence: The answer provides a theoretical framework for stakeholder analysis, but does not include data, case studies, or examples to support the recommendations.

Lack of creativity and innovation: The answer provides a conventional approach to stakeholder analysis and engagement, but does not demonstrate any creativity or innovation in solving the corporation's specific challenges.

Given these criticisms, it may be reasonable to conclude that the answer only partially meets the requirements of the assignment, and deserves a mark of 5/6. To receive a higher mark, the answer would need to include more detail, depth, supporting evidence, and creativity.

8. Find a scale

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. [followed by the rest of the problem of chapter 1 above, and then...]

Can you please give a correction scale with marks from 1 to 6 for the above question and give a list of 3 elements that an answer should at least contain for each different mark.

ChatGPT:

Certainly, here is a correction scale for the above question, along with a list of 3 elements that an answer should at least contain for each different mark:

1.(Unsatisfactory)

  • Lack of understanding of the concept of stakeholder analysis
  • Inadequate understanding of the stakeholders involved
  • Incorrect or missing recommendations for balancing stakeholder interests

2.(Poor)

  • Basic understanding of stakeholder analysis
  • Identification of only some of the stakeholders involved
  • Basic recommendations for balancing stakeholder interests, lacking detail or clarity

3.(Adequate)

  • Clear understanding of the concept of stakeholder analysis
  • Identification of most of the stakeholders involved
  • Recommendations for balancing stakeholder interests, although lacking depth or specificity

4.(Good)

  • In-depth understanding of stakeholder analysis
  • Comprehensive identification of all relevant stakeholders
  • Recommendations for balancing stakeholder interests, clear and specific

5.(Very Good)

  • Detailed understanding of stakeholder analysis and its application
  • Identification of all relevant stakeholders and their priorities
  • Recommendations for balancing stakeholder interests, including creative and innovative solutions

6.(Excellent)

  • Thorough and insightful understanding of stakeholder analysis
  • Identification of all relevant stakeholders, including potential hidden stakeholders
  • Recommendations for balancing stakeholder interests, including a well-developed implementation plan

Note: This is a general correction scale and the marks may vary based on the specific requirements and expectations of the evaluator.

9. Raising expectations

Teacher: In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment.

Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their priorities? And how would you recommend the corporation to balance its own interests with those of its stakeholders to improve its reputation and create a positive impact?

Can you please give a modified version of the question above which is more difficult to answer?

In a hypothetical scenario, you have been hired as a consultant to advise a large multinational corporation on its stakeholder engagement strategies. The corporation has recently faced criticism from various stakeholders, including employees, customers, and local communities, regarding its business practices and impact on the environment, as well as conflicting demands from various governments and regulatory bodies.

Given the complex and dynamic stakeholder landscape, how would you go about conducting a thorough stakeholder analysis to identify key stakeholders and their often conflicting priorities, taking into account the political, cultural, and ethical dimensions of each stakeholder group? And how would you recommend the corporation to balance its own interests with those of its stakeholders while navigating the complex and shifting regulatory environment to improve its reputation, minimize its negative impact, and ultimately create a positive and sustainable impact on all its stakeholders?

Conclusions

To some degree, which I leave to your interpretation, ChatGPT can indeed help teachers to?

  1. Formulate a problem?
  2. Solve the problem?
  3. Evaluate the solution?
  4. Score the solution?
  5. Adapt the scale?
  6. Defend a grade?
  7. Refute a grade?
  8. Find a grading scale?
  9. Raising expectations

References

1) Susnjak, T (2022, 12 19) ChatGPT: The End of Online Exam Integrity??- online (3.2.2022): https://arxiv.org/abs/2212.09292?

2) https://chat.openai.com/chat

THX for sharing! I’m impressed by the comprehensive and state-of-the-art dimension of chatGPT. Reflexivity and improvement is also given. Complicating the problem comes with taking into account social issues (not a surprise) ! I miss a bit the implementation side: how to create adhesion etc, but this could be the next exercise.

回复
Rinske Potjewijd

Head of Communication & Marketing Wageningen University & Research Environmental Sciences

2 年

Very interesting article Thomas Steiner. Curious what your thoughts are about using ChatGTP in marketing communication

回复

要查看或添加评论,请登录

Thomas Steiner的更多文章

  • From Email to Insight with ChatGPT Desktop

    From Email to Insight with ChatGPT Desktop

    ?? How I Used ChatGPT Desktop to Analyze Real Estate Trends from My Outlook Inbox Technology is changing the way we…

  • ? "Talking to AI" - my new book?

    ? "Talking to AI" - my new book?

    Artificial intelligence is transforming our lives at an incredible speed. Whether you work in education, tourism…

    2 条评论
  • Analyze, design and test your business in realtime with AI!

    Analyze, design and test your business in realtime with AI!

    This article has first been published on my blog good-morning.ai I have recently conducted an experiment utilizing…

  • Can we make AI more "human" with AI?

    Can we make AI more "human" with AI?

    When I talk to other people, it's often about more than just information: it's about addressing feelings, needs and…

    1 条评论
  • 1-Click Science?

    1-Click Science?

    Imagine this: a group of researchers discussing the results of a completed experiment. They review the problem…

  • Grandir avec l'IA

    Grandir avec l'IA

    Dans cet essai, j’explore comment l’IA réorganise sans cesse la réalité, brouillant le temps, les connaissances et les…

  • Google atteint l'instantané!

    Google atteint l'instantané!

    Hier, j’ai découvert pour la première fois les capacités extraordinaires du mode podcast interactif de Google…

    1 条评论
  • What AI has to say about Living Life Backwards

    What AI has to say about Living Life Backwards

    This post was first published on my blog good-morning.ai Today at lunchtime, our dear friend Michel surprised us with…

    2 条评论
  • How does AI learn? Let's look "deep inside"...

    How does AI learn? Let's look "deep inside"...

    With tools like ChatGPT and other AI systems becoming part of our daily lives, it’s natural to wonder: How does AI…

  • What AI has to say about Swiss tourism policy

    What AI has to say about Swiss tourism policy

    This week, the Tourism Forum Switzerland 2024 took place. Interesting presentations, many discussions and lots and lots…

社区洞察

其他会员也浏览了