Unlocking the full potential of ChatGPT: 2-step prompting. Text analysis reveals the chatbot’s action behind the scenes.

In this study, we investigated the ability of ChatGPT, a trained machine learning model, to give us an overview of a novel from different points of view. The analysis included examining the main character, his character traits, both internal and external features, his philosophical reflections, and the social circumstances surrounding him. Our experimental strategy was “2-step-prompting”: first, we let ChatGPT create a list of five possible prompts and then executed these. In addition, we compared the answers for two options: entering all prompts in the same chat or individual chats per prompt. The results gave us a look behind the scenes of ChatGTPs crafting, namely the balance of exploiting an extensive knowledge base and exploring new material. Read the full story on Medium.

1. Let ChatGTP generate five possible prompts for a text analysis

2. Use these prompts to get a more prosperous result

In English lessons, we read?A Part-time Indian?by Sherman Alexie, which includes summarizing individual topics, the characteristics of the characters, and their environment. With the advent of ChatGPT (November 30, 2022), the question is how the chatbot can help shape this work.

As we have learned from many publications since then, the key to success is the prompt, i.e., how you present your question to ChatGPT. We call our approach 2-step prompting: first, we let ChatGPT create a list of 5 possible prompts, then execute them.

Let’s get started: we ask ChatGPT to provide five possible prompts.

Prompt to let ChatGPT create five prompts

I intend to write a summary of A Part-time Indian with the help of ChatGPT. Content parts may be character traits and appearance, the plot, the social environment of Junior, his thoughts on life, and self-improvement. Please create a list of five possible prompts to enter for ChatGTP.

ChatGPTs answer

1. “Provide a summary of the character traits and appearance of Junior in A Part-time Indian.”
2. “Outline the plot of A Part-time Indian and its main events.”
3. “Explain the social environment of Junior in A Part-time Indian and its impact on his life.”
4. “What are Junior’s thoughts on life and self-improvement in A Part-time Indian?”
5. “Provide a summary of the main themes in A Part-time Indian and how they are portrayed in the novel.”

Find all answers of ChatGPT in out full article on ChatGPT: The Powerhouse of Text Analysis in the Classroom and Beyond

Now the ChatGPT created five articles, in two versions each, we let ChatCPT create five comparison criteria

What are the criteria to compare two texts on the same subjects if both are about a summary and description of a novel? Create a list of the five most important criteria.

ChatCPT’s answer

1. Clarity and conciseness: The text should be easy to understand and not contain any unnecessary information or jargon.
2. Accuracy: The text should provide an accurate summary and description of the novel, without any errors or inaccuracies.
3. Objectivity: The text should be free of bias and present a balanced view of the novel.
4. Style: The text should be well-written and engaging, with a clear and consistent style.
5. Analysis and interpretation: The text should go beyond a simple summary and description of the novel, providing insight and analysis into the themes, characters, and other elements of the novel.

Do ChatGPT's text analysis criteria comply with human criteria?

Let’s dig deeper! Can we measure ChatGPT’s creativity — without sophisticated AI tools, on the basis of our experiments? Good news!

We realize immediately that new chat one and new chat five are similar. Their vocabulary is similar, too — new one 90 vs. new five 81 words, which means that in this case, the new chat answer is even longer.

The comparison of same chat one and same chat five is more interesting (check the chart’s legend)! While same chat one is in the same range as new chat one and new chat five in terms of word count and vocabulary, same chat five has become the shooting star, which is apparent from two features:

Nr. 1:?In the range of frequently used words (including stop words), same chat five is significantly leading; see the gray line. It means that ChatGPT creates more phrases, and there is a higher level of content creation.

Nr. 2:?This is the practical result on ChatGPT’s creativity: for a vocabulary of 137 words up to 165 words same chat five clearly outperforms the other three. One-third of his vocabulary is exclusive to same chat five, which shows that ChatGPT lives its creativity.

A tentative explanation of the observed phenomena

Technically, ChatGPT (Chat Generative Pre-trained Transformer) is fine-tuned with supervised and reinforcement learning techniques (Wikipedia). In Reinforcement Learning (RL, Wikipedia) the focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).

When comparing the same chat and new chat results in our classroom experiment, we have observed nothing but ChatGPT’s reinforcement learning process. With the same chat, we observe exploration and thus a certain ingenuity, often fascinating, and with new chat, we test ChatGPT on his capability of current knowledge.

The comparison could explain the strikingly more extensive and elaborate answers regarding summaries; see below: in these cases, ChatGPT switches into the “exploitation mode.”

Some insights and tips for teachers on how to use ChatGPT in the classroom for text analysis

• With 2-step prompting, you can access more of ChatGPT’s knowledge base. Let first ChatGPT decide on how he wants to be prompted, then enter your prompts accordingly.
• If you enter multiple prompts within the same chat, you can unlock ChatGTP’s conversational potential. He will provide insights and writing suggestions. That chatbot knows to connect the dots and offers surprising insights. Same-chat answers are significantly longer than new chat answers.
• If you enter multiple prompts within a new chat each, you, in tendency, employ ChatGPT as an information system, and its limitations become evident. For clarity and conciseness, though, the new-chat strategy might be preferable.
• 2-step prompting offers findings on two levels. The first prompt reveals what is interesting in the overview. It shows the relative importance of different aspects before tackling details.
• 2-step prompting also offers a more standardized workflow, for instance, for text analysis in the classroom, whereas a single prompt (or subsequent single prompts) tends to look like trial and error.
• Technically inclined teachers could learn the AI behavior of ChatGPT and introduce students to rethinking the analysis of texts within a new framework.

We've talked about text analysis, but is that the right term? Or should it be Text Analytics? Let's ask ChatGPT: "what is the difference between text analysis and text analytics"

ChatGPT's answer

Text analysis and text analytics refer to the process of using computer algorithms and software to extract insights from large amounts of text data. The difference between the two terms lies in the scope and focus of the analysis:

• Text analysis is a broader term that encompasses various techniques for analyzing and interpreting the content and structure of text data, including text mining, sentiment analysis, and topic modeling.
• Text analytics, on the other hand, refers specifically to the application of statistical, linguistic, and machine learning techniques to discover actionable insights and meaningful patterns in text data. It is often used to support business decision-making and to improve organizational performance.

New prompt: is ChatGPT more related to text analysis or text analytics?

ChatGPT is a type of language model developed by OpenAI that uses advanced machine learning algorithms to generate human-like responses to text input. It is trained on large amounts of text data and uses this training to generate answers to questions and generate text that is relevant and coherent to the input.

In this context, ChatGPT can be seen as a tool for text analytics, as it enables organizations to extract insights and make predictions from large amounts of text data. ChatGPT can be used to support various text analytics tasks, such as sentiment analysis, named entity recognition, and language translation, among others.

Can ChatGPT's text analytics capabilities be helpful for language learning in the classroom?

Yes, ChatGPT's text analytics capabilities could potentially be helpful for language learning in the classroom.

ChatGPT's ability to understand and generate natural language text can be leveraged to support language learning in the classroom. For example, ChatGPT could be used to:

• Provide instant feedback and corrections on written or spoken language assignments
• Generate practice exercises and tests to help students reinforce their language skills
• Engage students in conversational practice, allowing them to apply and develop their language skills in a natural, interactive setting
• Provide personalized language learning support by recommending resources and exercises based on students' strengths and weaknesses

However, it is important to note that ChatGPT is a tool, and the success of using it for language learning will ultimately depend on the quality of the educational materials and the effectiveness of the teaching methods.