How will ChatGPT affect Knowledge Workers like You and Me?

In Industry 4.0, we will be increasingly delegating our tasks to machines and focusing on critical thinking/reasoning. Anyone denying to be a life-long learner and critical thinker shall be replaced by machines. ChatGPT has just proven it ruthlessly.

Introduction

Let me quote few lines from on ChatGPT from Wikipedia:

1. ChatGPT (Generative Pre-trained Transformer) is a chatbot launched by OpenAI in November 2022.
2. In a December 2022 opinion piece, economist Paul Krugman wrote that ChatGPT would affect the demand for knowledge workers.
3. In the Nature journal, Chris Stokel-Walker pointed out that teachers should be concerned about students using ChatGPT to outsource their writing, but that education providers will adapt to enhance critical thinking or reasoning.
4. Joanna Stern with The Wall Street Journal described cheating in American high school English with the tool by submitting a generated essay.
5. Professor Darren Hick of Furman University described noticing ChatGPT's "style" in a paper submitted by a student. An online GPT detector claimed the paper was 99.9% likely to be computer-generated, but Hick had no hard proof. However, the student in question confessed to using GPT when confronted, and as a consequence failed the course. Hick suggested a policy of giving an ad-hoc individual oral exam on the paper topic if a student is strongly suspected of submitting an AI-generated paper.

As you can understand, while academia will adapt to make sure that they can test candidates against academic plagiarism meaning the output of ChatGPT has been as good as human, why will corporates hire and/or keep knowledge workers like you and me if our task can be performed by machines seamlessly? This article tries to find an answer to this question based on my testing of ChatGPT.

My Testing Methodology

Testing on Bookish and Caselet-Based Questions

I thought to test ChatGPT with bookish questions as well as with caselet based questions. While bookish questions can be answered by simply studying few books, caselet-based questions require critical thinking.

Question Pattern

Since we all know about excellent performance of ChatGPT in writing essays, I thought to test ChatGPT with very pointed questions – MCQ (Multiple Choice Questions). MCQs require critical thinking and it is almost always difficult to choose the right answer/answers from the given choices all of which seem to be correct even to untrained human eyes.

My Test Topics

Based on my knowledge and experience, I chose to gather and/or frame questions on the below topics to test ChatGPT on “easy” topics:

1. Financial Management: I have chosen a relatively easy topic – Investment Analysis and Portfolio Management.
2. Technology: I have chosen Microsoft AI Fundamentals courseware which Microsoft defines as “Easy”
3. Process: I have chosen Product Ownership and Scrum Knowledge based on Scrum Guide of scrum.org. Scrum Guide (Edition: November, 2017 Section: Definition of Scrum Page No.: 3) says that scrum is “Simple to understand” (though “Difficult to Master”).

My Test Results

Below presentation captures, analyzes and discusses about my test cases and test result of ChatGPT:

Financial Management – Investment Analysis and Portfolio Management

• ChatGPT scored 100% in Financial Management bookish questions (all 5 out of 5 correct)
• ChatGPT scored 25% in Financial Management caselet-based questions (1 out of 4 correct)

Technology – Microsoft AI Fundamentals

• ChatGPT scored 100% in Technology – AI bookish questions (5 out of 5 correct)
• ChatGPT scored 20% in Technology – AI caselet-based questions (1 out of 5 correct)

Process – Product Ownership and Scrum Knowledge based on Scrum Guide of scrum.org

• ChatGPT scored 63% in Process – Scrum bookish questions (5 out of 8 correct)
• ChatGPT scored 33% in Process – Scrum caselet-based questions (2 out of 6 correct)

ChatGPT’s Report Card

The test data size has been small since ChatGPT, being in trial version, doesn’t allow too many hits from one user login at the time of writing of this article. Also, my intention has never been pulling all possible questions and perform a thorough testing of ChatGPT.

Though I have used a small sample of questions across Finance, Technology and Process (to cater to the audience based on their comfortability and preference), I have successfully proven the importance of reskilling, critical thinking and reasoning as essential skills for today’s knowledge workers. And, that's undeniable.

Below is a glance of ChatGPT’s report card in my testing:

Interpretation of ChatGPT’s Report Card

ChatGPT has been able to answer most of the bookish questions correctly but has been performing poorly on caselet-based questions requiring critical thinking. Exception being process (scrum) which requires higher amount of empathy and human intelligence where ChatGPT performed poorly even while answering bookish questions.

Knowledge workers like you and me have to focus towards enhancing our critical thinking abilities and increasing on-job learning to remain relevant in this rapidly changing Industry 4.0 world. Instead of thinking AI as our job-taker, let’s think AI as our order-taker. Actually, AI is looking at us to delegate our lower value chain activities to it so that we can do higher order critical thinking and reasoning.

What did ChatGPT say about my observations

I asked ChatGPT to answer below questions, and surprisingly ChatGPT echoed my point of view:

Why do you fail to answer caselet-based questions correctly?

ChatGPT responded as below:

As a language model, I can answer questions based on the information I have been trained on, but my understanding of specific caselets may be limited or out of date. Additionally, caselet-based questions often require a deeper understanding of context and real-world knowledge, which may be difficult for a machine learning model to fully grasp. I strive to provide accurate and helpful responses, but there may be times when my understanding is not complete. If you have a specific caselet-based question, feel free to ask and I will do my best to provide an answer.

How will you change the future of job for human?

ChatGPT responded as below:

Language models like me, have the potential to automate certain tasks and make them more efficient, which could lead to changes in the job market. For example, the use of natural language processing (NLP) in customer service could lead to increased automation of repetitive tasks and allow human representatives to focus on more complex issues. Similarly, NLP can be used to analyze large amounts of unstructured data, such as customer feedback, which could help companies make better business decisions.

However, it's also important to consider that the use of AI in the workforce will also create new jobs and opportunities. The development, maintenance, and improvement of AI systems will require a new set of skills and expertise.

Overall, it's likely that the use of AI will lead to both challenges and opportunities in the job market. It's important that society continues to closely monitor the impact of AI on employment and take steps to ensure that the benefits of AI are widely shared and that those who are negatively impacted are supported.

Will programmers be replaced by you?

ChatGPT responded as below:

As a language model, I am a tool that can be used to automate certain tasks, but I do not have the ability to replace programmers. While I can assist in certain programming-related tasks, such as code generation and data analysis, the creation and maintenance of complex software systems still requires human expertise.

Programming is a complex task that requires critical thinking, creativity, and problem-solving skills. While AI models can automate some parts of the programming process, it's still a highly skilled task that requires the ability to understand and work with the nuances of the specific programming languages and frameworks.

It's likely that the use of AI in the programming process will lead to some changes in the way that software is developed, but it is not likely to replace the need for human programmers altogether. Instead, AI will likely augment the abilities of human programmers, making them more efficient and allowing them to focus on more complex tasks.

Conclusion

In the past 200+ years, as we have transcended from Industry 1.0 to Industry 4.0, our job profiles have changed multiple times. The commonest thing throughout these years have been machines performing our tasks freeing us to perform tasks requiring more thinking/reasoning. AI will bring the next big wave freeing us for doing things those require even higher order thinking/reasoning. While we will perform increasingly complex but enjoyable activities at work delegating mundane works to machines, we have to be lifelong learners to be relevant and employable.

Disclaimer: The views expressed in this article are of my own and solely based on my experience. The test data size has been small since ChatGPT, being in trial version, doesn’t allow too many hits from one user login at the time of writing this article. If you think it’s more of an anecdotal test result and/or feel that any portion of this article needs to be revisited, please leave a comment. I’ll revisit the article.

Reference: Most of the questions are taken from NIFM Investment Analysis and Portfolio Management mock test (nifm.in), Microsoft AI-900 Exam Sample questions (learn.microsoft.com), scrum.org Open Assessments and thescrummaster.co.uk PSPO II practice test. All these question sources are available free of cost to public. I have used the questions to test their robustness in front of ChatGPT, and hence it’s not a copyright violation but a fair use for a limited and “transformative” purpose.

Sambit has 17+ years of experience in IT Services. He is currently working as a Presales Consultant/Solution Architect in Tata Consultancy Services Ltd. which is one of the largest multinational information technology services and consulting company headquartered in Mumbai, India. He has a special interest in BFSI Domain, Open Banking, Agile, Scrum, Product Ownership, AI, Metaverse, Blockchain, NFTs. He has deep expertise in Presales Consulting, Solution Architecture, IT Consulting, Implementing Value-Driven Agile Methodology, Organizational Change Management, Program/Project Management and Delivery Management. He lives in Kolkata with his wife and a small kid. He loves to travel; loves to spend quality time with family; loves to learn-unlearn-relearn, hypothesize-test-feedback, help-mentor-get mentored and most importantly hates to procrastinate. He is reachable through LinkedIn chat and usually responds within 24 hours.

Keywords: chatgpt ai impact knowledge_worker IT_professional IT_engineer

#chatgpt #ai