The Imperfect Tool: Comparing ChatGPT to Human Performance

At the beginning of the year, when ChatGPT first started gaining commercial attention, I decided to explore its capabilities. While it has been an enlightening experience, I've come to realize that ChatGPT, like any tool, is far from perfect. It can be compared to Microsoft Excel, requiring substantial input, checking formulas, knowing the correct formulas to use and validating the output.

Here is what I’ve learned about its limitations and how it compares to human performance.

The Limitations of ChatGPT:

Lack of Contextual Understanding: One of the primary limitations I have observed is ChatGPT’s lack of true contextual understanding. While it can generate coherent responses based on patterns in its training data, it does not possess genuine comprehension.

Dependence on Input Quality: ChatGPT’s output quality is directly tied to the quality of the input it receives. ?As I referenced earlier this is similar to using Microsoft Excel: a powerful tool, but only as good as the data and formulas you input. If my input to ChatGPT is ambiguous, misleading, or incorrect, the resulting output will reflect these issues. I’ve also felt that I needed to fact check and validate the data quite frequently. Even when it came to making sure %’s to total added up to 100%. So that is a lot of scrutiny and fact checking.

Real-World Examples

Creative Writing: When I tried using ChatGPT for creative writing, it could mimic a range of styles but often lacked the depth, originality, and nuanced understanding of themes that a human author brings to their work. As a human writer, I draw from personal experiences, cultural knowledge, and emotional insight to craft stories that resonate on a deeper level. ChatGPT’s text felt formulaic and derivative, much like having a well-read parrot.

Technical Support: In technical support scenarios, human technicians can ask probing questions, troubleshoot in real-time, and draw from hands-on experience to resolve issues. ChatGPT, while capable of providing general advice and solutions, struggled with complex, multi-faceted problems that require on-the-spot critical thinking and problem-solving skills. Humans excel at adapting solutions dynamically, whereas ChatGPT operates more like a tech support agent reading from a script.

?

Conclusion

While ChatGPT represents a significant advancement in artificial intelligence, my experience with it has underscored the importance of recognizing its limitations compared to human performance. Its lack of contextual understanding, inability to experience emotions and intent, and dependence on input quality highlight the necessity of human oversight and interaction. By using ChatGPT in conjunction with human skills and judgment, we can harness the power of AI while ensuring that human insight and expertise remain central. With proper guidance and oversight, it can be a highly useful tool. However, it is crucial to remember that it cannot replace the nuanced, empathetic, and sophisticated capabilities of a real human being.

ChatGPT 4.0 assisted me in writing this article.