ChatGPT is Bullshit?
There has been some controversy around the value of Large Language Models (LLMs).? In particular whether ChatGPT is simply a "bullshit machine" generating plausible-sounding text without concern for truth or falsehood. (See Hicks, M.T., Humphries, J. & Slater, J. ChatGPT is bullshit. Ethics Inf Technol 26, 38 (2024). https://doi.org/10.1007/s10676-024-09775-5)
We prompted ChatGPT-4o to respond with a rebuttal to the 'ChatGPT is bullshit' paper.
It's quite good.
Table 1: Response by ChatGPT to the prompt: If you were writing a short rebuttal piece to the paper Hicks, M.T., Humphries, J. & Slater, J. ChatGPT is bullshit. Ethics Inf Technol 26, 38 (2024). https://doi.org/10.1007/s10676-024-09775-5 what might it look like?
ChatGPT is Not Bullshit: A Rebuttal
In a recent argument, it’s been claimed that ChatGPT and other large language models (LLMs) are "bullshit machines," generating plausible-sounding text without concern for truth or falsehood. While this critique raises important points about how we frame AI behavior, characterizing ChatGPT’s outputs as "bullshit" is ultimately misguided. Here's why:
1. ChatGPT’s Purpose: Generating Coherent Text, Not Deceit
The core claim in the "bullshit" argument hinges on Frankfurt’s definition of bullshit as speech that shows indifference to the truth [1]. ChatGPT, according to this critique, is a quintessential bullshitter because it doesn’t care about the truth of its utterances. But this argument imposes a framework of human intentionality on a system that doesn’t possess it [5].
ChatGPT doesn't aim to mislead or deceive. It is designed to produce coherent, contextually appropriate responses based on patterns in vast amounts of text data [2]. It’s not indifferent to truth—it’s indifferent to everything, because it lacks intention, motivation, or awareness altogether [5]. This crucial difference means that ChatGPT is more akin to a tool, like a calculator or a text generator, rather than a deceptive actor engaged in "bullshitting" [7].
2. Truth is Incidental, But Not Irrelevant
Though ChatGPT isn’t a truth-seeking engine, the truth is far from irrelevant to how it operates. The accuracy of its outputs is directly linked to the quality of the data it was trained on [2]. When prompted to provide factual information, ChatGPT often generates accurate responses, because the patterns of language it has learned include significant amounts of true statements [5]. It may produce inaccuracies, but calling these "bullshit" ignores the system’s fundamental mechanism: it attempts to align responses with the input, which often leads to factually correct results [7].
Errors, including so-called "hallucinations," arise not from indifference or deception, but from inherent limitations in the model’s design [4]. These errors should be seen as a challenge in refining and guiding the system, not as evidence of systematic "bullshit" [12].
3. Misleading Language Undermines Understanding
Labeling ChatGPT's output as "bullshit" misrepresents its function and risks confusing both the public and decision-makers [11]. The metaphor of bullshit conjures an image of deliberate deception, which ChatGPT is categorically incapable of [1]. This could lead to unnecessary distrust of AI systems, obscuring their genuine capabilities and potential for useful applications [9]. ChatGPT isn’t trying to deceive or appear human—it’s following an algorithmic process that can be improved and aligned toward greater accuracy [5].
By contrast, more precise language—such as calling the inaccuracies "errors" or "statistical noise"—reflects the model’s limitations without anthropomorphizing it [6]. LLMs are tools, not actors. Tools can be inaccurate, but they aren’t capable of dishonesty [11].
4. Collaboration with Humans is Key
Ultimately, ChatGPT is a tool designed to work with human oversight [8]. It is not intended to be an independent arbiter of truth or a replacement for human judgment [12]. When used in the right context—augmented by human verification and refinement—it can produce valuable insights, summarizations, and assistance [9]. Misunderstanding its function risks missing the collaborative potential between humans and AI, where humans provide the critical thinking and judgment that models like ChatGPT lack [7].
Conclusion: Refining, Not Dismissing
Rather than dismissing ChatGPT as "bullshit," we should acknowledge it for what it is: an advanced, but imperfect, tool for generating human-like text [3]. Its outputs may sometimes be flawed, but labeling those flaws as "bullshit" distorts our understanding of its purpose and capabilities [5]. The challenge lies not in dismissing ChatGPT but in improving it, developing better ways to guide its output toward truth, and using it responsibly in a human-centered context [10].
领英推荐
Calling it "bullshit" is a catchy critique—but one that misses the mark.
References
1. Frankfurt, H. G. (2005). On Bullshit. Princeton University Press. ISBN: 9780691122946. DOI: https://doi.org/10.1515/9781400826537
2. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610-623). DOI: https://doi.org/10.1145/3442188.3445922
3. Marcus, G., & Davis, E. (2020). Rebooting AI: Building Artificial Intelligence We Can Trust. Pantheon Books. ISBN: 9781524748253.
4. Bommasani, R., et al. (2021). On the Opportunities and Risks of Foundation Models. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.2108.07258
5. Floridi, L., & Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. Minds and Machines, 30, 681-694. DOI: https://doi.org/10.1007/s11023-020-09548-1
6. Mitchell, M. (2023). Artificial Intelligence: A Guide for Thinking Humans. Farrar, Straus and Giroux. ISBN: 9780374257835.
7. Kiela, D., Firooz, H., & Grave, E. (2021). Dungeons and Data: How Language Models Fail on Opinion Prediction. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.2102.06590
8. Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. DOI: https://doi.org/10.12987/9780300252392
9. Brundage, M., et al. (2020). Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.2004.07213
10. Floridi, L. (2021). The Logic of Information: A Theory of Philosophy as Conceptual Design. Oxford University Press. ISBN: 9780198833635.
11. Bryson, J. J., & Theodorou, A. (2019). How Society Can Maintain Human-Centric Artificial Intelligence. AI & Society, 34(4), 577-589. DOI: https://doi.org/10.1007/s00146-018-0835-0
12. Amodei, D., et al. (2016). Concrete Problems in AI Safety. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.1606.06565
13. Mitchell, M. (2019). Artificial Intelligence: A Guide for Thinking Humans. Farrar, Straus, and Giroux. ISBN: 9780374257835
An Open Access preprint of the full paper can be downloaded from https://www.researchgate.net/publication/384357459_ChatGPT_is_Not_Bullshit DOI: https://dx.doi.org/10.13140/RG.2.2.16618.53448
Making Scientists Make Sense.
5 个月And the exception - where ChatGPT underperforms - is most interesting. https://hbr.org/2024/09/ai-can-mostly-outperform-human-ceos
Senior Managing Director
5 个月Dennis Lendrem Very insightful. Thank you for sharing
Product Manager, PhD, Statistics & AI Implementation | Design of Experiments | Digitalization | Machine Learning
5 个月Very interesting! These models vary a lot in the quality of their responses and on the whole I typically say that most LLMs get the text about 80 % right, the rest needs editing. This reply is on the high side of that for sure. I personally agree with the sentiment of the rebuttal as well.