The AI Bullshit Detector: 10 tips and prompts for better knowledge flows

In today's rapidly changing business environment of artificial intelligence (AI), organisations must ensure the accuracy and dependability of Generative AI content, particularly for organisational Knowledge Management (KM) and Learning and Development (L&D) flows. Essentially, the output quality is directly proportional to the quality of the input: rubbish in = rubbish out (RI=RO)! But where do you start??

As KM and L&D professionals, we must equip individuals with the necessary tools to discern truth from bullshit. Why? Knowledge is a human condition that needs to be activated for a positive impact - for me; knowledge is a function of people and their experiences in a given space and time or "Knowledge is a PEST" K=f(PxExSxT)

With AI and human cognitive processes becoming more intertwined, critical thinking skills (experiences) are crucial to positively influence the Momentum, Attitude, Speed, and Heading (MASH) of our organisational knowledge and learning flows. In a recent keynote, I argued that these 'power' skills are part of our human advantage, allowing us as humans to thrive as we compete and collaborate with AI. Therefore, we must develop and deploy effective filters to help people sift through the vast amount of information produced and presented by AI systems.

Those who have followed my work over the years will know it is grounded in Actor Network Theory (ANT), which highlights the interdependence between humans and technology. While AI is a powerful tool, it still has limitations, such as hallucinations, particularly in complex knowledge domains where knowledge and learning are emergent. This poses a challenge, as AI tends to produce inaccurate or deceptive information in such situations.

If you haven't seen it already check out David Snowden's Cynefin framework : AI tends to perform well in simple and complicated knowledge domains where knowledge is widely known but struggles as the level of complicatedness and complexity increases (the number of known and unknown variables and their connectedness and connectivity causing significant hallucinations). In this journal article , you can also check out my approach to knowledge and learning domain.

The emergence of AI has opened up a Wild West of business opportunities for improving decision-making, problem-solving, efficiency, and innovation. However, to fully realise this potential, we need to develop a set of critical filters to evaluate the information presented by AI and amplify our human advantage. For example, the following filters, inspired by Sagan's Baloney Detection Kit, can help us discern the accuracy of AI-generated content. By doing so, we as KM and L&D professionals can influence knowledge and learning flows to enable people to make informed decisions quickly based on factual information.

10 Filters and GAI prompts for better knowledge and learning flows

My ten filters for scrutinising AI-generated business content are as follows:

1. Independent Confirmation: Seek verification of the facts presented by AI from independent sources. GAI prompt: "Verify the accuracy of [specific claim] by searching and comparing it with information from reputable sources like academic journals, government reports, or trusted news outlets"
2. Encourage Debate: Promote discussions and AI prompts encompassing various viewpoints to ensure a well-rounded understanding. GAI prompt: "Generate three arguments for and against [specific topic], summarising perspectives from different stakeholders or experts to facilitate a balanced discussion"
3. Authority Is Not Truth: Recognise that popularity or authority does not necessarily equate to correctness (e.g. Wikipedia or, worse, TikTok truths). GAI prompt: "Provide examples of popular beliefs or widely accepted 'facts' in [specific field] that have been debunked or called into question by recent research or data."
4. Multiple Hypotheses: Search and challenge various explanations rather than settling on the first plausible theory presented. GAI prompt: "1. For [specific phenomenon], research and list three competing theories or explanations. 2. Outline and contrast these various perspectives?"
5. Critical Inquiry: Maintain a questioning attitude towards all hypotheses, including your own. GAI prompt: "Regarding [specific hypothesis], what are the main arguments supporting it, and what questions or challenges could be raised against it based on current knowledge or evidence?"
6. Quantification: Attempt to quantify claims to assess their validity more accurately. GAI prompt: "Find statistical data or research findings that quantify the impact or prevalence of [specific issue], and explain how these numbers were derived?"
7. Chain of Reasoning: Ensure that each link in an argumentative chain is solid and logically consistent. GAI prompt: "For the argument that [specific claim], break down the logical steps or evidence supporting this claim, and identify any potential weak points in the reasoning"
8. Occam's Razor: Favour simpler explanations when multiple hypotheses explain a phenomenon equally well. GAI prompt: "1. Among the explanations for [specific phenomenon], which one is the simplest while still accounting for all known facts? 2. compare your findings to more complex alternatives"
9. Falsifiability: Prefer hypotheses that can be tested and potentially disproven. GAI prompt: "1. Explain how the hypothesis about [specific topic] could be tested or disproven. 2. What kind of evidence or experiment would challenge this hypothesis?"
10. Awareness of Fallacies: Be vigilant of logical fallacies and rhetorical devices that may mislead.GAI prompt: 1. "Identify any common logical fallacies in the arguments surrounding [specific debate or claim]. 2. How do these fallacies affect the validity of the arguments?"]

• My top tip: try supplementing each prompt with the following instructions to improve the outputs: "Take a deep breath and solve the problem step by step" and "My career depends on this".

Our KM and L&D journey with AI in business requires balancing technological capabilities and human experiences using power skills (e.g. critical thinking). We must embrace this journey with an open mind, equipping people with the tools to discern fact from fiction, promoting a culture of informed decision-making and innovation. By developing and adopting these filters, we can improve the reliability and accuracy of our knowledge and learning flows and, ultimately, the information informing our business decisions and innovations.?

Separating meaningful insights from misleading information is not an option; it is a must to leverage AI in business effectively. This means we must practice critical thinking and scepticism to discern the wheat from the chaff. Please remember these principles as you navigate this complex landscape and ensure your organisation's engagement with AI is productive and grounded in truth.

Remember, RI=RO: "rubbish in equals rubbish out."