Are We Too Trusting: AI Agents and the Black Box Problem

The AI News Articulator Issue #5 (May 24 - May 31)

Hello, all! The AI News Articulator completes one month today! Yay! To all those who have subscribed, and reached out to me during this initial month, a huge THANK YOU to you all. Your support and kind words keep me going :)?

Also, I truly hope this newsletter continues to deliver on the goal of adding to your knowledge of AI, and it is something you look forward to reading each week. I’d love to hear what worked for you, what didn’t, and what you would like to see more of. Tell me, I’m all ears.

As we head into June, let’s take a look at the last week in AI news. While it was a bit relaxed (a welcome change of pace, no doubt) things were…interesting.?

Google’s AI-powered search summaries gave hilarious advice, ranging from nuggets of nutritional wisdom (telling people to eat a rock a day), to bold recipe suggestions (instructing users to add glue in order to make cheese stick to their pizza).?

There was also talk of AI Agents, and researchers taking a look inside an AI brain.

This week, I explore how these two seemingly separate topics intersect, so as to give you context and a deeper understanding of the latest in AI news.?

What Are AI Agents? (No Double Os, Just Zeros and Ones)

AI Agents — no, they are not suave, sharply dressed, shaken-not-stirred martini drinkers, who defeat ALL the bad guys, every time. In fact, you may have interacted with an AI agent in your day-to-day life, and not have known it. Let me break it to you, your Roomba is a type of AI agent. So are Siri and Alexa. You’re surrounded by AI agents. (side-eyes surroundings warily)

AI agents, aka, ‘bots’ or ‘virtual assistants’ are systems or softwares that are designed to be autonomous in their interaction with their surroundings. They are driven by goals and do not require prompting at every step. They are different from chatbots, like Chat GPT or Google’s Gemini, that require detailed prompts at every step to get to a conclusion.?

In an interview with the MIT TechnologyReview, OpenAI Chief, Sam Altman, said helpful agents will be AI’s “killer app.”?

“What you really want is just this thing that is off helping you.” - Sam Altman?

Going back to the Roomba example, all you need to do is set a schedule for it and it automatically maps your floor, remembers the floor plan, ensures it gets every inch of space it can, and finds its way back to its dock, without you telling it to do so. This is an example of an autonomous thinking AI agent.?

Some more examples of AI Agents are:

• Self-driving cars.
• Virtual healthcare assistants that can check symptoms and schedule doctor appointments at healthcare providers, eg., Babylon, Whereby, OHMD etc.
• AI investment portfolio managers that can help recommend stocks to you based on your personalized requirements, eg. Charles Schwab’s Intelligent Assistant.?
• Customer service bots that can understand requirements and provide custom responses.?
• Physical robots powered by AI that work on assembly lines in tasks that require precision and speed.?
• Software programs that help protect against bank fraud by analyzing huge troves of data to identify anomalies.?

All of these systems have varying degrees of autonomous intelligence and possess very specific skills. This leads us to the question, can AI think on its own??

How Does AI ‘Know’?

At the very basic level, how does it know, apples from oranges? How are these large language models (LLMs) learning and developing an understanding of the world?

This thought process that an AI system uses to reach conclusions or predictions remains largely unknown. Even as AI systems and agents integrate increasingly into our lives, researchers don’t know how they know stuff.?

This opacity into decision-making by AI systems is referred to as the ‘Black Box Problem’

Usually, the algorithm, i.e. the code that powers the LLM, is available publicly. But the training data that the LLM is fed to make it knowledgeable, can be kept hidden by developers. This makes it difficult to understand how an AI model actually reached a set of answers or conclusions. Because we don’t know what it knows. How did it connect the dots?

For instance, an AI model that is fed training data about the varieties of fruit, could? be used as a learning tool for a wide range of users, to help ID fruits.?

But in more complex cases, is it safe, or even intelligent on our part to hand the reins to a system which makes decisions on its own, even though we do not understand how it reached those decisions? (Do we know if it is plotting world domination or cooking up glue-and-cheese pizza recipes?)

There’s good news and not-so-good news. The not-so-good news is that researchers do not fully know just yet.?

The good news is, they recently made a breakthrough in this area. Last week, researchers at Anthropic, the company that built the Claude family of AI models, revealed that they found a way to start deciphering the inner workings of an AI system’s brain.?

The whole point of it being that it would be easier to trust an AI brain, if we just knew what logic it used to derive meaning and reach a conclusion.?

Why Is It a Trust Problem?

These are some of the issues that arise when we consider AI agents and the black box problem:

• Doing something too well. A popular example of this is an AI agent so focused on completing a task that it ignores rules of the human world, or overlooks nuances, including, a self-driving car that takes a rough terrain road just because it decides that it is the fastest route to get to a destination. Or worse, runs red lights in order to reach the destination in the least amount of time.?
• Explainability and trust. How can we hold something accountable, when we do not know how that system reached a decision? This is important when we consider high-stakes situations like healthcare, stocks-trading, loans etc.?
• Fairness and bias. AI training data is data that has been produced by humans, and due to the wide swathe of data that an AI model is exposed to, it is difficult to pinpoint if bias was introduced, how much of bias was introduced to the system, and if the responses that it gives us are rooted in fairness or are skewed.

What Does This Mean for Now and Going Forward?

AI agents take AI use in daily life a step ahead, from passive chatbots to becoming active participants. This raises a whole host of ethical and societal concerns.?

Perhaps, having clear frameworks in place for AI agents and those who create these systems accountable would be a start. At an AI summit in Seoul last week, companies including Amazon, Microsoft, Meta, Google, and OpenAI, made a voluntary safety commitment for the responsible development of AI, even agreeing to a ‘kill switch’ in case of extreme risk to society.

The task of implementing guardrails for AI systems is the most important one when it comes to further development of AI systems, in my opinion. As with any other product we use, we need to have a clear understanding of the product’s potential to malfunction. Only when we know what causes a problem can we begin to fix it.

And now, as promised, here is your weekly selection of curated AI news articles from top sources.?

AI Tools News and Updates?

Meta Introduces Vision Language Models, Shows Superior Performance Over Traditional CNNs (Analytics India Mag)?

OpenAI Says It Has Begun Training a New Flagship A.I. Model (The New York Times)

Google AI Overview controversy — why there’s a big backlash (Tom’s guide)

5 useful AI features Google just unveiled for Chromebook Plus (ZDNet)

Email Innovator Superhuman Says Its AI Search Is Twice as Fast as Gmail's (Inc.com)

Custom GPTs open for free ChatGPT users (The Verge)

Three Huge AI Tokens Are Merging—Here's How and When (Decrypt)

AI Ethics and Safety?

Inside OpenAI’s 9-Person Safety Committee Led by All-Powerful Sam Altman (The Observer)

OpenAI researcher who resigned over safety concerns joins Anthropic (The Verge)

OpenAI Says Russia and China Used Its A.I. in Covert Campaigns (The New York Times)

Some big media bosses are fighting AI — and others are cutting deals (The Washington Post)

Why regulating AI can be surprisingly straightforward, when teamed with eternal vigilance (World Economic Forum)

Vox Media and The Atlantic sign content deals with OpenAI (the Verge)

AI and Big Tech??

Big Tech develops AI networking standard but without chip leader Nvidia (Reuters)

OpenAI’s Sam Altman vows to give away most of his wealth through the Giving Pledge (CNN)

Ex-OpenAI Director Says Board Learned of ChatGPT Launch on Twitter (Bloomberg)

Elon Musk’s xAI secures $6B to challenge OpenAI in AI race (Artificial Intelligence News)

AI News – The Bad and The Bizarre??

The 7 most shocking Google AI Overview answers we’ve seen (Fast Company)

OpenAI says Russian and Israeli groups used its tools to spread disinformation (The Guardian)

People Share the Dumbest Google Search Results Using AI Tool (Complex)

How to turn off Google AI Overviews: Here are the tricks to avoid seeing bad AI advice (LaptopMag)

'I was vulnerable’: Artificial intelligence work-from-home job scams targeting victims (WBSB-TV)

AI is shockingly good at making fake nudes — and causing havoc in schools (Politico)

AI for Good?

New AI tool may help detect early signs of dementia (UT Southwestern Medical Center)

New AI tool helping detect weapons in schools (WNDU)?

With hallucinations waning, AI is diving deeper into scientific research (The Next Web)

This Entrepreneur Is Using AI To Find Solutions For Climate Change In Troves Of Data (Forbes)

AI and the Future of Work?

Hardly any of us are using AI tools like ChatGPT, study says – here’s why (TechRadar)

Anthropic’s AI now lets you create bots to work for you (The Verge)

These manager roles could be the most at-risk from automation and AI (Birmingham Business Journal)

This AI-Powered Fuel Pumping Robot Could Soon Be Fueling Your Car Up Autonomously (Amaze Lab)

Alphabet, Meta Offer Millions to Partner With Hollywood on AI (Bloomberg)

AI Is Creating Transportation Jobs--Not Taking Them (Inc.com)

AI and Investing

This Record Stock Market Is Riding on Questionable AI Assumptions (WSJ)

Nvidia Stock Jumped After Stock-Split News. These 14 Names Could Be Next. (Barron’s)

AI darling Nvidia's market value surges closer to Apple (Reuters)

Palantir (PLTR) stock could grow 5 times in value, says expert (Finbold)

(Disclaimer: The information contained in the "AI and Investing" section is for informational purposes only and should not be considered financial advice from the author. Please consult with a qualified financial advisor before making any investment decisions.)

#AI #ArtificialIntelligence #AINewsletter #AINews #MachineLearning #Technology? #Newsletter #FutureofWork #AINewsArticulator #AIAgents #IntelligentAgents #AutonomousAgents #BlackBoxAI #ExplainableAI #TransparentAI #AIEthics #AISafety #ResponsibleAI #EthicalAI #AIforGood #AI #ArtificialIntelligence #MachineLearning #DeepLearning #GenerativeAI #AIinHealthcare #AIinFinance #AIinMarketing #FutureofAI #AIRevolution #AIandSociety #AIandtheFuture

That's a wrap for this week's edition of the AI News Articulator. I hope you found the insights and curated news on AI valuable. Feel free to share your thoughts or suggest topics you'd like to see covered in the future.?

Also, subscribe to the AI News Articulator for weekly deliveries of curated AI news and analysis, straight to your inbox.?

See you next week!