AI is the Cluster Bomb we Think it is on Our Current Path, but it Doesn’t Have to be this Way

There has been a ton of talk in the news about the cluster bombs we’ve provided to Ukraine. Are they ethical? Is the dud rate 1% as widely reported or a much higher 14% or more with leaked behind the scenes data? Should we really be providing them if civilians may suffer the consequences of the duds for many years to come? If we don’t know really know the human cost or even the total cleanup cost, should we be considering them at all?

I was a guest at the Reuters Momentum AI Summit last week in Austin, and I thank Reuters the opportunity to attend this timely, compelling conference. ?One of the panels there wrestled with the question – ‘who is responsible when AI goes awry?’?

The very unsatisfying conclusion from primarily the audience was that we all are. The group seemed to arrive at this answer by eliminating those who might otherwise be responsible. Government is ill equipped to regulate AI, which everyone easily agreed upon. More difficult to stomach was that those who leverage the underlying algorithms in their solutions can’t be responsible because they really don’t know what hallucinations these algorithms might come up with. I am still scratching my head on this one.

Does this mean that Oppenheimer bares no responsibility for the suffering of millions of people and a global arms race that now allows us to destroy this and many other planets (if we could get to them) many times over because he didn’t know what his atomic bomb could really do? I have to call BS here. Further, one of the facilitators of the session, Cortnie Abercrombie, the CEO of ?AI Truth, an AI information provider, said that after studying the user agreements of many AI solutions, that the person that is almost exclusively is responsible for AI outcomes is the end user. What? So apparently, we can pin the nuclear arms race exclusively on the pilots of the Enola Gay. Preposterous.

I questioned the audience as to why we are unwilling to hold the line that AI must be explainable. I errored in that I didn’t clarify that in my definition of explainable, if you can’t explain what the black box is doing, you need to at least be able to explain the output through rigorous testing. The response was that the ‘ship has sailed on explainable AI’ meaning we are unwilling to reign it in or simply can’t, and we are content to build and provide solutions we don’t understand. Again, I have to call BS. We can at least understand what the black box is outputting and with what degree of accuracy. This is the most basic accountability of any piece of software.

To me, sweeping software provider responsibility for what AI is doing under the rug is exactly same story as the global Financial Crisis of 2008 – same move different subject. As a friend of mine likes to say, Speed 2 is just like Speed 1 but on a boat. In the Financial Crisis that virtually brought the global economy to its knees, our banks and financial institutions hired quants primarily from top physics programs who wrote algorithms they didn’t understand the implications of to help their companies package and sell "credit default swaps" to each other. No one understood what they were doing, but they were putting their jobs, their companies’ survival, people’s savings and retirement, and whole economies at risk. According to Investopedia, 8.8 million people lost their jobs and households lost 19.2 trillion dollars of wealth in the US alone. Quite a cluster bomb that literally impacted almost everyone.

Well, here we go again. Let me be clear though, I am a huge proponent of AI, but I am also a big proponent of knowing what we are doing.

Well, here we go again. Let me be clear though, I am a huge proponent of AI, but I am also a big proponent of knowing what we are doing. AI today, has a very high error rate or dud rate like a cluster bomb. It can hallucinate. It can make up fake references. It will use its math & statistics to fill just about any information hole we tell it to. Managed effectively and applied appropriately, this is not an issue. If you gave it control over a nuclear arsenal today, however, big problem. Today, it is best applied to things that can handle a significant error rate. A chatbot, as long as it is not spewing out legally binding advice, sure. Anomaly detection, if a human is in the loop, bring it on. What I see people doing consistently, which is scary, is personifying AI to reason that it actually knows anything. People seem to forget that the “A” in AI is for artificial. Just because you got a readable paragraph out of the prompt you inputted doesn’t mean that the machine or the code it is running know anything beyond the statistical fit of the second word following the first, and the third following the second for that particular prompt (in the case of generative AI). Fed some huge chunk of the world’s accessible information, ostensibly a big dump of the public Internet, it is as flawed as we all are together. More right than wrong, be still flawed. Unlike querying the Encyclopedia Britannica of old, there is no editor for the training data most AIs are trained on and some of this information is completely incorrect.

This doesn’t have to have the impact of a cluster bomb, however. If we apply it to things it is good at, we can unlock perhaps the biggest productivity gain in human history, but we have to at least know what it is good at. This, of course, will change at a blinding pace as specialty training sets can be developed and errors in this data or the AI output can be corrected.?

For those of us who build these solutions, we are either the Oppenheimers or the Edisons.

At the conference, the first commercial use cases being deployed or considered by the companies in attendance specifically for Generative AI included first drafts for lawyers, chatbots, data sets for personalization, and iterations on images – arguably all forms of synthesized search replacements and all requiring a human in the loop to be useful. All this is perfectly fine. As we push forward into more mission critical applications, we need to precisely know the dud rate, so to speak, and understand its impact. Some bristle at referring to this as explainability. That’s fine. We can come up with a new term for it, but if it is a black box, we need to at least test it enough to understand the accuracy and usefulness of its output. For those of us who build these solutions, we are either the Oppenheimers or the Edisons. We can’t pass the buck to the people who will fly what we build. I welcome this responsibility regardless of whether or not the path is entirely crystal clear for how get there and with the full knowledge that regulators won’t be able to help. I am willing to do the work to understand what I am unleashing.

The simple tenets for what might be deemed “Responsible AI” might read:

1.????Truly understand the user pain you are trying to solve.

2.????Test various approaches to solving that pain inclusive of AI approaches.

3.????Determine what end user, ethical, and/or regulatory accuracy is required for the solution to solve that pain sufficiently without causing harm. Accuracy is entirely use case dependent.

4.????Test output extensively with live or representative synthetic data (as determined by actual end users) to validate performance.

5.????Deploy if output is reliable at the required confidence level. Do not if it is not & revisit approach and/or apply it to a different use case.

I believe it is blindingly obvious that the right side of history will be to deploy solutions we, at the very least, fully understand what we will reliably get out of them even if we don’t understand exactly how the black box works. If we can’t at least hold this line, it is sure to be a cluster.

?#ai?#responsibleai?#software?#softwaredevelopment?#productmanagement?#leadership?#cpo?#ceo?#oppenheimer?#edison

Ken Pulverman is a Silicon Valley software executive with more than 20 years of leadership experience in large and small software companies including multiple C-Suite roles in both Product and Marketing in growth companies resulting in two successful exits and an IPO. Ken is currently taking a sabbatical year as part of his commitment to lifelong learning. He is pursuing a second masters in Product Management at Carnegie Mellon University where in addition to completing the degree, he is also teaching and writing a book on product management. Simultaneously, he is pursuing a yearlong CEO certificate program through the University of California, Berkeley. At the end of this year, Ken will begin a part-time Doctorate of Business Administration at SDA Bocconi in Milan. Ken will return to a full-time role starting in 2024.