Questions to ask your vendor about AI

Often the phrase "standing on the shoulders of giants" is credited to Isaac Newton, but it originated even earlier. However, here we are a millennium later and it still works. Earlier this week, I was attending the keynote speech of the CISO XC conference presented by Chris Roberts who presented some wonderful material about questions to ask your vendor about AI. I enjoyed the presentation so much, I have taken the liberty to reduce his presentation to a straight out regurgitation of his questions- and then add some of my own.

• Do we need A.I. to solve our problem? If so, why?
• Do we have sufficient quantity/quality of data to use AI?? If not, how do we get there?
• Do you have domain expertise in our field?
• What is your delivery model?
• Who is training (or trained) the solution?
• Do you have any human(s) in the loop?
• Is the system interpretable and auditable?
• Who is liable if your solution fails?
• Where is our data stored??
• Do you share / own our training?
• Why did you choose A.I. for this product?
• Can you explain how your advertised machine / deep / supervised / unsupervised learning works?
• Do you use rules and / or search-based techniques? If so, when, where how and why?
• What are the precision, recall and F1 scores for your product?
• What qualifications does your technical team possess?
• Are clients in production or POC with this solution?? If only the latter, why?
• What ROI are clients seeing?? How do they / did you measure this?
• How much training / time was required for that ROI?

As a co-founder of a company that is deeply intertwined with using AI to improve incident investigation, triage and response- I am highly excited about our own answers to these questions (either currently or future state). However, I would offer some of my own additional questions too:

• What happens if the underlying AI fails? Is there still value?
• Can I control the recipients and level of data sharing?
• Can I revoke or delete data that I no longer wish to share?
• Do you provide transparency and citations for any AI provided answer?
• How is new training performed and who does that?
• How does the system handle feedback?
• Do you use training data from external sources? If yes, how do you validate that data?
• Can I see the training data I have provided from my company? How can I perform monitoring or get notifications against inadvertent or intentional data leakage?
• If you are using generative AI, how do you prevent hallucinations (or do you)?

I am certain I could come up with more questions than this, but the list is already long. I may well release a V2 of these questions in the future, but I wanted to get this out before I start my weekend.

To close this article, I'll quote the closing statement from Chris that I liked:

We have ONE job

Protect those around us.

Let’s do it right