“You don't want data to hallucinate”– data experts talk model tuning and the need to understand AI better

The rapid rise of AI, and Generative AI in particular, has given SaaS companies cause to consider some of the risks these new technologies could potentially expose them to, particularly around data.

Research from Panintelligence revealed in its new report AI value or vanity? How SaaS companies are approaching innovation shows that two-thirds (67%) of SaaS companies have already added AI capabilities to their products, and more than a third (38%) have launched Generative AI functionality within the last 12 months.

However, companies who still need to prioritise data quality could be training their models on data that compromise prediction accuracy and create unfair or discriminatory outcomes.

We asked three data experts to consider this at our recent Data Pioneers event in London.

“We are going to go through a period of frustration where everybody realises that hallucination [in AI] is a feature and not a bug,” said Leon Rees, Startups and Venture Capital Advisor at Amazon Web Services (AWS) . “We are going to have to mix it in with something else when we know that we don't need the creativity of a model, we need the accuracy of a different model.”

“There are lots of scenarios where you might look at a dashboard and instantly be able to say based on your experience, “oh, that's not right”,” said Sam Strong, EMEA Sales Director at Rivery , a SaaS platform that provides supporting data ingestion, transformation, and orchestration. “The next steps are going to be to start to see technology that helps enable us to do that.”

“Maybe I'm a bit old fashioned, but data is data,” adds Svetlana Tarnagurskaja, CEO and Co-founder of The Dot Collective , a data and cloud consultancy. “You don't want data to hallucinate. If you're counting something, you want your number to be accurate… I think what we are going to see is people kind of cooling down of the hype a little bit and getting real.”

“There are going to be some use cases where understanding the reasoning behind the model is absolutely key to both being able to explain it internally and get that buy-in inside the organisation, but also to be able to actually say “is this right or is this wrong, and is this just missing the point”,” added Sam Strong.

“I think we are going to head into a world where Causal AI is blended alongside Generative AI", Leon Rees concludes. “We are building technologies that allow us to fine tune models, and customers can fine tune models in their own VPCs (Virtual Private Clouds). And then we are looking at ways of how do we mix those models with Generative inference and Causal inference so that we can get real accurate data when we need it, and real creative summarisation when we need it as well.”

Watch highlights from this expert discussion here. Alternatively, you can download a copy of our report: AI value or vanity? How SaaS companies are approaching innovation here.