TO AI or not to AI, that is the question

Nearly every panel discussion across financial services today has at least one question on Artificial Intelligence.? My favorite question on this topic comes from Goldman Sachs' Head of Global Equity Research in GS’ June 2024 publication on the topic: “We estimate that the AI infrastructure buildout will cost over $1tn in the next several years alone, which includes spending on data centers, utilities, and applications. So, the crucial question is: What $1tn problem will AI solve?”

Companies operating in the Private Credit world and in the broader Investment Management business have been wrestling with that question as well. Our Private Credit team is pleased to share the four insights our conversations in the industry have yielded while we also want to be transparent about our knowledge limitations.

Insight #1: Only Use AI Where You are Trying to Generate Alpha

Most CTOs we spoke with believe that the cost and effort required to support AI initiatives only reaches a positive ROI when looking for alpha. GenAI models are being utilized for portfolio analysis, research synthesis and various risk/compliance functions while large language models are being utilized to generate and analyze client communications and documents.

“We think about data in two categories: alpha and non-alpha”

??????????????????????????????????????????????????????? - Anonymous CTO ($25B+ AUM)

?

Our comments here are limited to our conversation and share some of the lessons learned by early adopters. There are multiple layers of cost to AI solutions: Organization of internal data, purchase of external data, engineering cost to construct and operate the models, data scientists to build and test the models and, of course, the costs of bad decisions.

• The importance of organization of internal data is explained by the well-worn adage garbage in / garbage out but the further insight is that real differentiation comes when you can successfully link your data with external data. ?Assuming tools can extract internal data cleanly that is part of a general data lake has been one of the most common mistakes, leading to months if not years of delay to get AI projects initiated. Best practice is to determine internal data requirements as a first step and to test its viability prior to an external spend. Establishing a data inventory and investing on the organization of internal data first has been a game changer for many.
• CTOs nearly unanimously agree that external data is their #1 expense and has been one of the fastest-growing expenses in Asset Management for over a decade.? Best practices include confirming completeness of data sets, especially with regards to historical data where there are frequently gaps prior to signing any agreement. Pricing based on utilization and eliminating data duplication are also effective ways to control expenses.
• Constructing data labs has been done by quant shops for the last 20 years and some teams process in excess of 70 million data points each day. Although many organizations have fully migrated to the cloud, some of the most sophisticated quant shops recommend keeping your model code in-house as it is intended to be a primary point of differentiation and should be treated with the highest degree of confidentiality.
• The cost of bad decisions is a bit harder to pin down as it can come in so many flavors.? You can always find correlation in data and AI will always provide you with an answer, so the most important aspect of your data science is ensuring that you are asking the right questions (aka, prompts). Best practices fall in the scope of AI governance below.

?

Insight #2: Open AI ChatGPT is a Powerful Solution to those entering the Wealth Channel

The democratization of private investments (making them accessible to wealth management clients) has created an expense that some providers may not have fully thought through prior to launch: creating a service channel to answer client questions. Fortunately, many quickly adopted ChatGPT tools that have been maturing in the service industry for nearly 10 years.? Best practice is to extend these tools to reinforce internal training.

?

Insight #3: Operational Efficiency Can Be Achieved without AI

To many, this statement may be a lightning rod.? Product managers around the world have been adding AI components to nearly every platform imaginable. While it is true that machine learning will aid in the identification of breaks and can suggest fixes, the reality is that the fundamental capabilities to identify data inconsistencies have existed for years. 50% efficiency improvements can be achieved with good data organization, forecasting and workflow-based exception management. To really manage risk, we’ll take hard thresholds on operational data over a shiny new AI engine eight days a week.

It should be mentioned that there is some good work being done with Large Language Models for data extraction on simple, consistent documents such as Agent Notices. Market leaders in this space have been able to extract 80+ data points from these documents with 85%+ accuracy for a couple of years now. Credit agreements are another story.? Although many foundational AI models are being used to digitize credit agreements, the results have been somewhat lackluster to this point. Coming soon, however, is the verticalization of AI engines. This is where the AI model is designed for a specific task and requires much less training. The clearest technology analogy to this is the proliferation of specialized chips from Tesla, AWS and Meta that are fit-for-purpose.

?

Insight #4: AI Governance is Mandatory

AI Governance, like any Risk Management program, should exist on several levels.? Establishing the framework, assembling and aligning the team and executing the management and monitoring are well-worn paths for many organizations with strong Enterprise Risk Management (ERM) programs and we would argue that the COSO framework would be a good roadmap here as well.

Many consulting firms (including our own) offer workshops and start-up plans to get AI governance frameworks in place.? These begin with simple concepts such as Access Management, not dissimilar to the learning curve most firms went through when migrating to the cloud 10-15 years ago.? However, there are still a lot of firms out there that do not realize that employees are utilizing free ChatGPT engines and unknowingly sending Confidential and Internal information and communications into the public domain. First rule of AI governance: You need to know what data is leaving your system.

Access Management should also include controls regarding how external data is accessed by internal resources.?? Again, unknowingly, many firms throw money out the window allowing employees to access the cheapest of data through the most expensive channels. Again, the lesson here is back to the first step of any AI program, getting your data organized and inventoried so you can manage it.

Establishing Compliance Policies and then monitoring them might seem like a daunting task but there are a few emerging platforms that facilitate this governance.? This includes the governance of the data science workflow and enables investment committee-level discussions and approval on AI-generated output before investment dollars have left the building. Of course, another reason to consider an AI governance program is that AI is one of the SEC Top 7 Examination Priorities for 2025.

?

Wherever you are on your AI journey and whether you agree or not with the positions we have outlined above, we are always open to a conversation.

?