Managing the risk of Large Language Models (LLMs) in Financial Services

The financial industry is witnessing an emerging trend of Large Language Models (LLMs) applications to improve operational efficiency. This article, based on a round table discussion hosted by TruEra and QuantUniversity in New York in May 2023, explores the potential use cases of LLMs in financial institutions (FIs), the risks to consider, approaches to manage these risks, and the implications for people, skills, and ways of working. Frontline personnel from Data and Analytics/AI teams, Model Risk, Data Management, and other roles from fifteen financial institutions devoted over two hours to discussing the LLM opportunities within their industry, as well as strategies for mitigating associated risks.

The discussions revealed a preference for discriminative use cases over generative ones, with a focus on information retrieval and operational automation. The necessity for a human-in-the-loop was emphasized, along with a detailed discourse on risks and their mitigation.

The roundtable discussion was a joint effort by TruEra and QuantUniversity and commenced with presentations given by Agus Sudjianto , EVP and Corporate Model Risk Head at 富国银行 , Sri Krishnamurthy, CFA, CAP Krishnamurthy, CAP, the founder of QuantUniversity , and Anupam Datta , co-founder of TruEra and a former 美国卡内基梅隆大学 professor.

Key Takeaways:

1. Use Cases for Large Language Models (LLMs) in Financial Institutions (FIs):

Leaders in the financial industry discussed the best use cases for LLMs in FIs. The preferred applications include information retrieval, contextual search, internal chat-bots, data extraction, operational automation, and summarization. Notably, there is a cautious approach towards direct client-facing use cases due to the perceived risk and the current technology reliability.

2. Deep Dive into Information Retrieval Use Case:

The round table included an in-depth discussion on the information retrieval use case, where the ability to control the LLM's operation baseline is a key motivator. The process includes LLMs creating embeddings on a trusted internal corpus of documents, translating user queries into embeddings, and matching them. The critical role of the human-in-the-loop was stressed, particularly for ensuring the answers' adequacy.

3. Risks Associated with LLMs:

Several risks were identified, including accuracy/reliability, hallucinations, security/IP/privacy risks, and reputation risk. There were specific concerns about LLMs' mathematical abilities and the potential for misuse of public interfaces like ChatGPT.

4. Approaches to Manage the Risks of LLMs:

The round table revealed the need for robust controls for LLMs. Suggestions ranged from end-to-end risk views, banning unsupervised LLM use, to applying robust MRM controls. While there is confidence in managing risks for information retrieval use cases, challenges remain in areas where outcome-based testing is the only option.

5. Implications for People/Skills/Ways of Working:

There were differing opinions on LLMs' impact on job roles. However, all agreed that job roles, especially data scientists and coders, would change significantly. LLMs democratizing the data science space could have both positive and negative implications for individuals and organizations.

Conclusion:

The round table discussion underscored the potential of LLMs to enhance operational efficiency in the financial industry. Leaders agreed on the need for a judicious balance between harnessing the technology and managing associated risks. As LLMs continue to evolve and improve, their adoption could redefine job roles and ways of working, making it imperative for professionals in the sector to adapt and evolve in this rapidly changing landscape. The journey with LLMs in the financial sector is just beginning, with much learning, exploration, and innovation to come.

The roundtable was moderated by Shameek Kundu from TruEra who also put together this detailed 4-page synopsis titled "Managing the Risk of LLMs in Financial Services" is available for download here.

Note: In spring 2023, I taught two graduate courses at Northeastern University where many projects involved AI and Generative AI related themes. There were many lessons learnt which I would like to share. The AI Risk Management Newsletter took a pause and is now back! Thanks for all the support and the amazing 3000+ community interested in AI Risk related topics. If you have ideas or themes you would want us to cover, please DM me!

Sincerely,

Sri Krishnamurthy

QuantUniversity