Will future generative AI models be larger or smaller?

Welcome to NRI's LinkedIn newsletter, a monthly curation of content and analysis from our team of experts. Be sure to subscribe to stay current on the most important topics in the financial services industry!

In 2017, eight AI researchers published a groundbreaking paper titled “Attention is All You Need,” introducing a new AI architecture known as the Transformer. This innovation significantly advanced AI by enhancing the performance of existing deep learning models and enabling the development of large language models (LLMs).

The Transformer architecture has since become integral to nearly all LLMs offered by major AI providers such as OpenAI, Google, Microsoft, and Meta. However, there are concerns about the sustainability of LLM progress.

1. Hallucination: Transformer-based LLMs generate textual responses probabilistically, which means they have a certain likelihood of making errors when interpreting training data or producing responses.

2. Resource Scarcity: The deployment of LLMs demands substantial resources, including electric power, computing capacity, and extensive training data. The scarcity of these physical resources threatens the continued advancement of LLMs.

To address the resource consumption of Transformer models, novel technologies are emerging. One notable example is the development of small language models (SLMs), which are trained on smaller datasets and require less computing power than LLMs, yet still deliver highly accurate outputs. Read the full article.

Generative artificial intelligence improves how insurance firms operate and enhances customer experiences by generating human-like content and accelerating knowledge-related tasks. According to the International Data Corporation, Gen AI global spending will reach a staggering $143 billion by 2027.?

The Japanese insurance industry has mostly been integrating Gen AI to increase operational efficiency and for internal uses such as employees FAQ, collecting information and ideation. In contrast, banks, including Capital One and J.P. 摩根 , increasingly use Gen AI for other operations such as risk modeling in fraud prevention and detection, credit scoring and sophisticated risk scenarios that leverage financial data to ensure more accurate and informative predictions.

According to an August 2023 Japan Data Scientist Society survey, a noticeable gap exists between the US and Japan regarding general business' adoption of Gen AI (in trial and in use): US 25.9% vs Japan 10.9%. NRI conducted surveys in May and October 2023 that illustrate different industries' Gen AI adoption across Japan, with the financial and insurance industry showing faster growth compared to other industries: 8.9% in May and 15.8% in October.? Read the full article.

Thanks for reading! Be sure to subscribe to NRI's newsletter here for monthly updates and contact us to start a conversation about working together in the future