The rise of open source LLMs

A Warm Welcome to OS LLM

The sudden explosion of generativeAI owes much to LLMs, whose "self-attention" architecture allows for massive data parallelization. From Retrieval Augmented Generation (RAG) to personalised automated customer support and new autonomous agents, the variety of business use cases for LLMs is already vast, offering an exciting source of business competitiveness and productivity gains rarely seen in the past.

Along with this explosion, the first AI as-a-service providers (Anthropic Claude, OpenAI ChatGPT or Googler Bard, etc ) were proprietary.? But if you look at the history of IT, the introduction of proprietary software has been rapidly matched by the development of so-called "open source" (OSS) alternatives. This is also happening in the LLM space.

In fact, OSS has gained weight since the introduction of the Linux operating system in 1991, which quickly gained popularity against proprietary market tenants such as OS2 and Sun OS, among others. Similarly, there was the early Internet browser war between Netscape and Microsoft and the emergence of the Mozilla open source project, which developed the Firefox browser.

Since then, open source softwares such as Android, WordPress or Apache have often dominated proprietary applications. In databases, OSS such as MySQL and MongoDB are leading the way. In e-commerce, Woo-commerce and OpenCart OSS are often ahead of, or at least on a par with, proprietary software such as Shopify or Wix.

LLMs face the same IT trend, with the venue of the likes of Mistral AI, GPT-NeoX-20B built by EleutherAI, or LLaMA by Meta AI among others. From the use cases that have leaked, those models seem also revolutionary (Exhibit 1).

Exhibit 1: OS LLM use case examples

Is Open Source a fad?

Open source refers to a development model in which products are created and maintained by a public community rather than by a single company. The source code for these products is ?thus publicly available, allowing anyone with the skills and interest to contribute to the development and improvement of the software.But if OSS is typically freely available for anyone to use, modify and redistribute, this removes a large opportunity pool of value capture, in terms of traditional licensing and SAAS revenue.

Is then OSS doomed to fail? Not necessarily. For one, an open source model can be a pure deterrent, e.g. when IBM blessed Linux, it also blocked Microsoft from the Blue Giant's bread and butter. Google did the same trick when it pushed Android to block Apple in the mobile ecosystem. ?Second, open source is also a smart "attacker" strategy: it allows ?to build critical mass quickly and to always be at the top of the development cycle - two aspects (performance and scalability) that make all the difference in software. The OSS development model also eliminates the need for significant up-front investment in the development of new products, and is often cheap because the work is done by passionate volunteers. Open source development is also free from vendor lock-in, giving organisations greater flexibility and control over their technology stack.

In a nutshell, the OSS strategy, then, is to create a large pool of value for a possible small appropriation of value, but the multiplication makes it larger than if the company had chosen a high proprietary source and high price. The key is nevetheless to find multiple complementary revenue streams. Red Hat, an open source pioneer before it was bought by IBM, generated most of its revenue by selling service contracts and complementary software applications for the Linux operating system. Typical OSS revenue streams include selling support and consulting services, offering hosted or managed services, or creating proprietary add-ons or plug-ins that extend the functionality of open source software.

?

Selecting the right LLM

The future of open source is therefore here to stay - and this raises the final question: should one choose OSS over proprietary, and if so, which one?

The choice between open and closed software depends on a number of factors. As shown in Exhibit 2, typical criteria include security, cost, performance, scalability, etc. In general, proprietary systems win on security and reliability, but this has been changing in recent years in favour of OSS.

Exhibit 2: comparing OSS and prop software

Regarding LLMs, some important points have emerged:

1) Open source LLMs have significantly narrowed the quality gap with proprietary closed LLMs in a variety of tasks such as chatbots, etc. Performance metrics such as token task completion and accuracy rate are now on par with proprietary models.

2) OS LLMs provide access not only to the source code, but also to its architecture and training data. This facilitates rigorous testing and customisation of models.

3) Furthermore, the use cases for LLMs are only now emerging and are often being created by the users themselves. Thus, open source can drive innovation by harnessing diverse expertise, creativity and ideas.

4) Licensing fees associated with proprietary LLMs can be a significant financial burden. However, open source does not mean free. Organisations using open source LLMs should expect to pay for operational costs such as infrastructure and cloud services.

5) Security and privacy may be the Achilles heel of open source LLMs. Cases of data breaches and unauthorised access to sensitive information characterise LLM. On the one hand, open source LLMs place the responsibility for data protection on the user, and thus the user can ensure better security and privacy measures. However, newer tool-augmented LLMs mostly rely on closed LLM APIs, exposing internal company workflows and information to closed LLM APIs. Finally, the issue may not be privacy per se, but maliciousness.? This risk may be much higher in the case of OSS, as third parties may have access to the source code. As cybersecurity risks and ethical AI are the biggest issue for many companies, open source may create a lot of uncertainty, even more than proprietary LLMs.

6) Over time, generative AI technology may rely on more standardised and modular building blocks within software libraries (such as prompt templates that allow easier adoption and customisation in downstream applications). The interoperability of pre-trained models across platforms should then dramatically reduce the need to retrain large models and make LLM a natural software element of many business cases.?

The speed of this standardisation and modularisation will also determine how open source LLM will be used in the future.?

?

References

Ahmed, T., Bird, C., Devanbu, P., & Chakraborty, S. (2024). Studying LLM performance on closed and open source data. arXiv preprint arXiv:2402.15100.

Blancaflor, E. B., & Samonte, S. A. (2023). "An analysis and comparison of proprietary and open source software for building an e-commerce website: A case study". Journal of Advances in Information Technology, 14(3), 426-430.

Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at work (No. w31161). National Bureau of Economic Research.

Bughin, J (2023), ?Is the impact of generative AI overhyped? Insights from one hundred AI business success stories, Medium.

Finlayson, M., Swayamdipta, S., & Ren, X. (2024). Logits of API-protected LLMs leak proprietary information. arXiv preprint arXiv:2403.09539.

Kogut, B and Anca Metiu,(2001). "Open-Source software development and distributed innovation", Oxford Review of Economic Policy, Volume 17, Number 2, pp. 248-264.

Irshar, M.; Ali, A.; Ibrahim, S. A (2019), Comparative Analysis Between Open Source And Closed Source Software in Terms of Complexity and Quality Factors.

Lerner, J. and J. Tirole, (2002). "Some simple economics of open source", Journal of Industrial Economics, Volume 50, Number 2, pp. 197-234.?

Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The impact of AI on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590.