The Continuing Relevance of Open Source

As medieval fortifications were developed and built, devices had to be devised to ensure the safety of those inside. To counter the possibility of siege towers, architects and engineers devised several additional external barriers that should help protect the walls of the castle. One such device was a moat. Usually, a ditch full of water and/or spikes that circled the walls of the castle and made sure that siege towers and ladders were kept at a safe distance from the walls.

An update of the idea of the moat was made by Warren Buffett in a 1999 Fortune Magazine article where he said:

“The key to investing is not assessing how much an industry is going to affect society, or how much it will grow, but rather determining the competitive advantage of any given company and, above all, the durability of that advantage. The products or services that have wide, sustainable moats around them are the ones that deliver rewards to investors.”?<link>

The concept of the moat was to remain an investor only concept if a?Google memo?had not leaked to the web and been verified by Dylan Patel and Afzal Ahmad. Whether the conclusions of the engineer are valid or not, is a question for later but there is a point mentioned in the memo that is worth exploring: the relevance of Open Source on democratizing knowledge and “drying moats”.

In 1999, Eric Raymond published “The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary” where he said that “Every good work of software starts by scratching a developer’s personal itch.” Later on the text, Raymond acknowledges that “while coding remains an essentially solitary activity, the really great hacks come from harnessing the attention and brainpower of entire communities.” This network amplification of knowledge is responsible for the rise of Linux as a major operating system from a personal project in 1991 to the major corporate server OS today supporting a myriad of hardware and uses. The same goes for other Open-Source projects such as Apache, Kubernetes, and Rancher, among others, which have come to be the?de facto?standards for the markets they cater to.

A good explanation for the growth of communities around such projects is that the “entry cost” is low for individuals and negligible for organizations. One only needs a computer and an Internet connection to begin “hacking away” on any Open-Source project. This was true, to a certain extent, of all Open-Source projects with the notable exception of the use of expensive GPU hardware for AI projects, especially those exploring Large Language Models (LLMs). In this highly specialized area, companies with high investment capacity thrived. The public release of OpenAI’s ChatGPT was quickly followed by Microsoft’s announcement of investment and use of their platform and by Google, playing catch-up, announcing Bard.

With no visible signs of IBM attempting to revive Watson, the LLM market was destined to become a private arena for a few, very deep, pockets.

Given the attention that ChatGPT, Bard and?Sydney?were gathering from the press, one can only imagine how much individual developers were scratching the itch to expand their Py Torch and TensorFlow experiments to the next level, being stopped only by the cost of the hardware they would need to reach a fraction of the capacity required by these massive models to be trained.

Two unrelated events were able to open the floodgates and allow the rapid development of communities resulting in the accelerated development of use cases for AI in a way that business architects and commercial analysts of those large companies had not expected. A new AI Bazaar was made possible thanks to the Crypto market crash and Meta.

Cryptocurrencies have always been a high volatility game. With investors and miners pouring significant resources in a system that could yield stellar profits or losses in a matter of seconds. Processing capacity was the name of the game and miners would snatch GPUs off distributors’ stocks even before these boards could reach gamers and developers. The crash of crypto exchanges and specially the investigations regarding FTX have drastically lowered the trust in the crypto market, reducing liquidity and making the maintenance of specialized mining hardware a costly endeavor.

In January 2023, Nikkei Asia published a piece where they?say:

The price for graphics cards – a computer component crucial for data-intensive activities like gaming – has dropped about 60% from a 2021 peak amid a slowdown in cryptocurrency mining.

Although gaming was quoted as the main beneficiary of this sudden cost reduction, the individual AI explorer also benefited from an array of both new and used GPUs flooding the market at reasonable prices.

And along came LLaMa

On February 24, 2023, going against the proprietary control of the LLM models established by OpenAI, Microsoft and Google, Meta publicly released their Large Language Model Meta AI (LLaMa) and made it available to researchers on a case-by-case basis hoping to expand the reach of their model and method. By the end of the month, LLaMa had already?leaked?through an anonymous message board and became suddenly available to anyone with an Internet connection. As usual in these cases, Meta has been diligently attempting to bring down mirrors of their code but once the genie is out of the bottle, it is usually a difficult endeavor to put it back in.

In a brief period, “AI explorers” were in a position where they not only had cheaper access to the hardware required but also to a working Large Language Model that could be studied and cheaply trained in common hardware for new, more specific needs.

In less than a month, according to Google’s leaked e-mail, not only were new implementations and methods devised, but recent studies, such as?LoRA?– Low-Rank Adaptation of Large Language Models were implemented and made available for?public use. This is an especially important step since it solves the question of scale in the training of these models. Miniaturization efforts were also seen in the form of LoRA trained LLaMa models that can run on hardware as small as a?Raspberry Pi.

I believe that this amount of innovation in such a small period proves that community is key not only to produce knowledge but also, to develop new technological implementations that have the potential to change society as we know it. Google’s leaked e-mail brings an interesting timeline of what happened since Meta released LLaMa to the public.?

We are about to see changes in a speed that we haven’t yet experienced. It took roughly 10 years from the commercial launch of the TRS-80 and the Apple ][ (both from 1977) to AOL in 1985, and it was only in the 1990’s that commercial Internet access really caught on. When Apple released the iPhone, in 2007, they unleashed a societal change that has taken us to where we are today, accessing the world from a device in our pockets. AI and language models are posed to be the next step and I seriously doubt it will take another 20 years to see the effects of this technology given what we have seen in the last few months.

So, buckle up for a fast, and bumpy, ride, and if you don’t want to be just a passenger in this ride, please consider joining or supporting one of the several Open-Source projects that have already sprouted and will require extensive participation to develop code and guidelines on bias-free (as much as possible) datasets for AI training. AI deployments have reached the point where they will cause changes in real life. The Open-Source approach is key in keeping the genie under control.