Build Vs Buy: A Key Initial Decision in Generative AI Initiatives

The decision between "build your own stack Vs buy" significantly influences an organization's initial Generative AI journey. This decision saves organizations a lot of time; time means money. Making the right decision at the beginning of the Generative AI journey positions the organization ahead of the competition by saving crucial amount of time.

Currently, many commercial models such as OpenAI, Google, and Amazon are available on the market, offering faster product development and deployment with minimal infrastructure requirements. However, they do have drawbacks, and any organization deciding to use such LLMs should conduct a detailed analysis that is tailored to their specific use case.

When trying to build Generative AI stack on your own for your organization, the first step is to choose foundational open-source LLMs (like Mistral..). You could then add your data to achieve better accuracy and performance. However, you will have to set up on-premises or cloud infrastructure for the model to run. This needs computing resources, including powerful GPUs, CPUs, and data storage, as well as resources to manage the infrastructure and data. Organizations could obtain all these resources from AWS, Microsoft, or Google cloud providers.

Even though many factors go into the decision of "build your stack Vs buy," organizations should consider the following key factors:

Time-to-market

When you use commercially available models (like OpenAI, Google, etc.) and their APIs, you can quickly focus on building the product rather than spend time and money on infrastructure. This allows organizations to scale quickly, significantly increasing the time-to-market. This still comes with a few drawbacks. For example, organizations are dependent on the reliability and availability of the models, which might not be ideal for some of their applications and their usage.

Building the Generative AI stack on your own means you must first work on procuring and building the resources needed, setting up the infrastructure needed for the resources, developing the architecture, and keeping up with the pace of this fast-changing technology. This is a lot of work before even starting to develop the product.

Cost

The commercially available models (like OpenAI, Google, etc.) and their APIs are expensive. Organizations must pay every time a user queries the model and asks for the datasets to be retrieved, and they must pay for the token (input and output) cost for the model. There are some charges for the code interpreter as well.

Building your own stack is also not cheap. Organizations have to pay cloud providers (AWS, Aura, GCP..) based on usage in terms of compute time and storage. For example, GPU instances on AWS could cost organizations anywhere between $2 and $20+ per hour, depending on the instance type. Storing and managing the data costs vary depending on the size of the training dataset and redundancy needs. Storage using AWS S3 can vary from $0.021 to $0.023 per GB per month, with extra costs for operations. And after organizations have successfully integrated and deployed their AI product, maintenance costs, both in terms of infrastructure and resources, have to be accounted for as well.

Overall, if the dataset is small and the use cases don't require the use of APIs that many times, then the cost will be lower. Typically, usage, datasets, and changes keep increasing in the product journey. Whether an organization opts to build or buy, it should run through all of its use cases, assess the costs associated with employing these models before implementation, and ensure that it aligns with the budget.

Switching between vendors

Once an organization starts developing products and applications on top of the commercially available models, switching between different vendors is challenging. The process involves migrating applications, the data and code used to build the product, integrations, and embeddings to the other vendor, requiring many months of work at the very least (and with no guarantee that the migration will be successful).

There are potential alternative strategies to facilitate a smoother transition between vendors, such as leveraging SaaS products like Chroma, Weaviate, and Pinecone for vector databases. Another approach involves initially using commercial models solely for testing products; upon confirmation of their effectiveness, organizations can proceed to construct their own models. While this approach grants organizations greater control, it necessitates comprehensive efforts on their part.

Data control

When organizations use these commercial models, they transfer their proprietary data, documents, and files from their environment to the vendor, relinquishing control over their data. The process involves intricate details, such as the vendor chunking the data before embedding and storing it as vectors, which now become their data. In the future, if organizations wish to switch vendors, retrieving these documents, files, and datasets may prove challenging, as the current vendor might retain them. Failure to preserve data within their environment renders migration to a different vendor impossible.

In contrast, when organizations build their own stack, they retain control over all their data, infrastructure, and resources. This is a crucial consideration for highly regulated industries like banking, finance, and healthcare, where storing data in an external vendor environment might pose security and privacy risks.

Summary

For organizations with limited resources, opting for commercially available models and APIs (e.g., OpenAI, Google) offers a rapid development solution without the complexity of building their own model infrastructure. However, drawbacks, such as vendor-switching challenges and data control concerns, should be considered.

In contrast, organizations with ample resources might find building their own stack a preferable option. This strategy enables better visibility into the infrastructure-building process and ensures complete data control. However, this approach requires significant technical know-how, and building and maintaining these models is a laborious and expensive process.

Finally, the decision to "build your own stack Vs buy" should factor in various considerations, including business use cases and specific product needs, which vary for each organization.

Want to dig into all of this in even more detail? In these two videos, I explore the decision-making that goes into the Generative AI products, specifically "build your own stack Vs buy.”