Navigating the Costs of Generative AI for eCommerce

In eCommerce, generative AI has emerged as a powerful tool to enhance customer experience, streamline operations, and drive sales. However, harnessing this technology comes with a variety of costs and considerations. This is a guide focusing on the financial aspects of using generative AI. I’ll explore the costs of large language models (LLMs), APIs, human resources, new processes, machine learning operations (ML Ops), and foundation model operations (FM Ops). Furthermore, we'll discuss advanced topics like training and retrieval-augmented generation (RAG).

The Choices in Generative AI Deployment

When integrating generative AI into your eCommerce platform, you have three primary options: hosting your own model, using APIs to access a hosted model like ChatGPT, or utilizing APIs to access services with business user interfaces designed for specific use cases. Each choice carries its own set of costs and benefits.

Hosting Your Own Pre-Trained Model

I believe that training a model from scratch for ecommerce companies would be very costly, running into millions or tens of millions of dollars for data preperation, pretraining, training, testing, and tuning. Instead, I believe many companies may opt to use an open-source model like the Llama or Miatral models as a base. This approach allows for complete control over costs and the ability to fine-tune the model for specific purposes, such as to align with the company's brand voice.

Cost of Service APIs, Infrastructure, and Subscription

Hosting your own model involves significant upfront investment in infrastructure. You'll need powerful servers with ample GPU capabilities, which can cost anywhere from tens of thousands to hundreds of thousands of dollars. ?Planning the capacity and scaling these services can also be tricky as you need to balance the usage vs precautionary headroom.

Total Cost of Ownership: AI Operations / ML Ops

Maintaining your own model requires a dedicated team for AI Ops and LM Ops. These experts manage the deployment, scaling, and maintenance your models. Salaries for these professionals can be substantial, with experienced engineers earning six-figure incomes. The total cost of ownership includes not only their salaries but also the software tools and platforms they use.

Development Effort and Cost

While you bypass the extensive process of training the model, setting up and integrating the pre-trained model into your system still requires development work. This includes configuring the environment.? The most significant costs are in building software that is focused on applying the model to all the actual use cases you wish to support.

Cost of Business Change and Development of Business Tooling

You'll need to develop custom business tools and interfaces to configure, monitor and manage the implementations of applied LLM use cases and integrations with your ecommerce applications and architecture. This in turn requires significant changes to your business processes as well as redefining roles and responsibilities on your teams.? This development effort translates to additional costs and time.

Ongoing Development, Maintenance, and Support for Integration

Once deployed, the all custom developments, integrations and models need continuous monitoring, updates, and maintenance. Ass the business changes and new more capable models are released developments need to be refactored, change or redeveloped.? This adds to the recurring costs and necessitates a dedicated support team.

In-House Expert Knowledge Required

Running a self-hosted model demands in-house expertise. Your team must possess deep knowledge of AI, ML Ops, and FM Ops. Training and retaining such talent can be expensive and challenging.

Using APIs to Access a Hosted Model (ChatGPT, AWS Bedrock, Gooogle)

Cost of Service APIs, Infrastructure, and Subscription

Using APIs to access a raw model from a third-party provider can be more cost-effective than hosting your own model. Providers typically offer subscription plans based on usage, ranging from a few hundred to several thousand dollars per month. This approach eliminates the need for expensive infrastructure.

Total Cost of Ownership: ML Ops and AI Ops

While you may not need a full ML Ops and AI Ops team, you will still require some staff to handle API integration and monitor its performance. This helps in reducing the total cost of ownership when compared to hosting your own model. Unlike traditional cloud operations, your teams will need to estimate and negotiate future capacity requirements with your model providers to ensure that you have the capacity you need, when you need it.

Development Effort and Cost

The development effort is lower when using APIs. ?However, integrating the API with your existing systems and customizing it for your specific use case will still requires significant development work.

Ongoing Development, Maintenance, and Support for Integration

API providers handle most of the maintenance and updates. However, you must continuously ensure the API integration works seamlessly with your platform. This requires periodic development and support efforts.

Cost of Business Change and Development of Business Tooling

Using APIs often necessitates fewer business changes than hosting your own model. However, you'll still need to develop business tools and processes to fully utilize the AI capabilities.

In-House Expert Knowledge Required

While the need for in-house expertise is lower than with a self-hosted model, you'll still require some knowledge to manage the API integration and troubleshoot issues. This might involve training your existing staff or hiring new talent.

Business-Specific SaaS AI Applications with APIs and Business User Interfaces

Cost of Service APIs, Infrastructure, and Subscription

This option involves subscribing to a service that provides generative AI capabilities through a user-friendly interface. Subscription costs vary based on the provider and the level of service, typically ranging from a few hundred to several thousand dollars per month.

Total Cost of Ownership: ML Ops and AI Ops

The total cost of ownership is minimized as the service provider handles most of the ML Ops and AI Ops tasks. This reduces the need for in-house expertise and lowers overall costs.

Development Effort and Cost

Development effort is minimal as these services are designed for easy integration and use. You won't need to build or train models, significantly reducing development time and costs.

Ongoing Development, Maintenance, and Support for Integration

Ongoing maintenance and support are primarily the responsibility of the service provider. Your team will need to handle only the integration and ensure it meets your business requirements.

Cost of Business Change and Development of Business Tooling

Business changes are minimal with this option. The provided interfaces are usually designed to fit seamlessly into existing workflows, reducing the need for extensive business tooling development.

In-House Expert Knowledge Required

The need for in-house expertise is the lowest with this option. The service provider's user-friendly interfaces allow business users to leverage AI capabilities without deep technical knowledge.

Training Your Own Model

Self-Hosted Training

Training your own model involves acquiring vast amounts of data, preprocessing it, and using powerful computing resources to train the model. This process can take weeks or even months and requires substantial computational power, which can be costly. Additionally, you'll need a team of data scientists and machine learning engineers to oversee the training process.

Training Through a Provider Like ChatGPT

Providers like ChatGPT offer pre-trained models that you can fine-tune for your specific needs. This approach is more cost-effective and time-efficient than self-hosting. You can leverage the provider's infrastructure and expertise, reducing the need for extensive in-house resources.

Full Managed Service

A fully managed service handles all aspects of training and deploying the model. This option is the most expensive but offers the greatest ease of use. The provider manages the infrastructure, training, and maintenance, allowing you to focus on utilizing the AI capabilities for your business needs.

The Risks of Constant Evolution of Generative AI

The rapid evolution of generative AI means that new models and techniques are continuously emerging, rendering existing models obsolete. To stay competitive, you must invest continually in updating and retraining models, which can be quite costly, particularly for self-managed models. Even when integrating directly with model providers, you must account for development costs to utilize new model capabilities. If opting for a SaaS business AI solution, it's crucial to ensure that the provider is not dependent on specific models and has proven capability to swiftly integrate new innovations. Otherwise, your AI solution risks becoming outdated.

Conclusion

Choosing the right generative AI strategy for your eCommerce business involves balancing costs, risks, and capabilities.

Self-Hosting a Pre-Trained Model

Self-hosting offers maximum control and customization but requires significant investment in infrastructure, skilled personnel, and ongoing maintenance. It also involves high costs to stay current with AI advancements.

Using APIs to Access a Hosted Model

Using APIs reduces infrastructure costs and simplifies operations but still demands significant integration effort and some in-house expertise. It provides a balance between cost and flexibility.

Subscribing to Business-Specific SaaS AI

SaaS AI solutions have the lowest total cost of ownership and require minimal in-house expertise. They reduce development and maintenance efforts but offer less customization and control. It's crucial to choose providers that can quickly adopt new AI innovations to avoid obsolescence.

Balancing Costs, Risks, and Flexibility

Your choice should align with your business's needs, capabilities, and budget. Self-hosting is ideal for those needing maximum control, APIs offer a middle ground, and SaaS solutions are best for ease of use and lower operational burden. Evaluate these factors carefully to leverage generative AI effectively and stay competitive.

?