The Challenge of Managing Cloud Spend in the AI Era

Being a CIO gives you unique insight into the latest and greatest of enterprise tech. It also shows you the problems that arise alongside these technological shifts.

For example, in the mid-2010s, when I was the CIO of Pure Storage , The Great Cloud Migration was all the rage. But, as more companies moved their workloads to the cloud, an issue surfaced: effectively managing cloud spend became incredibly difficult if not downright impossible. Startups took note. Soon, my inbox was flooded with companies promising to reduce my cloud costs.

These offerings worked fine in the short term and provided valuable insights in to our cloud spend. This worked for a while but did not have considerable impact. What we and other companies needed something for the long haul. How could we predict better, and subsequently prepare, for what our cloud costs might look like in the future? More importantly, what options were available to us as customers to manage costs when we knew that we would be using more cloud services?

It took a few more years for me to find such a solution. I was a customer of a few different companies and had seen several solutions come to market.? One of these products, Archera , resonated deeply with my inner CIO, so much so that I became an early supporter and then, once at Ridge , moved to become an early investor.??

Just like the widespread migration to cloud computing, the emergence of Archera has now coincided with another seismic technological breakthrough: the AI takeover. Cloud costs, more than ever, are through the roof. Not only are many companies unprepared, but some might not notice until the decimal has moved quite a ways to the right.

At its core, Archera wants cloud cost management to be automated, data-driven, and clearly impactful. The company's solution moves away from current cost management processes —tedious and unrewarding manual work for engineering teams that are usually out of date the moment a decision gets made. Engineering teams are and should not be in the cost management business. Software, data, and machine intelligence should solve this.

Here’s a closer look at the difficulties of managing cloud costs, and the new approach that’s bolstering bottom lines and clearing up the cloudy dispositions of CTOs and CIOs.

Problem #1: Cloud spending up, governance down

Cloud spending was already on the rise but AI’s coming-out party is further emptying companies’ wallets as they try to remain competitive and boost efficiency. It’s unfair to pin the overspend on the software engineers using most of these cloud resources; after all, their job is to train and deploy models that impact the company, not track cloud costs.

Managing and planning for cloud spending is a challenge that’s plagued companies since the beginning of the modern cloud era. The variables, and the number of ways businesses can lose money, are practically endless. Customers are breaking the bank on cloud platforms that provide more services than in recent years. Meanwhile, governance and best practices for using these cloud services are not up to par.

The cloud overspending problem is often a slow burn. Cloud costs start off small at first and then slowly get lost in the sauce, even as cloud storage increases along with spending. The inability of companies to predict and budget for cloud costs eventually takes its toll. Multiple siloed business areas—customer success, marketing, engineering—expand their usage, and cost optimization for cloud services becomes an uphill battle.

Of course, it doesn’t always take several weeks for a company to suffer losses from poor cloud cost management. One company lost a cool $300K —in a weekend—due to a minor keystroke mistake. Yikes!

Problem #2: Cost optimization, where art thou?

As I referenced in an earlier piece , co-authored by myself and Palvi Mehta , cost optimization for cloud spend requires engineering agility, financial control, and predictability. The cloud provides plenty of the former, but control and predictability have been missing in action for a while. With OpEx costs escalating rapidly, bottom lines, margins, and earnings are all taking a hit.

I previously mentioned the engineering teams, the ones predominantly using and managing public cloud resources. They’re software engineers, not fiscal savants concerned with optimizing spend. Their priorities differ from the finance folks. For engineers, reliability and usage outweigh cost considerations; and minimizing cloud spend can drastically impact innovation, delivery, and customer service.?

On the other side, financial teams are unable to see and comprehend cloud costs enough to act in the best possible way. Yes, they hold the budgetary reins, but can’t operationalize changes without help from their engineering and DevOps colleagues. While I tried hard to empower finance teams to do this in my past roles, the reality is engineering teams are using more and more cloud services every day. It is therefore unrealistic to expect finance teams to become experts in how cloud spend surges every month when a new service gets used.?

Long-term commitments may seem like the logical solution to the cost-optimization issue. The steep discount of a three-year reserved instance can look mighty tempting. But what if the unexpected happens over that three-year stretch? A global pandemic, for example, or the litany of business-related speed bumps that can pop up out of nowhere? And what about all the excitement around AI? Aside from the monetary losses, companies in this predicament won’t be able to change infrastructure based on the needs of the product. Agility: down the drain.

How can businesses effectively balance financial concerns with engineering concerns? How can they achieve flexibility and savings that align with business projections, a task that requires weighing thousands of variables and infrastructural purchasing strategies?

Don’t let the cloud cost you

If CTOs/CIOs are looking to avoid overspending on the cloud, I always point them to Archera’s cloud resource automation solution that proactively manages and de-risks an organization’s cloud resources.

Archera is the glue between finance and engineering teams, optimizing their mix of instances and maximizing financial efficiency without tarnishing customer success. Archera’s continuous cost optimization ensures peak flexibility that guarantees buy-back of unused commitments. Customers can reap all of the discounts of three-year reserved instances and still get out of the contract if circumstances change.

Zooming out, I believe Archera delivers in the three key areas that help get cloud spending back on track:

1. Visibility

• The level of finance-DevOps collaboration needed to successfully manage cloud resources is unrealistic. Different teams, different priorities. And they simply don’t have the time. The answer is to give finance the necessary visibility sans DevOps involvement.

2. De-risking of commitments

• Guaranteed buy-backs of commitments. This is the future of cloud spending: higher savings, no risk. Customers shouldn’t get stuck in a long-term commitment—and the infrastructure that goes along with it—if their needs are no longer met. When business cycles change, engineering teams should not be punished as a result but should have optionality. Archera gives them this.

3. Seamless adaptability

• Cloud environments are incredibly fluid. Pricing models, infrastructural needs, budgetary constraints—all of these factors are changing in real time and algorithms such as Archera’s enable users to maintain an optimized set of commitments across multiple cloud platforms.

Automating and optimizing cloud resource management is a must for companies that wish to improve efficiency and remain competitive. It’s all about the long term, the ability to predict and prepare for future cloud costs and, with AI pushing enterprise cloud spend to almost $74B in Q4 2023 , CTOs and CIOs need to act fast.