The New Backbone of Enterprise AI: Why Physical Infrastructure Will Shape the Future of a Software-Driven Economy

The following article is an 'Anonymous' Contribution written by a former Enterprise Seller at IBM.

Note from the Author:?

Enterprise tech might not sound thrilling, but as someone in sales, I’m always looking ahead to what’s next (and how to sell it). Let’s face it—we’re all in sales in some way, right? cue product and engineering screaming in the background.?

But seriously, enterprise tech trends don’t just shape the corporate world—they ripple through our economy, society, and everyday lives. From seamless device connectivity to frustrating data breaches or canceled flights, these innovations touch everything. So, let’s dive into how these trends affect us and where they’re taking us next.

The Data Center Era

For some general market context, Gartner estimates the worldwide enterprise IT spend for 2024 at approximately $5.2tn, broken down as follows:

IT Services (managed services, consulting, support, integration, BPO, IaaS/PaaS) – $1.6tn

• IaaS/PaaS - ~$300bn
• Cloud-related IT services - ~$700bn
• On-prem IT Services - ~$200bn
• 7.1% CAGR

Communications Services (telecom, network, content delivery, carrier services) – $1.5tn

• 3.0% CAGR

Software (not including IaaS/PaaS or software in datacenters) – $1.1tn

• Most concentrated in CRM, DBMS and Security software
• Networking is the fastest growing category of software at 15.6% CAGR
• 12.6% CAGR

Devices (computers, mobiles, displays, peripherals) – $730bn

• 5.4% CAGR

Data Centers (servers, storage, routers, switches, cooling, DC software, buildings) – $293bn

• Jensen says that total DC assets will double over next 5yrs
• 24.1% CAGR

IT organizations are expensive to run and it’s only rising: tell me something I don’t know.

1. Niche knowledge workers are expensive and we don’t have enough of them. (IT Services expenditures are only going up, most of it being captured in BPO, consulting and professional services)

I talk about this problem constantly, but it’s dwarfed by the daunting employment market. Companies need niche skills. Earlier this year I asked a client, a Corporate Director of IT, what he needed for his team to be successful. He laughed and asked if I knew any full-stack web3 developers that knew RPG (lol). Even if we ignore my anecdote, job seekers are currently encountering job requirements like: expertise in bioinformatics (e.g., Bioconductor, BLAST), legacy programming languages like Perl or R in biotech contexts, cloud platforms tailored for biotech (AWS for Genomics). Everyone that has unique and rare technical skills, listen to me: get your money now.

As the baby boomer generation retires, our national skill base will erode from the sheer number of workers that will leave the market, and I think our C-suites know it. Forbes claims boomers make up “nearly a third of the entire workforce and 56% of retiring boomers being in leadership positions” with 46 million expected to retire soon (2.2 million retired early in 2023). Specialized skills are becoming more specific with a dwindling population of workers who can support the need.

Now, I’m going to ruin the article: AI. Yup, there I said it. AI is here to supplement the retiring productivity we are about to lose, market-wide. Think about it. Most of the first-world is facing a population decline and our capital markets have begged the labor question for a while. “How does the company make more money if it can’t grow? Theoretically, we are going to have more labor (physical or information) and less workers.” All I’m saying is, the ROI is high enough that companies will prioritize better code-assistants, autonomous virtual assistants, ML models that optimize for specific outcomes, agentic AI and many more.

So, am I being replaced? Probably not. This is a long topic; I don’t want to get into it and I’m not a good enough writer to explain it. They are going to need you to do other things. There are a lot of examples but “specific generalists” is the only term I can think. Imagine, people who intimately understand processes, businesses, organizations or industries - ecosystems. And no, there will still be a need for subject matter experts (monitoring, customizing, maintenance).

Succinctly, enterprises face growing expectations while the job market tops out on skills/manhours. JPM grew Assets Under Management by 24% ($656 bn) in 2023, yet they hired just 5.52% more employees into the workforce. We are witnessing the great exchange of manhours to kWh.

2. The complexity of businesses is exponential. (See IT Services and Software making up half of the tax income our federal government will receive this year; Software 12.6% CAGR)

This always makes me think of the first time I saw architecture diagrams for a big company. I’m not saying this to be critical of businesses, but rather to emphasize the raw size and reach of these organizations. They have to do so much to function daily. How many daily alerts do you think the average F100 sees in their ITSM? How many VMs are they actively running? How many vulnerabilities are they managing? This is where AI/ML thrives.

CIOs are prioritizing the reduction of technical debt that's holding back their organizations. Outside of finance forcing the move to automation, the market will have to use these tools to manage the increasing volume of data (2.5 quintillion bytes) and the shrinking talent pool. AI applications are going to make their lives easier. Even if our talent pool was growing, it would have to be growing at an unrealistic rate to keep up with the need. Ecosystems of integrators, consultants and professional services will be in high demand as companies connect, prepare, process and test data in repeatable, scalable and optimized workflows. Seth Cohen (CIO of P&G) coherently describes the use of AI across business units and why companies “can’t lead with technology” in The Business of AI.

Ultimately, it’s going to take teams of people that understand the systems, processes, personnel and customers involved in physical or virtual product demand/delivery. The list of features, technologies, capabilities, markets, customers, partnerships and requirements will continue to grow for enterprises. This type of organizational exponential complexity can only be solved by a more efficient type of work, an exchange of power -- manhours to kWh.

One thing I’m sure of: those making software decisions that define businesses will need flexibility, scalability and the ability to integrate. Especially if we’re spending time and money to reduce technical debt, we are (hopefully) not going to add right back to it by buying monolithic software. AIaaS will be the future of software with plug-and-play token-based services. I want to talk more about AI factories, but they should be their own topic. In short, software will capture more revenue as AI increasingly fulfills business functions.

3. Datacenters and Communications Services are going to get more expensive. (See Nvidia’s market cap) (Just Kidding. 24.1% CAGR while the market can’t physically get ahold of more chips)

Datacenters are enjoying the highest demand they’ve ever seen and I don’t think it’s slowing down soon. At a happy hour with clients, I met a guy who recently left an enterprise AE role at a big ISV to sell for a regional datacenter. Truthfully, I had no clue what selling for a datacenter was like, so I asked how it compared. He said the hardest thing about it was finding more to sell. His company was sold out and actively trying to buy usage back from clients to resell to others at a higher price.

This is going to affect energy. I live in the datacenter capital of the world (shoutout NoVa/worst drivers in the US) and they are being built everywhere. In 2022, E2 Optics published an article outlining the 70% of global internet traffic flowing through Northern Virginia. What does E2 mention next? A power shortage in Loudon County: a county in Northern Virginia well known for horses and data centers. New datacenters can account for power usage comparable to cities and they consume it at a steady rate. Energy now poses a threat to availability.

It's not just energy, either. Google used over 4 billion gallons of water, globally, in 2021. By 2023, Google consumed over 1.1 billion gallons of water at ONE datacenter in Council Bluffs, Iowa. Smaller datacenters? No one can be sure as only 51% of datacenter operators track their water usage. This isn’t a condemnation of tech’s water usage (it's not not) but more of a comment on the amount of water necessary to make wheels go 'round the compute carousel.

In March, Amazon bought a datacenter next to a nuclear site in PA. In June, the NRC is deregulated through Accelerating Deployment of Versatile, Advanced Nuclear for Clean Energy (ADVANCE) Act of 2024 cutting both time and money required to stand up a new site. In September, Microsoft struck a deal with three-mile island to provide power. In October, Google signed an agreement with Kairos Power (Small Modular Reactors) announcing, “this deal will enable up to 500 MW of new 24/7 carbon-free power.”

The match made in heaven, peanut butter (data center) and jelly (nuclear). Nuclear plants need a significant water source, specific electrical infrastructure, secure facilities and are sensitive to location. They provide consistent and substantial energy (=431 utility scale wind turbines) at a stable rate. Data centers need everything that a nuclear plant needs...and produces. After 50 years of deadlock in nuclear, it changes almost exactly when it benefits our golden boy corporations? This is called Regulatory Capture.

Tech may, accidentally, be a hero for the average American. Companies like Oklo (Sam Altman invested in 2015) and Kairos will face less regulation and more capital allocation from vested interests in tech. As modular reactors become more available, they can be placed closer to draw and reduce transmission losses. Power could be more affordable and efficient in the future? Maybe. Or demand outruns production.

Physical infrastructure will decide the winners. Those that have access to power, cooling, network and compute resources will be able to build, deploy and service AI solutions.? There will be winners in the “shovels” (any segment of the compute supply chain) category that revolutionize power, optimize networks and provide powerful compute resources. There will be winners in the “gold” category (hyperscalers/DCs) that employ the cheapest cost structures for power delivery, the best SLAs for bandwidth/latency and the fastest compute resources.

The federal government is investing money across the US in the form of infrastructure upgrades that will directly affect our ability to keep up with demand. It’s going to continue to rise. After Covid exposed supply chain vulnerabilities in vital sectors of the US economy, the US government has started to fund infrastructure improvement thru acts like CHIPS. Issues with other aspects of our hard infrastructure are just now starting to come to light (Telecom; Energy) pushing additional investment from private and public sector to prevent future interruptions of service. The type of regulatory capture (or deregulation) I mentioned earlier will be used to secure the necessary assets or to build more.

AI is here but it feels more like way a hurricane arrives: everyone is talking about it, some are getting ready for it, but we haven’t seen the rain, yet. Soon the trees will be blowing sideways, and we’ll wish we had more sandbags. In the future, enterprise AI applications won’t be dictated by intellectual property but rather by the compute supply chain that will be delivering new solutions. AI factories will furiously produce new applications that will be designed and implemented by process/domain specialists. As we prepare for the world's next great power competition, we are watching our IP-driven economy change into a market defined by real assets: infrastructure.