Trust in Data Centers: Promises vs. Performance
Dr. Eric Woodell
World's #1 expert in data center resilience. I audit and certify colocation facilities, ensuring secure, continuous operations—insured by Lloyd's of London.
Like it or not, colocation companies now own the majority of data centers in the USA.? This shift to colocation sites, away from enterprise data centers, has fundamentally changed the industry in both good and bad ways.
The positive effects largely revolve around the logistical and financial advantages that enterprise IT organizations obtain, by using colocation sites, versus building their own; speed to deployment, enhance business agility, avoidance of sunk costs, reduced operational costs, and the ability to directly write-off variable costs.? All of these factors make colocation sites not only a viable option, but THE financially smart move to make.?
Indeed, when I worked as the Critical Facilities Manager for Southern California Edison in 2010, I performed a series of Net Present Value (NPV) calculations (below), which showed that colocation facilities (for the specific circumstances of SCE at the time) were a financially sound approach through operational years 12 (for a premium colocation provider) or 16 (basic colocation provider).
The green line represented a stripped-down, purpose-built data center that eschewed any “luxury” accommodations, and the yellow line represented an owned data center with high-quality accommodations like executive meeting rooms, equipment rooms, offices, etc.? I pushed the executives- HARD- to seriously consider colocation as an alternative to the massive costs associated with building a new data center, because financially, it was the right move to make for the time.
I have no doubt many other managers of IT organizations within large corporations made the same sorts of calculations and came to the same conclusion.? This, of course, is why colocation has become a major industry, and billionaire investors are getting into the act.
But this brings into play several negative aspects of colocation data centers.
First of all, IT organizations that outsource their data center needs run into a huge problem that typically remains hidden, a complete lack of agency. ???You see, those clients have no insight (or oversight) to make sure the data centers where they’ve put their companies crown-jewels are being properly maintained or have adequate cybersecurity protection.? The clients are utterly blind to the potential risks of improper maintenance management and are therefore exposed to unplanned outages.
This lack of agency in turn leads to all sorts of shenanigans in the colocation market; improper (or nonexistent) energy tracking and trending, overselling space and power in the hopes that no particular customer will ever reach maximum power load in their contract, and improper, deficient or nonexistent maintenance of mission critical facilities assets.
All of these things, I’ve personally seen.
But you say “wait a minute, I’ve got the SOC-2 certifications stating that my colocation company is properly handling their maintenance and cybersecurity!”
My response is the Hindenburg Research investigation into Equinix, which has resulted in an investigation by the Securities and Exchange Commission, and Faces Class Action, Multiple Regulatory Investigations Related to Manipulated Accounting Accusations.
And despite the latest proclamations saying, “we investigated ourselves and found we did nothing wrong!” the latest data from their own financials, tells the story:
In 2015 the cut their maintenance budgets in half.
In the news as of May 9 2024, they halved it AGAIN.
This is a recipe for disaster.
The data center industry is very similar to the aviation industry; superb design and meticulous maintenance are required to achieve excellent performance for a long period of time.? In the aviation industry, the Federal Aviation Administration (FAA) oversees the design, testing and safety of all commercial aircraft.?
The data center world enjoys well-established design standards, thanks to the Uptime Institute’s Tier Topology papers.? And frankly, there are thousands of engineers that can design a data center, and do it very well.
Going back to the aviation industry, the FAA oversees all maintenance practices, and has auditors that have extensive knowledge and experience in the things they are to audit.? In contrast, the data center industry has no standards whatsoever for auditors.? Colocation data centers buy “SOC-2” certifications to assure enterprise IT organizations that they’re properly maintaining the mission-critical facilities- the safety nets for their IT presence- properly, and that other (minor) things like cybersecurity are being properly done.
What’s the qualification to be a SOC-2 auditor?? You have to be a Certified Public Accountant (CPA), that’s IT.
Imagine, you’re about to board a Boeing 737 for a flight, and a CPA comes running up to you as you’re standing in line, and saying “don’t worry, I’ve audited this company, THIS plane is certified safe!”? Would YOU trust that “certification?”? Of course not, and neither would I.
You want actual proof?? Equinix continued to obtain SOC-2 certification, even as they slashed funding for maintenance.
But they're not the only company doing this sort of thing... they're just the ones that got caught. This sort of thing is happening across the industry, and the natural result is that IT clients are now experiencing LOTS of unplanned outages.
领英推荐
Mathematically, 99.982% uptime works out to odds of an outage in a given year of 1 in 5,555.? But looking at the results above, the real odds of a data center outage in a given year is 1 in 5.5.? Put another way, actual performance is >1000 times worse than expected, due to maintenance mismanagement.
If the aviation performed as poorly as the data center industry is (just counting "severe" outages), you'd see commercial plane crashes every day.
It REALLY is that bad.
What can you do if you’re in a colocation data center that isn’t maintaining the safety nets you need?? Cancel your contract and go to other colocation companies? ?
What guarantees do you have that it will be any better at another colocation company??
All of this at a time when the electric grid is suffering from growing demand and decreasing supply, which increases the risk of utility interruptions:
WHEN (not if) your colocation facility has an unplanned outage- assuming your company survives that punch in the face- the colo company will offer you “credits” …? for more of the same!?
What. A. Deal.
It would appear, at this point, that many large IT organizations are trapped between two unhappy options. ?You either stay with the company that has put your business at risk, and hope it somehow gets better (the maintenance expenditures graph above should tell you how THAT’S working)…? or try to find some other colo company that’ll be better, and hopefully not worse.
Hope ain’t much of a plan.?
You need a third option; the Amerruss Resilience Program.?
Our success story with Vanguard,?a global financial giant with over $7.5 trillion in assets, epitomizes our capability. ?Over 6 ? years, Vanguard's extensive portfolio of more than 60 sites, including Tier-II and Tier-III facilities, experienced zero unplanned outages, a feat considered statistically impossible according to industry standards.
The Amerruss Resilience Program offers a profound value proposition: guaranteeing data center uptime in line with their Tier level specifications, with the ultimate aim of achieving 100% uptime. ?Our approach is not just about compliance but about exceeding industry standards and expectations, a necessity underscored by the current landscape marked by potential rolling blackouts and escalating cyber threats.
Our program is based on deep technical expertise, real-time feedback through quarterly audits, and a proactive stance on maintenance and risk management.
It is an economical, extremely effective system that allows you to wrest agency back for your organization and have direct visibility of any risk elements that would threaten.? And it should be noted that any required repairs are the financial burden of the colocation company to mitigate.?
The cost for a small location, such as a POP site, would typically be in the low five-figures, and a medium-sized enterprise IT presence in a colocation facility would typically be in the low six-figure range, depending on size, complexity, amounts of records to be reviewed, etc.? Insurance coverage for the program is easily accessible and also economical, if needed.
Compared to moving to a new colocation company, where you may just get more of the same, these costs are orders of magnitude smaller, and it mitigates the risks your company faces.? ?
Call to Action:
As IT and C-suite executives, you face a critical decision point.
The integrity of your operations hinges not just on the promises of uptime but on the actual reliability delivered by your colocation data center.
Don’t settle for uncertainty and risk—choose a proven solution.
The Amerruss Resilience Program provides not only a safeguard against operational disruptions but also a strategy for real assurance and control. ?Our approach is about far more than compliance; it’s about setting new standards in reliability and operational excellence.
Ensure your data infrastructure is not just maintained but thrives. ?
Contact Amerruss today to secure the resilience your business deserves.
Research Analyst, Author of Security Yearbook 2024 stiennon.substack.com
6 个月This is eye opening Eric. Thanks.