DATA CENTERS, QUANTUM PHYSICS and SKYNET…?
12 September 2021
? I’ve been casually following the almost exponential growth in Data Centers for a while now, mainly because I was fascinated by how the way we use data is changing our behavior and our culture. We’d already been seeing quite a bit of Data Centers growth in central Ohio, and Google just announced a major expansion. Between that and a conversation I had with a nice young lady last week, I was inspired to write something about what’s driving it and why it’s a story that will matter a lot more to you in the next decade than you might imagine.
???Fair warning… In the course of writing, I got a little self-indulgent and meandering.
?????A little over 400 years ago, William Shakespeare wrote a play called The Tempest, which contained a fragment of a sentence that said; “Whereof what's past is prologue”. In that spirit I want to take a trip down memory lane, and we’ll be taking the scenic route.
?????I was born in 1965, the same year that Gordon Moore came up with his famous law. Normally, I would take a moment to do some exposition on Moore’s Law, but let’s face it… You’re reading a blog post about Data Centers, which means I can reasonably assume that you’re as big a nerd as I am and need no explanation.
?????Six years later, about the time I was causing my 1st grade teacher to question her career choices, Intel introduced the Intel 4004 Microprocessor, that being a chip that holds the data processing logic and control on a single integrated circuit. Intel’s advertising campaign claimed that they were "Announcing a new era in integrated electronics" and this was one of those rare moments when the advertising was understating the facts by a considerable amount.
?????The Intel 4004 had a Minimum Feature Size, which is the smallest scale at which a feature can be manufactured, of 10,000nm. The wavelengths of electromagnetic radiation that the human eye can see ranges from roughly 700nm (red) to 400nm (violet). If you’re trying to manufacture chips with an MFS much smaller than 400nm, you’re into the Ultraviolet spectrum and that can be a real catastrophe. (Science Humor. Sorry.)
?????I mentioned the Minimum Feature Size for reasons I’ll get to shortly, but there are two other small details I have to touch on first. About a decade after Intel introduced the 4004; in possibly the most insanely brilliant and prescient moves in computing history, Bill Gates bought the rights QDOS (Quick and Dirty Operating System) from Seattle Computer Products for a song. Different versions of the story have him paying anywhere from $50,000 to $75,000 for it. He then filed off the serial numbers and hired the same programmer who wrote QDOS (Tim Paterson), to re-write it into MS-DOS, which he then licensed to IBM as PC-DOS. The full story is fascinating, and I encourage you to Google it for yourself when you have the time. The point of course, is that this is how Microsoft came to own the desktop; or to put it more accurately, the USER EXPERIENCE.
?????The stage is almost set. We have a real desktop computer and before the decade was over the Big 3 Killer Apps for computers running a Microsoft Operating System; Word, Excel & PowerPoint, would be gobbling Market Share like Pac-Man gobbling cookies. Apple was definitely a competitor with a compelling product of their own, but the commercial success of Microsoft was so pervasive that even decades later they still effectively own the Desktop, especially in business applications.
?????The only thing missing is the Internet. Not the World Wide Web, which is actually just a verbal conceptualization of the content stored on Web Servers connected by the Internet. No, I am referring specifically to the Internet itself.
????Time out for a Trivial Pursuit question: “How many websites are on the Internet?” ?Hint: It’s a Trick Question; and I’ll give you the answer later in this exposition. Back to the story…
?????In order for the modern Internet as we know it to exist, we needed one crucial and indispensable piece of equipment. The piece of equipment was a modern Router. In the beginning there were networks that connected computers, but those networks couldn’t necessarily talk to each other because of incompatibility between systems. What was needed was a way for these individual networks to talk to each other regardless of whether they were otherwise compatible. We needed a network of networks; literally an "Internetwork", which is where we get the word “Internet”.
?????Now this next part is going to be incredibly oversimplified. I have to do that because the full story is even more complicated and fascinating than the Microsoft story, and it remains incredibly controversial even up to today, albeit among the relatively small group of people who either know or care. Again, I encourage you to Google it when you have a chance and read about it for yourself.
?????Leonard Bosack and Sandy Lerner were a young couple at Stanford in the early 1980’s, where there was more than one computer network, but they couldn’t all talk to each other. There was a lot of development work going on with the TCP/IP protocol and network interconnectivity at the time, much of which could be traced back to ARPANET, and Xerox was continuing work on Ethernet out of their Palo Alto facility. LAN’s, or Local Area Networks were already developed, but Wide Area Networks (WAN’s) weren’t really a thing yet, not in the way we understand them today.
?????Legend has it that Leonard Bosack and Sandy Lerner, who met in Graduate School, fell in love, and got married, but the jobs they took managing computer networks at the Stanford campus were in different locations on the campus. They wanted to keep in touch during the work-day, but couldn't, because each of their local networks were incompatible.
?????There were already many people working on creating a campus-wide network, but Leonard and Sandy were motivated by romantic interests, and they went to work on the problem with some enthusiasm. First, they built "Bridges" to connect their networks, then after working out the kinks, built those Bridges into what would be considered a modern Router. The specific technical points that differentiate Hubs, Switches, Bridges and Routers get a bit esoteric, but suffice to say that the Stanford campus effectively became the first truly interconnected WAN in the modern sense, because two young people in love wanted to be able talk to each other.
?????All the basic pieces were now in place, and when Tim Berners-Lee developed an easy-to-use bit of software in the early 1990’s that allowed ordinary people to easily use a “Web Browser” to access the connectivity of the Internet to communicate, usage exploded in a way had to be seen to be believed.
?????By the end of the 20th century there were about 400 million users on the Internet, which is part of the reason the Y2K problem was so scary. In the blink of an eye, a big chunk of the global economy had become reliant on the Internet, and the computers that connected them. Five years later the number of users more than doubled, surpassing a billion.
?????According to Datareportal.com, as of July 2021 there were 4.8 billion active Internet users worldwide, approximately 61% of the global population. According to an April 2021 report from Forbes, global e-commerce is expected to reach $4.2 trillion this year, with U.S. consumers accounting for close to one-quarter of that.
?????Until fairly recently, global Internet usage was primarily traditional home computers and mobile devices such as mobile phones and tablets, along with a variety of business servers, mostly in retail and banking. There used to be a theoretical growth problem with the Internet because important component of Internet functionality called IPv4 used a 32-bit address space, which only provided enough IP addresses for about 4.3 billion connected devices. IPv4 was crutched with two strategies; Classless Inter-Domain Routing and Network Address Translation (CIDR & NAT). What they did was allow a LAN to have a single IP address for connecting to the Internet, while having a private, and non-routable IP address scheme internally. This actually turned out to be incredibly effective at slowing down the exhaustion of IP Addresses, which are necessary for information to be routed on the Internet.
?????Unfortunately, there were still limits and it had been known for some time that IPv4 needed a replacement. That replacement was IPv6, which uses a 128-bit address, theoretically allowing 3.4×10^38 addresses. That’s 3.4 times 10 to the 38th power, or 340,000,000,000,000,000,000,000,000,000,000,000,000. I’ve been told that there is an English language word for that number, but I couldn’t tell you what it is.
?????IPv6 is important for one main reason. It contains enough theoretical unique addresses to truly support “The Internet of Things”, which is usually abbreviated as IoT, which may be the most clumsy and unintuitive name abbreviation for something in the history of technology.
????????Meanwhile, Moore’s law has been hard at work. If you recall, way back at the beginning of this Blog post that is turning into a novel, I had mentioned that the granddaddy of microprocessors, the Intel 4004, had a Minimum Feature Size of 10,000nm (nanometers). At that size, you could use practically use a good magnifying glass, or a cheap microscope, to view the circuit tracings on the chip. The SPARC64 from 1995 had an MFS of 400nm, and as best as I can tell was the last chip that had tracings that could be seen using an optical device operating in the visual light spectrum. The Pentium Pro from the same year was down to 350nm. In the early 21st century, Microprocessors were dropping below 100nm, and in 2018 the Intel Cannon Lake chip hit 10nm. A year later, the AMD Zen 2 was down to 7nm.
?????This was Moore’s Law in action; processing power going up while the cost kept dropping. While all this was happening, what would become the modern Blade Server was developing. The roots of Blade Servers actually go back to the 70’s, but there were changes in the physical architecture of the “box” just before the turn of the century that became the ancestor design to what we have today. According to Wikipedia; “A blade server is a stripped-down server computer with a modular design optimized to minimize the use of physical space and energy.”?
????That last bit, “physical space and energy” is important. Modern Data Centers are consuming about 1% of global energy at this point, maybe a bit more. Many people of the opinion that it will hit 3% before the end of the decade, maybe well before the end of the decade. That doesn’t sound like a lot, but with global energy usage approaching 25,000TWh, even a small percentage is a lot. For perspective, that’s well over 20 million times the energy it took for Doc Brown’s DeLorean to travel back in time.?
????Interestingly, Data Centers power usage has been relatively flat for several years. Well, Flat-ish. The reason for that is because Moore’s law has seen processing POWER increasing dramatically, which the EFFICIENCY of Server Systems has also gone up just as dramatically. That’s continuing as I write this. Apple’s A14 chip has a 5nm architecture, which is about the size of 10 silicone atoms.?
????Just a few years ago, most people would have told you that we couldn’t even get down to the 7nm level without running into problems, but we are currently able to produce 3nm chips and IBM announced this past summer that they will be producing a 2nm chip in 2022, for which they claim a performance increase of 45% or a reduction of power consumption by 75%, compared to the 7 nm node chip. And yes, there are labs out there right now that are trying to reach 1nm through the use of Carbon Nanotubes and other new materials. In May, a joint effort between MIT and the National Taiwan University, announced that they believe they have a path to 1nm. Reading about research by them and others makes me think we’ll hit 1nm by 2025.?
????The thing is, there are limits to how small you can go without Quantum Physics becoming a problem, and that’s where things get weird. It’s pretty well known that labs in many places around the world are working on Quantum Computers, but until that research bears commercially viable fruit, we’re about to run into a wall. ?
????Simply put, Moore’s Law, at least as we’ve known it, appears to be coming to an end. The problem, in a nutshell, is Quantum Tunneling. That’s the sub-atomic version of putting your toddler in his room behind a child safety gate so you can take a nap, then waking up and finding him on top of the refrigerator, eating cookies. I can neither confirm nor deny that this ever happened to my parents.?
????Essentially, when you get down to a certain size, the universe doesn’t behave the way Isaac Newton said it does. In fact, it behaves in ways that Einstein found quite annoying. Ways which mean that beyond a certain point, the way we build Microprocessors now becomes unreliable. One nanometer appears to be shaping up as something of a limit to how we do things now, though there will probably be a fair amount of room for exploiting efficiencies that will continue to carry Moore’s Law forward for a while, even as we reach the physical limits of how many transistors we can squeeze onto a chip.
?????I think that sums up the problems that the computer side of the equation is facing, but there is an equally big problem on the network side and this one may be insurmountable. The problem is the speed of light. You can get to a maximum speed that’s right about 186,282 miles/second or a bit under 300,000 meters/second. And that’s it, at least as far as we know. That’s why VoIP phones often don’t work very well over satellite connections; VoIP being very sensitive to any delay in the signal. Starlink shows some promise there, because their birds fly lower, but that remains to be seen.
?????The practical concern here is that until we figure out how to use Quantum Entanglement in communications, those communications take time. Light travels so fast that it took humans until the 17th century to figure out that it travels at all. Galileo figured it out early in the 17th century, then around 1676, Danish astronomer Ole Roemer became the first person to prove that light travels at a finite speed when he noticed and documented that Jupiter’s moons eclipses took place at different times than predicted whenever our orbits around the sun changed the distance between Earth and Jupiter.
????The time it takes for light to travel around the circumference of the Earth is 133 milliseconds. 150ms is normal latency for VoIP phone systems. This means that when you speak into the phone, it should not take longer than 150ms for the end user to hear you. 300ms or more would be unacceptable. ?
????It would be easy to assume that everything is fine, because 133ms is less time than 150ms, but there’s a certain amount of lag time that it introduced by the equipment itself. It varies, but what is important to understand is that it takes time to convert your voice all the way down to a binary signal that can travel on a network, time to route it to where it’s going, and then time to convert those 1’s and 0’s back into a human voice. So, you’re already using up some of your “lag budget” before you even factor in the speed of light.?
????It would be easy to think that these miniscule amounts of time are so trivial that it’s silly to worry about them. So, let me relate a story. In March of 2014 I was visiting family back on the east coast in Rockland County NY, which is right by Mahwah in NJ. I was talking with a friend who was in networking, and he was telling me about an NYSE Data Centers in Mahwah that was in the process of setting up a laser-link system to a NASDAQ Data Centers in Carteret, which is about 35 miles due south of Mahwah, across the Arthur Kill from Staten Island.?
????This sort of thing is more common than people may realize. There’s a laser network link between the London and Frankfurt stock exchanges that was completed the year before this conversation, and a lot of these laser installations are about replacing the microwave links that had only been put in a few years before. It would be reasonable to ask why anyone would go to all the effort when communications already move at the speed of light on fiber optic networks; but the thing is, they don’t. ?
????The refractive index for single-mode fiber, if I round up, is about 1.5, so when you divide lightspeed by 1.5 you get about 2/3 of the speed of light. What THAT means is that going from Fiber to Laser means about a 50% increase in speed. Currently, the fastest supercomputer in the world is Japan’s Fugaku, which a maximum sustained performance level of 442,010 teraflops per second on the LINPAC benchmark. And remember… Today’s Supercomputer is tomorrow’s Data Centers Server. When your processing speed is at those levels, a 50% increase in transmission speed is a big deal.?
??And this is where Skynet comes in, though hopefully without those pesky Terminators.
?????In the first movie of the series, The Terminator, we are introduced to the idea of a self-aware computer intelligence that decided the humans needed to go. The movie, intentionally or not, gave the impression that Skynet was some huge computer system housed in a building somewhere, like something out of 1950’s Sci-Fi. But the movie was released in 1984, a year before the CRAY-2, and massively parallel multiprocessing wasn’t really a thing yet. 20 years later, when Terminator 3: Rise of the Machines came out, we discovered that Skynet wasn’t the Hardware, it was the Software. When Kristanna Loken’s Terminator extended a metallic probe from her finger and injected software into a computer at a secret military base, she was uploading Skynet to the whole system. By that time, she had already infected computer systems worldwide with the software, and as a result, Skynet didn’t exist in a specific place, it existed everywhere.
???Which, after my absurdly long exposition, brings us to the modern Data Centers.
?????When I was a network engineer for the Columbus Ohio public school system, we had three Data Centers around the city that were linked by multiple 100meg fiber optic connections. That was a traditional Data Centers model, where there were one or more secure locations which housed the hardware that held the software, all of them “In-House”. That particular setup worked fine for CPS and honestly, there’s no real need for them to change anything, other than keeping the hardware up to date. They have a fairly specific need, and that need isn’t going to change much over time. ?
????I do want to make sure I mention another factor here. You can think of a public school system as a business, where they have sold a service to some portion of the general public and those consumers will be using the service during a pretty well-defined portion of the day. In this context, I mean “day” as a 24-hour period. There are some businesses that effectively have 24-hour days; Amazon and Walmart being two that come to mind immediately. A public school system has traditionally had a roughly 9-hour peak operating period where usage is high, with a sharp drop outside that period. A university campus will also have a roughly 9-hour day, but whereas K-12 will have a tightly defined peak from about 7:00am to about 4:00pm; universities will have a sort of mid-range usage period running well into the evening and continuing on weekends.
?????The general idea here is that different businesses keep different hours based on customer demand patterns; and this is without even getting into their IT related operations. But of course, this is 2021 and practically every business in the world has some sort of computer operations, even if it’s mainly for bookkeeping and their web presence is essentially just a Facebook page. The process of businesses becoming ever more dependent on the Internet had been proceeding at a steadily increasing pace since the mid-1990’s, but then came 2020 and Covid-19. Suddenly, pushing more and more operations into “The Cloud” accelerated like Mad Max hitting the Supercharger switch, all of it driven by the sudden switch to working “remotely” and the movement of sales deeper into the “on-line” segment.
?????Speaking of which, did you ever wonder where the term “The Cloud” come from? My first experience with computers was back in the 1970’s when my father bought a “Trash-80” from Radio Shack. I went through the usual clichés, including writing a program that would display “Hello, World”, before losing interest. I wouldn’t have a computer again until about 1994, when I bought a 386SX to keep track of the books for a contracting business I was in that was working on Time-Warner’s “Physical Plant” overhaul for the first implementation of Road Runner.
????The Operating System was Windows 3.11 over DOS 6.22, and if I remember correctly, you had to feed the computer 9 different floppy disks to install them. I had soon swapped out the chip for a 386DX, which has a math co-processor on the chip, and hacked the 14.4k Modem board so that it would run at 19.2k, which at the time was big jump. Eventually I installed a second modem so I could dial into two accounts at my ISP. If memory serves, I could have run up to four, but having two phone lines was already expensive enough.
???About 8 years later, I was sitting in a classroom getting a formal education on networking, and when the instructor was diagraming network connections between Routers, he would draw a few circular Router symbols on the board and have lines going from the Routers into a crudely drawn cloud that was supposed to represent the Internet. The Cloud symbol actually goes back to ARPANET in the late 70’s. The important point is ?that The Cloud wasn’t a “thing”, it was a representation of all the interconnected “things” the collectively make up the Internet.?
????Remember when, earlier in this meandering post, I asked you a trick question; “How many websites are on the Internet”??
????Well, the answer is None. Zero, Zip, Zilch and several other synonyms starting with a Z. The reason why that’s the answer is because ALL of the content you find on the Internet isn’t actually ON the Internet. It’s on individual computers, whether a box sitting in someone’s office, or a Blade server housed in a Data Center, which are CONNECTED to the Internet. Metaphorically, the Internet is all the roads you drive on and The World Wide Web lives in the houses and buildings that you get to by driving on the roads. ?
????In the technology business in general, things are roughly divided into two groups: Networking and Software. That’s a HUGE oversimplification of course, but I need to explain an important concept. Software is the broad category of things that enable you to DO stuff. It doesn’t matter if it’s an app on your iPhone, a web browser on your laptop, or a spreadsheet on your workstation. Somebody wrote a program in some sort of high-level computer language that runs on your device’s operating system, and it lets you do things. Write a document, send an e-mail, download a movie… whatever.?
????Those things that you use software to do are dependent on being connected to a network in order to move data from one place to another. Having an application that lets you send an email or download a book isn’t much good if that email can’t actually go anywhere, or if your e- book has no way to get to your device. That’s what the network is for, and it’s the network that ties it all together so that it’s useful.?
????There’s a concept in networking known as “Multi-Homing”, which is basically just the practice of connecting your LAN to the WAN through two or more connections. You COULD have both connections running to a single ISP, like I did with my dual-modem setup 25 years ago, but that would defeat the purpose. I did what I did back then because I wanted more speed and I couldn’t afford an ISDN modem, let along the absurd cost of a T1 connection. The purpose of Multi-Homing is redundancy. If one connection goes down, you are already connected to a back-up. This is related to Data Centers in a way I will explain shortly.?
????So, what we call Cloud Computing actually goes back to the 1970’s but comparing what it was then to what it is now is like comparing a Volkswagen Beetle to a Ferrari 812. Today’s usage is completely ubiquitous. If you use Apple’s iCloud or Microsoft’s OneDrive, you’re using the cloud, though if we stick to the automotive metaphor, that’s like driving a Honda Accord. It’s a perfectly nice car, reliable and comfortable, but it’s still just basic transportation, and until relatively recently, that’s all that Cloud Computing was in general. It was a useful place for back-ups, and maybe you were serving up a connection and application to a remote user, but it was still pretty basic.?
????The major trend today is actually binary, and by that, I mean that it needs to be looked at from two perspectives; the customer side and the provider side. The reason for doing that is that I’m trying to look at overall trends, and those trends are being driven by forces with very different perspectives. On the customer side it comes down to a simple question; “What can you do for me that is going to be a benefit to me?”. On the provider side it comes down a more complex one; “What solutions do my current and future customers need, so that I can provide them with those solutions and KEEP those customers?”?
????It’s probably worth pointing out that people create names for things so that they can talk about them. Yes, that’s a little “Captain Obvious” of me, but my point is that the NAME is not the THING, and it’s worth thinking about what we mean when we use those names. The big buzzword these days is Hybrid-Multi-Cloud. The Hybrid part refers to a blend of public and private Data Centers and the Multi-Cloud part refers to using more than one public Data Centers. ???
???WARNING: More Oversimplification Coming Up.
?????Remember when I mentioned Multi-Homing and that it relates to Data Centers? Well, this is how it relates. The difference is that where Multi-Homing is done mainly for redundancy, the motivation for Multi-Clouding (Is that even a word?) is much more complicated. As to the Hybrid part, Public & Private basically have to do with who owns the Data Centers.?If you’re a car Dealership ?and you have a small Data Center on-site to hold your customer data, accounting, etc., that’s Private. It doesn’t really have to be in the same building, it just has to be in a building that the Dealership ?either owns or leases, because what defines the Private side is who directly controls, maintains, and has access to the server.
?????The DATA on the server is different. Not so much on the Private side but let’s say that this Dealership ?has contracted with an outside web service company to handle their website. It doesn’t have to be AWS, it could be “Bob’s Web Services” down the street. So, the Dealership ?buys some Co-Location space on one of Bob’s servers, and that might even be a sub-lease from a much bigger provider that Bob is buying space from.?Both the Dealership and Bob are benefitting from only buying what they need, and the Dealership ’s website has access to a much fatter pipe than it would if it was on a server that was On-Premises at the Dealership . Neither of them has to worry about physical maintenance or security, but the Dealership?still owns their own website. If they want to move it elsewhere, they can. If Bob goes out of business, the Dealership could move it or perhaps even leave it on the same server, just with a new company taking care of it. THAT is Hybrid. I’m getting to the Multi-Cloud part…?
????A little while ago I told you about MAN (Metro Area Network) that I used to help care for. If memory serves me correctly, it consisted of 3 Data Centers connected by an MPLS Ethernet Ring, which is not to be confused with the older Token-Ring networks. Basically the 3 Data Centers were connected to each other such that they provided mutual redundancy, and while each Data Center was designed to serve approximately 1/3 of the overall network, they were set up so that any of the 3 could theoretically serve the entire school system. That setup provided redundancy, data backup, load-balancing and higher speeds.
?????In the case of this school district, everything ?is On-Premises, so it’s Private, not Public. However, they have 3 individual Data Centers, each of which is intended to serve 1/3 of the remote sites but could serve all of them if needed. THAT is Multi-Cloud. Well, in a manner of speaking. Technically, when people talk about Multi-Cloud, they are referring to multiple Public Cloud providers, not multiple Private Clouds, but you get the idea. If you combine the two concepts, you get Hybrid Multi-Cloud. I did warn you that I was going to over-simplify again, didn’t I?
???Getting back to the market forces driving Hybrid Multi-Cloud, let’s consider the customer viewpoint first.
?????When I was running a construction business, we had purchased a couple of 7’ wide grading machines that mounted on the back of a tractor. We also bought an excavator with multiple buckets, a tracked skid-steer with a half-dozen hydraulic attachments, a couple of dump trucks and one of those great big motor graders that weighed about 55,000 pounds and required a specialized tractor trailer to haul it anywhere at $400 per round trip, plus the oversized load permits you’d better not forget to pull.
?????The two 7’ graders aren’t germane to this example because they were proprietary, but the other equipment added up to about $70,000 in Capital Investment, plus the lease costs on the 6,000 square foot building we needed to store everything in; and the carrying costs associated with insurance, maintenance and repairs. And, if something broke down on a job-site, that was OUR problem, and ours alone. ?
????We eventually sold the excavator, skid steer and dump trucks and began renting instead; because we simply didn’t use that equipment enough to justify ownership. If any particular job required equipment we didn’t have, we rented it. An excavator of the same size and power as the one I owned would be about $400/day, $1,000/week, or $2,300 a month; and the rental company would provide both delivery and pickup. If a machine broke down, they would either repair it on site that day or bring me a replacement. More importantly, when I needed a smaller or larger machine, all I had to do was ask. The savings on insurance alone was over $1,000 a month, and it was far easier to control costs.?
领英推荐
????In the same way, the value proposition for going to a Hybrid Multi-Cloud solution is that you offload the maintenance and security costs to your provider, where the cost is shared among all the provider’s customers. If your needs grow, you buy more capacity. If your loads are “peaky”, you don’t have to buy a larger pipe that you won’t use most of the time, your Cloud provider can handle it. There’s another very important value proposition, but I’m saving it for the end.?
????From the provider side, Hybrid Multi-Cloud is driven by the same basic need that customers have, which is flexibility. Fundamentally, a Data Center is a structure to safely house Servers and supply them with sufficient power, cooling, and access to the Internet. Within that basic context, the structure can be as small or as large as you can imagine. You can have a Hyper-Scale facility custom built, you could repurpose an old retail space, you can even have modular solutions that can be trucked to a location and hooked into local resources like an RV at a campsite. That flexibility means that a provider doesn’t need to worry about what a customer’s needs are, because whatever they are, they can be met.
?????All of this brings us back to the Skynet comparison. Older science fiction in general tended to have dangerous AI (Artificial Intelligence) represented by a single individual machine, going back at least as far as 1956’s Forbidden Planet. I think it was probably about the turn of the century when AI in entertainment began to be conceptualized as software, rather than hardware; at least in terms of fiction that was widely consumed. The Matrix and Terminator 3 are obvious examples; and a reflection of how the general public was beginning to view computer systems more in terms of the software, rather than the hardware. To put it another way, people were thinking less in terms of needing to HAVE a computer, and more in terms of a specific need they needed to have filled. The hardware was just a means to an end.
?????To be honest, I would be hard pressed to tell you when Cloud Computing as we know it today showed up on the scene, though perhaps Akamai’s content delivery network (CDN) that went online near the end of the 20th century is as good an origin point as any.
????Shifting focus a bit, it is probably fair to say that most of the general public is at least marginally aware of Cloud Computing, but they don’t necessarily understand what it is, of grasp it’s real power.
????What makes Cloud Computing arguably the most significant advance in computing since Desktop Computers because a common household item almost 30 years ago, is the concept of Edge Computing. This is not to be confused with The Internet of Things, even if philosophically there is a common thread; the pushing of technology as close to the end user as possible. Restating a fundamental reality; the most powerful computer in the world is of limited use if the people who need to use it have to physically be in the same location to do so. In this context, the Edge is where the User is. Metaphorically, the Head of the Table isn’t a function of the table itself, or how it’s situated in a room. It’s defined by who sits there. It is the person who defines the thing, not the thing that defines the person. As IDC puts it:
“The edge is a location, not a thing. It is the outer boundary of the network—sometimes found hundreds or thousands of miles from the nearest enterprise or cloud Data Centers, and as close to the data source as possible. The edge is where real-time decision-making takes place.”
??- International Data Corporation
????Three years ago, International Data Corporation predicted that the sum of the world’s data would grow to 175 Zettabytes by 2025. I actually had to Google that, because I had no clear idea what a Zettabyte was. You ready for this? A zettabyte is a measure of storage capacity, which equals 1000? or in plain numbers, 1,000,000,000,000,000,000,000 bytes. One zettabyte is equal to a thousand exabytes, a billion terabytes, or a trillion gigabytes. So, we’re talking about 175 Trillion Gigabytes. Take a look at a typical Thumb Drive and think about that. On a side note, I have a sawbuck that says their prediction will turn out to be an underestimate.
????As it happens, they just published a report which stated that by that same year we can expect that “total worldwide spending on cloud services, the hardware and software components underpinning the cloud supply chain, and the professional/managed services opportunities around cloud services – will surpass $1.3 trillion by 2025 while sustaining a compound annual growth rate (CAGR) of 16.9%.”
????That sort of growth obviously means a lot of Data Centers are going to get built, but the key word in “Data Centers” is DATA. That is a LOT of 1’s and 0’s to move around. It’s a little difficult to get a handle on just how much traffic is flowing across the Internet at any given time, but we’re probably at about 4 Billion Zettabytes per year right now, or likely will be by year’s end. By 2025, we can expect that to be at about 5 Billion Zettabytes.
????This next bit is difficult to overstate; the power of Cloud Computing is that the Software and the Data itself, can be anywhere in the world, any time it needs to be.
????Years ago, Cisco conceptualized a Three Layer hierarchical network model, which consists of a Core layer, a Distribution layer, and an Access layer. The idea is that the Core is intended for the fastest possible transit, the Access layer should be as close as possible to the people accessing it, and the distribution layer is how they are linked. There’s a lot more to it than that, but that veers off into a lot of networking minutia.
????In Cloud Computing, it’s conceptualized as categories; Core, Edge, and Endpoint. In their Rethink Data Report from 2020, IDC puts it this way:
Core
This includes specialized computing Data Centers for enterprises and cloud providers, which covers all types of cloud computing, including public, private, and hybrid clouds. In addition, it also includes Data Centers operated by enterprises, such as Data Centers running power grids and telephone networks.
Edge
Edges are enterprise-enhanced servers and devices that are not located in the core Data Centers. And it includes server rooms, front-line servers, base stations, and smaller Data Centers in various regions and remote locations to speed up response time.
Endpoint
Endpoints include all devices on the edge of the network, including computers, telephones, industrial sensors, networked cars, and wearable devices.
????So, in a sense, Cloud Computing is following the Cisco paradigm of logically segregating the total system in a way that promotes the best functionality.
????A hypothetical user might be a large health care institution like Cardinal Health. They have certain HIPAA obligations, so that would likely be something they keep On-Premises. They also have all sorts of web portals, collaborative work going on in dozens of areas, and Applications to track supplies and shipments; the possibilities being essentially endless. So, if they really want to leverage Cloud Computing to save costs, drive productivity and lower energy consumption; they’ll think hard about who is doing what, where they are doing it and what would be the best place to physically house the data that any given end-user needs.
????Paying attention to these questions works. Taken from the www.section.io website, we have these data points:
????Beyond that, you can start to ask questions about what the data itself needs. A business web sales portal might need a lot of bandwidth, but it may not need a lot of computing horsepower, at least not relative to any one customer. What about a business intelligence app; perhaps an AI that is sifting through decades of maintenance records kept by an Airline in an attempt to learn something that might help the Airline to avoid costly unexpected delays, or something worse. That AI needs a lot of processing power, but how big a data pipe does it need? How about an AI that is analyzing the stock market and looking for opportunities that it can take advantage of in the blink of an eye? Certainly, it needs processing power to examine all the data that goes into its decisions, and it will need a connection to the Internet with high availability and quality. But what’s more important? How much data it can send at once, or how fast that data can get to its destination? Location starts to be very important when tiny fractions of a second matter. Another outgrowth of the rapid increase in Data Centers is new concepts and services that weren’t possible until Cloud Computing made it possible.
????We now have Infrastructure as a service (IaaS) which delivers virtualized computing resources over the Internet. Virtual machines, Load Balancers, etc. There is also Platform as a service (PaaS), which delivers hardware and software tools. Windows Azure is a good example, though far from the only one. We also have Software as a service (or SaaS), which delivers applications as a service. You no longer need to buy, install, and maintain the software, you just access it online. Microsoft Office 365 is a great example. I'm using it to write very essay right now.
????All this pressure from all this data has led to a new term, or at least it’s new to me: Data Gravity. The term was coined about a decade ago by GE Software Engineer by the name of Dave McCrory. I’ve never spoken to him personally, but I think he was trying to convey the idea that the sheer “mass” of data affects everything around it, particularly services related to Data.
????Which brings me to an interesting example: Apparently, something approximating 70% of the world’s Internet traffic passes through a place known as the Dulles Technology Corridor, in Northern Virginia. There are a variety of reasons for this, the big ones being cheap land (or at least it used to be), and a cost per megawatt of power that’s been about 20% below the national average. And fiber. Lots and lots of fiber. But then, would it have happened at all if not for the explosion of Data Centers in Northern Virginia and elsewhere? It starts to be a real Chicken vs. Egg question if you look at it too closely. It certainly brings a whole new meaning to “Data Driven”.
????Maybe I was wrong a little while ago. Maybe the real power of modern Cloud Computing isn’t in the benefit derived from putting it wherever the best location happens to be. Maybe it’s the way it’s changed how we think about data, and how we use it.
????These three items are from that same IDC report I mentioned earlier:
The last 3 words in that last sentence are; “Data in Motion”. At the heart of it, that’s really what we’re talking about. Stockpiled data doesn’t do anyone any good. Putting it to use does, which I would argue is the real point to all this.
????WAY back at the beginning of this little essay I mentioned in passing that I had been interested in this subject for a while. Part of that is because I’m a nerd, for which I blame Asimov, Heinlein, and Roddenberry. My recent interest was spiked by Google’s announcement that it was planning to expand their New Albany, Ohio Data Centers by building out new locations in Columbus and Lancaster. AWS already has a Data Centers in the area, as does Facebook and several other companies like Racksquared, Cologix, and Cyxtera. A lot of Dark Fiber that was installed years ago is getting lit.
????All of that being said, my interest is mostly to do with the social changes that are simultaneously driving and being driving by the mobility of Data, it’s Non-Locality. I’m not interested in this change as either a sociologist or a politician, I’m interested as an observer. I really do believe that we’re in a period akin to the advent of the Industrial Revolution or the Age of Exploration; and that while it’s fascinating to watch it unfold, where it goes isn’t as predictable as we might like it to be.
????Consider; Mark Zuckerberg at Facebook perceived a problem in his social network in the way the algorithms that determine how people connect to each other tended to cause them to silo into groups that were highly reflective of their pre-existing network of family and friends. He changed the algorithm to create more cross pollination in order to widen the social network of Facebook’s users. Instead, it tended to broaden the silos but instead of building better communication and understanding (and revenue!), it created more hostility and tension between groups.
??I’ve mentioned Data Mobility, which is about getting data where you need it, and when you need it there. Interestingly, there’s also a concept called Mobility Data. Google spends a lot of time paying attention to where each of us are, how we got there, how long we stay, what we do and where we go next. Yes, that can feel a little creepy, but they are after all in the business of accumulating information, mining it for insights and monetizing both the raw data and what can be learned from it.
????For example, Google published some data in a report dated September 11th, 2021, that examined the effects of the pandemic on our behavior. The report revealed that our nationwide presence at locations related to retail, recreation, workplaces, and transit stations was down between 7% and 13%; but our usage of parks, beaches, marinas, dog parks and public gardens was up a whopping 65%. If asked beforehand I would not have predicted those particular changes; but that information is now useful information for businesses.
Knowing where we are going and what we are doing is useful for several interested observers:
????These factors, these drivers, which are in turn driven by our ability to collect and use both Data and Meta-Data, will continue to change how we live, work, and even govern ourselves. Not to insert myself into the various controversies involving elections; but it’s not hard to imagine the popularity of leveraging Blockchain technologies to vote online, as well as to track and verify that your vote is counted as you cast it. It has always been true that when people become aware of something they see as beneficial, they tend to want to have it; whether it’s as consequential as election access and integrity, or as mundane as ordering socks online.
????Many of the changes will be predictable, many will not.?Of course, any change will tend to produce at least some surprising results, but one prediction I feel pretty confident in making is that the revolution in Data Mobility is still very young. To steal a line from a famous 20th century Englishman; we are really only at the end of the beginning.
??There’s one last thing to talk about. Power. Not computing power, but electricity. Power to run the servers that are the heart of any Data Centers, and power to cool them.
????You may or may not have noticed that the Data Center industry has found it convenient to refer to the size of any given facility not in terms of data capacity, but in terms of power consumption, or perhaps better to say in terms of a power supply capacity. There are some practical business and industry reasons for this, but the truth is that part of it is simply that it’s more relatable and more useful over time.
????Let’s say that you have two Blade Servers; one built in 2015 and one built in 2020. Chances are high that the newer server will store more data, process it more efficiently and serve it up faster than the older one; all while consuming less power. You could talk about a Server, a Rack, or an entire facility’s capabilities in terms of storage capacity or processing speed, but that’s more or less constantly changing. There’s also the question of what you’re doing. Are you running one or more AI’s to for data mining and/or analysis? Are you storing data for archival purposes? Are you mainly running virtual machines? There’s a good chance you’re tasking resources to multiple applications, especially if we’re talking about a public Co-Lo facility. In this context, what your Servers can do is less important than those Servers having the resources available to do them.
????In a way it’s like the EPA mileage ratings on new car window stickers. There’s some utility for comparison, but usage tends to trump measurements. If two people buy a brand-new Suburban and one uses theirs to ferry the kids around while the other is retired and pulls a large camping trailer around the country; you’re going to see very different real-life results.
????Rating a Data Centers in terms of power capacity gives you a far more stable number over time, but it also does something far more useful for both planning and life-cycle.
????A Data Center is a physical structure, frequently built of concrete and steel. As such, the facility will have a lifespan measured in decades, if not centuries. When the physical structure itself is eventually torn down, it’s unlikely going to be because the building itself is falling apart. Almost certainly it will be to replace it with something else, maybe a newer and larger Data Center, maybe something else entirely.
????The hardware inside is another story altogether. Its lifespan is measured in a small number of years, sometimes even in months. The pace at which technology advances is simply too fast and obsolescence comes too quickly. You might succeed in keeping a home computer for most of a decade, perhaps replacing the hard drives at least once, and adding more RAM, but sooner or later the operating system itself becomes obsolete and the vendor ceases to support it.
????Of course, the equipment inside is not limited to just the Servers. There’s also the power distribution equipment, the uninterruptable power supplies, the racks and wire management structures, the cooling systems and even the on-site power generation equipment. The lifespan of those items ranges from years to decades with proper maintenance, and some of it has an essentially indefinite lifespan. Powder-Coated metal racks installed inside climate-controlled buildings and cable management structures made of ABS or HDPE are going to last at least as long as the building itself. Power supply, distribution, and back-up equipment can be maintained and even upgraded on an ongoing basis. The Servers themselves are a different story entirely.
????What all this means is that it’s far more useful to rate a Data Center for its power consumption capacity, the ability it has to support the Hardware and Applications that produce revenue. That capacity will remain relatively stable over time, which is helpful when you’re planning the life-cycle of a facility and trying to make sure that you’re maximizing the returns on your capital investment.
????So how much power does a Data Consume? The flippant answer is that it depends on the facility. There’s too much variability to give a single answer, unless of course we look at the larger trends.
????According to a White Paper published by Schneider Electric, A 1 MW high-availability Data Center can consume about $20,000,000 of electricity over its lifetime; and recent articles suggest that for some customers the cost of electricity is greater than the cost of the IT hardware that the Data Center houses.
????Worldwide power usage by Data Centers is currently in excess of 200TWh, or about 1% of global electrical use. The IEA estimated it was around 250TWh in 2019 and projects it to be about 279TWh in 2022; though I’ve seen higher figures, with an article from 2020 on www.vxchnge.com claiming over 400TWh globally. In the US, the information I can find seems to point to usage in the mid-70’s Terawatt-Hour range, with Data Center power consumption being around 2% of the US total consumption.
????Hyperscale Data Centers are leading the energy efficiency edge, with a lot of facilities in the 1.4 – 1.5 PUE range, while many facilities around the world are closer to 2.0 PUE. Google actually has an impressive PUE of 1.11 across its Data Centers worldwide; with a theoretically perfect score being 1.0. Of course, with the current interest and focus on Edge computing, there will still need to be a lot of smaller facilities pretty much everywhere, and those won’t hit the same economies of scale that the larger Data Centers will. Also, a lot of the efficiency pick-ups from recent years were due to the increasing use of higher efficiency chips and Solid-State memory devices, so sooner or later that will taper off.
????Ultimately, most of the energy pouring into Data Centers will be released in the form of heat pouring off the cooling systems. Regardless of which particular component of the complete Data Centers system produces the heat initially, it is eventually pumped back outside from the Refrigeration Units and the Chillers. There are a lot of strategies for reducing the electrical consumption of Data Centers, ranging from simply insulating Server areas to the point that they become akin to commercial refrigerators, to identifying and shutting down “Zombie Servers”.
????According to a 2020 study published by Energy Innovation LLC, Data Center cooling is responsible for about 43% of power consumption, the Servers accounting for another 43%, Storage Drives for 11% and the Network hardware for the last 3%.
????Figures like this vary because official statistics aren’t compiled, at least not in any widely organized way, so a lot of this is done by mathematical modeling. Regardless, it’s probably a safe bet that no matter the efficiencies, Data Centers themselves will continue to rapidly multiple to meet the soaring demand for data management and mobility. As a consequence, power consumption as a percentage of global use will continue to climb, as will the drive for better solutions.
????I’ll wrap up with a little wild speculation; because what’s the fun of talking about these things if we don’t indulge in some speculation.
????I find myself wondering two things:
And at long last, we’re at the end of my musings for today. Thank you for staying with me through this unusually long and meandering essay.
???
Chief Engineer at Woods Hole Oceanographic Institution
2 年@.