Navigating the Shift from Hyperscale to Microclouds
emma | Cloud Management Platform
Unlock all benefits of cloud without the complexity.
During the early 2000s, Amazon's retail business was expanding rapidly, driving the need for a reliable and scalable infrastructure to handle the increasing online traffic and sales volume. Realizing the potential in monetizing its infrastructure expertise and underutilized data center capacity during non-peak times, in 2006, AWS officially launched its first services, including Simple Storage Service (S3) for storage and Elastic Compute Cloud (EC2), for scalable computing without the overhead of managing physical infrastructure. The services were available on a pay-as-you-go basis, revolutionizing the way companies handled their infrastructure forever.?
Following Amazon's vision, Google started offering cloud computing services with the launch of Google App Engine in 2008, and Microsoft launched Microsoft Azure in 2010. Around the same time, other significant players, like IBM, Oracle, and Alibaba, also entered the cloud computing space.? Over the years, these platforms continued evolving, adding services and features to cater to diverse needs, such as IoT, machine learning, serverless computing, and more. Competition between these major providers intensified, leading to price reductions, geographical expansions, and service innovation.?
Today, the key initiators have evolved into hyperscale giants with new vendors cropping up and maintaining their own niche markets. As cloud models evolve, extending from the vast hyperscale data centers to more compact, localized microclouds, CXOs and cloud teams face the crucial task of selecting the most fitting cloud model for their business needs. In this paper, emma explores hyperscale and microscale cloud computing paradigms to aid informed decision-making during this transformative phase of cloud adoption.
What’s Hyperscale?
Hyperscale is a system or infrastructure’s ability to rapidly handle growing demand and scale.? In cloud computing, hyperscale refers to massive cloud data centers with an unusually large number of servers, storage systems, and networking equipment that can deliver cloud computing services on an extraordinary scale.?
Leading cloud providers, like? Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), and IBM, have built data centers with thousands of servers to provide affordable, on-demand hyperscale computing. They are sometimes referred to as “hyperscalers”. Large distributed internet companies like Meta (Facebook), Apple, Netflix, and TikTok have also invested in hyperscale computing to address the challenges of scale, efficiency, global reach, and innovation inherent in their operations.?
Background
The exponential growth of data and the increasing demand for real-time processing and analytics, fueled by trends like social media, IoT, big data analytics, and multimedia content, necessitated computing infrastructures and architectures capable of handling vast amounts of data. The demand for scalable, flexible, and high-performance computing infrastructure skyrocketed. Cloud pioneers like AWS, GCP, and Microsoft Azure responded by continuously expanding their infrastructure, building massive data centers across the globe.?
In addition to the sheer scale of these data centers, the evolution of these CSPs (Cloud Service Providers) to hyperscalers involved developing highly efficient and scalable architectures using commodity hardware, advanced networking, and innovative software solutions to handle massive workloads, save energy, and offer competitive rates. It also involved building a diverse service portfolio beyond basic computing and storage, including machine learning, AI, IoT, serverless computing, analytics, and more.?
Currently, there are? over 900 operational hyperscale data centers, with several hundred more in development and expected to be operational by the end of 2024. Given the rapid, large-scale penetration of GenAI, analysts expect the global hyperscale capacity to almost triple in the next six years.?
领英推荐
Characteristics of Hyperscale Cloud Data Centers
Hyperscale data centers operate on a vastly different scale and possess several characteristics that set them apart from traditional data centers.?
Hyperscale data centers can span hundreds of thousands or even millions of square feet, hosting a vast number of servers, memory and storage systems, and networking equipment. A data center should span 10,000 square feet, accommodate 5000 servers, and provide 40MV of capacity counts to be considered a hyperscale data center. However, this is just the bare minimum.? Google’s 375-acre (16.3 million sq ft.) data center in Midlothian, Texas is one of the largest in the entire country.?
Hyperscale data centers utilize a modular design approach to scale horizontally, allowing seamless expansion by adding thousands of servers and storage nodes as needed. Because of this modular design. if a component fails or requires an upgrade, it can be replaced or upgraded independently without affecting the entire system.?
Each component within the hyperscale data center is pre-fabricated and standardized to streamline installation, upgrades, repairs and replacements. Unlike standard data centers that house a mix of standard and specialized, proprietary hardware, hyperscale data centers use off-the-shelf, commodity hardware for cost-efficiency and flexibility.?
These characteristics collectively enable hyperscalers to cater to the vast computational, storage, and networking requirements of modern enterprises and applications, offering scalability, reliability, and performance at a massive scale.
Why Hyperscale Computing Matters?
Big data analytics involves processing and analyzing massive datasets, typically generated in real-time. It requires the computational power and distributed architecture of hyperscale computing. For instance, analyzing vast genomic datasets, conducting bioinformatics research, and supporting personalized medicine initiatives require the computational resources and storage capacity provided by hyperscale architectures. Training and deploying ML models also requires substantial computational resources and parallel processing. Hyperscalers can provide the infrastructure and resources needed to train ML models on large datasets to businesses that couldn’t have otherwise invested in dedicated ML platforms. Essentially, complex workloads with dynamic resource requirements need the scalability and computational power of hyperscale data centers.?
Read the full white paper: https://www.emma.ms/files/Navigating-the-Shift-from-Hyperscale-to-Microclouds.pdf