Cash or Cache Cloud? Cloud Journey Series
Cloud journeys don’t happen overnight. Typically, they are multi-year journeys with a few bumps, twists, and changes along the way. We’ve discussed how you can prepare, plan, and monitor the progress in the Know Before You Go article. What I see are a lot of, are organizations needing to operate in a hybrid state while they build out their cloud footprint. This is especially true when you consider the data that applications and processes rely on is often shared. Some applications create the data, others manipulate it, another application analyzes it, while another presents the findings (in one way or another). To decouple all of this is not an easy task and, in many cases, impossible.
A few years back, the term “data gravity” was frequently used. This referred to data as being heavy, cumbersome to move. It often was such a challenge that it became a limiting factor in adopting new technologies or processes. In general, it’s still true but the term is less used. NetApp’s Data Fabric was coined about 5 years ago with the intent of addressing the challenges with data gravity and other data management issues. We will discuss more about Data Fabric in the future however it’s a design principal NetApp has adhered to, allowing customers to be less burdened by data locality and harness the power of their data in a simple and fluid fashion.
As applications move, migrate, or get refactored to be cloud-native, then the data may become closer however, what do you do in the time being? This is especially true when data access is time-sensitive. Do you have time to wait for replication or is immediate access required? Can your application or users tolerate the latency when accessing data from a remote location? What impact do egress costs have on your budget? This is where caching comes in. Caching provides the ability for applications and users to be next to their data however have the “gold copy” or “point of record” copy kept in a primary location. This way, data access remains consistent, changes automatically synchronize across geographic boundaries, and data protection is simplified using the primary copy. In addition, it reduces overall costs and reduces risk exposure. The costs extend beyond just the storage costs, having a significant impact on bandwidth requirements (more bandwidth = higher monthly costs), cloud egress costs, and additional infrastructure costs like capacity, backup software licenses, backup hardware, etc. It quickly gets to be very expensive.
NetApp provides a caching capability as a built-in feature of ONTAP, NetApp’s primary storage operating system. ONTAP is available as scalable appliances or a subscription service (Keystone) for on-premises and near cloud deployments and software-defined for both on-premises and cloud-native deployments. ONTAP is available natively in AWS as FSx for NetApp ONTAP, Azure, and Google Cloud Platform (GCP) as Cloud Volumes ONTAP (CVO) and also native in IBM Cloud. ONTAP is multi-protocol serving NAS workloads like NFS and CIFS/SMB, SAN protocols like NVMe-oF, Fibre Channel, and iSCSI in addition to the S3 object protocol as well. As ONTAP is highly flexible, it makes it easy for organizations to deploy a single storage platform to meet their wide variety of needs while enabling a native hybrid-cloud experience as well. Regarding caching, ONTAP uses a feature called FlexCache. FlexCache works for both NFS and SMB/CIFS protocols. It is block based though, meaning while users and applications interact with files via NFS/SMB/CIFS, ONTAP stores these files in 4KB blocks. Why does this matter? It’s highly efficient and enables the minimum amount of data to be stored or transmitted, which we’ll get into later. A final point about FlexCache (and ONTAP) is any data traffic between ONTAP systems (replication, caching, etc.) is encrypted so you don’t create an exposure point while data blocks are sent between the ONTAP systems.
For organizations that only have SMB/CIFS requirements, FlexCache is a great solution however NetApp has another option called Global File Cache (GFC). GFC works with an ONTAP instance and Windows File Servers. GFC runs as a small VM providing caching, lock management, replication, and more. If Windows Servers are already providing remote file access at branch offices, GFC will work with them to include them in the enterprise cache solution. Even better, FlexCache and GFC can happily co-exist. FlexCache for larger or multi-protocol locations and GFC for SMB/CIFS only and/or small sites where a small footprint is optimal.
In the picture above, we see primary data stored in Microsoft Azure and AWS. Cached copies of this data are in use by users, applications, and systems all over the world. Sometimes in remote data centers, edge instances, other clouds, etc. It can really be anywhere. The 180TB volume of data (blue folder) may only be 80GB in Mumbai, 19GB in New York, and 1TB in Seattle. This would represent the “hot data” that those locations need to use the data for their purposes. This may be reading, writing, or modifying the data. The primary data maybe resides in a company data center or colocation facility like Equinox. Again, it can be anywhere and change over time.
In the above depiction, we see two primary data repositories with one being in Microsoft Azure and another in AWS. Azure has three primary file systems which are 25TB, 180TB, and 2PB in size. AWS also has three primary file systems which are 60TB, 100TB, and 10TB in size. These also have cache copies of the primary data stored elsewhere. It doesn’t have to be this way however I chose to picture it this way. The important take-home point is the primary data can reside where it makes sense for you today. Where the data needs to be accessed can be from the primary instance or the cached instance. As applications move from a data center to the cloud, the primary copy may move from the data center to the cloud and we reverse the cache/primary relationship. The bottom line is NetApp makes data access easy and flexible. Get your data where you need to work with it or write it next to where it’s being created, resulting in optimal performance.
Protecting the primary data is now very easy. We perform backup, replication, ransomware monitoring, data governance, data compliance, etc. from these primary volumes. The caches have read/write access and are extensions of the primary volumes. In Houston and Tokyo, we have IoT devices that are creating data and storing it on its cache copy which is then synchronized back to the primary copy in Azure (Green folder). As data scientists in New York use this data, they have a 2TB cache copy for fast, local access making their modeling and analysis much faster.
AWS is storing a 100TB (Purple Folder) of user and team shares as the primary instance. Teams across the globe have access to the full 100TB however based on how they interact with the data, each site has a different size cache copy based on how the local users are accessing their personal user share and team/workgroup data. The same can be said for any of the other data represented.
领英推荐
How does this work for cloud migrations?
As workloads start primarily in an on-premises state, the primary copy will most likely start on-premises as well. A cache copy is setup in the cloud so as workloads, users, and data are migrated. The cache copy provides localized access to the data they are accessing. As the threshold is crossed where 50% or greater workload exists in the cloud, it’s possible to establish the primary instance in the cloud and transition the on-premises access to cache.
This gets around the common challenge of shared data access during migration and maintaining optimal performance for the weeks/months/years a migration may take. In addition, it simplifies the migration by maintaining pathing and mounting throughout the process. Another challenge addressed is egress costs. Reducing the amount of data sent out of the cloud reduces cloud costs. With data being stored near the application, it reduces the amount of data needing to leave the cloud, reducing egress costs. In addition, while FlexCache (ONTAP’s file caching feature) is block-based, only the unique blocks of data need to be transmitted. This reduces bandwidth requirements and costs in addition to reducing egress costs since only a subset of blocks for a file may need to be transferred. Say there is a PowerPoint presentation that is stored in Azure using Cloud Volumes ONTAP (CVO) as the primary copy. It is cached in London, Frankfurt, and Chicago where users read it and make changes to it. The PowerPoint file is 23MB in size. Simon in London reads it for the first time (23MB coming from Azure), makes a few small changes and saves it (1MB sent back to Azure as ingress so no Azure cost). Margo in Frankfurt opens the presentation and while it was already cached in Frankfurt, only the 1MB of changes get sent from Azure to Frankfurt (1MB egress vs 23MB) and makes a few changes resulting in 2MB of unique data, sending that 2MB of data back to Azure (ingress so no Azure costs). When Chris in Chicago opens their cached copy, only the subset of unique data is sent to Chicago from Azure which is between 1MB to 3MB of egress from Azure based on the specific changes Margo made. The result of using ONTAP FlexCache and/or Global File Cache (GFC) on egress costs is a reduction from 69MB to only 27MB (26-27MB based on changes). This represents about a 61% reduction in egress costs. That can be very significant when looked at across a larger dataset. Egress costs seem to be between $25-$95 per TB so this represents a savings of $15-$56/TB/month. At 100TB in a month, this represents a savings of $1,500-$5,600 each month, or $18,000 – $67,000 per year.
Let’s consider another aspect of how this helps organizations. ONTAP has built-in data efficiency (deduplication, compression, & compaction) in addition to automated cold block tiering. How does this work? Well, your primary data stored on an ONTAP system will reduce the actual footprint as much as possible. This reduces costs in several ways. When you consider anything deployed in the public cloud as an ongoing monthly cost based on usage, shrinking that down as much as possible is a huge benefit. Using the AWS cost calculator for AWS FSx for NetApp ONTAP, making some common assumptions around data reduction and tiering, reduces the monthly cost of storing your data in the cloud by 70-90%. You can compare for yourself using a few easy online calculators: AWS FSx for NetApp ONTAP Pricing, Cloud Volumes ONTAP for Azure, or Cloud Volumes ONTAP for GCP. These calculators will let you get an estimate of your storage costs using ONTAP natively in AWS/Azure/GCP and compare it to other options.
Bringing it all together
Using caching as part of your cloud strategy provides several benefits.
While this was focused on cloud, FlexCache and Global File Cache are widely deployed for on-premises workloads as well. Many organizations rely on FlexCache and Global File Cache to optimize workloads like media rendering, AI/ML, user/group shares, and much more. NetApp is committed to helping organizations get the most of their data while reducing the costs and complexity out of their environment. NetApp is unique in how many native offerings are available across all the primary public cloud providers (AWS/Azure/GCP/IBM Cloud).
Quick Links:
AWS FSxN Storage Sales LATAM
1 年Rodrigo Ferreira Marques
AWS FSxN Storage Sales LATAM
1 年Michel Marquizeppe Santiago
Client Executive at NetApp - Helping clients harness today’s wealth of data and apply it to create new value across the entire organization.
2 年Timely topic and good content, Troy. Nicely done!