Introduction to Azure Elastic SAN

Azure Elastic SAN was made Generally Available a few days ago, on 13th February 2024. Here is my initial write-up covering an introduction to the service and what I've played around with so far. This blog post is that of my own opinion and information; it does not reflect or guarantee any best practices or recommendations from Microsoft. Refer to the official Azure Elastic SAN Microsoft Documentation for your decision making.

See the GA release announcement post here: Introduction to Azure Elastic SAN | Microsoft Learn

EDIT April 2024: You can now resize Elastic SANs up and down. When I first wrote this, you could only increase them. I have tried to remove references to this outdated claim in the article. See the documentation for Resizing an Azure Elastic SAN.

So, what is Azure Elastic SAN?

Azure Elastic SAN is a fully integrated solution that simplifies deploying, scaling, managing, and configuring a storage area network (SAN) in Azure. It uses the iSCSI protocol for connecting the volumes meaning we get support for multiple types of compute resources, such as Azure Virtual Machines, Azure VMware Solution, AKS and database offerings. ?

The service has high availability, durability and security options for your data. There are redundancy options offering locally redundant storage (LRS) or zone-redundant storage (ZRS). At time of writing, not all of the capable regions have ZRS support. Make sure you check limits in your region. I’m sure these will start getting rolled out to wider geographies very soon, the service is only 3 days into GA as I’m writing this.

We also get encryption capabilities for the data held, being able to use Microsoft Platform-managed Keys (PMK) or Customer-managed Keys (CMK) if you want to mange your own using Azure Key Vault. At time of writing, you can only configure the CMK option by using Azure CLI or PowerShell. This can’t be done in the Azure Portal right now, PMK is the only option for creating within the Azure Portal.

Elastic SAN allows you to optimise your performance and cost by provisioning storage in bulk and sharing the SAN’s performance across all its volumes. We are able to scale with two different options, Base Units which include capacity and performance, or just capacity units which add capacity, for lower cost.

We can manage those volumes at scale with a feature called volume groups, which inherit settings and configurations from the SAN level. With volume groups we can configure networking options and encryption options for the volumes that sit within it.

What are the benefits of Azure Elastic SAN?

Compatibility: You can use Azure Elastic SAN volumes as backend storage for various workloads and top-tier databases, such as SQL, MariaDB, and others, without having to configure storage for each compute option. It’s also great for clustering use-cases with each volume being able to support 128 total iSCSI sessions. Each connection is recommended to use 32 sessions for the best performance, meaning you can connect 4 targets to each volume.

Simplified provisioning and management: You can create and manage an Elastic SAN with a few clicks in the Azure portal, or use Azure PowerShell or Azure CLI for scripting and automation. You can also use volume groups to apply settings and policies to multiple volumes at once. I span one up and had it connected within 10-15 minutes.

Performance: You can scale your performance up to millions of IOPS, with double-digit GB/s throughput, and have single-digit millisecond latency. You can also adjust your performance and capacity as your needs change, without downtime or data migration. The dynamic scaling is extremely powerful in my opinion. The IOPS and throughput also scale very aggressively and are hugely competitive against Premium SSD or Ultra Disk options, especially on price! Note: With iSCSI being a network connected target, the performance will comply with the network limits of your Virtual Machine and not the usual VM Disk IOPS/Throughput limitations. It’s recommended to enable Accelerated Networking on the Virtual Machine Network Interface Card for best performance, if supported. It is also recommended to use Gen 5 Virtual Machines for best networking performance. Check the network limits on whatever Virtual Machine SKU you plan on using with Elastic SAN. Check the tables in VM sizes for Max network bandwidth limits. Example table can be seen showing this here: Dv5 and Dsv5-series - Azure Virtual Machines | Microsoft Learn

Cost optimization and consolidation: You can increase your SAN storage in bulk, and choose to increase your performance along with the storage capacity, or increase the storage capacity without increasing the SAN's performance (capacity on it’s own is cheaper than the base capacity units if you don't need added performance). You can also avoid overprovisioning volumes, as they share the performance of the SAN, which you’d more likely end up having to do for comparable performance on managed disks. A database use-case is one of the main reasons I’ve looked at this following a customer ask. It is going to require a 5TB Ultra Disk to be provisioned costing from ￡2100+ a month, whereas an Elastic SAN can achieve the same capacity, IOPS and throughput from ￡562 a month with ZRS redundancy.

How to plan for deploying an Elastic SAN?

Before deploying an Elastic SAN, you should consider the following aspects: how much storage do you need, what level of performance do you need, and what type of redundancy do you require. Use performance monitoring tools such as Azure Monitoring Agent (AMA), Azure Migrate and other tools to figure out what these numbers might look like for you.

You should also determine the optimal size of the Elastic SAN you deploy, based on your workload's IOPS and throughput requirements, and the cost of base and additional capacity. This will help ensure you don’t overspend; remember, you can always resize later if you need to.

You should also consider the networking options for your Elastic SAN, such as enabling or disabling access to the public endpoint. Using the Public Endpoint does not mean it's accessible over the public internet. It can be only accessed with a Service Endpoint configuration and setting network rules. You can also use Private Endpoints, as expected, which is what I used in my example below.

Refer to the planning section for more details: Planning for an Azure Elastic SAN | Microsoft Learn

How to create and connect to an Elastic SAN?

To create an Elastic SAN, you can use the Azure portal, Azure PowerShell, or Azure CLI. You need to specify the name, region, resource group, redundancy, base capacity, and additional capacity for your SAN. I’ve created an Elastic SAN in UK South using LRS, as ZRS is currently not supported in this region ?

I’ve created mine with 1 base unit giving me 1 TiB of capacity, 5000 IOPS and 200MB/s throughput. Each additional base unit gives me 1 extra TiB of capacity, 5000 IOPS and an extra 200MB/s throughput. You can see how quickly this performance scales to incredible amounts!

I’ve deployed mine in Availability Zone 1. This matches where my Virtual Machine is deployed, which will be it’s main consumer and ensures I get the lowest latency possible. It’s worth noting that LRS redundancy will offer you slightly lower latency than ZRS too. Consider this for extremely tight latency demanding workloads.

To create volume groups and volumes, you can use the same tools as above (Portal, CLI, PowerShell). You need to specify the name, capacity, and encryption settings for each volume group and volume. You can also configure network access for each volume group.

To connect to an Elastic SAN volume, you need to use the iSCSI initiator on your compute client, and enable multipath I/O for optimal performance. You can use the connection scripts provided in the Azure portal, or use Azure PowerShell or Azure CLI. It’s stupidly easy to configure using the scripts provided in the Azure Portal. Before proceeding, ensure your Private Endpoint configuration and Private DNS Zones are resolvable from your Virtual Machine (if using PE), then run the script and you’re done! Let’s see the connection steps below:

Above is the connection script generated in the Portal. I'll copy and paste this into a PowerShell .ps1 file and run it as administrator on my VM, then we’ll see it connect our volume.

We see a bunch of connection operations completed successfully. These are the 32 iSCSI sessions being connected to ensure we get full performance. If we open up iSCSI Initiator we can see my target is connected. If I look at the properties, the 32 are displayed sessions in here:

Finally, checking Disk Management we see our newly connected volume. From here we're able to format it, assign a drive letter and then start using. Simple as that! ??

How to monitor and manage an Elastic SAN?

You can use the Azure portal, Azure PowerShell, or Azure CLI to monitor and manage your Elastic SAN resources.

You can view the status, performance, and usage of your SAN, volume groups, and volumes. We have some Metrics available to us for monitoring the Elastic SAN, here is a screenshot of the currently available ones below:

Dynamically scaling Elastic SAN and Volumes

It is very easy to scale up your Elastic SAN if you need more performance or capacity. To modify the capacity and performance of the Elastic SAN itself, you can do this from the Configuration blade, as shown below:

We can also dynamically scale up our individual Volumes by going into the Volumes blade, clicking into the Volume we want to change, and updating the numbers, as shown below:

Snapshots

Elastic SAN can also create snapshots (currently in Preview) which are incremental point-in-time backups of your volumes. The first snapshot is a full copy of the volume, then every subsequent snapshot consists only of the changes since the last snapshot (deltas). Snapshots of your volume don’t have any separate billing, but they can reside in your Elastic SAN and consume the capacity. Consider using additional capacity units instead of base units if you start to run out on space holding snapshots.

You can’t change the state of the existing volume with snapshots, you can only use them to deploy a new volume or export the data to a managed disk snapshot. If a volume is larger than 4 TiB, exporting to managed disk snapshot is not supported.

Read up on the full considerations and limitations for snapshots here: Backup Azure Elastic SAN volumes (preview) | Microsoft Learn

Here I’m going to take a snapshot of my volume1 that I created earlier:

Cool, that's easy. I now have a snapshot of my volume.

My volume is 100 GiB, so within the capability of exporting to a managed disk snapshot. Let's see what this looks like in the Portal. Again, extremely easy:

You can't schedule volume snapshots from the Portal that I can see as a baked in feature. So if you want to do this, you would need to create an Azure Function or a Logic App to trigger this by running an Azure CLI or PowerShell command.

Closeout

Thank you for reading! I think Elastic SAN is a really exciting service. It’s use cases for?high IO databases and workloads look great. Additionally, migrations to the cloud from traditional on-prem SANs is now much easier to manage and scale going forward in Azure with Elastic SAN.

The very high performance metrics you can achieve with Elastic SAN for the low cost is the most exciting part in my opinion. You can scale to millions of IOPS extremely easily! When comparing Managed Disks, such as Ultra Disk or Premium SSD v2 for high performance workloads, they can get quite expensive! Elastic SAN offers a more flexible option for provisioning capacity and performance.