Migration with DataCore SANsymphony (Expanded Edition)

INTRODUCTION

Switching gears a bit from the performance discussion, I wanted to cover another aspect of DataCore SANsymphony: Migration. Migration is something that every storage administrator has to face at some point. A few reasons for this inevitability come to mind: 

• Storage hardware is approaching end-of-life
• Expanding storage resources due to growth
• Moving datacenter locations

In this highly virtualized world we now live in, most compute hypervisors (i.e. VMware, Microsoft Hyper-V, etc.) allow for the online migration of virtual machines from one storage device to another. However, what happens when you have systems or applications which are not virtualized either due to them simply not being virtualized yet or because they cannot be virtualized? This often ends up becoming a very manual and tedious process.

In this article, we will direct our attention to this type of migration. But before we review the steps involved in performing the migration, let's cover some background details on what makes all this possible.

BACKGROUND

As you may already be aware, SANsymphony at its most foundational level is a storage hypervisor. A hypervisor is an abstraction or translation layer which provides decoupling between the two layers in consideration. For example, VMware provides abstraction of the compute layer from the underlying hardware to allow applications to execute regardless of the hardware chassis in use. Similarly, SANsymphony provides abstraction of the data layer from the underlying storage hardware to allow data to freely exist across any common storage device.

However, abstraction isn't enough. The system must be compatible with the underlying storage platform. Just as VMware requires an industry standard x86-64 based compute platform, SANsymphony requires a T10 standards based storage device. All storage vendors which adhere to this standard are in principle, compatible. The combined characteristics of compatibility and abstractability opens a door to some interesting possibilities, one of which happens to be the topic of this article.

Consider for a moment we have a T10 compliant storage device underneath a T10 compliant software intelligence (i.e. SANsymphony). First this means they communicate using the same language. Second, the upper-layer software intelligence would have access to the lower-layer raw data structures on the storage device (i.e. disk sectors) which contain the data (regardless of what the data is). If both of these statements are true, then in order to relocate the data from one T10 storage device to another, all one needs is a transport mechanism to facilitate the move. As it turns out, the SANsymphony framework provides this transport mechanism. Let's take a look.

MIGRATION TYPE #1: SYNCHRONOUS MIGRATION

DataCore SANsymphony provides both synchronous and asynchronous operations and we will explore both, starting with synchronous.

Synchronous migration refers to migrating the data between two systems that have a relatively-near proximity to each other (typically within 100 km, or 62 mi). In most scenarios, this would represent a migration within the local datacenter, between campuses, or even between nearby cities.

The diagram above shows two SANsymphony engines each with their own respective storage system attached. In this scenario where we will be performing a migration, the underlying storage systems will be placed into a special mode called Pass-Through mode. I will explain this mode in more detail later in this article.

Let's take a look at what this configuration would look like in the real world within the DataCore Management Console.

The screenshot above shows two SANsymphony Engines, DCSSV01 and DCSSV02 respectively. Under Physical Disks in the tree to the left, notice that the volume (labeled Existing Storage LUN 0) which contains the data we want to migrate, appears as Locked. This is an automatic protection mechanism that SANsymphony imposes to ensure that the LUN remains unaltered until directed by the operator.

The is also true of the other volume on DCSSV02. Under Physical Disks in the tree to the left you will see the volume which will be migrated to (New Storage LUN 0) is also Locked.

IMPORTANT NOTE: Although I am not showing it here, you can re-present the "Existing Storage LUN 0" back to the application server(s) so they can continue working as normal while the migration occurs.  

In order to begin the process of migrating the data from the LUN connected to DCSSV01 to the LUN connected to DCSSV02, we must first create a DataCore Virtual Disk. Simply right-click on the Virtual Disks object in the tree under DCSSV01 and select Create Virtual Disks.

The Create Virtual Disk wizard should now be visible. Simply give the virtual disk a friendly name which denotes its purpose, select a disk type of Mirrored, and set a size for the virtual disk to any non-zero value. The size of the disk will be automatically resized later based on the geometry of the source volume. Click Next to continue.

The next screen is where you will designate the physical disk which is backing the virtual disk. Normally this would be a disk pool, however in this case since we are migrating a disk with existing data blocks, we will need to select Pass-Through Disk in the dropdown box for both DCSSV01 and DCSSV02.

 

The last screen is simply an acceptance screen. Verify that the storage source for each DataCore Engine is correct. In this case, we will be migrating the data from DCSSV01 to DCSSV02.

You will also be reminded that the second pass-through disk (which is DCSSV02 in this case) will be overwritten and the size of the virtual disk will be adjusted to match the source volume.

Once the process starts, you will be presented with the progress of the migration on the Info tab as shown below. Notice that the source volume status is immediately set to "Up to date" and the destination volume status is "In full recovery". This indicates the migration of data from the "Existing Storage LUN 0" to "New Storage LUN 0". The rate of migration is also shown in the progress bar on the right-hand side.

Once the migration has completed, the both statuses will show as "Up to date".

At this point, the data is fully mirrored on both sides. You can now proceed with the migration of additional LUNs. When completed with the migration, you can either split the mirrors, continue migrating other volumes, or simply shutdown the DataCore Engines and remap the New Storage LUNs to the original application servers. All the data structures have been migrated and are now available on the new storage system.

MIGRATION TYPE #2: ASYNCHRONOUS MIGRATION

Asynchronous migration refers to migrating the data between two systems that are greater than 100 km (62 mi) apart. In most scenarios, this would represent a migration between datacenters, cities or even countries. There is technically no distance limitation to this approach as long as you have a standard TCP/IP connection between the sites.

Just as before with the synchronous approach, we are going to start out with a LUN that has existing data on it ("Existing Storage LUN 0" on DCSSV01) and a LUN that will be used as the destination LUN ("New Storage LUN 0" on DCSSV02). This process can be accomplished from engines within the same group or across different groups.

In order to begin the process of migrating the data from the LUN connected to DCSSV01 to the LUN connected to DCSSV02, we must first create a DataCore Virtual Disk. Simply right-click on the Virtual Disks object in the tree under DCSSV01 and select Create Virtual Disks.

The Create Virtual Disk wizard should now be visible. Simply give the virtual disk a friendly name which denotes its purpose, select a disk type of Single, and set a size for the virtual disk to any non-zero value. The size of the disk will be automatically resized later based on the geometry of the source volume. Click Next to continue.

IMPORTANT NOTE: With the asynchronous approach, we will not be selecting a disk type of Mirrored. The disk type of Single will be used here instead.

The next screen is where you will designate the physical disk which is backing the virtual disk. Normally this would be a disk pool, however in this case since we are migrating an existing disk and its associated data blocks, we will need to select Pass-Through Disk in the dropdown box for the DCSSV01 engine.

The last screen is simply an acceptance screen. Verify that the storage source is correct.

You will also be reminded that size of the virtual disk will be adjusted to match the source volume.

IMPORTANT NOTE: You will need to do the exact same process on the destination engine (DCSSV02) for the destination volume (New Storage LUN 0). Once completed, both volumes will be designated as single pass-through volumes.

Now that both volumes on both engines are designated as pass-through volumes, we can proceed with setting up the replication process. Right-click on the Virtual Disk named "Existing Storage LUN 0 - Source" and select Replication -> Create Replication.

You will be presented with the replication wizard. Simply select the destination DataCore engine (DCSSV02) and the destination virtual Disk (New Storage LUN 0 - Destination) from the respective dropdown boxes. Finally, click Create to start the process.

You will reminded the destination volume will be overwritten with the data from the source volume and that the destination volume will not allow access during the process unless activated or split. For the purposes of migration, we will be splitting the volume when we are done.

When the process begins, a consistency check will be performed on the source volume.

Once the consistency check is completed, a full initialization will occur. This is equivalent of a full seeding of the data structures from the source volume to the destination volume.

Once the full initialization is complete, the volumes will remain in sync until a split occurs severing the source from the destination.

To remove SANsymphony once the migration is complete, stop I/O on the source side to ensure that no changes are occurring during the transition. Once I/O is stopped ensure that the data buffer is empty (as shown on the screen above). This will be depicted by the "Remaining Data" having a value of zero. Then you can proceed with splitting the replication simply by clicking on the Split Replication button in the toolbar at the top. Once split, both DataCore engines can be shutdown and the new volume(s) on the new storage system can be presented directly to the application server(s) on the destination side.

MIGRATION TYPE #3: PASS-THROUGH SNAPSHOT

Pass-through Snapshot refers to migrating the data between two physical storage arrays attached to the same DataCore engine using DataCore Snapshot. Pass-through is a feature that allows attaching an existing storage array with data present to the DataCore engine for the purposes of "passing-through" the original volume to the original application server(s). Once in this mode, features such as Snapshot are available to be used against the newly passed-through source volumes.

In this scenario, a primary volume from one array (i.e. Volume_SRC) is snapped to a secondary volume from another array (i.e. Volume_SRC_Snap).

The full snapshot operation copies all data from the primary over to secondary while the primary is being accessed by the application server(s). Additionally, once the full snap operation is completed, DataCore tracks the changes that occur on the primary volume. At a later time, a Snapshot Update command can be issued to the snapshot bringing the secondary volume up-to-date with the changes that occurred on the primary volume.

The method of performing a snapshot from one storage system to another is an effective way of migrating large amounts of data at a local site from one array to another with only one DataCore engine deployed while maintaining access to the original data.

CONCLUSION

DataCore SANsymphony provides very powerful data migration features which emerge from its ability to interoperate with common block storage devices. In this article, we covered the concept of migrating existing storage devices in a mode called Pass-Through. DataCore can also perform data migration in its native Pool Mode (where the backing storage devices remain under full control of SANsymphony). Pool Mode provides the following benefits:

• Allows non-disruptive storage hardware upgrades
• Clears and reclaims space occupied on the original drive(s)
• Allows decommissioning of active physical disks non-disruptively from pools and redistributes their contents among the remaining disks in the pool
• Maintains redundancy and performance during major maintenance operations, upgrades, and expansions 

For more information on DataCore SANsymphony, please watch the video below: