Manual LUN Creation vs. Watching Paint Dry

Data migration involves many painstaking tasks. One of the most tedious of these is creating new LUNs on the new storage system. We recently posted two blog articles, “There is No “N” in LUN,” and “Why is lun Not a LUN” in preparation for a series of additional posts which will describe the advanced features DMS v4.00 provides. One of these features pertains to creating new LUNs as a destination for the source LUNs to be migrated. To appreciate the power of this new feature, we will walk through this operation in detail.

To prepare new LUNs in the new storage, the exact size of each source LUN must first be determined. Each new LUN must then be created using the same or larger size. The new LUNs are then assigned to the migration appliances by LUN-masking them to the initiators on the appliances. After this, these LUNs are “discovered” in the appliances, then paired up to the source LUNs one by one so the migration software can copy the data from one side to the other. Keep in mind that there are no human readable “names” associated with these LUNs. The only identity for a LUN is the GUID, as explained in the previous posts.

Further complicating the matter, there may be thousands of LUNs involved in just one migration operation.

During the stone age of data migration, the only tools available for this process were human eyes and hands. It was a painstaking and time-consuming job, fraught with potential pitfalls and mistakes. Also, creating the new LUNs is just one of many steps in the entire migration operation – one of many onerous, boring, and fastidiously demanding chores that storage industry professionals have been dealing with since the dawn of data migration projects. If we conducted a poll of data migration experts, I am sure a majority of them would indicate their preference for watching paint dry rather than for laboring over these tasks.

Even before the new LUN creation process starts, most migration tools require administrators to manually survey, identify, and verify all source LUNs – down to individual paths – before selecting the set of source LUNs to start with. Remember, there may be thousands of these LUNs of different sizes, from different client hosts, and with different paths. As we discussed in the previously mentioned blog posts, in a fibre channel environment, a path comprises the initiator WWPN, the target WWPN, and the logical unit number (lun) of the LUN, along with the WWPN (a 64 bit entity normally represented by a 16 digit hex number). Yes, watching paint dry may not be that bad after all.

When the Cirrus Data Migration Server (DMS) arrived, it already had the capability to perform auto-discovery, identification, and verification to provide visual and verbal information, which allowed the storage admin to organize the source LUNs with a few simple mouse clicks. These powerful functions cut out 90% of the manual process required by existing tools, because DMS is transparently inserted into the SAN storage links. Once in-band, DMS can analyze the I/O and determine the entire storage topology and configuration, with every detail of every component involved, completely error-free. The whole discovery and display process takes only a couple of seconds, which alone cuts down tremendously on the amount of work required.

Nevertheless, until recently, administrators were still required to manually create the destination LUNs in the new storage, then assign this storage to the DMS appliances through LUN Masking to the correct WWPNs of the initiators in the appliances. Although DMS provided the batch utilities to rename the new LUNs by GUID, as well as the fuzzy auto-paring capability to suggest the correct pairing, LUNs still had to be created manually in the new storage.

That is, until now.

The new auto-allocation feature recently added to DMS automates almost the entire process. Beginning with DMS 4.00, a storage interface plug-in module is introduced, allowing specific plug-ins to be created for any storage system that provides API for LUN creation and assignment. Using the plug-in, DMS automatically creates and assigns the destination LUNs in the new storage. The LUNs are then discovered, named, and paired with the source LUNs. The functions can be provided as REST API, CLI, or any other format through the network. A standard plug-in can be created quickly for any new storage, using the specific mechanism provided, and the same automatic allocation mechanism works for both local and remote migration.

Our engineers came up with an ingenious solution to allow the user to specify customized parameters for the LUNs to be automatically created in the most natural manner. The administrator can create one or more small template LUNs in the new destination storage using its own management tool (GUI). These LUNs are identified by the prefix “DMS-” and are used as an example for the subsequent LUNs to be created by the DMS migration process. The template LUNs are retrieved by DMS during auto-allocation. The administrator will only be prompted to select the template LUN to use. The same parameters from the selected template LUN will be used for LUN creation. This vastly simplifies the process for LUN creation, with any desired parameters specific to any storage.

With the auto-allocation feature, the complete migration process is now as simple as the following:

• Insert DMS into the environment (typically completes in 20 minutes or shorter)

• Automatically discover the entire SAN configuration (typically, a few seconds)

• Inspect and verify the discovered SAN configuration displayed (time spent depending on how much one enjoys looking at all the details provided both graphically and in tables)

• Select the hosts/LUNs to migrate (can be accomplished in few minutes, depending on how familiar the administrator is with their storage)

• Create template LUNs in the destination storage using the storage’s own management GUI, if needed

• Specify the IP address and credentials for the new storage system, then select the template to create the destination LUNs

• The host LUNs are now created and assigned automatically to the DMS appliance

• The source and destination LUNs are automatically matched and paired up

• Data copy begins

The entire preparation process – from installation to LUN creation to initiating migration – can be completed within an hour. In comparison, traditional migration methods can take weeks of effort before a single byte of data is copied.

When we presented this new feature to our storage partners, they eagerly and enthusiastically provided their APIs in the hopes that we would quickly create plug-ins for their own storage systems. We will publish the specifications and a template for the plug-in so that anyone can create a plug-in for any storage. Keep watching our news page for an official announcement on this.