Free Data Migration Tools? Not So Fast (Or, For That Matter, Not So Effective)

There is no shortage of tools available in the market that claim to perform data migrations with near-zero downtime during cutover. Many of these tools are even offered as freeware or are included as part of the operating system. But the old saying of “you get what you pay for” definitely applies where data migration is concerned. In fact, tools that claim to perform data migration on the cheap probably aren’t meant for real data migration at all.

Let me explain. Data in an enterprise can be lumped into two basic types. The first type is “lots of small user files that hardly change.” Examples are: business presentations, office documents, spreadsheets, images, etc. These are typically KBs or MBs in size, and once the file is created, it seldom gets reopened and changed. The second type is what I call “a few large files that are active and always open for read, write, or both.” These are typically database files for Oracle, SQL, or blobs of homegrown applications that are always in use as long as the app is online. These files tend to be gigabyte or even terabyte sized files. As we’ll see, when it comes to migration, these two data types differ in more ways than size.

If you need to copy a few hundred (or a few thousand) small files that hardly change, you could actually get away with using free system tools like Microsoft Windows’ xcopy /m command. These tools are effective at copying entire files and setting their “archive” bit when successfully copied. By repeating the exact same command, only the new/modified files are then copied. So, for Type 1 data, the xcopy tool is usable for a data migration job because it will copy most of the static files to the new storage, and repeated use will only copy the changed files and the ones that were skipped previously due to activity. In fact, Microsoft has directory/folder level synchronization tools that can automate the entire process. These tools work well because the bulk of the data is static, and a brief cut-over outage (putting the directory/folder offline to guarantee no more changes) will complete the migration without missing any files.

But not so with Type 2 data: big database files. They’re always in use. And so, your free migration tools will always skip over them. Even if you find a way around the “open file” issue (for example, by taking a snapshot of the disk and mounting it as a static file set), these active files are always changing (albeit only a small portion of the file, like 1% per day). But free tools like xcopy do not KNOW which records have been changed, and can only copy the entire TB-sized files over and over again. In summary, if you want to move a 200TB database, you’ll need to re-copy all 200 TB every time because the freeware tools have no way of knowing what has changed within the file. Practically speaking, using these free tools for migration can only be possible if you shut down the database throughout the copying process.

That brings us to the third problem with freeware migration tools: performance. Because freeware tools copy files from the host, they impact the performance of other host-level applications. That’s not much of a problem if you’re moving 500GB of files at midnight. It is, however, a big problem if you’re moving 200 TB of data, since you’re now impacting users on the system who may have to wait two minutes for their Excel file to open while your freeware tool is hogging all of the host’s bandwidth.

I’m not saying that you shouldn’t use freeware for file migration. I’m just saying you shouldn’t use it for data migration. There’s only one solution on the market today that is designed to move large, active data (and smaller, inactive data too) without disrupting application performance or causing extended periods of downtime: Cirrus’ Data Migration Server (DMS). We know this because we designed it from the ground up to do what other tools can’t do. DMS offloads all the heavy-lifting of data copying away from the host to avoid impacting host system performance. And DMS is designed to migrate not just the Type 1 data set, but also the massive, active databases, recopying only those blocks that have changed since the last sync, so you don’t need to move 200 TB every time.

So, for day-to-day Type 1 file migration, xcopy or Robocopy or whatever freeware tool you use is probably fine. But for real data migration scenarios, there’s really only one choice: Cirrus Data Solutions’ DMS. While there may be other tools (such as virtualization appliances) that can also perform block-level data migration, DMS is the only tool built with the patented Transparent Data Intercept (TDI) technology that eliminates all the normally risky changes and downtime required to deploy the migration appliances. In all cases, DMS can be installed and migration can start while your apps are running at full speed. In some cases, even the final cut-over process requires no outage, resulting in a totally transparent data migration project that has zero downtime from beginning to the end. How do we get away with real zero-downtime migration? Well that’s another topic for another blog…