We’ve all heard the buzz about deduplication in recent years.  But what is deduplication really, and what’s the benefit?  An official definition would be the intelligent elimination of redundant data, in order to optimize storage capacity.  But the challenging part for users is that it can be implemented and delivered in many different forms.  File-level, block-level, inline, post-process, client-side, target-side… it can all get quite confusing.

It’s no surprise that we’ve heard even more about deduplication lately, as organizations experience increased pressure to develop new, low-cost approaches for optimizing their storage environment.  As we’ve discussed in previous blogs, Data Protector 6.1 is all about lowering total cost of ownership.  So this was a major area of focus for the recent Data Protector 6.1 release… and fortunately, several deduplication options are now offered that allow you to choose the best solution for your needs.

First, we have virtual full backup.  For those of you who are familiar with Data Protector – ok, I admit it… this is not really new.  Virtual full backup was first introduced with version 6.0.  But it is probably the most under-utilized and little-known feature in Data Protector today, so it’s worth mentioning here.  In fact, the virtual full concept is something that is completely unique to Data Protector, and can dramatically reduce the size of certain backups on disk-based target media – by up to 95%.  How is this possible?  Virtual full backups are actually just a series of pointers to data that already exists on the backup media – they contain no backup data themselves.  This allows the creation of a new full backup quickly and easily, with no added burden on the clients or the network, and minimal storage capacity requirements.  Even better, users have a create an easy restore point, as well as a point from which to consolidate prior backup jobs and move them to tape.  And best of all – this feature is completely free of charge.

The second option available with Data Protector is the one that is most popular with today’s users – target-side deduplication.  Data Protector supports a wide variety of third-party storage devices using inline (“on the fly”), block-level deduplication features such as Data Domain, NetApp, Quantum, and others.  And to make support of these devices more cost-effective, Data Protector’s Advanced Backup to Disk (D2D) licensing was changed last year from being source-based to target-based as well.  This allows users to base their licensing cost on the physical capacity of the device, rather than the logical capacity – essentially allowing 40 TB, 50 TB, or more to be backed up with only a 10 TB Advanced Backup to Disk license.  Another great way to lower costs.

Finally, of course, there are the deduplication features in HP’s own D2D and VLS products.  The D2D series provides inline deduplication, with easy integration into any backup environment, while the VLS series employs a post-process deduplication method (waiting for all data to be written to disk before deduplication begins), for maximum backup performance.  Each of these methods has strengths and weaknesses, but HP provides the flexibility of both choices.  And in keeping with HP’s “Better Together” theme, using Data Protector with the D2D series provides the added capability of a special Low-Bandwidth Replication feature.  This feature leverages the power of deduplication to reduce the amount of bandwidth required for replicating data from one location to another – for example, from a remote/branch office to a central site.  By deduplicating the data at the remote office, only the changed data must be transferred again to the central site.  Then, the original data set can be “rebuilt” at the central site and offloaded to tape for long-term storage.

In the coming years, analysts expect the importance and usage of deduplication technology to increase.  It will be a key element in the buying criteria for data protection solutions.  It is seen as significant enough even to accelerate the transition from tape-based backup technology to disk-based, due to its cost-savings potential.  As vendors look to enhance their capabilities in this area, Data Protector will be no exception – with multiple options today, and much more to come in the future.