StoreOnce Algorithm is HP’s First Step Toward Delivering Deduplication Unification

Deduplication is indeed a technology with tremendous benefit for organizations, and a single deduplication algorithm across an entire enterprise will certainly revolutionize backup and disaster recovery operations. Although realization of that vision is still some time away, the technology building toward it can be seen in the HP D2D4324 Backup System with StoreOnce deduplication.

As particle physicists focus on a generally accepted grand unified theory, likewise the technologists at HP work toward a vision of “grand unification” in deduplication.

“Imagine a world in which you have one deduplication algorithm that can be leveraged across the enterprise,” said Jay Livens, senior product manager, Enterprise Nearline portfolio in the Americas, during a recent podcast on the HP Storage Guy blog. He called deduplication a “game changer” in backup and data protection because it enables “orders of magnitude” improvements in space savings and bandwidth while bringing down costs in those areas. But while deduplication is being applied primarily in the backup space now, the real promise is enterprise wide data deduplication.

The  latest step towards HP delivering on that vision came on March 1st when HP announced the release of its D2D4324, the company’s latest generation of its StorageWorks D2D Backup Systems featuring StoreOnce deduplication. The D2D4324 contains the most up-to-date StoreOnce deduplication algorithm that nudges HP closer toward delivering a single unified deduplication algorithm that can be deployed across an entire enterprise.

Before getting to the algorithm enhancements included in this update, however, let’s first take a look at the hardware and software improvements built into the new D2D4324. These upgrades increase capacity and throughput, thereby allowing the appliance to back up more data more quickly than previous generation products could.

Hardware upgrades in the D2D4324 include 2 TB SATA disk drives that are double the capacity of the 1 TB SATA drives of the previous model, the D2D4312. The new system comes with 18 TB of useable capacity and can be scaled up to 72 TB useable through the addition of more SATA disk drives. This combination of faster disk drives and faster CPUs has also resulted in a near doubling of HP StoreOnce’s throughput from 2.4 TB/hr. in the D2D4312 to 4 TB/hr. in the D2D4324.

Software updates in HP StoreOnce include new monitoring and reporting features that provide aggregate and granular views of throughput, storage utilization, and deduplication ratios as well as real-time and historical insight into storage trending and activities. The granular views zoom in to the level of individual network-attached storage (NAS) and virtual tape library (VTL) devices.
Other updates include enhanced VTL emulation that can support a shared storage pool with up to 200 virtual tape drives per library as well as replication throttling that limits the amount of bandwidth used by the D2D replication engine.

Most significant to the HP vision, however, has been updates to its StoreOnce deduplication algorithm.
The algorithm lies at the heart of HP’s vision of creating a standards-based deduplication deployment across the entire enterprise to enable the most efficient and effective implementation and centralized management of deduplication possible.

In this release HP Labs has made enhancements to the algorithm to provide more effective deduplication of data types that have traditionally proven difficult to handle. This new feature of the algorithm facilitates modularity in the deduplication solution as well as its mobility from one HP platform to another.

While the algorithm is presently available only on StoreOnce D2D appliances, HP’s ultimate goal is to integrate and deploy it across more of its solutions which will contribute to eliminating the inefficiencies of the data “rehydration” process that results from separate deduplication algorithms at each point in the enterprise. Livens explains the impact of this. “This means you can move data from one system to another without having to constantly rehydrate the data first–that is, expanding the data to its full un-deduplicated fashion, and then re-deduplicating it on another appliance.”

Furthermore, many of the enhancements available in the new algorithm are also available as a free software upgrade for all currently deployed StoreOnce D2D appliances.

HP has a vision of enterprise-wide deduplication but to deliver on that vision HP chose to start with new deduplication algorithms that have an impact  across its entire HP StoreOnce install base.  While in the short term its practical benefits are seen primarily in the ability of the HP D2D4324 Backup System to achieve up to 4 TBs/hour of backup speed, longer term HP is putting a foundation in place for a single deduplication algorithm to be used across the enterprises it serves.


Click Here to Signup for the DCIG Newsletter!


DCIG Newsletter Signup

Thank you for your interest in DCIG research and analysis.

Please sign up for the free DCIG Newsletter to have new analysis delivered to your inbox each week.