Broader Adoption of Inline Deduplication for Primary Storage May be Closer than you Think

While attending SNW last week and receiving a briefing from WhipTail Technologies CTO James Candelaria, Bob Farkaly, Director of Marketing for Exar’s Storage System Products, was never far away. Because of the brevity of my meeting with Candelaria, I never really had a chance to formally talk with Farkaly at the show and connect all of the dots between Exar and WhipTail, other than to assume some component of Exar was an integral component of WhipTail’s Racerunner solid state storage appliance.

It was not until after I got home this week and had a chance to do a little research that I could connect some of the dots. I first found a blog entry on Storage Switzerland‘s site that WhipTail had integrated Hifn’s BitWackr (Exar acquired Hifn in February 2009) so companies could compress and deduplicate data stored to WhipTail’s Racerunner appliance. Because of the speed of the solid states drives, it is now becoming feasible to introduce deduplication into the primary data path without negatively impacting application performance.

However not knowing much about Exar, I followed up with Farkaly and had a chance to talk with him this past Wednesday. Here are some of the insights he had to share:

  • The Hifn card that the WhipTail Racerunner uses to do compression and deduplication on has its origins in virtual tape libraries (VTLs). Many VTLs already in use in enterprise production environments use Hifn’s Express DR 1600 cards for compression.
  • The next generation of cards from Hifn will do compression, deduplication and data encryption.
  • It is this next generation of cards and its deduplication technology that were still in development when Exar acquired Hifn that contributed to Exar’s interest in purchasing Hifn.
  • These cards operate at the block level and are currently available for Windows and Linux servers (Racerunner has a Linux kernel).
  • There is nothing that precludes these cards from eventually being used by UNIX operating systems though no one has come Hifn/Exar asking them to develop drivers and test them for UNIX operating systems
  • Doing inline deduplication for primary storage is very different than doing it for VTLs and backup applications as data redundancy is much less common. Hifn is so far seeing the best results when used in conjunction with NAS interfaces where it is seeing data reductions in the 4:1 range

So what does this conversation with Farkaly tell me? Well, I don’t expect any of the major storage vendors to be announcing inline deduplication for primary storage in the next month. But that said the combination of WhipTail and Hifn prove that it can be successfully done and Farkaly indicated that more vendors were looking at this technology. While he could not give any indication of who was looking at it or if there were any time lines associated with it, the fact that Exar essentially delivers a turnkey way to implement compression and deduplication for primary storage makes it much more likely that you will more storage arrays coming to market sooner rather than later with this functionality.

Have a good weekend!

Click Here to Signup for the DCIG Newsletter!


DCIG Newsletter Signup

Thank you for your interest in DCIG research and analysis.

Please sign up for the free DCIG Newsletter to have new analysis delivered to your inbox each week.