Over the last couple of weeks my weekly recap blogs touching on the subject of cloud storage has prompted a lot of emails and phone calls to me in the background to discuss this topic so I wanted to touch on that again this week. In addition, I’ve also been doing a little research into some of Data Domain’s claims (and the counterclaims of its competitors) in regards to the advertised performance numbers on its new DD880 and under what conditions enterprise users might expect to achieve those numbers. Finally, I wanted to comment on some of the statements that I made last week about a CEO change and a corporate acquisition and end up with a new rumor that is circulating in the storage industry.
First, after my blog on cloud storage posted last week, I received a call from my friend Stephen Foskett. Anyone who knows Stephen knows that this man never sleeps and I secretly suspect he is hard wired into somebody’s storage matrix. Right now one of the many jobs that Stephen holds is that of Nirvanix’s Director of Consulting and I’m sure Nirvanix’s noticeable absence in my previous couple of blogs about cloud storage did not go unnoticed by him. However Stephen can rest easy this week as Nirvanix is now mentioned in the same sentence as cloud storage.
Much of our conversation focused on the differences between “private clouds” and “public clouds”. Previously I had brought out that most storage cloud providers are developing their own proprietary implementation of the REST API which essentially locks users into a specific cloud storage implementation. There was also another good entry earlier this month on ParaScale’s blog (another start-up storage cloud provider) on this exact topic. It points out how each application vendor has to write support for each cloud storage vendor’s implementation of the REST API. However ParaScale failed to mention in that particular blog what it is doing to make the situation any better.
Stephen pointed out to me that the story does not end there. Rather storage cloud offerings are breaking into three distinct camps – those that just do private clouds, those that do public clouds and those that can do both.
Private clouds essentially just provide NFS and CIFS interfaces. These are the easiest to deploy since many applications can already connect to these types of “clouds” since they use standard network protocols. The downside is that they negate some of the benefits of cloud storage since users have to first buy some of the storage infrastructure whereas public storage clouds do not require an upfront capital expenditure for hardware and software.
Conversely, public clouds are starting to adopt the increasingly popular but proprietary REST API implementation that is needed to navigate through corporate firewalls. In an article that I wrote for SearchStorage.com that appeared earlier this week, Jon Martin, director of product management at EMC’s Cloud Infrastructure group, even went so far as to say, “Eighty-five percent of the market is turning to REST while SOAP is fading away.”
However what Stephen brought to my attention is that companies like EMC are making its cloud storage offering available in two flavors, an internal cloud storage offering and the other as a public cloud storage offering. The internal one uses a CIFS/NFS interface while the public cloud uses a REST interface. So even if users want to go exclusively with EMC, they should not assume they can easily migrate data between the two of them.
This is how Nirvanix differentiates itself and represents a third class of storage cloud provider. It installs its own file system on servers and first caches writes intended for the cloud to local disk. The benefit of this approach is that whether users want to create an internal or external storage cloud, any application can take advantage of it since Nirvanix masks the storage cloud (private or public) by presenting a consistent file system interface to applications that it understands and to which it can write data.
This does not come without some risk. Since Nirvanix caches the data to local disk, there is no 100% guarantee the data ever makes it to the cloud since if a connection to the cloud is never established or something catastrophic occurs to that server’s disk cache, the data can be permanently lost. To get around this, Nirvanix encourages users to create disk caches on highly available disk systems for this purpose of caching writes before they go to the cloud. This is an interesting approach but I only see this approach (as I see all of them) as appealing to a certain segment of end-users.
Now I want to turn my attention to Data Domain. As many are aware, about a month ago Data Domain released its new DD880 that was specifically targeted for the enterprise backup space. I personally feel that this product achieved that ends in terms of the performance and throughput benchmarks that it published and what real production environments actually require.
Of course, like every vendor’s published benchmarks, there are caveats to achieving those performance numbers. The two that Data Domain’s competitors most often like to point to are that to achieve these benchmarks, users must implenent Symantec’s OST API and a 10 GB Ethernet connection to achieve these throughput levels.
However NEC makes the case that in order for users to achieve these throughput numbers all backup data sent to the DD880 appliance must be a duplicate of data that is already deduplicated on the DD880. In this way, no new data actually needs to be stored to the DD880. Instead it only needs to write pointers to existing deduplicated data.
In an email I received from Gideon Senderov, NEC’s director of product management and technical marketing, he wrote the following:
The third criterion is also implied to some extent by Data Domain’s claim on the DD880 data sheet: “The high-throughput, inline deduplication data rate of the DD880 is a direct benefit of the Data Domain Stream-Informed Segment Layout (SISL) Scaling Architecture. A single DD880 system achieves single-stream throughput of up to 1.28 TB/hour, performance that is imperative or protecting large, business critical databases in the data center. This performance is achieved by a CPU-centric approach to deduplication, which minimizes the number of disk spindles required to achieve throughput needed for critical single-stream operations.”
The only way performance is CPU-centric and minimizes the number of spindles required to achieve throughput needed is when the amount of data actually committed to disk is so minimal that it is not gating the data flow that is being processed inline. This happens when the data is duplicative and only pointer information is committed to disk.
Over a year ago, we (NEC) actually conducted tests with Data Domain’s 5xx hardware (DD580) and verified that their maximum stated numbers were indeed *only* applicable for 100% duplicate data. For 0% duplicate data (i.e. all data actually written to disk), the max throughput numbers we measured with the same DD system were 45% lower. Note that at the time, their published numbers did not incorporate OST or
10GbE, which co
upled with some software enhancements for duplicate data attributed to increasing their published numbers by 90% as specified in their press release a few months ago.
“While performance can increase on all systems across all protocols, large data centers using Data Domain’s flagship DD690 system with Veritas NetBackup OpenStorage (OST) by Symantec and 10 Gb Ethernet, can now support accelerated backup throughput of up to 750 MB/s, or 2.7 TB / hour. This is approximately 90% faster than the DD690’s best throughput when introduced in May 2008.”
If we apply the same ratios to the DD880 (dividing by 1.9 to account for the 90% mentioned in the press release and then multiplying by 55% to account for the 45% lower previously measured throughput for non-duplicate data), the published max number of 5.4 TB/hr or 1,500 MB/s with OST and 10GbE and 100% duplicate data suggests their max throughput for non NBU/OST applications with 1GbE connectivity and 0% duplicate data would be approximately 435 MB/second.”
I checked in with Data Domain’s Senior Director of Product Management, Ed Reidenbach, to get his response on this to see if Data Domain achieves these benchmarks by sending a copy of data to the DD880 that is already deduplicated on the DD880. (In Ed’s defense, he did not see the entire text of Gideon’s message referenced above. Gideon and I discussed this topic at a high level on the phone before Gideon sent me this additional information via email a few days later.) This was Ed’s response to my initial, shorter inquiry:
“To get this number we (Data Domain) use the harmonic mean of 10 full backups to the DD880. Each Full after the first has some new data added, data changed and data deleted. The average global dedupe rate after doing daily fulls for a relatively long retention period (40 to 45 days), is about 15-18x, not the hundreds x that might be implied.“
The term “harmonic mean” was a new one even for me so I did a little digging on the Internet to figure out it meant. I found a layman’s definition on mathforum.org and sent Ed this response in an attempt to clarify his response:
“To ensure I understand ‘harmonic mean’ correctly, this is how I am interpreting it based upon an explanation I found on the Internet:
If you want the harmonic mean of 10 and 20, you first take 1/10 and 1/20, find their average, which is 3/40, and then take the reciprocal of that which results in 40/3 or 13-1/3.
In algebra, the harmonic mean (h) of two numbers (a and b) is 1 / ((1/a + 1/b) / 2) or in other words 1/m = 1/2 (1/a + 1/b).
In the example you cite, instead of there just being two numbers used to determine the harmonic mean, you use 10 numbers. Is that correct?”
He responded to that email with the following:
“You are correct. The harmonic mean is more conservative. The harmonic mean is always less than the geometric mean, which is always less than the arithmetic mean. There would be a total of 9.“
At this point you can draw your own conclusions as to whether or not the high end of Data Domain’s benchmark numbers are achievable in most real world environments. It seems rather unlikely but then again, I have found very few vendors can ever deliver on their performance claims in real world environments. My thought is that most users are unlikely to achieve either Data Domain’s published high end numbers or bottom out at the low end numbers that NEC published in anything but highly unusual circumstances.
Finally, I wanted to turn my attention to a couple of items that I discussed in my blog last week. I mentioned that the CEO had stepped down at a company and that the company had confirmed and would issue a press release this past week to that effect. Of course, just to prove me wrong, the company did not do that. I checked in with them again this week and it now plans to issue a press release after it hires a new CEO. Once it does, I will mention the company in my blog. In the meantime, the company assures me it is doing fine and that it already has a candidate for the job.
Also, there are a few who privately contacted me and asked me if it was Copan Systems that I was referencing when I mentioned last week that I heard a company was going to be acquired. It is not Copan though I certainly can understand why based upon an article that Chris Mellor at The Register posted this past Wednesday. Unfortunately I have no news to share at this point in regards to that company.
In regards to vendor announcements that caught my eye this week (actually, it went out last week, I just didn’t see it until this week), Xiotech is offering a “Cash for Disk Clunkers” program. Xiotech is giving a $1000 credit for each terabyte of disk traded in towards the purchase of a new Xiotech Emprise storage system.
Finally, one rumor that I did pick up is that EMC may still be on the acquisition trail. Apparently it is on the hunt for an eDiscovery company but no one seems to know why or what company or companies it might be interested in. While I have not verified this with EMC, it wouldn’t surprise me if EMC purchased a company like Kazeon Systems if for no other reason just to further tweak NetApp.
Have a good weekend!