In recent weeks and months, I have more than once raised the question about the safety and integrity of corporate data that is stored on systems that use RAID, especially SATA-based storage systems. The reason this problem is becoming more acute with each passing day is three-fold:
- Deduplicated data. When you deduplicate data, only one chunk of data is stored that 10s, 100s or even potentially 1000s of files may have in common. The problem emerges during the restore since all of these files need access to the same chunk of data in order to restore the file. If that chunk of data in any way becomes corrupt or unreadable, the impact of the corruption becomes far reaching and may lead to the inability to recover many files.
- Bit Error Rates. This is a problem that shows up on RAID systems when a bit of data on a disk drive becomes unreadable. It occurs so infrequently that it not really considered a problem in small environments, but as corporate data stores grow into the 10s or 100s of TBs, the likelihood of this error appearing turns from a remote possibility into almost a certainty. Reaching these multi-TB thresholds is quite possible for companies who need to keep their archived and backup data stores for years or that archive or back up data for hundreds or even thousands of applications or servers. Typical data growth rates of 50%+ per year further contribute to the likelihood of companies reaching these higher storage capacities.
- Disk Drive Rebuild Times. SATA disk drives are already 1 TB in size and 1.5 TB disk drives were announced just last week. The problem that emerges is rebuild times should a disk drive fail. Using traditional RAID, storage systems need to reconstruct every block of the failed disk drive to a spare disk drive during the rebuild. Recovering a failed 1 TB drive will minimally take hours if not days (up to a week if some stories are to be believed). It’s conceivable that another drive could fail in that RAID set causing the whole RAID array to fail. While RAID 6 partially alleviates the problem with protection against two concurrent drive failures in the same RAID set, it is likely only a band-aid, not a long term fix.
However my intent is not to leave readers hanging or fretting as to what storage systems they can select that take this problem into account. The NEC HYDRAstor is one such product that has taken steps to address this issue. HYDRAstor includes a feature called Distributed Resilient Data™ (DRD) that is able to offer more protection than RAID 5 or RAID 6 without their rebuild performance drawbacks. Because HYDRAstor is based on a grid storage architecture, it can by default survive the failure of not only multiple disk drives but also multiple Storage Nodes. The default setting is 3 disk drives or 3 Storage Nodes if multiple nodes are present (based on the video on the HYDRAstor web site, it looks like a company needs at least 12 nodes for a company to have assurance it can recover from the failure of 3 different nodes).
Here is how DRD works:
- As data is ingested by the HYDRAstor, it breaks up an incoming chunk of data into 9 fragments plus also creates 3 additional fragments that act as parity fragments for a total of 12 fragments
- At this point another HYDRAstor feature called DynamicStor™ kicks into gear. DynamicStor distributes these fragments across the disk drives and/or nodes
- In the event of a disk and/or node failure occurs, DynamicStor can reconstruct the data from the fragments on the remaining disk drives and/or nodes
Companies with growing data stores (and what company does not have growing data stores?) need to come to terms with the fact that traditional RAID technologies are no longer sufficient to protect their data long term. Features like DynamicStor and DRD on the NEC HYDRAstor take into account the growing capacity of SATA disk drives and the new risks that higher capacity storage systems create while providing companies new disaster recovery options. In so doing, HYDRAstor provides companies a means to safely and confidently store and protect their backup data on SATA disk drives.