Permabit’s RAIN-EC Architecture Ready for Petascale Archived Data Stores

As previously discussed in a DCIG blog entry on the real impact of losing 1 bit in 100 trillion, non-recoverable bit errors on SATA disk drives have the potential to become a too frequently occurring problem as organizations dramatically increase the scale of their disk stores. Imagine what will happen when data stores expand to petabyte sizes with more frequent access?
Right now commercial data stores are on track to achieve the petascale range sooner rather than later. According to multiple sources, data collected and stored is doubling every year for most businesses; a rate of growth that has held fairly constant over time. In the 1990s, a 100 GB database was large enough to stress most systems – back when disk scanning speeds were 30 MB/s and database tools were relatively immature. In the current decade, terascale data stores are already common – and managing 100 GB is now considered somewhat trivial. In the coming decade, truly massive petascale systems can be expected to dwarf today’s large multi-terabyte stores – requiring a similar leap in the technology being used to store and retrieve the data.
The gap between a TB of data and a PB of data is significant. This amplifies technology “glitches” which may be considered trivial in terascale stores but which will become intolerable at petascale levels. We can catch glimpses of the problems presented by examining how the scientific community has handled petascale datasets, such as in the report Lessons Learned from Managing a Petabyte by scientists from the Stanford Linear Accelerator Center. Microsoft researchers have also looked at this problem working with scientists at Johns Hopkins University, storing images from the Sloan Digital Sky Survey.
What these sources report in common are challenges not only with moving the data onto storage fast enough, but also with numerous attendant processes, including:

  • Finding and retrieving individual objects from within the massive number of objects stored – cited in one case as requiring three years for a single node to scan the objects held in a single PB of storage using 2005 technology
  • Ensuring sufficient parallel processing access to enable hundreds of people and applications to have simultaneous access to the data store
  • Ensuring availability of data with sufficient resiliency and redundancy
  • In short, every step from data collection to data mining and distribution is made more difficult by the scale of the data

All of which makes Permabit’s products, designed to solve this entire range of problems, all the more impressive. Permabit addresses the problems of managing petascale disk stores effectively with its advanced RAIN-EC methodology, and does so using inexpensive hardware components. RAIN-EC was introduced as part of the Permabit Enterprise Archive Data Center Series, launched in early 2008. It builds on the RAIN grid architecture by providing a methodology for efficiently and effectively protecting data without paying large performance and scalability penalties while maintaining brief times (just a few milliseconds) to seek and find an object from within a massive store.
Permabit clearly understands the threat of potential data loss that using petascale archive systems presents to companies using current RAID and SATA drive technologies. Through its RAIN-EC technology and grid storage architecture found in its Enterprise Archive Data Center Series product, Permabit offers a viable means to overcome this threat and make it possible for companies to plan for the coming decade of petascale archive data stores. Permabit Enterprise Archive more effectively protects archived data stores than any currently available technology and provides companies the secure, scalable foundation that they need today to meet the needs of their enterprise archives tomorrow.

Click Here to Signup for the DCIG Newsletter!


DCIG Newsletter Signup

Thank you for your interest in DCIG research and analysis.

Please sign up for the free DCIG Newsletter to have new analysis delivered to your inbox each week.