A Zettabyte of Data Puts New Premium on Scale-out Storage Solutions

In 2010 the amount of digital information created in the world exceeded a zettabyte for the first time. Now, in 2011, enterprises have to deal with the reality of managing and storing that amount of data. As they do so, they are coming to the realization that the majority of that data may only be actively accessed for a short period of time.

Archiving was traditionally how they managed this amount of data as it provided a cost-effective means to store critical but less frequently accessed data for future use. But as the amount of data grows, traditional approaches to archiving in the areas of data storage and data storage management are breaking down. This is putting a new premium on external scale-out storage systems that are optimized for economically, efficiently and securely storing this amount of data.

It almost goes without saying that a zettabyte of digital information is a lot of data. It is the equivalent of 1 trillion gigabytes (GBs), 1 billion terabytes (TBs), 1 million petabytes (PBs) or 1,000 exabytes (EBs). Granted, it is unlikely that many enterprises will store a zettabyte or even an exabyte of data this year or next. But enterprises that store petabytes of data are becoming more commonplace as many different applications are contributing to this explosive growth of data.

The examples are all around us in our daily lives: Healthcare organizations are dealing with ever increasing amounts of image and patient data,  Financial services organizations must handle a growing number of transaction details,  Enterprises are coping with rapidly increasing volumes of email and customer data.

This is not the type of application data that enterprises can afford to ignore as they are liable for all of it. So as data volumes zoom into the PBs, the costs,  complexity, and liabilities associated with managing and storing it are catching the attention of more C-level executives as they are coming to the realization that they need to store and manage these PBs of data for potentially long periods of time.

This creates problems on multiple levels for these enterprises.

  • They need a storage solution that can scale to hold that much data.
  • The value of this data may be unclear so the cost per GB for the storage needs to stay low
  • They do not want to hire more people to manage the systems storing this data
  • The frequency of data access may and likely will vary over time
  • The data may need to be stored for a long period of time
  • The data may have compliance or regulatory requirements associated with it

Complicating the issue, archiving solutions that are traditionally associated with storing and managing this type of data are really not built to handle this volume of data. While a number of them scale to store a PB or more of data, I was recently made aware of one user that needed a storage system that could handle an initial ingest of 6 PBs of data and then grow it from there. Still other large shops are also privately telling me they are generating half a PB or more of data per month. In these situations traditional approaches to archiving are simply inadequate.

This explains the increased interest that enterprises have in scale-out storage systems like the HP IBRIX X9000 which scale to over an exabyte and possesses the attributes of both traditional archiving and production storage systems.  

On the archiving side it can economically store inactive and/or static data on SATA disks and then enterprises can optionally lock it down using its WORM feature. On the production storage side, enterprises may optionally store active data on the IBRIX X9000 on high performance SAS drives. Then to optimize data placement, they can use the X9000’s built-in auto-tiering feature so archiving occurs automatically without necessarily requiring third party software or ongoing administrative intervention. 

Few if any enterprises have to deal with managing and storing zettabytes or even exabytes of data in the next few years. But storing and managing tens and potentially hundreds of petabytes of data is becoming more commonplace in large enterprises.

It is for this reason that more enterprises are looking to scale-out storage solutions like the HP IBRIX X9000. It provides many of the benefits associated with traditional archiving solutions, the performance associated with production storage systems, and the flexibility to deliver both features concurrently to help minimize the struggles associated with managing petabytes and even exabytes of data.

Click Here to Signup for the DCIG Newsletter!

Categories

DCIG Newsletter Signup

Thank you for your interest in DCIG research and analysis.

Please sign up for the free DCIG Newsletter to have new analysis delivered to your inbox each week.