As we have been working on the development of a DCIG Buyer’s Guide for Hybrid Storage Arrays, it has been interesting to see the different approaches that the vendors are taking as they seek to leverage flash memory plus traditional hard drives to deliver previously unheard of IOPS and ultra-low latencies at a cost per GB that makes sense to a broad range of businesses. The “secret sauce” varies from vendor to vendor, but in every case it involves sophisticated caching and/or automated storage tiering software.
Many business initiatives are hampered by inadequate storage system performance. In the last several years, flash memory has offered tantalizingly low latency and high-IOPS performance, but at a cost per GB that has put flash-based storage out of reach for many of these businesses.
More than a dozen companies are now addressing this challenge by delivering hybrid (Flash Memory + HDD) storage systems that are generally based on some combination of the following ingredients:
- Sophisticated caching and/or automated storage tiering
- Storage optimization software (Deduplication, compression, and thin provisioning commonly achieve a 6x to 12x multiplication of effective capacity.)
- Metadata stored separately from file data on high-performance media
- Large DRAM caches (Some vendors deduplicate and compress the cache, further increasing its effective capacity and the percentage of IO requests completed from cache.)
- Flash memory to provide low-latency storage, sometimes including both SLC and MLC in a single array
- Multi-terabyte hard drives (HDD) to provide low-cost capacity
Flash as Cache
Some hybrid array vendors use flash memory in a way that is unambiguously cache. Nimble Storage uses separate flash memory devices for write cache and read cache. The write cache consists of fast SLC flash drives in a RAID 0 configuration. Multiple random IOs are cached and coalesced into a single sequential IO operation, while large sequential IOs bypass the flash cache to be written directly to traditional hard disks. Nimble’s separate read cache consists of larger capacity, non-RAIDed MLC drives.
Flash as Storage Tier
Other hybrid array vendors use flash memory in a way that is unambiguously a storage tier. Although the normal operating mode for Tegile Zebi arrays combines a large DRAM cache and the writing of all data to flash followed by off-load to HDD, Zebi arrays also permit administrators to “pin” particular volumes into flash so that 100% of all read and write operations are guaranteed to be completed from low-latency flash memory storage. These “pinned” volumes may live exclusively in flash or be set to replicate to HDD in the background.
Blurring of Caching/Tiering
A third group of hybrid array vendors use flash and HDD in ways that blur the distinction between flash as cache and flash as storage tier. As noted above, Tegile Zebi arrays do this in their normal operating mode. So do Tintri arrays. The Tintri file system does what they call “automated placement” to ensure only active data is kept in flash; evaluating factors like available flash capacity, disk I/O and the access patterns of virtual disks over time to decide where each block of data should reside.
Divergent Expertise and Views of the Future
I find the different approaches interesting because they reflect both the technical expertise of the people who came together to conceptualize and then bring a new generation of storage to market, and their divergent views of the future of both solid state and spinning disk storage–and its role in the data center over the next five years. Some approaches may only make sense in a world where large price and performance gaps persist between storage tiers with HDD providing low-cost capacity, while other approaches seem to apply across all-solid-state as well as hybrid solid-state/HDD storage systems.
Which vendors have created a technology that will readily adapt to ongoing performance advancements in solid state storage technology and a future of all-solid-state storage systems? (See the recent announcements regarding Samsung 3D V-NAND, Crossbar RRAM, Diablo Memory Channel Storage, and Micron Phase Change Memory.) Which storage system vendors have developed a technology that is more or less tied to NAND flash memory plus HDD? Time will tell.
In the meantime, hybrid storage arrays are getting much deserved attention for bringing low-latency, high-capacity storage to market at entry price points that make sense to many businesses. These hybrid arrays are enabling those businesses to move forward on initiatives that have otherwise been hampered by legacy storage system performance, and simultaneously enabling them to significantly improve the return they are seeing on their data center investments.