DCIG recently updated its research into enterprise all-flash arrays and will soon publish the DCIG 2020-21 Enterprise All-Flash Array Buyer’s Guide. As we reflect on what has changed since we published the prior edition in 2018, five advances in all-flash array technology seem especially significant. Any business planning to refresh its storage infrastructure should take these advances into consideration.
NVMe and NVMe-oF Accelerate Storage Access
A new generation of enterprise all-flash arrays are delivering order of magnitude latency improvements compared to the first generation of all-flash arrays. Three developments account for most of these improvements:
- Vendors optimized storage operating systems to take advantage of NVMe parallelism
- Use of the NVMe protocol to transfer data to flash storage devices (SSDs)
- Use of the NVMe over Fabrics protocol (NVMe-oF and NVMe/FC) to transfer data to application servers
The first generation of all-flash arrays (AFAs) connected to flash memory via disk-oriented SATA or SAS protocols. These arrays often achieved I/O latencies as low as one (1) millisecond. For many enterprises, this provided a 10x improvement over legacy HDD-based arrays.
A new generation of all-flash arrays that take advantage of both NVMe and NVMe-oF are claiming latencies under 100 microseconds. (1,000 microseconds = 1 millisecond.) This represents another 10x improvement over the first generation of AFAs. Between 2018 and 2020, NVMe adoption jumped from 17% to 41%. NVMe-oF adoption jumped from 4% to 23%.
Replacing legacy arrays or even a first-generation all-flash array with an array supporting end-to-end NVMe will deliver more performance with fewer servers, in less data center space, using less power. The economic return on investment can be substantial.
Storage Class Memory
Some enterprise storage systems have used NVDIMMs as a low-latency, yet persistent, cache for many years. These NVDIMMs marry DRAM and flash memory on a card that fits into a normal memory slot on the system motherboard. During normal operations, all I/O activity occurs in DRAM. On power loss or shutdown, the contents of the DRAM are copied to the on-board flash memory. While very useful for safely accelerating storage operations, this technology has been limited to no more than 32GB per NVDIMM.
New storage class memory (SCM) technologies are making their way into enterprise storage systems in both NVDIMM and SSD formats. SCM is slower than DRAM, but faster than standard flash memory. An Intel Optane DC persistent memory DIMM can place up to 512GB per memory slot. Only one array DCIG researched currently uses this technology. However, seven of the arrays use storage class memory in SSD format. In the SSD format, capacity increases to 1.6 TB. Some arrays use SCM as a storage tier; others use it as an extension of the DRAM cache. Properly implemented, either use of SCM can produce additional reductions in latency and an increase in IOPs.
Persistent Storage for Containers
Much of the new software development in enterprises targets containerized environments. Unlike many of the workloads native to public clouds, many of these applications require persistent storage. Since 2018, the ability of enterprise all-flash arrays to provide persistent storage to these new containerized workloads has increased from 61% to 76% of all AFAs.
New Deployment Options
All of the arrays DCIG researched are available as traditional physical appliances. What has changed is that 16% of these products are available as a software-defined array instance in public cloud infrastructures. Thus, all the data services businesses rely on are now available in the cloud and with performance that is much better than traditional cloud storage. This change is a significant enabler for enterprises to move critical workloads to the cloud.
Storage-as-a-Service Options
Storage vendors have made significant changes in how enterprises can acquire and manage enterprise storage. They are offering new storage-as-a-service (STaaS) options. In STaaS deployments, the vendor is responsible for maintaining the underlying storage system, freeing business IT staff to manage the business’s data rather than managing the infrastructure. More than one-third (36%) of the AFAs we researched are available for on-premises STaaS.
Many vendors also offer STaaS in colocation facilities that are near public cloud data centers and provide high-bandwidth, low-latency connectivity to the public cloud. This is a way for the vendor to provide their customers with hybrid cloud and multi-cloud storage. Twelve percent (12%) of the AFAs we research are available as STaaS in colocation facilities.
Why it Matters
Any business planning to refresh its storage infrastructure should take these advances into consideration. These advances can increase the performance density of any data center, align IT infrastructure with current software development practices, and support hybrid-cloud and multi-cloud business initiatives while freeing up IT staff from day-to-day management of the underlying storage system.
DCIG will continue to cover developments in enterprise storage and data protection. If you haven’t already done so, please sign up for the weekly DCIG Newsletter so that we can keep you informed of these ongoing developments.