At a recent analyst briefing, Micron Storage leaders identified at least three critical transitions that must take place in order to unleash the full potential of flash memory in the data center:
- the transition from planar to 3D NAND, enabling a jump in global production capacity and in the capacity of individual storage devices
- the transition from SATA and SAS interfaces to PCIe/NVMe to increase bandwidth and reduce latency
- the transition to treating flash—and other future non-volatile RAM technologies—as a large pool of persistent memory rather than as a disk replacement
The Transition to 3D NAND
The transition to 3D NAND production is well underway, with 3D NAND displacing planar flash by the end of 2016 for all four NAND manufacturers and across every product category. 3D NAND will lead to greater storage densities. At the August 2014 Flash Memory Summit we saw a prototype PCIe card carrying 64TB of NAND, but no date was given for general availability. Although no specific product announcements were made during the briefing, one Micron staffer suggested we might see flash devices with 16TB raw flash capacity by the end of 2015.
The Transition to PCIe/NVMe
The existing SATA/SAS disk-drive interface is already a bottleneck for flash storage performance, and the continuing growth in per-device flash capacity makes that bottleneck more and more an issue. Happily, the transition of the flash storage ecosystem to PCIe/NVMe during 2015 will allow for up to 4 lanes of PCIe to each flash device along with much improved queuing mechanisms. NVMe drives have the potential to deliver 10x the performance of current SATA SSDs.
The Transition to Treating Flash as Persistent Memory
The transition to treating flash as persistent memory will be much more complex, but has the greatest potential to transform the data center through another 10x-100x jump in storage performance. Micron’s multi-pronged strategy to accelerate this transition includes:
- creating high-trust partnerships with OEMs,
- engaging with enterprise end users to gain a deeper understanding of their critical workloads, and
- collaborating with key players up and down the technology stack—from operating system providers to application developers—to achieve a virtual re-integration of the entire data center technology stack.
In Micron’s view, this reintegration will not be based on owning all the parts–as it was in the early days of mainframe computing–but through a reinvigorated collaboration and a set of win/win partnerships.
No doubt there will be many challenges and naysayers along the way, but Micron Storage seems to be well-positioned to facilitate just such a virtual reintegration, and has taken multiple steps in the last 12 months to realize this vision. Micron Storage has assembled a dream team of technology pros from leading enterprise storage/server/compute providers including Dell, EMC and Intel. These professionals understand enterprise requirements, and all appear to have left their prior companies on good terms and with relationships intact.
Creating High-trust Partnerships
Progress will be much quicker if key participants can learn to trust one another and collaborate much more deeply, especially in research and development. When I took over the leadership of IT at Buena Vista University I told my staff and my peers over and over, “If we will trust one another and think strategically, we can accomplish phenomenal things together.” They believed me and put that belief into action, creating the nation’s first wireless community (eBVyou) just 12 months after the original 802.11 WiFi standard was adopted.
The first fruits of the “high-trust partnerships with OEMs” element of Micron’s strategy were revealed during the February 19 announcement of the IBM FlashSystem V9000 and the FlashSystem 900. IBM credited Micron’s responsive collaboration during the research and development process with accelerating development; in particular helping IBM to gain the fine-grained visibility into flash cell health that enables the arrays to place the hottest data in the healthiest flash cells. This more intelligent approach to wear leveling enabled IBM to confidently transition from eMLC to Micron’s MLC flash while increasing the warranty on the resulting flash modules to a full 7 years.
Engaging with Enterprise End-users
Realizing the full potential of flash memory will require vertical integration across the technology stack—a process that will probably take years to play out. Nevertheless, there are near term opportunities to transform the performance of specific workloads by better understanding those workloads and then collaborating with two or three companies to bring a solution to market.
For example, Ed Doller, VP of Storage Technology, told me his team experimented with running a search directly on the CPU embedded in their SSD controller and achieved an 8x improvement in performance compared to running the search using a server’s primary CPU. This experiment demonstrates a truism that Ed shared with us, namely that as hardware gets more sophisticated, programmability emerges. Micron wants to leverage that programmability to address enterprise requirements with and through suitable partners.
Collaborating to Accelerate Storage Evolution
In order to achieve orders-of-magnitude improvements across all types of workloads, the data center technology stack will need to stop addressing non-volatile memory through block-oriented disk-based constructs and start addressing it through highly parallel page-based persistent memory constructs. This will require changes in operating systems and in applications.
Although moving forward in some areas will require high-trust partnerships, Micron can move the operating system component forward by making direct, meaningful contributions to the Unix/Linux storage stack. Micron recently joined the Linux Foundation to signal this intention, and at the February 19 briefing announced that they are creating a storage software design center in Austin, TX to foster collaborations.
The Necessity of Unlearning and Relearning
Re-inventing the data center infrastructure will be a complex, long-term task. The greatest barriers to progress may well be human rather than technical. One barrier to progress is what we already know—our customary approach to thinking about storage. The people designing next generation data center technologies will need to unlearn a lot of what they know about current systems design and return to the origins of computing, an environment characterized primarily by compute and memory.
Darren Thomas, VP of the Storage Business Unit at Micron, has articulated just such a strategic vision—redefining the future of storage—based on recognizing the opportunities and then creating high-trust win/win partnerships with key participants outside of Micron. As one of just a handful of DRAM and NAND manufacturers in the world, Micron clearly has a lot at stake in this transformation; and just as clearly Micron has decided to take an active role in shaping and accelerating that future.
All of the dynamics highlighted above suggest that storage will continue to be a locus of innovation and a lever for transformation in the enterprise data center for years to come. DCIG will continue to provide periodic snapshots of this dynamic marketplace through its feature-oriented buyer’s guides, including the forthcoming DCIG 2015-16 Hybrid Storage Array Buyer’s Guide in late March and the DCIG 2015-16 Flash Memory Storage Array Buyer’s Guide in May of this year.