Today, more than ever, technology providers focus their messaging on how their solutions better equip organizations to implement AI. This messaging certainly makes sense in today’s AI-obsessed world. Organizations must often minimally upgrade, if not replace, their existing technology to ready their datacenters to run AI workloads.
However, as organizations do so, they must accept another reality. The assumptions, constructs, and paradigms which they have used to build their datacenters have also shifted. Maintaining high availability (HA) and securing their data and datacenters certainly still matters. But organizations must now focus on creating highly resilient datacenters that facilitate data movement and data recoveries.
Huawei Storage Innovation
This week I am in Paris, France, attending the Huawei Innovation Data Infrastructure (IDI) Forum. In attending this event, I expected to learn more about Huawei’s storage portfolio and innovations that enable AI workloads. Huawei delivered on that expectation.
While Huawei presented many enhancements to its storage portfolio during the event, one specifically stood out to me. It often seems that when providers pitch the benefits of AI, they include many disclaimers. A common one includes organizations must replace their existing infrastructure with new infrastructure to support AI workloads.
While organizations can certainly do that, organizations that already use the all-flash Huawei OceanStor Dorado may have another option. Huawei offers a Data Engine Node that connects to an existing OceanStor Dorado.
Deploying this Data Engine Node gives organizations the flexibility to start running AI workloads against their existing data. Once deployed, the Data Engine Node starts reading the data already stored on the OceanStor Dorado. It then reads this data regardless of how organizations store the data (block, file, or object protocols).
Once read, the Data Engine Node then creates and stores the vector embeddings and RAG pipelines that AI workloads require. The AI workloads then interface with the Data Engine Node to retrieve the information they need.
However, organizations that need new infrastructure to run AI workflows should find that the Huawei OceanStor A800 better meets their needs. Huawei delivers the A800 as a storage platform that includes all the features that the Data Engine Node and Dorado deliver separately. Further, the A800 can keep the AI metadata updated in real-time. This contrasts with the Data Engine Node + Dorado solution, which must use a post-processing approach to do so.
Data Movement and Nascent Language AI Emerging as AI Challenges
New innovations in Huawei OceanStor solutions, such as those described above, I fully expected to encounter at this event. However, I did not expect some of the Huawei users presenting at the IDI Forum to share other AI challenges they have encountered. These issues represent AI challenges for which no one yet has a fully satisfactory answer. For example, the National Library of Norway has two AI challenges that it wants to overcome.
First, Norway has a law requiring centralized digital preservation of all content produced in the country. That includes all public and copyrighted content that must be copied and stored within its National Library. This encompasses content such as books, newspapers, radio, TV, and web content.
Storing and retaining all this content creates the first challenge. It stores all this content on archival media such as tape. However, preparing this data for AI use requires moving it from media such as tape to flash storage. Moving one or two terabytes alone takes time. Moving multiple petabytes of data to flash so they can be processed can take forever. So, the National Library still must figure out how to quickly and effectively move all that data to flash so it can then process it.
Even once all this data gets moved, the National Library faces another challenge. The books, websites, newspapers, etc., all use Norwegian. Only about 5 million people worldwide speak Norwegian. This creates a problem.
How do LLMs and AI tools that primarily use English handle nascent languages like Norwegian? No one trained really has any good answers for this yet. Further, it poses a challenge to the sustainability of nascent languages in today’s English-speaking AI world.
Highly Resilient Datacenters Replacing Highly Available Ones
Everyone wants their data stored in a highly available datacenter, correct? Well, maybe, but storing one’s data in a highly resilient datacenter may now represent a more prudent strategy.
Storing data in an HA datacenter comes with one significant risk in today’s ransomware-filled world. If ransomware infects one datacenter, there represents a chance the ransomware then infects the other datacenter. In fact, software that keeps two or more HA datacenters in sync may replicate ransomware from one to the other.
Hence, enterprises should prioritize creating highly resilient datacenters over creating HA datacenters. For people like me, who used to work in enterprise datacenters, this represents a bit of a paradigm shift. However, this redesign will represent the best approach for many enterprises to pursue.
Benoit Fix, Huawei Datacenter Solutions CTO, recommends enterprises pursue the following four steps:
- Mitigate system unavailability risk by selecting reliable storage that offers minimum guaranteed performance.
- Strengthen multi-protocol, multi-site, and multi-active DR, and conduct regular DR drills for recovery.
- Use multilayer ransomware protection to ensure that a last clean data copy is always available.
- Embrace technologies like AgenticOps to shift from reactive response to proactive risk prevention.
Data May be Awakening but Datacenter Infrastructures Definitely Evolving
“Data Awakening, Infra Evolving” was the theme of this year’s Huawei IDI Forum in Paris, France. Further, the material and insights presented here represented the best content of the three IDI Forums that I have attended.
While I appreciated the technology advancements that Huawei announced, I also respected its willingness to let users share the AI challenges they yet face. This combination of insights highlights how enterprises should continue to expect their datacenter infrastructures to evolve for the foreseeable future.
SUBSCRIBE TO DCIG RESEARCH
One Subscription. Full Access. Better Technology Decisions.
DCIG’s one-year individual subscription provides full access to all DCIG TOP 5 reports.
Subscribe today at https://www.dcig.com/subscribe.
KEEP UP-TO-DATE WITH DCIG
To be notified of new DCIG articles, reports, and webinars, sign up for DCIG’s free weekly Newsletter.
To learn about DCIG’s future research and publications, see the DCIG Editorial Calendar.
Technology providers interested in licensing DCIG TOP 5 reports or having DCIG produce custom reports on their behalf, please contact DCIG for more information.
Editor’s note: Huawei is a client of DCIG.