Using cluster file system software on virtual machines (VMs) in VMware environments has always been a bit problematic at best. While it could be done with techniques like Raw Disk Mappings (RDMs) and 3rd party cluster file system software, organizations need to sacrifice “desirable” virtualization features like vMotion to achieve it.
This has now changed. VMware quietly made available a new multi-writer flag for VMFS that, when combined with a solution like the VERITAS Cluster File System (CFS) from Symantec, enables VMs on the same or different physical ESX servers to access the same VMDKs.
The VMware Virtual Machine File System (VMFS) and its Virtual Machine Disks (VMDKs) included with VMware vSphere abstract away the underlying storage hardware making them some of VMware vSphere’s more desirable features. Together they make VMware features like vMotion possible so a VM may move between different physical ESX machines.
The one “gotcha” with vSphere is its inability to virtualize highly available applications that already use or want to use cluster file system software. This software needs to directly access and communicate with the underlying storage to maintain a highly available cluster.
However if these applications with cluster file system software are virtualized, vSphere inserts VMFS and VMDKs between the cluster file system software and the physical storage rendering the cluster file system software inoperable.
To date the workaround to VMFS and VMDKs has been to use VMware’s RDMs. Using RDMs, the cluster file system software can directly access and communicate with the storage as RDMs expose the native properties of the underlying disks that the cluster file system software needs to operate.
By using RDMs and bypassing VMFS and VMDKs, vSphere features like vMotion become unavailable. This decreases VMware’s value proposition since access to vMotion is often a big motivation for virtualizing highly available applications in the first place.
VMware’s availability of its VMFS multi-writer flag makes virtualizing highly available applications that use a cluster file system a viable option. Prior to the availability of this multi-writer flag, sharing a VMDK was disabled by VMware vSphere to prevent a VM from inadvertently accessing another VM’s VMDK and corrupting the data on the VMDK. By turning this flag “on,” a single VMDK may be shared among multiple VMs whether the VMs are on the same or different physical ESX hosts.
This is not to imply that enterprises should by default turn this multi-writer flag “on” when they create a new VMDK. Turning on this flag makes a VMDK available to many VMs as vSphere relinquishes its control over the VMDK. This control is now given to the guest VMs that must have software in place which can assume the responsibility for managing which VMs have access to the VMDK and when.
This granular level of control is exactly what these highly available applications that use cluster file system software need. Now as VMs are created and applications virtualized, the VMDK associated with the VM is instead presented to the cluster file system software.
By enabling this multi-writer flag, any VM with the cluster file system software installed on it may communicate with any VMDK(s) associated with any VM in the cluster. What makes using VMDKs particularly appealing is that the VMs in the cluster get access to VMware’s vSphere vMotion feature since they are using VMDKs as opposed to RDMs.
They also get access to all of the benefits associated with cluster file systems in addition to the benefits afforded to them by VMware vSphere. For example, using the VERITAS Cluster File System (CFS), organizations can:
- Control file access. VERITAS CFS ensures that reads and writes to a file by each VM occur in the correct order to maintain the integrity of the data in the file.
- Achieve sub-minute application recovery times. The high availability component of VERITAS CFS detects application failures instantaneously and initiates the service recovery on any other node which currently has access to the application data, providing a very fast service recovery architecture.
- Get efficient storage utilization. Applications and work-flows can now be distributed across different VMs, providing a single pool of storage where all of them can immediately read and write, avoiding data duplications and eliminating the need to transfer data between the VMs.
- Do in-place migration from physical to virtual. As many organizations are looking to virtualize their highly available applications, VERITAS CFS gives them flexibility to first introduce some VMs into their existing cluster of physical machines and then non-disruptively migrate physical nodes over to VMs over time.
- Offer access to up to 64 VMs to the same file. Some applications require multiple VMs that are part of its cluster to access the same file at the same time. Using VERITAS CFS, up to 64 nodes (physical or virtual) may currently access the same file at the same time.
- Take application consistent snapshots. Using VERITAS CFS application consistent snapshots may be taken across all of the nodes in the cluster to improve the application’s protection and recoverability.
- Experience little to no performance degradation. Preliminary testing that Symantec has done in-house on VMDKs and RDMs with the same application reveals little to no performance degradation when going from RDMs to VMDKs. In some cases, performance even improved using VMDKs. It is Symantec’s current belief based upon this testing that the performance impact caused by VMDKs will be minimal at best and only observed by the most performance sensitive applications.
Now before every organization that currently runs VERITAS CFS or is considering its use in vSphere environments rushes out to deploy it, it is worth noting that not all of the features that disks and RDMs offer are yet available from VMDKs.
For example, VMDKs do not yet support SCSI-3 persistent group reservation (PGR) bits. While this may seem trivial, highly available applications that use I/O fencing rely on this bit to manage access to a disk from different VMs. In this example, organizations will need to continue to use RDMs when virtualizing these applications until such a time this feature becomes available. Aside from this cautionary note, VMware has finally thrown open the doors for organizations to use cluster file systems for virtualized highly available applications with the availability of its VMFS multi-writer flag.
Using the VMware VMFS multi-writer flag, organizations no longer have to make unpleasant trade-offs when looking to virtualize applications using cluster file systems. Rather they may virtualize their existing applications that use VERITAS CFS that reside on physical machines (or introduce VERITAS CFS on applications that are already virt
ualized) and maintain the levels of application performance needed once they are virtualized even as they introduce the even higher levels of flexibility and mobility that VMware vSphere and VERITAS Cluster File System collectively offer.