As I write this blog entry, I am currently on a flight to New York City to attend the last day of the fall 2008 Storage Decisions conference. While I intend to post a blog entry about my experiences at SD this Friday, the flight is giving me some time to go back to last week and share some additional thoughts and insights I gained while attending the InfiniBand Trade Association (IBTA) Tech Forum in Las Vegas on Monday, Sept 15. While infiniband was obviously covered as part of this forum, it was done so in the larger context of what virtualizing the corporate infrastructure means and how that will contribute to how companies construct and manage their data centers in the future.
First, I want to highlight some tidbits in the keynote that Gartner’s John Enck gave at the opening of the IBTA Tech Forum. Initially he challenged the assumption that VMware is a secure platform that is immune to attacks and viruses. He mentioned that security is still a point of immaturity in VMware and that VMware issued a fix for known vulnerabilities in its product just a week prior to VMworld. While I believe that VMware is more secure than Windows was in its early days, more vulnerabilities are likely to be discovered as it is more widely deployed.
Another interesting tidbit that Enck shared had to do with some of the new management functions that server virtualization makes possible. Enck specifically mentioned that because the images of each individual VMware virtual machine (VM) is stored in what is called a VMDK (Virtual Machine Disk Format) file, it opens the door for companies to assign metadata to these files including service oriented metadata. In so doing, companies can begin to manage and assign service priorities to VMDK files in the same way they manage other files. It is not yet clear how this will evolve over time but it makes sense that priority levels will become part of the VMDK file’s metadata. In this way, VMs will be automatically assigned to the appropriate tier of server and storage hardware simply by looking at the setting at the metadata.
Personally I found Enck’s presentation one of the best I have heard on the current challenges of server virtualization as well as its future promise. His comments were on-target based upon what I know while providing an honest and realistic assessment of the infiniband market as it relates to virtualization in general.
Another presentation that I also wanted to comment on came from Citrix’s CTO, Simon Crosby, who is responsible for XenServer’s oversight and direction. Those of you who think that VMware has already won the server virtualization battle may be less convinced after hearing Crosby talk. While I am not going to dispute that VMware has market momentum and is gaining corporate mindshare, Crosby said that XenServer has the world’s largest high performance computing (HPC) cluster deployed at the Department of Defense (10,000 nodes deployed worldwide) and Google also has a very large Xen implementation going on.
One of the more interesting thoughts that Crosby shared during his presentation had to do with what occurs when the operating system becomes aware it is virtualized. Operating systems are now oblivious to the fact that the relationship between it and the underlying hardware no longer exists as before. Using operating systems that are “virtualization aware” creates a paravirtualization environment which opens the door for a profound change in how companies manage their infrastructure. However Crosby did not get into the details as to what “profound” changes he saw occurring.
Another point that Crosby shared had to do with hypervisor performance and how the different virtualization hypervisors scale as the number of cores in the physical machine increase. In his opinion, performance on all hypervisors is about equal where there are only 2 cores in the physical machine; when 4 cores are present you start to see performance on one hypervisor start to fall and when 8 cores are present, he said that Xen starts to separate itself from the crowd. He attributes this to two things. First, Xen is fully 64-bit aligned and said that Xen helps to drive the 64-bit roadmap. Second, its core code base is under 100,000 lines which make for faster processing such that the performance overhead on the physical machine from supporting Xen hypervisor can be under 1%.
Crosby concluded his presentation with a in-depth discussion of how important I/O is going to become in the coming years. While I want to dedicate a separate blog entry to this after I have done some more research on this topic, he stressed that it is important that companies begin to put the right network interface cards (NICs) in their virtualized servers. There are now essentially 4 classifications or levels of NIC cards (0, 1, 2 & 3) and it is NIC cards that support levels 2 & 3 that companies should give priority. He said the level 2 is where the big jump was made as the hypervisor can now begin to directly access buffers in the NIC card and reserve specific buffers for specific VMs. In so doing, this can provide a tremendous boost in performance on specific VMs.
Some final thoughts I wanted to share come from Jacob Hall, the Chief Architect for Wachovia Corporate and Investment Banking. First, he is creating an infrastructure where the I/O is decoupled from the chassis as he sees embedding I/O in the chassis as decreasing the life time of the chassis. By decoupling it, he can scale and upgrade processors and I/O independently of one another and take advantage of advances in each without impacting the other. Second, he thinks the entire industry needs to accelerate its adoption of advances in I/O technology. Advances in other areas of technology happen in over 1 – 4 years while advancements in I/O take 7-8 years to find their way into the datacenter. In his opinion, this is too long. By increasing the adoption rate of new I/O interconnects, it not only increases performance but reduces energy costs.