This has been a bit of a quiet week in terms of blog entries on the DCIG website but I did not want to leave everyone hanging on the Friday before going into the Memorial Day weekend. So for this week’s recap blog I opted to reflect on a conversation that I had with Hosting.com’s Backup Operations Manager a few weeks ago. In that conversation, he provided some interesting perspectives in terms of how Hosting.com is using R1Soft in its environment.
The individual I spoke to was Cliff Pankonien, Hosting.com’s Backup Operations Manager. Pankonien and his team are responsible for protecting all of Hosting.com’s client data and use R1Soft’s Continuous Data Protection (CDP) Enterprise Edition on both its Linux and Windows servers.
To do the backups, the R1Soft CDP software takes daily snapshots of each application server and then copies the data contained in that snapshot to an R1Soft CDP Server. Each R1Soft CDP Server in his environment is either a 2U SuperMicro 6025 or 6026 with a 3WARE 9650 8-port disk drive bay. Each bay has two 80GB drives in a RAID 1 configuration for the CDP Server operating system and six 2 TB 5400K RPM drives in a RAID 10 configuration that are used for storing backup data which gives each CDP server 5.4 TBs of useable space.
One of the major reasons that Hosting.com chose R1Soft CDP Enterprise Edition was that it needed to complete backups successfully regardless of the type of environment that Hosting.com needed to protect, physical or virtual.
Hosting.com nightly achieves a 99% or greater backup success rate on both its physical and virtual machines using R1Soft. Even among those few servers that do have errors, rarely is Pankonien able to attribute any of these problems to R1Soft.
Rather he finds that errors are due to factors outside of R1Soft’s control such as problems with Microsoft’s Volume Shadow Copy (VSS) or Hosting.com clients accessing their application servers during backup periods and performing such tasks as rebooting them or changing administrative passwords.
These errors are typically detected by members of his team when they arrive to work in the morning. At that time, his team simply re-runs the backup job which fixes the problem.
Pankonien cites R1Soft’s scalability as one of its most desirable features. Hosting.com went from an initial implementation of two Control Servers and less than 30 CDP Servers to seven Control Servers with over 100 CDP Servers across its three data centers.
He says, “As Hosting.com continues to grow, we build another Control Server and start putting CDP servers under its management. Once the Control Server is managing 15 CDP Servers (which is what R1Soft recommends), we simply build another Control Server and continue scaling out. We can continue in this manner regardless if we are protecting virtual or physical application servers.”
A single CDP Server can typically protect about 70 to 125 application servers regardless if the application resides on a physical or virtual server. However Pankonien finds that the amount of internal storage that a single CDP server needs drops substantially when the application servers to be protected are virtualized.
He found that he needed to change the configuration of the CDP Servers that are used to protect VMs. When he started doing VMware backups, he built these CDP Servers like the others that had 5.4 TBs of available storage capacity.
The problem was that the sizes of the VM backups were so small that a single CDP Server theoretically had enough storage capacity to backup 190 VMs. However there were so many VMs to backup that the CDP Server could not complete all of the backups in 24 hours.
Now he configures new CDP Servers intended for VM backups to have only two 2 TB disk drives in a RAID 1 configuration with 1.9 TBs of available storage capacity. This is sufficient capacity for a CDP Server to backup about 100 VMs while enabling Hosting.com to lower the storage costs associated with building each CDP Server.
So what I found most insightful about Hosting.com’s implementation of R1Soft is how it is using a lesser known data protection solution in an enterprise environment that has thousands of both physical and virtual servers with great success. Pankonien finds that R1Soft CDP is scaling just fine and able to keep up with its backup demands in both his physical and virtual environments.
That’s it for this week. I realize this was a relatively low key blog entry as I head into the Memorial Day weekend but hopefully you found it informative.
Be sure to stop by next week because even though not many blog entries posted this week on DCIG’s website, I was busy at the keyboard getting up a bunch ready for next week.
Have a safe and happy Memorial Day Weekend!