The situation confronting a VMware and Windows architect that I recently spoke with is probably one to which many system administrators can relate. On one hand, he had a requirement to make patches and updates to his company’s systems to keep them in compliance with PCI DSS regulations. On the other, making such changes could result in system downtime and disrupt his company’s operations (i.e. – stop its flow of income.) To resolve it, he turned to a new technology called the Thin Capture appliance from Kubisys.
The Payment Card Industry Data Security Standard (PCI DSS) is a compliance standard to which all companies that accept credit card payments must comply. These standards comprise of twelve (12) requirements, two of which are: developing and maintaining secure systems and applications (#6) and regularly testing them (#11).
Satisfying these two PCI DSS requirements fell to this VMware and Windows Architect to which I spoke. These two requirements called for him to apply Microsoft hot fixes shortly after their release to ensure his servers were protected against the latest security threats.
It was applying these hot fixes that put him in a quandary. A Microsoft hot fix could break his production point-of-sale (POS) software as occasionally it was not compatible with the hot fix. This could result in an outage that would negatively impact his company’s daily sales and disrupt its back end supply chain.
To avoid that scenario, he tried using VMware to create clones of his Windows virtual machines (VMs) to test these hot fixes before applying them. He knew that by using clones, he could avoid scheduling down-time to the production application, and thoroughly test the hot fixes. But he discovered that creating clones and then conducting these tests could take up to a week to complete which then would have left his company out of compliance with the PCI DSS standards and vulnerable to security threats.
Uncertain as to what to do next, he turned to one of his preferred solutions providers and described these issues. His provider suggested he check out the Kubisys Thin Capture™ appliance so he invited Kubisys onsite to explain how it worked and do a demo.
In a previous blog entry I described the Kubisys Thin Capture appliance but, in brief, the Thin Capture appliance communicates with Windows servers and initiates snapshots on Windows production server using the Windows native Volume Shadow Copy Service (VSS). The snapshot remains on the Windows servers with only needed data accessed by the Kubisys appliance.
This snapshot is used as the source by the Thin Capture Appliance so OS patches or application software upgrades can be tested against it. As the test runs and additional data is needed to complete the test, the Thin Capture appliance accesses the snapshot on the production server over the IP network and only moves the data that is needed in order to complete the test.
This was exactly what he needed as it solved problems for him on multiple levels. During his testing he was able to:
- Discover his physical and virtual machines inside of his firewall and recreate them on the Kubisys Thin Capture appliance because of its AD integration.
- Reconfigure his firewall to grant the Kubisys Thin Capture appliance secure access to his web-facing POS systems
- Test application upgrades and OS patches with near real-time copies of his production systems
- Identify and troubleshoot problems prior to applying fixes, patches or upgrades to his system
- Run snapshots of the application on the Kubisys Thin Capture appliance so he could provide screen shots of the application without touching the application to prove needed updates and patches had been applied so his company could remain in compliance.
Possibly the biggest benefit he realized was that it eliminated his dependency on VMware clones. While many may say VMware clones can provide these types of benefits, this VMware architect says that is not the case. He still had to change networking protocols to do these types of tests using VMware clones plus assign storage to each clone. Further, he was dependent on his networking and storage teams to do tasks such as DNS changes and assigning and reclaiming IP address and storage capacity.
The Kubisys Thin Capture appliance had its own internal storage and could host application servers. So once he deployed it he only had to bother his network team once to do the initial network configuration and he did not have to bother his storage team at all. He told me, “Using the Kubisys Thin Capture appliance I am now self-sufficient and do not need to bother anyone to test applications. This has reduced my PCI compliance testing setup time from days to an hour or less.“
Now some of you may wonder why I did not cite the name of the person or the company in this blog entry but hopefully the reasons are obvious. This is a real problem that many companies supporting mission-critical, revenue generating applications and who are subject to PCI DSS compliance standards deal with on a regular basis.
However it would not be prudent for anyone to say on the record that they are “out of compliance”, even if they are only out of compliance for brief periods of time until testing is done. Say this “on the record” could put them in hot water with auditors.
However companies go through this balancing act all of the time. They are often put in the undesirable position of having to balance the need to apply fixes, patches or upgrades to mission critical applications to remain in compliance with PCI DSS standards while attempting not to bring the company to its knees should they only find out after the patch is applied that it is incompatible with the application and causes it to fail.
Server virtualization has clearly helped in minimizing the risks associated with applying fixes, patches and upgrades but, as this VMware architect discovered, it still has its limitations. This is why the Kubisys Thin Capture appliance could quickly become a must-have solution in environments that are concerned about making the changes that they need to remain in compliance with security standards such as PCI DSS without putting their mission critical applications unnecessarily at risk.
To get more details about this company’s experience with the Kubisys Thin Capture appliance, you may download a DCIG Case Study on this Kubisys implementation here.