A common question when cloud storage is discussed is “will my applications run slower with cloud storage?”. It’s a very valid question, and the answer to that question requires examination of many key elements of the application behavior, the network, any intermediary devices (including cloud storage gateway devices or on-ramp devices like StorSimple), and of course, the cloud storage provider.
Looking at typical application deployment scenarios, people tend to deploy storage to support those applications in the data center using one of three methods:
– Directly-attached storage (DAS), where the storage is captive to the server (either within the server chassis, or a drive tray that is dedicated to the server). This is generally a SATA, SAS, or other SCSI connection (FC-AL, etc)
– Network-attached storage (NAS), where the server accesses a shared pool of storage using file system protocols (such as CIFS or NFS) over the network
– Storage-area networks (SAN), where the server accessed volumes that appear as DAS, even though they are accessed over a network and generally provisioned from a shared storage system
Generally speaking, these storage attachment methods exhibit high levels of bandwidth, low levels of latency, and low levels of packet loss (one exception being NAS over the WAN, but we won’t get into that here).
In the case of cloud storage, your server accesses its storage in one of four (primary) ways:
– Using file protocols directly, which is identical to NAS as described above (CIFS or NFS)
– Using RESTful HTTP-based APIs, which requires that the application or operating system be prepared to handle any intermediary translation between HTTP and one of the protocols mentioned above
– Using SOAP APIs, which again may require translation
– Using a cloud storage gateway or on-ramp device
In all of these cases, accessing cloud storage occurs over a network, and that network has some measurable amount of latency, packet loss, and some limitation on bandwidth capacity. In the data center, latency, loss, and bandwidth may not be much of a concern, aside from environments using applications that are either high-throughput or transactional in nature. Interestingly enough, in many cases people find that their application servers present storage-related performance challenges of their own as well.
In any case, however, decreases in bandwidth, increases in latency, or increases in packet loss can have a severely adverse affect on performance – and this is not just true for storage.
With cloud storage, the effects of latency, packet loss, and bandwidth can be amplified, as you are effectively inserting the Internet in between your server and your storage for every IO (assuming you are using a public cloud storage service). Whether or not applications will run more slowly is a function of the BDP (bandwidth delay product) of the network, along with the number of IOPS, average block sizes, and response time requirements that your application would drive or require in an optimal environment. If the BDP of the network cannot provide sustained performance under those metrics, it is highly likely that your application will indeed run more slowly with cloud storage.
So what is a person to do? Cloud storage has obvious benefits – pay-on-use, lower cost per GB, elasticity, simpler management, and so on. How can you overcome the possibility of using cloud storage slowing down your data center application?
Performance is one of the many benefits of a cloud storage gateway or on-ramp device. Using such a system is not a guarantee that cloud storage will work for your application, as some applications may have storage I/O or access patterns that will not work well even in environments where a gateway is present. However, the majority of applications today that power corporations exhibit two characteristics that make cloud storage gateway devices like StorSimple attractive:
– They exhibit high locality of reference in the data they access
– They work with data that is typically compressible or able to be deduplicated
For example, email data contains text (highly compressible) and attachments (able to be deduplicated), and most people tend to work with email that is the most recent, meaning locality of reference is high (i.e. a ‘working set’ of data can be identified). Document management systems (like SharePoint), file servers, content archives and libraries, and many others, also exhibit these characteristics.
The reason cloud storage gateway devices are attractive for these applications is that:
– They provide internal storage capacity, which can be used as a high performance on-premises tier to support your working set of data. This means that the majority of IOs are serviced locally, mitigating the need to go to the cloud
– They automatically detect shifts in the working set (insertion of new data, older data becoming older and less frequently used), and transparently move that data between faster (integrated storage) and slower (cloud storage) tiers
– They compress and deduplicate data, meaning that data is stored efficiently, and fewer bytes are transferred into and out of the cloud. Writes and reads are performed faster, since the amount of data being moved is smaller, meaning less dependence on WAN bandwidth, and less impact in terms of the effect of loss or latency
As a side effect, what our customers have noticed is that:
– Their performance is consistent with what they had experienced with using traditional storage for the applications we are targeting
– The cloud storage capacity costs are reduced, due to deduplication and compression
– The cloud storage transfer costs are reduced, due to deduplication, compression, and the benefits of automated data tiering (we have a patented algorithm called Weighted Storage Layout, or WSL, which performs this task)
– They can leverage the myriad benefits of cloud storage without the performance worry that typically accompanies use of cloud storage
We have a number of customers that are beta testing our system in primary storage use cases for applications that are considered critical to the function of their business, and are finding that performance is consistent with what is provided by traditional storage for those particular applications.
If you’d like more information, I encourage you to drop me an email!
Cheers,
Joel
