In my last blog, we talked about how cloud bursting came about and the benefits of the private cloud. So, what about the public cloud? We have found that usually, the groups who provide Research Computing Infrastructure at universities lead the way in the uptake of new technology, but because the public cloud has been so widely publicly available, everyone has been trying cloud bursting at the same time.
There are large Enterprise organisations that have moved vast swathes of their infrastructure out to the public cloud because it’s much cheaper than holding onto a data centre and its staff and means they can grow and shrink that infrastructure as demand requires it.
With the many UK universities we work with, core IT computing is leading the way with public cloud bursting, unlike research computing at universities which is on par or slightly behind on its uptake.
Does it come at a price?
With cloud bursting, there can be attractive initial rates to run the servers, but on top of that, there are all the additional costs which, unless you’re experienced at running IT or cloud infrastructures, might not necessarily be noticed on the outset.
For example, there are costs around data egress whereby most companies will say it is free to put your data into the cloud, but then there will be a cost to store the data on a monthly basis and a cost to access that data. So essentially, you are paying for the bandwidth when you are accessing your data back out of the cloud.
When conducting a scientific experiment which involves huge amounts of data, you need to continuously access the terabytes of data to run analyses, you’ll get charged every time you retrieve that data.
In some instances, even if you just try and run a search through your file structure that counts as a data egress charge as you are still accessing the data. Unless all the potential scenarios have been considered in the use of cloud bursting, these sorts of costs can sneak up on you and it can become very expensive, very quickly.
We’ve seen this recognised by both public cloud providers and the UK’s provider of digital solutions for UK education and research – JISC.
There are ongoing efforts to provide special pricing agreements for universities, waivers for certain charges, and even large amounts of credits to help show the utility of using public cloud services to enhance and expand the research capabilities for universities.
Public cloud providers are surveying the market and partnering with companies, like ourselves, for our pedigree and experience in providing cutting-edge solutions to the UK Research Computing community for many years, in order to help universities take advantage of their products by integrating them with the existing infrastructure such as HPC clusters.
This leads us to the real argument for the public cloud. You no longer have to build a system big enough for your largest workload. Black Friday is a good example; you don’t want to build a server farm big enough for Black Friday. You build a server farm big enough for your “normal Wednesday” and burst out to the public cloud to deal with the additional workload. If there is no such thing as a “normal Wednesday” for your organisation (such as a consultancy with a very peaky workload) it makes even more sense for the whole workload to be in the cloud, so that you only pay for what you consume rather than having too much or worse too little compute available to you.