Backups should be stored in the cloud
Matthew Hardman
Hybrid Cloud | High Performance Applications | Data Ops | Strategy | Leadership
"Wait a second..., don't you work for an on-premise storage company?"
This was the reaction I got from a customer, when we were talking about the modernisation of enterprise data protection. How did we get to this point, and why am I, a representative from an "on-premise" storage company, advocating the use of the cloud, encouraging the customer to buy less storage on premise! (Just in case you go no further in this article, to be clear Hitachi Vantara, is a data company, not a storage company.)
Tape has been a major part of every enterprise organizations data protection strategy, it has helped them to recover data and get up and running as quickly as possible in the case of system failure. It has become ingrained in the business process for compliance as a check point. It has quickly and quietly become one of the major costs in IT expenditure. And for some customers, it has become a necessary overhead and burden, especially as the reliance on IT to deliver competitive advantage has increased year over year.
The reality is, the environment that we operate in has changed so much that "Backup to Tape" process has to change, in fact in saying this, I am sure to incur ire of many a tape advocate, but tape has become irrelevant.
What is the problem we are faced with?
What we back-up and protect in the past was pretty much structured data sources, data from IT systems that ran the business, but over time, there was a requirement to back up more data, user data, log data the list goes on. This has quickly blown out the size of backup for many organizations. Why? Well an organisation does not just do one backup, store that, and then replace the old one with the new one. Companies now have to manage retention policies, how often do I keep backups, do I do daily, weekly, monthly and how long do I keep it for? Consider the example of a 1TB backup of a database, and how many copies of all the data companies are managing.
How does it get so out of control? Well we are cautious beings, we don't want anything to go wrong, and if it does we have layered recovery options, but to go from 1 TB of data to 84 TB, it starts to become a challenge. Some organizations might be simpler, but many others might have a view on something more complicated.
Tape has a big problem, it cannot be deduplicated, and what I mean by that, is that it means that when we backup the data, it backups up every single piece of data one for one. As an example, let's imagine backing up a user file system, and they had five folders, and in each of those folders, the exact same file was stored. If backup software were storing the backup image created to disk, it would be able to recognise the fact that the file was exactly the same five times over, and would make a smart to decision to backup one copy of the data, and essentially add pointers to the file indicating other locations it should be recovered to. This results in to a 5:1 reduction. Tape though, tape cannot do this, tape will be required to keep a copy of every single file in every single location, making it no more protected than if it were stored to disk, but much more inefficient.
Getting off tape has now become a priority, everyone talks about Digital Transformation, but we tend to have a focus on some fo the more visible projects, the cooler things. Backup, unfortunately isn't cool, isn't sexy, you aren't going to get people looking to it for business innovation, but it is absolutely, undeniably a critical part of the operations of the business. Backup must get some Digital Transformation love of its own.
So... dump the tape and jet off to the cloud?
Yes and yes... in a manner optimised for the business. As I stated at the beginning, backup images should be stored in the cloud, but just to be an annoying word smith, I didn't say "all" backups should be stored in the cloud, but cloud does offer customers an element of cost operations versus running things on premise, but also adds complexity to the process.
In our diagram above, we already started to store our backup images to disk before casting them to tape, so why not interchange that step from casting to tape, to casting to cloud? When I say cloud, I refer to a hybrid cloud approach. The solution should enable you to store your most recent backup images close to your organisation in a private cloud infrastructure, but when those backup images meet a certain time period, they should be able to be tiered to the public cloud of your choice for longer term retention, enabling you to optimise your storage in your private cloud, and getting a better economy of scale.
You might be asking, why not go straight to the cloud, but those who use the cloud are aware of some of the challenges of costs in communicating the data from the organisation to the cloud, especially in recovering the data back in to the organisation, that's why having more recent backups available locally is more important. In addition to that though, having a cloud storage platform onsite, can give you even more enhanced capabilities to protect your data backups, things such as
- helping you to streamline and manage which cloud you tier your backups to with ease
- encryption of backup images before transmission across a public network
- integrity management of the data with health checks and recovery options in case the backup image gets corrupted, and more.
These are the essential capabilities that you need to protect... your data protection strategies.
So when a customer says to me, "We are not interested in on-premise storage, we are going to the cloud."
My response is, "Neither am I, let me help you get to cloud faster!"
Retired Product Marketing Professional
5 年Nice post, Matthew. If I may, I'd like to pile on with another very important reason to move away from tape for backup data. Tape backups are a huge liability in the context of data privacy regulations such as GDPR. My blog explains:?https://community.hitachivantara.com/people/rvining/blog/2018/07/26/gdpr-what-about-backups?
Great Post Matthew always an insightful read.
Data Protection Systems Engineer
5 年Why wouldn't you just use HDPS from Hitachi which offers all this natively in SW and can talk to Application backup API's as well.
Freelance IT Architect / Product Owner
5 年Nice story Matthew! One aspect of writing the backup data into the private cloud infrastructure is that it should not be done in 'container files'. This way of writing files is often used by different backup software vendors, and disturbes the deduplication process.
VP APJ at KnowBe4 | ex-Microsoft | ex-Oracle | Sales Leadership | Founder | Advisor
5 年You managed that within a quiet moment with the kids at the gym? Impressive!! You need more quiet moments!