Data storage is serious business. Customers entrust their data to their storage vendors with the understanding that it will be there when they want it. No excuses.\r\n\r\nI was a founder and CEO of Carbonite, a well-known backup company. Carbonite backs up about a half a billion computer files every day. So we know a few things about data loss in the cloud and what it can mean to customers. Here's what I can tell you.\r\nCloud storage is more reliable than physical storage\r\nPeople are inherently more comfortable with the notion of physical storage than they are with storing data in the cloud. That's understandable. We live in a physical world where losing something like your car keys is tangible and easy to grasp.\u00a0 Perhaps that's why some IT professionals (erroneously) think that having data stored in their own data centers is somehow inherently safer than storing it in a public cloud. When dealing with something as ethereal as millions of computer files, most people don\u2019t have a good gut feel for how reliable data storage needs to be in order to avoid costly and embarrassing losses.\r\n\r\nEven physical storage of paper documents or tapes is not foolproof.\r\nFrom Computer Weekly...\r\n\u201cData storage company Iron Mountain has admitted losing backup tapes containing the data of thousands of employees at one of its customers. This is the third major data breach to affect Iron Mountain.\u201d\r\n\r\nFrom The New York Times...\r\n\u201cTime Warner said the data, on 40 tapes in a container the size of a cooler, disappeared more than a month ago while being shipped to an offsite storage center. Iron Mountain issued a statement saying, \u2018Iron Mountain performs upwards of five million pickups and deliveries of backup tapes each year, with greater than 99.999% reliability.\u2019\u201d\r\nWhat does 5 nines of reliability mean?\r\nLet's do the math. If 99% reliability means that you will lose one object out of 100 every year, then 99.999% (5 nines) reliability means that you will lose one object out of 100,000 objects every year. Iron Mountain makes 5 million pickups and deliveries a year, so by their numbers you can expect them to lose 50 objects per year. That\u2019s probably consistent with the losses we read about in the newspapers.\r\n\r\nBy contrast, top-tier cloud storage vendors, including Amazon S3, Microsoft Azure, and my company, Wasabi, offer 11 nines of reliability\u00a0(or durability as they say in the industry). That makes such cloud storage 1 million times more reliable than Iron Mountain\u2019s physical storage. In other words, if you gave Amazon or Wasabi 1 million files to store, statistically they would lose one file every 659,000 years. You are about 411 times more likely to get hit by a meteor.\r\n\r\nLet\u2019s try to look at this in a more tangible way. At Wasabi, we store billions of \u201cobjects,\u201d or files that customers have sent us. On average, files are about 800 MB in size. So if your organization is storing 1 PB of data, it\u2019s likely that you have something like 1.2 billion objects.\r\n\r\nIf your storage were 99% reliable, that would mean that you would lose one out of every 100 objects every year. The least durable commercial cloud storage is Amazon S3 Reduced Redundancy Storage (RRS) which is spec\u2019d at 99.99%. Using RRS, you could expect to lose .01% of your files every year, or .0001 x 1.2B = 12 million lost files per year. Here\u2019s a table with some representative products and the expected data loss per year:\r\n\r\n\r\nActive Integrity Checking means extra protection\r\nWith either S3 RRS or lower reliability services like Backblaze B2, the problem is that you won't know you\u2019ve lost files until you try to use them. It\u2019s not like when you lose all your files and can restore them from a backup. Let\u2019s say you store your data for five years in Backblaze B2. After five years you would expect to accumulate 600 lost files (5 x 120). Backups from five years ago are probably gone, leaving you with permanent data loss. That\u2019s why many IT managers resort to annual (or more frequent) testing of all their data to create and test checksums on what they actually have in storage. If a mismatch is found, hopefully, there is another copy somewhere that they can access to restore a corrupted or missing file.\r\n\r\nWasabi does a checksum compare every 90 days--what we call Active Integrity Checking. Since there are effectively five copies of every piece of data to achieve 11 nines, any one copy that becomes corrupted or lost can be quickly and reliably restored. \u00a0With 11 nines of durability, the likelihood is that you will never experience data loss in your lifetime. \u00a0So why replicate data to a second data center?\r\nIt's all about availability\r\nReplicating your data in a second data center at a different location gets you two things: insurance against a local disaster (flood, fire, earthquake) that could physically destroy one of the data centers, and increased availability. Data centers can and do go offline from time to time due to power or local Internet failures. If a data center guarantees 99.9% uptime, that means that it will be offline .1% of the time, or about 9 hours per year. \u00a0Geographic replication would give you 99.9999% uptime, or 1\/1000th the amount of downtime. \u00a0This level of availability may or may not be worth the extra money; it really depends on your application and what any amount of downtime means to your business.\r\nNo amount of nines can prevent data loss\r\nThere is one very important and inconvenient truth about reliability: Two-thirds of all data loss has nothing to do with hardware failure.\r\n\r\nThe real culprits are a combination of human error, viruses, bugs in application software, and malicious employees or intruders. Almost everyone has accidentally erased or overwritten a file. Even if your cloud storage had one million nines of durability, it can\u2019t protect you from human error.\r\n\r\nFor this reason, Wasabi introduced the notion of the \u201cimmutable bucket\u201d\u2014storage that cannot be erased or modified by anyone--not even the admin or anyone at Wasabi. Once you write it, it\u2019s there until the hold time that you designated expires. If someone tries to erase or modify an immutable file, you just get an error message. I\u2019ve written a whole blog post on immutability if you'd like to learn more.