What Does 11 Nines of Durability Really Mean?

From Portland Press Weekly…

Or tell the residents of King's Landing from the Game of Thrones that… OK, if you haven’t seen the final episode, I’m not going to ruin that for you. Needless to say, life is risky business.

Fun with Statistics

What does 5 nines of reliability mean? That’s been a very standard “reliability” statistic that’s been thrown around years. But what does it mean?

Let’s do the math. If 99% reliability means that you will lose one object out of 100 every year, then 99.999% (5 nines) reliability means that you will lose one object out of 100,000 objects every year.

In the physical world, Iron Mountain makes 5 million pickups and deliveries a year, so by their numbers you can expect them to lose 50 objects per year. That’s probably consistent with the losses we read about in the newspapers.

By contrast, top-tier cloud storage vendors, including AWS S3, Microsoft Azure, and my company, Wasabi, offer 11 nines of reliability (or durability as we say in the industry) – 11 nines = 99.999999999% reliability.

Doesn’t seem like adding more 9s past the decimal would really matter, would it?

What Does 11 9s of Durability Mean, Really?

99.999999999% durability of objects over any given year makes cloud storage 1 million times more reliable than Iron Mountain’s physical storage, and without the hazards of getting caught in Boston’s rush hour traffic.

Breaking it Down the Wasabi Way

At Wasabi, we store billions of “objects,” or files that customers have sent us. On average, files are about 800 MB in size. So if your organization is storing 1 PB of data, it’s likely that you have something like 1.2 billion objects.

If your storage were 99% reliable, that would mean that you would lose one out of every 100 objects every year. The least durable commercial cloud storage is Amazon S3 Reduced Redundancy Storage (RRS) which is spec’d at 99.99%. Using RRS, you could expect to lose .01% of your files every year, or .0001 x 1.2B = 12 million lost files per year.

BTW – Did I mention that S3 RRS is 4 times more expensive than Wasabi? Granted that RRS is less expensive than S3, but even S3 is 5 times more expensive than Wasabi. (Read why more tiers aren’t better, and one hot tier is all you need)

Here’s a table with some representative products and the expected data loss per year:

Active Integrity Checking Means Extra Protection

With either S3 RRS or similar lower reliability services, the problem is that you won’t know you’ve lost files until you try to use them. It’s not like when you lose all your files and can restore them from a backup.

For example, say you store your data for five years in S3 RRS. After five years you would expect to accumulate 600 lost files (5 x 120). Backups from five years ago are probably gone, leaving you with permanent data loss. That’s why many IT managers resort to annual (or more frequent) testing of all their data to create and test checksums on what they actually have in storage. If a mismatch is found, hopefully, there is another copy somewhere that they can access to restore a corrupted or missing file.

Wasabi does a checksum comparison every 90 days – we call it Active Integrity Checking.

Since there are effectively five copies of every piece of data to achieve 11 nines, any one copy that becomes corrupted or lost can be quickly and reliably restored. With 11 nines of durability, the likelihood is that you will never experience data loss in your lifetime.

That leads us to a related topic – availability. If Wasabi has 99.999999999% object durability, should you replicate data to a second data center?

It’s All About Availability

Replicating your data in a second data center at a different location gets you two things:

1. Insurance against a local disaster (flood, fire, earthquake) that could physically destroy one of the data centers or take it offline, and

2. Increased availability.

Data centers can and do go offline from time to time due to power or local Internet failures. If a data center guarantees 99.9% uptime, that means that it will be offline .1% of the time, or about 9 hours per year. Geographic replication would give you 99.9999% uptime, or 1/1000th the amount of downtime. This level of availability may or may not be worth the extra money; it really depends on your application and what any amount of downtime means to your business.

The Inconvenient Truth of Reliability

There is one very important and inconvenient truth about reliability:

Two-thirds of all data loss has nothing to do with hardware failure, whether it’s YOUR hardware, or ours.

The real culprits of data protection are a combination of:

• Human error,

• Viruses,

• Bugs in application software, and

• Malicious employees or intruders.

Almost everyone has accidentally erased or overwritten a file. Even if your cloud storage had one million nines of durability, it can’t protect you from human error.

Enter the Immutable

For this reason, Wasabi was the first public cloud storage provider to offer the “immutable bucket” — storage that cannot be erased or modified by anyone – not even the administrator for your account or anyone at Wasabi. Once you write it, it’s there until the hold time that you designate expires. If someone tries to erase or modify an immutable file, you just get an error message.

I’ve written a whole blog post on immutability if you’d like to learn more.

Summary

• Reliability/Durability = stored data does not suffer from bit rot, degradation or other corruption.

• Availability/Uptime = the storage system is operational and can deliver your data when you request it.

Wasabi takes your data storage very seriously – we are designed, built and operate with one goal in mind, to be your trusted cloud storage utility – both highly durable and available, as well as inexpensive and high performance. To us, this is the foundation that all next generation cloud providers, like Wasabi, should be based on.

PS – Not Technical Enough?

If you are interested in even more technical details and mathematical models on how Wasabi is built for high levels of object durability, read our Tech Brief ebook “Wasabi Extremely High Durability Protects Mission-Critical Data.”