Archiving for LTO
Long-Term and Active Archiving
As data volumes grow exponentially, enterprises are having to find more efficient and cost-effective ways to store and maintain all of that information. And thanks to ever-evolving data regulations, it’s becoming more critical to retain that data for regulatory compliance, business rules and intellectual property protection.
While traditional long-term archiving solutions have typically involved on- or off-premises robotic Linear Tape-Open (LTO) libraries, advancing digital technology makes it clear that managing these growing volumes of data will require some a bridge to the cloud. But so far, the big cloud providers are still too expensive when compared to tape.
Tape isn’t going away
The reality is that annual sales of LTO tape continue to grow. Tape isn’t going away anytime soon. Case in point: The National Library of Medicine in D.C. recently sought a solution for a 35 PB LTO implementation, including 10 years’ worth of archived, multi-generational LTO tape. The primary objective of the library is to share research around the world.
Even the most loyal of tape users, however, acknowledge its downsides. Tape can be slow, particularly if it needs to be retrieved from off-site — and it can get lost, erased or damaged. For example: Requests to the National Library of Medicine can take days.
While tape is a great long-term medium with 15 to 30 years of usable shelf life, the drives themselves and constantly evolving standards render the ability to read older LTO formats nearly impossible unless you keep an inventory of old tape drives to access the old formats
LTO tapes provide between 15 and 30 years of storage, which may be less depending on usage and storage. That means the first couple generations (released in 2000 and 2003) are now entering the final phases of viability.
Drives supporting newer releases of the format are typically only compatible within two generations. However experience with the recently released LTO-8 suggest that it is only backwardly compatible to one generation.
The overall trend is towards greater risk when collections are not migrated. Extinction events for older formats, even perhaps as recently as LTO6, should be anticipated within two to five years.
The biggest users of LTO systems for archiving are moving to the cloud to keep second or third copies of data in addition to their tape archive strategy.
With Wasabi hot cloud storage, storing data in the cloud is truly cost-effective and can become a critical piece of your long-term archiving strategy. Having an additional location to store long-term archives keeps your ability to safely archive firmly in your grasp, by not putting all your archive eggs in one basket.
Long-Term Archiving Is More Critical Than Ever
Data regulations are evolving quickly—which is only making the need for effective, secure archiving more important for enterprises.
|Industry||Typical Data Retention Requirements|
|Pharmaceutical||Up to 100 years for R&D records and applications|
|Energy||Oil field lifetime, 50+ years for subsurface exploration data|
|Finance||Lifetime of the life insurance policy, plus up to 10 additional years|
|Aerospace||30+ years for design records (lifetime of aircraft)|
|Reference: Index Engines, 2016|
What’s Cloud Storage 2.0?
After 50 years of on-premises investment in data storage, businesses outgrew their ability to economically store all of their data on their own. That situation gave rise to Cloud Storage 1.0 solutions.
Cloud Storage 1.0 represents the first wave of what are now legacy enterprise cloud vendors. Their solutions began the migration from on-premises to cloud implementations more than a decade ago, beginning with storage, compute, and network capabilities. While revolutionary at the time, Cloud 1.0 capabilities have not kept pace with the needs of the market.
The data volumes kept in Cloud Storage 1.0 solutions have grown tremendously (nearly doubling year over year) and have proven that cloud storage can expand more readily than on-premises solutions. However, the price and performance of Cloud Storage 1.0 solutions have not adapted in step with the changing environment and dynamic needs of innovative new technologies such as artificial intelligence, the Internet of Things (IoT), and full-motion video.
Simply put, Cloud Storage 1.0 is both too expensive and too slow for the growth needs of most enterprises and their available budgets. And in many ways, it’s no less expensive than owning and operating your own datacenter, which may explain why many certain businesses have not embraced the Cloud First strategy: the risk and cost of migrating entirely to the cloud using Cloud 1.0 technologies vastly outweigh the benefits.
Cloud Storage 2.0 solutions radically reduce pricing while simultaneously taking advantage of technology innovations that did not exist a decade ago. The result: much higher performance and more sophisticated capabilities.
As you consider the potential value of storing more data affordably and storing it online where it’s immediately accessible, imagine what’s possible with an effectively unlimited, high-performance–yet much less expensive– Cloud Storage 2.0 solution like Wasabi Hot Cloud Storage.
Archiving Is Not the Same as Backup But Does That Matter?
People often speak about backup and archiving as if they are the same thing. But they actually serve quite different purposes. Backups are copies of data you can use to restore systems if the original data is lost, corrupted or suffer some form of disaster. They are updated or overwritten frequently to reflect the latest versions of data. Archiving is a process of storing unchanging data—that is no longer actively used—for long-term retention. Archives aren’t overwritten and the data doesn’t change.
Why is it important to make that distinction? Because your strategies for cloud storage might differ depending on whether the data is for backup or archive purposes. Legacy cloud vendors like AWS, Microsoft Azure and Google Cloud Platform offer various tiered storage services, with each storage tier intended for a distinct purpose—primary storage, backup storage or long-term retention. Each has unique performance characteristics and pricing schedules (typically, faster, more expensive storage for primary storage or backup and slower, less expensive storage for archival use cases).
With Wasabi hot cloud storage, however, the distinction between storing data for archiving and storing data for backup might not be so important.
We believe in simple storage, period—fast enough to use for backup and recovery, and cheap enough to use for archiving. By using a single, fast and economical storage solution to maintain both backup and archival data, you can simplify planning and operations, avoid surprises and save a lot of money.
Migration Strategies for Archiving
If you’re using LTO successfully for your long-term archiving needs, taking advantage of cloud capabilities doesn’t have to mean scrapping your existing system. It might make more sense at first to keep the existing LTO infrastructure and simply extend it to the cloud for use going forward or for specific use cases.
Cloud Storage 2.0 has become so fast and inexpensive that it pays to test out the cloud as a platform for future archives or to supplement your existing LTO installation. You could keep a secondary copy of your archival data in the cloud, which would allow you to store that data away from your primary off-site location. Having a copy stored in a cloud region across the country, for instance, would ensure that the archive can’t be destroyed by a local event. It’s essentially a backup of your archive data.
A cloud copy of your archival data would also allow you to distribute the data across the world as needed without having to retrieve and ship a physical tape.
Migration strategies include:
On-premises tape to cloud
Replacing LTO for any or all archive use cases with cloud storage as the destination
Hybrid on-premises AND cloud
With cloud storage as a second or third copy of your data
Using a Virtual Tape Library
Simulating a tape library (the drives and physical tapes) digitally in order to make a cloud storage destination appear to be a physical tape library
Using a storage gateway
Provides an interface between cloud storage and enterprise applications, usually a RESTful API over HTTP, rather than conventional storage area network (SAN) or network-attached storage (NAS) protocols.
Using an offline appliance, like the Wasabi Ball Transfer Appliance
Save time, money, and effort by moving large, existing archival datasets to Wasabi with plug-and-play simplicity
We want to get out of the storage business and focus on our revolutionary AI based video and audio detection processes and capabilities that directly serve our major clients. We’re moving over 6 petabytes of information through the Wasabi Ball as we migrate to the cloud and our hardware reaches end of life.”
— Lakshmi Venkataswamy, co-founder and CTO, IQ Media.
When the archiving target is Wasabi, clients often keep a second (off-site) copy and for extra protection, a third copy, which is an “immutable” copy that cannot be encrypted, deleted or modified by ransomware or other malicious or accidental behavior. Immutable storage is essentially the same feature as the optical WORM (write once read many) discs that were the standard media for archive purposes.
One significant advantage of a cloud-based immutable storage location is that the cloud is not limited to the amount of data that can be stored on a physical media such as an optical WORM disc, or for that matter, an LTO WORM tape (the same form-factor of a normal LTO tape, but that can only be written to once), and can be made more intelligent.
Active Archiving: Reawaken Archives for New Possibilities
Today’s cloud technology has advanced to a point where cloud providers like Wasabi are able to offer active archiving—super low-cost data storage that still allows you fast, efficient access to your data from anywhere, anytime.
With active archiving, long-term storage no longer means cold storage. It’s now possible to store archives extremely inexpensively while keeping that data accessible to your enterprise. Think of what it would mean if you could free up your budget AND make archive data available anytime. You could use years or even decades of archived data in analytics applications to deliver insights today. You could bring old data to life for new applications, or even find new ways to monetize that archived data.
By balancing speed of access with the cost of storage, you get the best of both worlds.
According to predictions by IDC, for the growth and classification of data by 2025, the “Digital Universe” will total 163 zettabytes, and if stored on traditional storage tiers, would be used for these reasons and percentages:
|Tier||Requirements||Destination||% of Data per Tier in 2025 with Traditional On-Prem Tiers|
|Tier 0||For High Performance||SSD||5%|
|Tier 1||Mission Critical and Online||HDD||15%|
|Tier 2||Mission Critical and Online||HDD||20%|
|Tier 3||Archive – Long-term and Active||Tape, Active Archive||60%|
|Cloud Tier||One Modern Tier for All Needs||Cloud Storage 2.0 (Wasabi)||n/a|
IDC’s research states that cloud storage can support all tiers as they are traditionally defined, and Wasabi’s design is specifically built with the goal of a single, predictable, low cost, high performance tier, fit for all purposes.
In order to take economic advantage of a traditional tiered storage approach, your data has to be categorized, either in advance, or as an ongoing process. That categorization then allows rules and processes to be put in place to move data from more expensive, higher performance tiers to less expensive, slower performance tiers. Manual categorization is possible, but hard to scale, leaving most data uncategorized.
Unfortunately, IDC estimates that by 2025, only 15% of data will be categorized, with the tendency of uncategorized data to be automatically moved to archived status, in order to keep pace with the raw growth of data.
If that data is archived in an offline, traditional “cold tier,” the odds of going back to categorize and analyze that data drops tremendously.
By choosing an Active Archive approach, particularly using LTO + Cloud Storage 2.0, or pure Cloud Storage 2.0, data that would otherwise go into archive status and effectively never be seen or used again, now becomes readily accessible to pull back into a productive state.
Benefits of Active Archiving
Ease of use
Provides end users with file-level access to all their data, all the time, without IT intervention.
Less media conversion
By archiving to the cloud, you can eliminate the time, effort and raw costs of migrating between physical media (whether the drive or media like tape) over time.
An active archive using Wasabi has essentially infinite scaling capabilities, enabled by the advantages of cloud object storage.
Improved primary storage performance
Because primary storage infrastructures contain a large amount of infrequently accessed or archive data, you can substantially improve system performance by partitioning archive data from active data and reducing the total amount of data to manage on primary storage.
Improved archival storage performance
Archive access time is significantly improved using Wasabi, because the single performance tier is always high performance, without the performance hits of multi-tier, slow access “cold” tiers of other public cloud providers.
Moving all data to a lower-cost, higher-performance tier like Wasabi’s hot cloud storage can provide a significant TCO reduction.
Your own energy costs for storage itself are eliminated by taking advantage of cloud offerings.
Reduced backup window
Even with backup using data compression and data deduplication, backup windows face constant pressure from data growth rates that typically exceed 40% annually. Archiving streamlines the backup process by removing terabytes of data from the backup set on primary storage while greatly reducing the duration of the backup window. Repeatedly backing up unchanged archival data is eliminated, removing redundant operations that many backup and archive strategies accidentally employ.
We were working on a new project for Mercedes-Benz when the client suddenly asked for a scene from an earlier video project. It was no problem. Wasabi is so fast, it’s like an active archive.”
— Stephen Skeel, 7 Wonders Cinema.
Benefits of Archiving with Wasabi Cloud
What if you could store ALL of your data in the cloud affordably?
NOW YOU CAN. Wasabi is here to guide you through your migration to the enterprise cloud and to work with you to determine which cloud storage strategy is right for your organization.