Cloud 101
Manufacturing Data Retention: What to Keep, How Long, and Where to Store It
In the manufacturing business, logs and records aren’t just administrative overhead. The data that the business retains may also serve as evidence in an investigation, an audit trail, or protection against potential liabilities.
However, while this data can have significant value, its sheer scale can make storage and retention complex and expensive. Data from Industrial Internet of Things (IIoT) devices, manufacturing execution systems (MES), and quality systems, along with large computer-aided design (CAD) files, can strain retention infrastructure.
While companies must retain certain records, the specifics of regulatory requirements vary based on industry, specific regulations, and geography, making a “one-size-fits-all” approach infeasible. However, this complexity can be managed by implementing a practical framework that considers best practices and common requirements for what to keep, how long to keep it, and the storage architecture that makes it manageable.
Why data retention is a manufacturing problem, not just an IT one
Failures in data retention can have numerous potential implications for the business, including regulatory penalties, failed audits, and product liability exposure. Manufacturing data, including production, quality, and traceability records, are legal assets, not just operational data.
The growing volumes of manufacturing data may make data retention seem like an IT problem because IT is responsible for implementing scalable and secure storage for these assets. However, the critical, cross-cutting importance of these records mean that the success of data retention efforts is a problem for the business as a whole, not just the IT department.
What manufacturing data actually needs to be retained
Manufacturing records may have various uses within the business, ranging from supporting operational improvement efforts to addressing potential liability and legal challenges. A manufacturing data retention strategy should include the following types of records:
Quality and inspection records
Production and process logs from manufacturing execution systems (MES)
Design files and engineering documentation
Audit trails and traceability records for recalls
Supplier and supply chain records
Surveillance footage from plant-floor cameras
How long do you actually need to keep data?
Retention windows for manufacturing data can vary significantly based on numerous factors, including industry, regulation, and geography. For example:
International Organization for Standardization (ISO) 9001 doesn’t mandate a specific retention period but requires organizations to define an appropriate retention period based on their customer, legal, and regulatory requirements.
U.S. Food and Drug Administration (FDA) has no single record-retention period either. Requirements vary by product, record type, and applicable regulation, so organizations should determine retention periods based on the rules that apply to their operations.
International Automotive Task Force (IATF) 16949 standard requires organizations to define and document a record-retention policy. Typically, the retention rule for different types of records is the lifetime of the product plus one year.
Depending on the regulation or standard, organizations may be required to retain records for 10 years or more, which is especially true if retention requirements are based on the lifecycle of the product.
In other cases, such as ISO 9001, manufacturers have a degree of latitude in determining an “appropriate” retention period. This means the company needs to balance the cost of keeping data for too long vs. the potential risks of deleting it too soon.
Why legacy and hyperscaler storage struggle with retention at scale
Legacy storage solutions often rely on on-prem storage infrastructure. With the length of manufacturing data retention windows, these solutions often age out before retention windows close, introducing additional complexity into data retention programs.
Storing records with hyperscalers introduces tradeoffs between cost and availability. Hot storage may be accessible but less cost effective, while cold storage can introduce latency problems when records are needed quickly (such as audits and so forth). Additionally, hyperscalers commonly impose egress fees, which can make long-term archives expensive to access if data is needed for audits, litigation, or other purposes.
What good retention storage looks like
Good retention storage is necessary for manufacturers who may need to maintain records for decades or longer. Some of the key features of a good retention storage system include:
Always-accessible object storage: While records may be archival, manufacturers might need to quickly access data in response to an audit or other event. Good retention storage allows access to all data without retrieval delays or tiering penalties.
Immutable storage: Manufacturing records are designed to help prove compliance or protect the company against potential legal risk, so data integrity is vital. Immutable storage is necessary to prevent tampering or deletion of stored data.
Flat-rate pricing: Data retention requirements mandate that manufacturers store records for years or decades. Flat-rate pricing is important so that companies can predict and budget for the costs associated with record retention.
S3-compatible integration: Integration with existing MES, enterprise resource planning (ERP), and compliance tools is necessary to facilitate data storage and access. Storage providers should offer S3-compatible integrations for easy data management.
Building a retention strategy that scales
Manufacturers need to balance cost, accessibility, and regulatory requirements when designing a record retention program. Some best practices include:
Map data to requirements: Some regulations, such as IATF 16949, implement different retention windows for various types of records. Mapping data types to retention requirements before choosing infrastructure helps to ensure that solutions meet business needs.
Separate active and archive data: Active production data and archival data serve different purposes and have different requirements. Separating the two helps to protect archived records from accidental modification and deletion.
Automate lifecycle policies: Manufacturing systems produce large volumes of data that organizations need to retain for varying periods. Automating lifecycle policies helps to reduce manual overhead, avoid mistakes, and minimize waste from keeping data past its deadline.
Take a risk-focused approach: Some manufacturing records are more important than others, such as the data required for regulatory compliance. Starting with this data and expanding from there helps to maximize the ROI of retention efforts.
Cloud storage for manufacturers doesn't need to be complex
Manufacturers face increasingly complex data retention requirements as data volumes continue to grow, forcing them to manage more information for longer periods.
Manufacturers need long-term storage that does not impose penalties for retaining or accessing data. Because audits or lawsuits may require immediate access to records that are decades old, retrieval delays and fees can harm the business.
Wasabi Hot Cloud Storage gives manufacturers a scalable, always-accessible archive with no egress fees, immutable storage options for tamper-proof recordkeeping, and flat-rate pricing that makes decades-long retention financially predictable. It also offers S3-compatibility to ease integration with existing backup, MES, and compliance tools, so no rip-and-replace is required.
solution
Cloud storage for manufacturing innovation
Resilient operations meet data-driven innovation with scalable, secure cloud storage to modernize manufacturing operations across plants, systems, and supply chains.
Frequently asked questions
Data retention periods vary based on industry, location, regulation, and data type. In some cases, the retention period is based on the lifetime of the product.
Backup storage is designed to protect against potential data loss or corruption, and backups are created frequently and retained for a short period. Archive storage is designed to meet regulatory obligations, which often require longer retention windows and have additional requirements, such as immutability or searchability.
During an audit, a failure to produce requested records could jeopardize the company’s compliance or trigger administrative action or penalties. In the case of a recall, a lack of records could force the company to choose between expanding the recall or accepting liability exposure.
Wasabi meets or exceeds all major industry compliance standards and government regulations including HIPAA, CJIS, FERPA, GDPR, SEC, and ISO 27001. Additionally, Wasabi Hot Cloud Storage is designed to provide 11 9s of object durability and supports immutable storage through Object Lock. These capabilities can help organizations protect retained data as part of a broader compliance strategy.
Immutable storage keeps data read-only for a defined period, preventing modification or deletion. This supports compliance by preserving a trustworthy audit trail and chain of custody for audits and litigation.
Egress fees make long-term retention costs unpredictable. Because organizations cannot anticipate what data audits, litigation, or other events will require—or how often data will be accessed—they cannot reliably forecast access-related charges.