INDUSTRY
Object Size and the Hidden Cost of Multipart Uploads
This article, part of our ongoing series about the total cost of ownership (TCO) of cloud storage, explores the connection between object size and hidden fees for multipart uploads.
When evaluating cloud storage costs, most people focus on the total volume of data they need to store. But one critical factor that often goes overlooked is object size—and it can have a massive impact on your total cost of ownership.
In cloud storage, not all petabytes are created equal. One file is never just one file, it’s made up of many smaller pieces (objects) that the system views as a whole. Understanding object size is fundamental to how your cloud storage interacts with your data, and how your storage provider charges you for that data.
The hidden cost of minimum object sizes
Many cloud providers impose a minimum billable object size, meaning even if your stored file is smaller than that minimum, you’ll be charged as if it were larger than it actually is. For example, certain AWS S3 and Azure blob storage tiers have a 128 KB minimum billable object size. If you store a 64 KB file, you’re billed as if it were 128 KB, effectively doubling your storage cost. This can be particularly painful for organizations dealing with massive volumes of small objects, such as IoT sensor data, archived documents, log files, and images.
&w=1200&q=75)
Object overhead charges (or header fees) are additional costs from the hyperscalers for the metadata that is added to stored objects to help retrieve them when they are archived to lower cost tiers. For example, each object stored in an AWS Glacier tier is appended with 40 KB of additional metadata for data access purposes—all of which the user pays for. Small objects mean more objects. More objects mean more object overhead. And that means higher total costs.
Multipart uploads cost you more than you realize
When dealing with large files in cloud object storage, multipart uploads are a necessary feature to ensure efficient data transfer over the public internet. Instead of attempting to upload a massive file all at once, multipart uploads break the file into smaller chunks—often in the range of several megabytes each—and upload them in parallel. This method improves reliability and speeds up the ingest process, particularly when dealing with large video files, high-resolution images, or massive datasets.
However, while multipart uploads make technical sense, they have additional cost implications when using hyperscale cloud providers like AWS. API requests are again an unexpected expense factor. Each chunk of a multipart upload is treated as an individual API request— specifically a PUT request. The hyperscalers also charge to initiate and complete a multipart upload.
eBook
Cloud Cost Optimization
Tired of going over your cloud storage budget? Download this informative eBook today!
The costs may seem trivial at first glance, but when dealing with thousands of files—especially in media-heavy workflows like video production, surveillance, or big data analytics—these fees accumulate quickly. If your workflow is upload-intensive, your API costs can spiral out of control, turning what seemed like an affordable storage solution into a financial burden.
These fees also vary by storage tier, meaning it’s more expensive to upload data to a cold storage tier than to a warm tier. The graphic below shows the costs to store 100 TB of data across various tiers of Amazon storage. Note how PUT fees alone expand the costs of seemingly lower priced Glacier Instant Retrieval storage and nearly double the costs of S3 Standard Instant Access and Instant Access One-Zone.
&w=1200&q=75)
How Wasabi helps you avoid these pitfalls
Unlike AWS, Azure, and Google Cloud, Wasabi doesn’t impose additional object overhead fees. Wasabi also has a very low minimum billable object size of only 4 KB. Whether you’re storing 4 KB or 4 MB files, you only pay for the actual data you use—without worrying about hidden fees or high minimum threshold costs.
Wasabi does not charge for API requests, including PUTs, which make ingest free for all data. No matter your file size, you can rest assured that your data will be uploaded quickly and at no additional cost to you.
Object size and multi-part uploads can be the source of hidden cloud storage fees. It’s a wise practice to understand your cloud storage provider's minimum object size and API request charges, or switch to Wasabi for a more predictable alternative, with flat storage charges and no fees for API requests.
ebook
Demystifying Cloud Object Storage Costs
The cloud was supposed to save us money. Instead, cloud costs are spiraling out of control.
Download the ultimate guide to hidden fees that can break your budget.
Related article
Most Recent
Onja, a social enterprise in Madagascar, trains students to be world-class software developers.
An inexpensive "cold” tier might look like an amazing deal at first, but your savings may be much less than you expect.
Security often comes at a price with hyperscale cloud storage, where charges for data security countermeasures can add up fast.
SUBSCRIBE
Storage Insights from the Storage Experts
Storage insights sent direct to your inbox.