What Is Multipart Upload? Splitting Files Into Chunks

Multipart upload is how S3-compatible storage handles files too large for a single request: split the file into independent parts, upload each one separately, then ask the server to stitch them back together. A single PutObject call tops out at 5 GB, so anything larger has to go through this process, not as an edge case but as the standard path for any sizeable file. This article covers the exact three-step process, the real size and part-count limits, and what happens to the parts if the upload never finishes.

TL;DR:

• A single S3 upload request maxes out at 5 GB. Multipart upload splits a larger file into parts (5 MiB to 5 GiB each) uploaded independently, then assembled server-side.
• The process is exactly three API calls: initiate, upload each part, complete. Parts can upload in parallel and in any order.
• An abandoned multipart upload keeps billing for its stored parts until it's explicitly completed or aborted, there's no automatic expiry.

Why Large Files Need Multipart Upload at All

A single PutObject request can carry at most 5 GB, per AWS's own S3 documentation, and AWS recommends switching to multipart upload well before that ceiling, at 100 MB, not right at the limit. Splitting a large file into parts and uploading them independently solves three real problems a single giant request doesn't: parts can upload in parallel across multiple connections instead of one slow serial stream, a failed part can be retried on its own instead of restarting the entire upload from byte zero, and an upload can begin before the final file size is even known, useful when a file is being generated on the fly rather than read from disk. None of this is unique to any one vendor, it's how the S3 API itself is built, and any S3-compatible client speaks the same protocol.

The Three-Step Process

Multipart upload is exactly three API calls, run in this order, per AWS's own documentation:

1. Initiate (CreateMultipartUpload): the client requests a new multipart upload for a given object key, and the server responds with an upload ID, a unique identifier that ties every subsequent call for this upload together.
2. Upload parts (UploadPart, one call per part): each part is uploaded with that upload ID and a part number from 1 to 10,000. Parts don't need to be uploaded in order, and don't need consecutive numbers, though most clients use them sequentially in practice. Each part upload returns an ETag that the client has to record for the next step.
3. Complete (CompleteMultipartUpload): the client sends the upload ID plus the full list of part numbers and their ETags. The server concatenates the parts in ascending part-number order into the final object, and only then does the object actually exist and become readable.

Nothing about this requires all parts to come from the same machine or the same network connection. A client can spread parts across multiple parallel workers, which is exactly what makes multipart upload faster for large files on a fast connection, not just more resilient on a flaky one.

The Real Numbers

AWS's own published limits: a part must be between 5 MiB and 5 GiB, except the last part of an upload, which has no minimum size. A single multipart upload can have up to 10,000 parts. Combine those two ceilings and the maximum object size multipart upload supports is 48.8 TiB. AWS's own worked example makes the shape of a real upload concrete: a 100 GB file split into 1,000 parts of 100 MB each requires exactly 1,002 API calls total, one CreateMultipartUpload, 1,000 UploadPart calls, one CompleteMultipartUpload. That's the entire protocol overhead, regardless of file size, since the actual data transfer happens inside the 1,000 part uploads, not as separate bookkeeping.

Multipart upload's parallelism is also why AWS pairs it with S3 Transfer Acceleration in its own documentation: routing each part through the nearest edge location instead of straight to the bucket's home region tackles a different bottleneck than parallel parts alone, network latency over long distances rather than single-connection throughput. AWS's own testing found the combination cutting upload time by up to 61% for large files moved across distance. That's an AWS-specific feature, not a generic S3 API capability every provider ships, worth knowing the difference between the protocol-level benefit (parallelism, works anywhere multipart upload does) and the vendor-specific one (edge routing, works only where that vendor offers it) before assuming both apply outside AWS.

What Happens If You Never Finish

An initiated multipart upload doesn't expire on its own. Every part already uploaded keeps sitting in storage, and keeps being billed for storage, bandwidth, and requests, until the upload is explicitly finished with CompleteMultipartUpload or explicitly cancelled with AbortMultipartUpload. A crashed script, an interrupted process, or a client that just never calls complete leaves those parts billing indefinitely with nothing to show for it. AWS's own guidance for this is a bucket lifecycle rule using the AbortIncompleteMultipartUpload action, which automatically cleans up any multipart upload left unfinished past a set number of days, rather than relying on every client to clean up after itself correctly every time.

Does AIOZ Storage Support Multipart Upload?

AIOZ Storage's own documentation doesn't mention multipart upload, chunking, or file splitting anywhere in its concept pages, so treat direct support as unconfirmed rather than assumed either way. If a specific integration depends on it, test against AIOZ Storage's actual S3-compatible endpoint directly before building around it.

Frequently Asked Questions

What's the maximum size for a single S3 upload without multipart?
5 GB per PutObject request. AWS recommends switching to multipart upload starting around 100 MB, well before that hard ceiling.

How many parts can one multipart upload have?
Up to 10,000, numbered 1 through 10,000. Combined with the 5 GiB maximum part size, that puts the largest possible multipart-uploaded object at 48.8 TiB.

What's the minimum size for a part in a multipart upload?
5 MiB, except for the last part of the upload, which has no minimum size at all.

Do the parts of a multipart upload have to be uploaded in order?
No. Part numbers don't need to be consecutive or uploaded sequentially, which is exactly what makes parallel upload across multiple parts possible.

What happens if a multipart upload is never completed?
The uploaded parts keep sitting in storage, and keep being billed, indefinitely. AWS recommends a lifecycle rule using AbortIncompleteMultipartUpload to automatically clean up unfinished uploads after a set number of days.

Is multipart upload only useful for unreliable networks?
No. It also improves throughput on fast, stable connections by uploading parts in parallel, and it lets an upload start before the final object size is known, three separate benefits, not just one.

Does AIOZ Storage support S3 multipart upload?
Not documented either way. AIOZ Storage's concept pages don't mention chunking or multipart upload, so this isn't confirmed as supported, only untested against the actual endpoint.

Related Reading

• What Is Amazon S3? Object Storage Explained - the bucket/object/key model multipart upload builds objects into.
• Using AIOZ Storage as a Dataset and Model-Output Backend - what AIOZ Storage's own published limits do and don't confirm about large-object handling.
• File Management API with FastAPI, Boto3, and AIOZ Storage - the boto3 client setup any multipart upload code would build on.
• AIOZ Storage Explained: S3-Compatible Decentralized Storage - the full picture of what AIOZ Storage's S3 compatibility does and doesn't cover.
• What Is a Checksum and How S3 Verifies Data Integrity - why a multipart upload changes how the object's ETag actually gets calculated.

References

• AWS Documentation: Uploading and Copying Objects Using Multipart Upload
• AWS Documentation: Amazon S3 Multipart Upload Limits
• AWS Blog: Uploading Large Objects to Amazon S3 Using Multipart Upload and Transfer Acceleration