Limits
Size, count and request limits on the hosted S3 endpoint.
| Limit | Value | Amazon S3 | What happens past it |
|---|---|---|---|
Single PutObject | 5 GiB | 5 GiB | 400 EntityTooLarge. Use a multipart upload. |
| Multipart part size | 5 MiB to 5 GiB (the last part can be smaller) | Same | 400 EntityTooSmall or 400 EntityTooLarge |
| Parts per upload | 3,000 | 10,000 | 400 InvalidArgument on the part that crosses the limit |
| Largest object | About 14.6 TiB (3,000 × 5 GiB) | 50 TiB | — |
| Object key length | 958 bytes | 1,024 bytes | Keys from 959 to 1,024 bytes fail with 500 InternalError |
Keys per ListObjects page | 1,000 | 1,000 | 400 InvalidArgument for max-keys above 1,000 |
Keys per DeleteObjects request | 1,000 | 1,000 | 400 MalformedXML |
| Clock skew | Under 15 minutes | 15 minutes | 401 |
Content-Length | Required on uploads | Required | 411 MissingContentLength |
| Presigned URL lifetime | 1 second to 7 days | 7 days | 400: "X-Amz-Expires must be between 1 and 604800 seconds" |
On the Free plan, an upload that would take you past your
10 GiB storage allowance is refused with 400 EntityTooLarge before any data is sent. The
console shows your usage.
Choosing a part size
Use parts of at least 10 MiB. Each part counts as at least 10 MiB of storage, so smaller parts make a file count as more than it holds. See The 10 MiB minimum. The AWS CLI and boto3 default to 8 MiB parts, and the AWS SDK for JavaScript and rclone to 5 MiB, so each quickstart sets a larger part size.
Because of the 3,000-part limit, your part size also sets your largest file:
| Part size | Largest file |
|---|---|
| 10 MiB | About 29 GiB |
| 64 MiB | About 187 GiB |
| 512 MiB | About 1.5 TiB |
rclone and DuckDB raise the part size for you when told the limit: max_upload_parts=3000 in
the rclone quickstart, and
s3_uploader_max_parts_per_file = 3000 in the
DuckDB quickstart.
Concurrent downloads
Your organization can have a limited number of downloads in flight at once, shared by all of its buckets and access keys. Every GET counts as one, including each range request and each GET through a presigned URL. HEAD requests don't count.
Past the limit, Shelby returns 503 SlowDown with Retry-After: 1. AWS SDKs retry it for you.
If it keeps happening, lower your client's parallelism. A web page that loads many presigned
images or videos at once can hit it too, because a video player makes a series of range
requests as it plays.
What isn't limited
Objects stay until you delete them. Shelby has no lifecycle rules or automatic expiry.