Troubleshooting
Common S3 errors on Shelby, what causes them, and how to fix them.
Authentication errors
Shelby checks credentials before a request reaches the S3 layer. A rejected request gets
401 Unauthorized with a short plain-text message, not an S3 XML error. boto3 and the AWS
CLI report it with no S3 error code, and the AWS SDK for JavaScript reports an XML parse
error instead. To see the message, run make doctor from
shelby-quickstart, or the AWS CLI with
--debug:
| Message | Cause | Fix |
|---|---|---|
Invalid S3 credential | The access key ID or secret is wrong, or they come from different access keys | Copy both again from console.shelby.xyz. The access key ID starts with SHLB. If they're right and requests still fail, contact us with your access key ID. Never send the secret. |
API key not found | The access key ID setting doesn't hold an access key ID, usually because the secret is in both settings, as older setups had it | Put your access key ID (SHLB…) in your client's access key ID setting and the secret in its secret access key setting |
Invalid signature | The access key ID or secret is wrong or revoked, or something rewrote a signed header | Copy your access key ID and secret again from console.shelby.xyz. The access key ID starts with SHLB. |
Request timestamp is too far from server time | Your clock is 15 minutes or more off | Turn on automatic time sync |
Anonymous requests are not allowed | The request wasn't signed. For a presigned URL, the usual cause is boto3 with region us-east-1, which makes legacy SigV2 links that Shelby doesn't accept. | Configure credentials in your client. Sign presigned URLs with region default. |
For ordinary requests, the region doesn't cause authentication errors: Shelby accepts a
signature for any region. Presigned URLs are the exception, and must be signed with
default. Set the region to default everywhere. It's the region Shelby reports, so clients
don't warn about a mismatch.
Presigned URL errors
A refused presigned URL also gets a short plain-text message. Presigned URLs lists every message with its cause and fix, including those for folder shares.
Errors by symptom
| Symptom | Cause | Fix |
|---|---|---|
501 NotImplemented: "aws-chunked transfer encoding is not supported" | Your client sends uploads in aws-chunked framing by default (AWS CLI 2.23+, botocore 1.36+) | Set request_checksum_calculation to when_required, as in each quickstart |
501 NotImplemented from a presigned link: "A query parameter you provided implies functionality that is not implemented" | The AWS SDK for JavaScript added x-amz-checksum-mode to the link | Set responseChecksumValidation: "WHEN_REQUIRED" on the client, as in the JavaScript quickstart, and sign again |
501 NotImplemented naming CopyObject | Server-side copy, from aws s3 mv, copy_object, or rclone updating a modification time | Download and upload again. In rclone, add --checksum. |
503 ServiceUnavailable reading a key just after writing or deleting it | Reads wait up to 10 seconds for a recent write to be indexed, and this one took longer | Retry with backoff |
503 SlowDown with Retry-After: 1 | Your organization has too many downloads in flight. Every GET counts, including range requests and presigned links. | Retry after a second. AWS SDKs do this for you. If it keeps happening, download fewer objects at once. See Limits. |
| Object missing from a listing just after upload, or still listed after delete | Listings catch up after a few seconds | Retry, or confirm with HeadObject |
DeleteObjects from the AWS SDK for JavaScript returns a plain-text 404 and deletes nothing | The SDK sends batch deletes in a form Shelby doesn't recognise yet | Delete keys one at a time with DeleteObjectCommand. See Known compatibility issues. |
412 PreconditionFailed on upload | You sent If-None-Match: * and the key already exists | Expected: the create-only check worked |
400 InvalidArgument on a list | max-keys is above 1,000 | Use a page size of 1,000 or less |
400 InvalidArgument partway through a multipart upload | More than 3,000 parts | Raise the part size. Retrying fails the same way. See Limits. |
400 EntityTooSmall | A part other than the last is under 5 MiB | Raise the part size |
400 EntityTooLarge on upload | The object or part is over 5 GiB, or, on the Free plan, the upload would take you past your 10 GiB storage allowance | Use a multipart upload for objects over 5 GiB. On the Free plan, delete objects or add a card. See Pricing. |
400 BadDigest | The Content-MD5 you sent doesn't match the body | Check that nothing changes the body in transit |
404 NoSuchBucket | The bucket name is wrong | Copy the bucket name from the console |
403 AccessDenied: "Bucket access denied" | The access key belongs to a different bucket | Use an access key from the bucket you're addressing. Each access key reaches only its own bucket. |
MinIO mc fails every request with invalid header field value for "Authorization" | mc looks up the bucket's region, and Shelby doesn't answer that lookup correctly yet | Set MC_REGION=default before running mc |
400 MetadataTooLarge on upload | More than 2 KB of x-amz-meta-* user metadata, the same limit as S3 | Keep larger metadata in your application |
An object comes back as binary/octet-stream, without the metadata it was uploaded with | It was stored before Shelby kept object metadata | Upload it again to store its metadata |
DuckDB fails with Failed to parse http_proxy | DuckDB reads your HTTPS_PROXY variable but can't parse a proxy URL that includes a username and password | Start DuckDB without the proxy variables set, then set http_proxy (host and port), http_proxy_username and http_proxy_password with SET |
Debugging requests
- Credentials, bucket, clock and tools at once: clone
shelby-quickstart and run
make doctor. It only reads, and it names the fix for each401. - AWS CLI: add
--debugto see the signed request and response. - boto3: call
boto3.set_stream_logger("botocore")before creating the client. - rclone: use
--dump headers.-vvdoesn't show HTTP traffic.
Successful responses and S3 XML errors include an x-amz-request-id header. Include it
when you contact support. The plain-text 401 responses don't have one, so include the
message, the time of the request and your bucket name instead.
Getting help
Ask in Discord, or send the team a message. Include the request ID or error details from above.