Skip to content

Troubleshooting

Common S3 errors on Shelby, what causes them, and how to fix them.

Authentication errors

Shelby checks credentials before a request reaches the S3 layer. A rejected request gets 401 Unauthorized with a short plain-text message, not an S3 XML error. boto3 and the AWS CLI report it with no S3 error code, and the AWS SDK for JavaScript reports an XML parse error instead. To see the message, run make doctor from shelby-quickstart, or the AWS CLI with --debug:

MessageCauseFix
Invalid S3 credentialThe access key ID or secret is wrong, or they come from different access keysCopy both again from console.shelby.xyz. The access key ID starts with SHLB. If they're right and requests still fail, contact us with your access key ID. Never send the secret.
API key not foundThe access key ID setting doesn't hold an access key ID, usually because the secret is in both settings, as older setups had itPut your access key ID (SHLB…) in your client's access key ID setting and the secret in its secret access key setting
Invalid signatureThe access key ID or secret is wrong or revoked, or something rewrote a signed headerCopy your access key ID and secret again from console.shelby.xyz. The access key ID starts with SHLB.
Request timestamp is too far from server timeYour clock is 15 minutes or more offTurn on automatic time sync
Anonymous requests are not allowedThe request wasn't signed. For a presigned URL, the usual cause is boto3 with region us-east-1, which makes legacy SigV2 links that Shelby doesn't accept.Configure credentials in your client. Sign presigned URLs with region default.

For ordinary requests, the region doesn't cause authentication errors: Shelby accepts a signature for any region. Presigned URLs are the exception, and must be signed with default. Set the region to default everywhere. It's the region Shelby reports, so clients don't warn about a mismatch.

Presigned URL errors

A refused presigned URL also gets a short plain-text message. Presigned URLs lists every message with its cause and fix, including those for folder shares.

Errors by symptom

SymptomCauseFix
501 NotImplemented: "aws-chunked transfer encoding is not supported"Your client sends uploads in aws-chunked framing by default (AWS CLI 2.23+, botocore 1.36+)Set request_checksum_calculation to when_required, as in each quickstart
501 NotImplemented from a presigned link: "A query parameter you provided implies functionality that is not implemented"The AWS SDK for JavaScript added x-amz-checksum-mode to the linkSet responseChecksumValidation: "WHEN_REQUIRED" on the client, as in the JavaScript quickstart, and sign again
501 NotImplemented naming CopyObjectServer-side copy, from aws s3 mv, copy_object, or rclone updating a modification timeDownload and upload again. In rclone, add --checksum.
503 ServiceUnavailable reading a key just after writing or deleting itReads wait up to 10 seconds for a recent write to be indexed, and this one took longerRetry with backoff
503 SlowDown with Retry-After: 1Your organization has too many downloads in flight. Every GET counts, including range requests and presigned links.Retry after a second. AWS SDKs do this for you. If it keeps happening, download fewer objects at once. See Limits.
Object missing from a listing just after upload, or still listed after deleteListings catch up after a few secondsRetry, or confirm with HeadObject
DeleteObjects from the AWS SDK for JavaScript returns a plain-text 404 and deletes nothingThe SDK sends batch deletes in a form Shelby doesn't recognise yetDelete keys one at a time with DeleteObjectCommand. See Known compatibility issues.
412 PreconditionFailed on uploadYou sent If-None-Match: * and the key already existsExpected: the create-only check worked
400 InvalidArgument on a listmax-keys is above 1,000Use a page size of 1,000 or less
400 InvalidArgument partway through a multipart uploadMore than 3,000 partsRaise the part size. Retrying fails the same way. See Limits.
400 EntityTooSmallA part other than the last is under 5 MiBRaise the part size
400 EntityTooLarge on uploadThe object or part is over 5 GiB, or, on the Free plan, the upload would take you past your 10 GiB storage allowanceUse a multipart upload for objects over 5 GiB. On the Free plan, delete objects or add a card. See Pricing.
400 BadDigestThe Content-MD5 you sent doesn't match the bodyCheck that nothing changes the body in transit
404 NoSuchBucketThe bucket name is wrongCopy the bucket name from the console
403 AccessDenied: "Bucket access denied"The access key belongs to a different bucketUse an access key from the bucket you're addressing. Each access key reaches only its own bucket.
MinIO mc fails every request with invalid header field value for "Authorization"mc looks up the bucket's region, and Shelby doesn't answer that lookup correctly yetSet MC_REGION=default before running mc
400 MetadataTooLarge on uploadMore than 2 KB of x-amz-meta-* user metadata, the same limit as S3Keep larger metadata in your application
An object comes back as binary/octet-stream, without the metadata it was uploaded withIt was stored before Shelby kept object metadataUpload it again to store its metadata
DuckDB fails with Failed to parse http_proxyDuckDB reads your HTTPS_PROXY variable but can't parse a proxy URL that includes a username and passwordStart DuckDB without the proxy variables set, then set http_proxy (host and port), http_proxy_username and http_proxy_password with SET

Debugging requests

  • Credentials, bucket, clock and tools at once: clone shelby-quickstart and run make doctor. It only reads, and it names the fix for each 401.
  • AWS CLI: add --debug to see the signed request and response.
  • boto3: call boto3.set_stream_logger("botocore") before creating the client.
  • rclone: use --dump headers. -vv doesn't show HTTP traffic.

Successful responses and S3 XML errors include an x-amz-request-id header. Include it when you contact support. The plain-text 401 responses don't have one, so include the message, the time of the request and your bucket name instead.

Getting help

Ask in Discord, or send the team a message. Include the request ID or error details from above.

What are you working on, and roughly how much data?

We use your details only to reply. See our privacy notice.