# Troubleshooting (/tools/s3-gateway/troubleshooting) ## Authentication errors Shelby checks credentials before a request reaches the S3 layer. A rejected request gets `401 Unauthorized` with a short plain-text message, not an S3 XML error. boto3 and the AWS CLI report it with no S3 error code, and the AWS SDK for JavaScript reports an XML parse error instead. To see the message, run `make doctor` from [shelby-quickstart](https://github.com/shelby/shelby-quickstart), or the AWS CLI with `--debug`: | Message | Cause | Fix | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Invalid S3 credential` | The access key ID or secret is wrong, or they come from different access keys | Copy both again from [console.shelby.xyz](https://console.shelby.xyz). The access key ID starts with `SHLB`. If they're right and requests still fail, contact us with your access key ID. Never send the secret. | | `API key not found` | The access key ID setting doesn't hold an access key ID, usually because the secret is in both settings, as older setups had it | Put your access key ID (`SHLB…`) in your client's access key ID setting and the secret in its secret access key setting | | `Invalid signature` | The access key ID or secret is wrong or revoked, or something rewrote a signed header | Copy your access key ID and secret again from [console.shelby.xyz](https://console.shelby.xyz). The access key ID starts with `SHLB`. | | `Request timestamp is too far from server time` | Your clock is 15 minutes or more off | Turn on automatic time sync | | `Anonymous requests are not allowed` | The request wasn't signed. For a presigned URL, the usual cause is boto3 with region `us-east-1`, which makes legacy SigV2 links that Shelby doesn't accept. | Configure credentials in your client. Sign presigned URLs with region `default`. | For ordinary requests, the region doesn't cause authentication errors: Shelby accepts a signature for any region. Presigned URLs are the exception, and must be signed with `default`. Set the region to `default` everywhere. It's the region Shelby reports, so clients don't warn about a mismatch. ## Presigned URL errors A refused presigned URL also gets a short plain-text message. [Presigned URLs](/tools/s3-gateway/presigned-urls#errors) lists every message with its cause and fix, including those for folder shares. ## Errors by symptom | Symptom | Cause | Fix | | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `501 NotImplemented`: "aws-chunked transfer encoding is not supported" | Your client sends uploads in `aws-chunked` framing by default (AWS CLI 2.23+, botocore 1.36+) | Set `request_checksum_calculation` to `when_required`, as in each [quickstart](/tools/s3-gateway/quickstarts/aws-cli) | | `501 NotImplemented` from a presigned link: "A query parameter you provided implies functionality that is not implemented" | The AWS SDK for JavaScript added `x-amz-checksum-mode` to the link | Set `responseChecksumValidation: "WHEN_REQUIRED"` on the client, as in the [JavaScript quickstart](/tools/s3-gateway/quickstarts/javascript), and sign again | | `501 NotImplemented` naming `CopyObject` | Server-side copy, from `aws s3 mv`, `copy_object`, or rclone updating a modification time | Download and upload again. In rclone, add `--checksum`. | | `503 ServiceUnavailable` reading a key just after writing or deleting it | Reads wait up to 10 seconds for a recent write to be indexed, and this one took longer | Retry with backoff | | `503 SlowDown` with `Retry-After: 1` | Your organization has too many downloads in flight. Every GET counts, including range requests and presigned links. | Retry after a second. AWS SDKs do this for you. If it keeps happening, download fewer objects at once. See [Limits](/tools/s3-gateway/limits#concurrent-downloads). | | Object missing from a listing just after upload, or still listed after delete | Listings catch up after a few seconds | Retry, or confirm with `HeadObject` | | `DeleteObjects` from the AWS SDK for JavaScript returns a plain-text `404` and deletes nothing | The SDK sends batch deletes in a form Shelby doesn't recognise yet | Delete keys one at a time with `DeleteObjectCommand`. See [Known compatibility issues](/tools/s3-gateway/known-issues). | | `412 PreconditionFailed` on upload | You sent `If-None-Match: *` and the key already exists | Expected: the create-only check worked | | `400 InvalidArgument` on a list | `max-keys` is above 1,000 | Use a page size of 1,000 or less | | `400 InvalidArgument` partway through a multipart upload | More than 3,000 parts | Raise the part size. Retrying fails the same way. See [Limits](/tools/s3-gateway/limits#choosing-a-part-size). | | `400 EntityTooSmall` | A part other than the last is under 5 MiB | Raise the part size | | `400 EntityTooLarge` on upload | The object or part is over 5 GiB, or, on the Free plan, the upload would take you past your 10 GiB storage allowance | Use a multipart upload for objects over 5 GiB. On the Free plan, delete objects or add a card. See [Pricing](/protocol/pricing#free-plan). | | `400 BadDigest` | The `Content-MD5` you sent doesn't match the body | Check that nothing changes the body in transit | | `404 NoSuchBucket` | The bucket name is wrong | Copy the bucket name from the console | | `403 AccessDenied`: "Bucket access denied" | The access key belongs to a different bucket | Use an access key from the bucket you're addressing. Each access key reaches only its own bucket. | | MinIO `mc` fails every request with `invalid header field value for "Authorization"` | `mc` looks up the bucket's region, and Shelby doesn't answer that lookup correctly yet | Set `MC_REGION=default` before running `mc` | | `400 MetadataTooLarge` on upload | More than 2 KB of `x-amz-meta-*` user metadata, the same limit as S3 | Keep larger metadata in your application | | An object comes back as `binary/octet-stream`, without the metadata it was uploaded with | It was stored before Shelby kept object metadata | Upload it again to store its metadata | | DuckDB fails with `Failed to parse http_proxy` | DuckDB reads your `HTTPS_PROXY` variable but can't parse a proxy URL that includes a username and password | Start DuckDB without the proxy variables set, then set `http_proxy` (host and port), `http_proxy_username` and `http_proxy_password` with `SET` | ## Debugging requests * **Credentials, bucket, clock and tools at once:** clone [shelby-quickstart](https://github.com/shelby/shelby-quickstart) and run `make doctor`. It only reads, and it names the fix for each `401`. * **AWS CLI:** add `--debug` to see the signed request and response. * **boto3:** call `boto3.set_stream_logger("botocore")` before creating the client. * **rclone:** use `--dump headers`. `-vv` doesn't show HTTP traffic. Successful responses and S3 XML errors include an `x-amz-request-id` header. Include it when you contact support. The plain-text `401` responses don't have one, so include the message, the time of the request and your bucket name instead. ## Getting help Ask in [Discord](https://discord.gg/shelbyserves), or send the team a message. Include the request ID or error details from above.