# Python (boto3) (/tools/s3-gateway/quickstarts/python)

Prefer to run it? [shelby-quickstart](https://github.com/shelby/shelby-quickstart/tree/main/clients/python)
has this page as a script that checks every step and cleans up after itself.

## Before you start

* Python 3.10 or later, with `pip install boto3`

Copy your bucket name, and an access key ID and secret for that bucket, from the
[console](https://console.shelby.xyz). Export them so the commands below can use them:

```bash
export SHELBY_ACCESS_KEY_ID="<your access key ID, SHLB…>"
export SHELBY_SECRET_ACCESS_KEY="<your secret>"
export SHELBY_BUCKET="<your bucket name>"
```

<Steps>
  <Step>
    ### Create a client

    ```python title="shelby_s3.py"
    import os

    import boto3
    from botocore.config import Config

    s3 = boto3.client(
        "s3",
        endpoint_url="https://s3.beta.shelby.xyz",
        aws_access_key_id=os.environ["SHELBY_ACCESS_KEY_ID"],
        aws_secret_access_key=os.environ["SHELBY_SECRET_ACCESS_KEY"],
        region_name="default",
        config=Config(
            s3={"addressing_style": "path"},
            request_checksum_calculation="when_required",
        ),
    )
    bucket = os.environ["SHELBY_BUCKET"]
    ```

    `request_checksum_calculation` stops boto3 from sending uploads in `aws-chunked` framing,
    which it does by default (botocore 1.36 and later) and Shelby doesn't accept yet. Without
    it, uploads fail with `501 NotImplemented`.
  </Step>

  <Step>
    ### Upload and read back

    ```python
    import hashlib

    body = b"Hello, Shelby\n"
    s3.put_object(Bucket=bucket, Key="examples/hello.txt", Body=body)

    data = s3.get_object(Bucket=bucket, Key="examples/hello.txt")["Body"].read()
    assert hashlib.sha256(data).digest() == hashlib.sha256(body).digest()

    # A new object can take a few seconds to appear in listings.
    for obj in s3.list_objects_v2(Bucket=bucket, Prefix="examples/").get("Contents", []):
        print(obj["Key"], obj["Size"])
    ```
  </Step>

  <Step>
    ### Share a download link

    ```python
    import urllib.request

    url = s3.generate_presigned_url(
        "get_object",
        Params={"Bucket": bucket, "Key": "examples/hello.txt"},
        ExpiresIn=300,
    )
    with urllib.request.urlopen(url) as response:  # sends no credentials
        print(response.read().decode())
    ```

    Anyone with `url` can download the object for the next 5 minutes, without credentials. The
    link works because the client signs with `region_name="default"`. Until it expires, treat the
    link like a password. See [Presigned URLs](/tools/s3-gateway/presigned-urls).
  </Step>

  <Step>
    ### Clean up

    ```python
    s3.delete_object(Bucket=bucket, Key="examples/hello.txt")
    ```
  </Step>
</Steps>

## Upload large files

`upload_file` switches to multipart automatically, in 8 MiB parts by default. Set a part size
of at least 10 MiB, because smaller parts each [count as 10 MiB](/protocol/pricing#the-10-mib-minimum) of storage, and large
enough for your largest file to fit in Shelby's 3,000 parts:

```python
from boto3.s3.transfer import TransferConfig

s3.upload_file(
    "dataset.parquet",
    bucket,
    "data/dataset.parquet",
    Config=TransferConfig(multipart_chunksize=64 * 1024 * 1024),  # 64 MiB parts
)
```

64 MiB parts fit files up to about 187 GiB.

## What's different with boto3

* **`copy_object` and `s3.copy` fail with `501`.** Shelby has no server-side copy. Download
  and upload again.
* **Paginators need `PageSize` of 1,000 or less.** Larger values are rejected.

[Compatibility matrix](/tools/s3-gateway/compatibility) ·
[Troubleshooting](/tools/s3-gateway/troubleshooting)