Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/routes/docs/products/storage/+layout.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@
{
label: 'Image transformations',
href: '/docs/products/storage/images'
},
{
label: 'S3 API',
href: '/docs/products/storage/s3'
}
]
},
Expand Down
240 changes: 240 additions & 0 deletions src/routes/docs/products/storage/s3/+page.markdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
---
layout: article
title: S3 API
description: Connect any S3-compatible client, SDK, or tool to Appwrite Storage. Configure credentials, then manage buckets, objects, and multipart uploads over the S3 API.
---

Appwrite Storage exposes an S3-compatible API, so you can use the AWS CLI, the AWS SDKs, and third-party tools like rclone or s3cmd to work with your buckets and files. The API speaks AWS Signature Version 4 and maps standard S3 operations onto Appwrite Storage, which means most existing S3 code and tooling works after you change three settings: the endpoint, the credentials, and the region.

{% info title="What you need" %}
The S3 API is available on Appwrite Cloud. To connect, you need a project ID and an [API key](/docs/advanced/security/api-keys) with the storage scopes listed below. No changes to your existing buckets or files are required, they are addressable over both the native Storage API and the S3 API at the same time.
{% /info %}

# Endpoint and credentials {% #endpoint-and-credentials %}

S3 clients authenticate with an access key ID and a secret access key. Appwrite maps these to your project ID and an API key secret. Configure your client with the following values.

| Setting | Value |
| --- | --- |
| Endpoint | `https://<REGION>.cloud.appwrite.io/v1/s3` |
| Access key ID | Your Appwrite project ID |
| Secret access key | An Appwrite API key secret |
| Region | `us-east-1` |
| Signature version | AWS Signature Version 4 (SigV4) |
| Addressing style | Path-style only |

The `<REGION>` in the endpoint is your Appwrite Cloud region (for example `fra` or `nyc`), the same host you use for the rest of the Appwrite API. The S3 `region` setting is separate. Set it to `us-east-1`, which is the region Appwrite reports for your buckets (`GetBucketLocation`) and keeps clients that require a region value consistent.

{% info title="API key scopes" %}
Create an API key in your project's **Overview** settings, then grant it the storage scopes your workload needs:

- `buckets.read` and `files.read` for read-only access (list and download).
- `buckets.write` and `files.write` to also create buckets and upload or delete objects.

Copying an object reads the source and writes the destination, so `CopyObject` and `UploadPartCopy` need both `files.read` and `files.write`. Grant all four scopes for full read and write access.

The project ID identifies your project, and the API key secret both authenticates the request and determines what it is allowed to do. Requests signed with a secret that does not match a project API key, or a key missing the required scope, are rejected with `AccessDenied` (HTTP 403).
{% /info %}

# Configure your S3 client {% #configure-your-s3-client %}

Point your client at the Appwrite endpoint, set the region to `us-east-1`, enable path-style addressing, and pass your project ID and API key secret as the credentials.

{% multicode %}
```bash
# AWS CLI
aws configure set aws_access_key_id <PROJECT_ID>
aws configure set aws_secret_access_key <API_KEY_SECRET>
aws configure set region us-east-1

# Appwrite serves path-style URLs only, so force path addressing
aws configure set default.s3.addressing_style path

# Pass the Appwrite endpoint on every command
aws s3 ls --endpoint-url https://<REGION>.cloud.appwrite.io/v1/s3
```

```js
// Node.js: @aws-sdk/client-s3 (v3)
import { S3Client } from '@aws-sdk/client-s3';

const client = new S3Client({
endpoint: 'https://<REGION>.cloud.appwrite.io/v1/s3',
region: 'us-east-1',
forcePathStyle: true,
credentials: {
accessKeyId: '<PROJECT_ID>',
secretAccessKey: '<API_KEY_SECRET>'
}
});
```

```python
# Python: boto3
import boto3
from botocore.config import Config

s3 = boto3.client(
's3',
endpoint_url='https://<REGION>.cloud.appwrite.io/v1/s3',
region_name='us-east-1',
aws_access_key_id='<PROJECT_ID>',
aws_secret_access_key='<API_KEY_SECRET>',
config=Config(signature_version='s3v4', s3={'addressing_style': 'path'})
)
```

```go
// Go: aws-sdk-go-v2
cfg, err := config.LoadDefaultConfig(context.TODO(),
config.WithRegion("us-east-1"),
config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
"<PROJECT_ID>", "<API_KEY_SECRET>", "",
)),
)
if err != nil {
log.Fatal(err)
}

client := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.BaseEndpoint = aws.String("https://<REGION>.cloud.appwrite.io/v1/s3")
o.UsePathStyle = true
})
```

```php
// PHP: aws/aws-sdk-php
use Aws\S3\S3Client;

$client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'endpoint' => 'https://<REGION>.cloud.appwrite.io/v1/s3',
'use_path_style_endpoint' => true,
'credentials' => [
'key' => '<PROJECT_ID>',
'secret' => '<API_KEY_SECRET>',
],
]);
```
{% /multicode %}

# How buckets and objects map to Appwrite {% #how-buckets-and-objects-map %}

S3 buckets map directly to Appwrite Storage buckets, and S3 objects map to files. Creating a bucket over S3 creates an Appwrite bucket with the bucket name set to the ID you provided and Appwrite's default settings. Any bucket you already have in your project is usable over S3 immediately.

Every object maps to a file. When you upload an object, Appwrite stores it as a file with an automatically generated file ID and uses the object key you provided as the file's name. The **canonical key**, the key returned by list operations and expected by read, copy, and delete operations, is:

```text
{fileId}-{fileName}
```

For example, uploading to the key `reports/january.pdf` creates a file whose name is `reports/january.pdf`, and the object is then listed under a canonical key such as `6710e2...f3-reports/january.pdf`.

{% info title="Reading back what you upload" %}
Because a new file ID is generated on every plain upload, uploading twice with the same key creates two distinct objects instead of overwriting. To read, copy, overwrite, or delete an object, use its canonical key from a list operation. Uploading to a canonical key that already exists overwrites that object in place.

This also lets the S3 API interoperate with files created through the native Storage API and Console: list the bucket to discover the `{fileId}-{fileName}` key, then address the file over S3 with it.
{% /info %}

# Supported operations {% #supported-operations %}

The API covers the bucket, object, and multipart operations that standard S3 clients and SDKs rely on. Operations are addressed with path-style URLs under `/v1/s3`.

## Buckets {% #buckets %}

| Operation | Description |
| --- | --- |
| `ListBuckets` | List the project's buckets. |
| `CreateBucket` | Create a bucket. The bucket ID becomes the bucket name, with Appwrite's default settings. |
| `HeadBucket` | Check that a bucket exists and is accessible. |
| `DeleteBucket` | Delete an empty bucket. Deleting a bucket that still contains objects returns `BucketNotEmpty` (HTTP 409). |
| `GetBucketLocation` | Returns `us-east-1`. |
| `GetBucketAcl`, `PutBucketAcl` | Compatibility responses only. See [Compatibility and limitations](#compatibility-and-limitations). |

## Objects {% #objects %}

| Operation | Description |
| --- | --- |
| `PutObject` | Upload an object. Returns an `ETag`. Honors `Content-Type`, `x-amz-meta-*` user metadata, and `x-amz-server-side-encryption`. |
| `GetObject` | Download an object. Supports `Range` requests (HTTP 206), and the `If-None-Match` (HTTP 304) and `If-Match` (HTTP 412) conditional headers. |
| `HeadObject` | Retrieve an object's metadata (size, content type, `ETag`, user metadata) without the body. |
| `CopyObject` | Copy an object using the `x-amz-copy-source` header. Supports `x-amz-metadata-directive: REPLACE` to replace metadata. |
| `DeleteObject` | Delete a single object. |
| `DeleteObjects` | Delete multiple objects in one request, including quiet mode. |
| `ListObjects` | List objects in a bucket, with `prefix` filtering. |
| `ListObjectsV2` | List objects with `prefix`, `delimiter`, `max-keys`, `continuation-token`, and `start-after`. |
| `GetObjectAcl`, `PutObjectAcl` | Compatibility responses only. See [Compatibility and limitations](#compatibility-and-limitations). |

Content type is taken from the `Content-Type` you send. When that is missing or generic (`application/octet-stream`), Appwrite infers a type from the file extension. Buckets keep their constraints under the S3 API: uploads that exceed the bucket's maximum file size or use a disallowed extension are rejected, and disabled buckets are treated as not found.

## Multipart uploads {% #multipart-uploads %}

Large files are uploaded in parts. Standard SDK multipart helpers, such as the AWS CLI's `aws s3 cp` and the SDK transfer managers, use these operations automatically.

| Operation | Description |
| --- | --- |
| `CreateMultipartUpload` | Begin a multipart upload and receive an `UploadId`. |
| `UploadPart` | Upload a single part. Returns the part's `ETag`. |
| `UploadPartCopy` | Upload a part by copying a byte range from another object, using `x-amz-copy-source` and `x-amz-copy-source-range`. |
| `CompleteMultipartUpload` | Assemble the uploaded parts into the final object. |
| `AbortMultipartUpload` | Discard an in-progress multipart upload and its parts. |
| `ListMultipartUploads` | List in-progress multipart uploads in a bucket. |
| `ListParts` | List the parts already uploaded for an `UploadId`. |

{% info title="Multipart requirements" %}
Parts must be contiguous and start at part number 1. Completing an upload with non-contiguous part numbers (for example parts 1, 5, and 100) is rejected. Sequential uploads, which is what SDK multipart flows produce, are unaffected. In-progress uploads are kept for 24 hours before they are cleaned up, so complete or abort within that window.
{% /info %}

# Examples {% #examples %}

Once your client is configured, everyday S3 commands work as usual. These AWS CLI examples assume you have run `aws configure` as shown above and set an alias or environment default for the endpoint.

```bash
# Create a bucket
aws s3api create-bucket --bucket my-bucket \
--endpoint-url https://<REGION>.cloud.appwrite.io/v1/s3

# Upload a file (uses multipart automatically for large files)
aws s3 cp ./january.pdf s3://my-bucket/reports/january.pdf \
--endpoint-url https://<REGION>.cloud.appwrite.io/v1/s3

# List objects to discover canonical keys
aws s3 ls s3://my-bucket --recursive \
--endpoint-url https://<REGION>.cloud.appwrite.io/v1/s3

# Download using the canonical {fileId}-{fileName} key from the listing
aws s3api get-object --bucket my-bucket \
--key 6710e2f3-reports/january.pdf ./january.pdf \
--endpoint-url https://<REGION>.cloud.appwrite.io/v1/s3
```

# Compatibility and limitations {% #compatibility-and-limitations %}

The S3 API targets the operations most clients depend on. Keep the following in mind.

- **Path-style addressing only.** Virtual-hosted-style URLs (`https://<bucket>.host/...`) are not supported. Enable path-style addressing in your client, as shown in the configuration examples.
- **Signature Version 4 only.** Requests are authenticated with SigV4, and the request timestamp must be within 15 minutes of the server's clock.
- **Reading back objects.** Reads, copies, overwrites, and deletes address objects by their canonical `{fileId}-{fileName}` key from a list operation, not by the raw key you uploaded with. See [How buckets and objects map to Appwrite](#how-buckets-and-objects-map).
- **ACLs are compatibility responses.** `GetObjectAcl`, `PutObjectAcl`, `GetBucketAcl`, and `PutBucketAcl` return a canned private ACL and are accepted for tooling compatibility. They do not change access. Manage access with [Appwrite permissions](/docs/products/storage/permissions) and API key scopes instead.
- **Folders are not supported.** Uploading a folder marker (a key ending in `/`) returns `NotImplemented` (HTTP 501). Object keys can still contain `/`, so prefixes behave as expected.
- **Object listings are built in memory.** `ListObjects` and `ListObjectsV2` load the matching objects from the database and order them in the API. `ListObjects` returns every matching object in a single response, while `ListObjectsV2` honors `max-keys` (capped at 1000) and paginates with `IsTruncated` and a `NextContinuationToken`. No objects are silently dropped, but listing very large buckets increases per-request latency and memory.
- **Custom file IDs with hyphens.** The canonical key is split on the first `-`. Objects created through the S3 API use generated IDs without hyphens, so this only affects interop with pre-existing files whose custom ID contains a `-`. A mismatch returns `NoSuchKey` rather than the wrong object.
- **Unsupported S3 features.** Object tagging, CORS, lifecycle, bucket policies and policy status, encryption configuration, ownership controls, notifications, and versioning are not supported. Requests to these sub-resources return `NotImplemented` (HTTP 501).

## Error responses {% #error-responses %}

Errors are returned as standard S3 XML error documents with an S3 error code and HTTP status.

| S3 error code | HTTP status | Meaning |
| --- | --- | --- |
| `NoSuchBucket` | 404 | The bucket does not exist, is disabled, or is not accessible. |
| `NoSuchKey` | 404 | The object key does not exist. |
| `BucketNotEmpty` | 409 | The bucket still contains objects and cannot be deleted. |
| `BucketAlreadyOwnedByYou` | 409 | A bucket with that ID already exists in your project. |
| `AccessDenied` | 403 | The signature is invalid, or the API key is expired or missing a required scope. |
| `InvalidRange` | 416 | The requested `Range` cannot be satisfied. |
| `PreconditionFailed` | 412 | An `If-Match` precondition did not hold. |
| `NotImplemented` | 501 | The requested S3 feature is not supported. |
| `InvalidRequest` | 400 | The request was malformed or violated a bucket constraint. |
| `InternalError` | 500 | An unexpected server error occurred. |
Loading