> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prequel.co/llms.txt
> Use this file to discover all available pages before exploring further.

# S3 staging bucket

> Set up an S3 staging bucket for efficient source or destination data transfer.

Some sources or destinations without built-in staging resources require a staging bucket to efficiently transfer or ingest data.

### Prerequisites

* By default, S3 authentication uses role-based access. You will need the trust policy prepopulated with our identifier to grant access. It should look similar to the following JSON object with a proper service account identifier:

```json title="Trust policy" icon="brackets-curly" expandable theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sts:AssumeRoleWithWebIdentity"
      ],
      "Principal": {
        "Federated": "accounts.google.com"
      },
      "Condition": {
        "StringEquals": {
          "accounts.google.com:sub": "<some_service_account_identifier>"
        }
      }
    }
  ]
}
```

### Create staging bucket

1. Navigate to the **S3** service page.
2. Click **Create bucket**.
3. Enter a **Bucket name** and modify any of the default settings as desired. Note: **Object Ownership** can be set to "**ACLs disabled**" and **Block Public Access settings for this bucket** can be set to "**Block all public access**" as recommended by AWS. Make note of the Bucket name and AWS Region.
4. Click **Create bucket**.

<Note>
  **Optional: Add a short retention lifecycle policy**

  You may configure a lifecycle rule on the staging bucket to automatically delete objects older than 2 days as the bucket is not used to persist data. In the bucket **Management** tab, click **Create lifecycle rule**, set an expiration action for current versions of objects with a 2-day age. Note that transfer logic automatically cleans up files after transfer completion, so this is an optional step.
</Note>

### Create policy

5. Navigate to the **IAM** service page, click on the **Policies** navigation tab, and click **Create policy**.
6. Click the JSON tab, and paste the following policy, being sure to replace `BUCKET_NAME`  with the name of the bucket chosen above.
   1. **Note**: the first policy applies to `BUCKET_NAME` whereas the second policy applies only to the bucket's contents, `BUCKET_NAME/*`, an important distinction.

```json title="Bucket access policy" icon="brackets-curly" expandable theme={null}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::BUCKET_NAME"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
              	"s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::BUCKET_NAME/*"
        }
    ]
}
```

<Note>
  **KMS encryption (optional)**

  If your S3 staging bucket uses KMS encryption (CMK), add the following statement to the `Statement` array of your IAM policy to allow data encryption/decryption with your KMS key. Encryption with SSE-C is not currently supported.

  ```json title="KMS statement" icon="brackets-curly" expandable theme={null}
  {
    "Effect": "Allow",
    "Action": [
      "kms:GenerateDataKey",
      "kms:Decrypt"
    ],
    "Resource": "arn:aws:kms:REGION_NAME:ACCOUNT_ID:key/KEY_ID"
  }
  ```

  Replace `REGION_NAME`, `ACCOUNT_ID`, and `KEY_ID` with your values.
</Note>

7. Click through to the **Review** step, choose a **name** for the policy, for example, `transfer-service-policy` (this will be referenced in the next step), add a description, and click **Create policy**.

### Create role

1. Navigate to the **IAM** service page.
2. Navigate to the **Roles** navigation tab, and click **Create role**.
3. Select **Custom trust policy** and paste the provided trust policy to allow AssumeRole access to the new role. Click **Next**.
4. Add the permissions policy created above, and click **Next**.
5. Enter a **Role name**, for example, `transfer-role`, and click **Create role**.
6. Once successfully created, search for the created role in the Roles list, click the role name, and make a note of the **ARN** value.

### You're done!

Use this configured S3 staging bucket during the connection of your preferred data source or destination.