> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prequel.co/llms.txt
> Use this file to discover all available pages before exploring further.

# SFTP

> Configuring your SFTP server.

## Prerequisites

* By default, SFTP uses keypair authentication for access. You will need a provided `public key` to configure your destination. It will look roughly like this:

```text title="Public key format" icon="file-lines" theme={null}
ssh-key <ssh_public_key_beginning_with_AAAA> some-comment
```

<Steps>
  <Step title="Create a user on the SFTP server">
    Login to the SFTP server and complete the steps below.

    1. Create group `sftpwriter`:

    ```shell title="Create group" icon="terminal" theme={null}
    sudo groupadd sftpwriter
    ```

    2. Create user `sftpwriter`:

    ```shell title="Create user" icon="terminal" theme={null}
    sudo useradd -m -g sftpwriter sftpwriter
    ```

    3. Switch to the `sftpwriter` user:

    ```shell title="Switch user" icon="terminal" theme={null}
    sudo su - sftpwriter
    ```

    4. Create the `.ssh` directory:

    ```shell title="Create .ssh directory" icon="terminal" theme={null}
    mkdir ~/.ssh
    ```

    5. Set permissions:

    ```shell title="Set directory permissions" icon="terminal" theme={null}
    chmod 700 ~/.ssh
    ```

    6. Navigate to the `.ssh` directory:

    ```shell title="Navigate to .ssh" icon="terminal" theme={null}
    cd ~/.ssh
    ```

    7. Create the `authorized_keys` file:

    ```shell title="Create authorized_keys file" icon="terminal" theme={null}
    touch authorized_keys
    ```

    8. Set permissions:

    ```shell title="Set file permissions" icon="terminal" theme={null}
    chmod 600 authorized_keys
    ```

    9. Add the public key to the `authorized_keys` file. *The key, including the "ssh-key" and comment, should be all on one line in the file, without linebreaks.*

    ```shell title="Add public key" icon="terminal" theme={null}
    echo "ssh-key <ssh_public_key_beginning_with_AAAA> sftpwriter-public-key" > authorized_keys
    ```
  </Step>

  <Step title="Add your destination">
    Use the following details to complete the connection setup: **host name**, **folder name**, **username**, **port** and preferred **delimiter character**.

    <Note>
      **Write permissions at the SFTP root are required**

      In addition to write access within your configured `<folder>`, this destination writes per-transfer manifest files under a `_manifests/` directory created at the root of the SFTP home/path. Ensure the SFTP user can create and write to `_manifests` at that root (even if your data lands under a subfolder). Manifests allow downstream systems to detect when a transfer is complete. See the FAQ below for how these files are organized.
    </Note>
  </Step>
</Steps>

## Frequently asked questions

<AccordionGroup>
  <Accordion title="How will the data appear in my SFTP server?">
    The data will be loaded with the configured file format (`Parquet`, `CSV`, or `JSON`/`JSONL`) in a predictable folder structure that can be easily parsed by downstream systems.

    <Tree>
      <Tree.Folder name="sftpwriter_home_folder" defaultOpen>
        <Tree.Folder name="some_provided_folder" defaultOpen>
          <Tree.Folder name="some_table_a" defaultOpen>
            <Tree.Folder name="dt=2024-01-01" defaultOpen>
              <Tree.File name="0_20240101181004.csv" />

              <Tree.File name="1_20240101184002.csv" />
            </Tree.Folder>

            <Tree.Folder name="dt=2024-01-02" defaultOpen>
              <Tree.File name="0_20240102180123.csv" />
            </Tree.Folder>

            <Tree.Folder name="dt=2024-01-03" defaultOpen>
              <Tree.File name="0_20240103182145.csv" />
            </Tree.Folder>
          </Tree.Folder>

          <Tree.Folder name="some_table_b" defaultOpen>
            <Tree.Folder name="dt=2024-01-01" defaultOpen>
              <Tree.File name="0_20240101186004.csv" />
            </Tree.Folder>

            <Tree.Folder name="dt=2024-01-02" defaultOpen>
              <Tree.File name="0_20240102185123.csv" />
            </Tree.Folder>

            <Tree.Folder name="dt=2024-01-03" defaultOpen>
              <Tree.File name="0_20240103187145.csv" />
            </Tree.Folder>
          </Tree.Folder>
        </Tree.Folder>
      </Tree.Folder>
    </Tree>
  </Accordion>

  <Accordion title="How is the SFTP connection secured?">
    Use SSH key-based authentication for a dedicated, least-privileged SFTP user. Restrict access to only the required directories (e.g., chroot), and allowlist the service's static egress IP at your network perimeter.
  </Accordion>

  <Accordion title="What file formats are supported?">
    `Parquet` (default/recommended), `CSV`, and `JSON`/`JSONL`.
  </Accordion>

  <Accordion title="How do I know when a transfer completed?">
    Each transfer writes a manifest JSON file per model under `_manifests/` at the root. Files follow the pattern: `_manifests/<model_name>/dt=<transfer_date>/manifest_{transfer_id}.json`. Use these manifests to trigger downstream processing.
  </Accordion>

  <Accordion title="Why do I sometimes see duplicates?">
    File-based destinations are append-oriented. The change-detection process uses a lookback window to prevent missed records, which can create duplicates across adjacent transfers. Downstream pipelines can deduplicate by primary key prioritizing rows in the most recent transfer window.
  </Accordion>

  <Accordion title="Can I provide my own public key? Where is the private key stored?">
    We do not support providing your own public key for security reasons. The private key is securely generated and stored in our system and is never shared externally.
  </Accordion>
</AccordionGroup>