SFTP
Configuring your SFTP server to load data.
Prerequisites
- By default, SFTP uses keypair authentication for access. You will need a provided
public key
to configure your destination. It will look roughly like this:
ssh-key <ssh_public_key_beginning_with_AAAA> some-comment
Step 1: Create a user on the SFTP server
Login to the SFTP server and complete the steps below.
- Create group
sftpwriter
:
sudo groupadd sftpwriter
- Create user
sftpwriter
:
sudo useradd -m -g sftpwriter sftpwriter
- Switch to the
sftpwriter
user:
sudo su - sftpwriter
- Create the
.ssh
directory:
mkdir ~/.ssh
- Set permissions:
chmod 700 ~/.ssh
- Navigate to the
.ssh
directory:
cd ~/.ssh
- Create the
authorized_keys
file:
touch authorized_keys
- Set permissions:
chmod 600 authorized_keys
- Add the public key to the
authorized_keys
file. The key -- including the "ssh-key" and comment -- should be all on one line in the file, without linebreaks.
echo "ssh-key <ssh_public_key_beginning_with_AAAA> sftpwriter-public-key" > authorized_keys
Step 2: Add your destination
Share your host name, folder name, username, port and preferred delimiter character with us to complete the connection.
Frequently Asked Questions
- How will the data appear in my SFTP server?
- The data will be loaded as CSV files, in a predictable folder structure that can be easily parsed by downstream systems.
sftpwriter_home_folder/ ├─ some_provided_folder/ │ ├─ some_table_a/ │ │ ├─ dt=2024-01-01/ │ │ │ ├─ 0_20240101181004.csv │ │ │ ├─ 1_20240101184002.csv │ │ ├─ dt=2024-01-02/ │ │ │ ├─ 0_20240102180123.csv │ │ ├─ dt=2024-01-03/ │ │ │ ├─ 0_20240103182145.csv │ ├─ some_table_b/ │ │ ├─ dt=2024-01-01/ │ │ │ ├─ 0_20240101186004.csv │ │ ├─ dt=2024-01-02/ │ │ │ ├─ 0_20240102185123.csv │ │ ├─ dt=2024-01-03/ │ │ │ ├─ 0_20240103187145.csv
- The data will be loaded as CSV files, in a predictable folder structure that can be easily parsed by downstream systems.
Updated 2 months ago