Skip to main content
Select your destination vendor below for connection setup instructions. For an overview of how destinations work in Prequel, see Destinations.

​
Supported destinations

You can find all currently supported destinations and documentation links below. You also may decide to self-host the destination configuration instructions on your own documentation site. If you prefer to do that, we maintain a copy of the source markdown files accessible at public locations below:
VendorTypeStatusDocsMarkdown (.md file)
snowflakeOLAPGAlinklink
bigqueryOLAPGAlinklink
redshiftOLAPGAlinklink
redshift_serverlessOLAPGAlinklink
databricksOLAPGAlinklink
athenaOLAPGAlinklink
clickhouseOLAPGAlinklink
motherduckOLAPBetalinklink
delta_lakeOTFGAlinklink
icebergOTFBetalinklink
postgresOLTPGAlinklink
aurora_postgresOLTPGAlinklink
mysqlOLTPGAlinklink
aurora_mysqlOLTPGAlinklink
mongodbOLTPGAlinklink
sql_serverOLTPGAlinklink
oracleOLTPBetalinklink
singlestoreOLTPBeta
s3Object StorageGAlinklink
s3_compatibleObject StorageGAlinklink
gcsObject StorageGAlinklink
absObject StorageGAlinklink
sftpObject StorageGAlinklink
google_sheetsSpreadsheetGAlinklink

​
Other available guides

GuideTypeStatusDocsDocs (.md file)
S3 Staging BucketStaging ResourceGAlinklink
Google Cloud Storage Staging BucketStaging ResourceGAlinklink
Azure Blob Storage Staging BucketStaging ResourceGAlinklink
AWS to GCP role assumptionAuthenticationGAlinklink
You should knowYou can use your discretion to decide what documentation to provide to your users that wish to connect their destination. To avoid confusion, we recommend working with your user to determine which database or data warehouse destination type they wish to connect, and then sending over the subset of documentation for them to work with.

​
Format of landed data

​
Data warehouses & databases (incl. Snowflake, BigQuery, Redshift, Databricks)

Data transferred to data warehouses and relational databases will be loaded as properly typed tables within a single schema. A _transfer_status artifact is created in the destination schema to record transfer metadata: specifically, a transfer_last_updated_at timestamp for each model. For most destinations this is a table. For BigQuery, it is a view backed by __TABLES__ metadata and a _transfer_status_cache table.

​
Object storage (incl. AWS S3, Google Cloud Storage, Azure blob storage)

Data transferred to object storage destinations will be loaded as Apache Parquet, CSV, or JSON files in Apache Hive style partitions. The default file format is Parquet, which we strongly recommend for performance reasons. This means data will appear in the following folder structure: <bucket_name>/<folder_name>/<model_name>/dt=<transfer_date>/<file_part>_<transfer_timestamp>.parquet Where:
  • <bucket_name> and <folder_name> are provided during destination configuration.
  • <model_name> is the name of the data model being transferred (this is equivalent to a table name in relational data destinations).
  • <transfer_date> and <transfer_timestamp> are generated at transfer time and based on the transfer’s start time. <transfer_date> is of the form 2006-01-01, while <transfer_timestamp> is of the form 20060102150405.
  • <file_part> is a monotonically increasing integer for a given timestamp, and does not carry any special meaning.
Prequel also provides a manifest file for each model within each transfer, mapping each transfer to its file contents. See Manifest files for object storage for details on the file structure.
What are Apache Hive style partitions and Apache Parquet file format?
  • Apache Hive style partitions are compatible with most popular query engines, and should make data easily queryable and transportable.
  • Apache Parquet file format is an open source, column-oriented data file format that offers efficient data compression and data integrity.
Note that individual files for object storage destinations will not exceed ~4 GB in size. Rows transferred to object storage destinations are treated as “append only”. This has the following effects for all object storage destinations:
  • Prequel will always write data based on the current data model, meaning Schema evolution and data model updates must be handled downstream of transfer.
  • Deduplication, merging updated rows will not take place during transfer, all updates will be written as unique rows. These updates must be also be made downstream of transfer.
  • Full refresh transfers will append the full dataset without deleting existing data.
However, object storage locations configured as a Delta Lake destination will receive the native schema evolution, deduplication, and ACID transaction behavior that all data warehouse destinations receive.

​
Spreadsheets (incl. Google Sheets)

Data transferred to spreadsheet destinations will be loaded as a newly created tab per data model. Where possible, the tabs will be created as protected tabs (or “read-only”) to prevent accidental modification.