What's a source?

"Sources" refers to the databases or data warehouses from which data will be transferred. Prequel supports many different sources: you can find the exhaustive list in our Add A Source API Endpoint.

Connecting your source

We require different permissions and set ups depending on what source you are using. For a step-by-step guide on how to configure and add your source to Prequel, select the source you're using from the left-hand menu under the Sources heading.

Multiple sources

By default, Prequel works with a single source. When set up in that way, all models pull data from the single available source.

However, there are cases where you might want to add multiple sources. For example, you may want to sync certain tables from your data warehouse, and a few other tables from a Postgres database running elsewhere. To facilitate those use-cases, Prequel can be run in multiple sources mode.

In a multi-source setup, models need to specify which source they intend to read data from. For example, model foo.prql might read from source A, and bar.prql might read from source B. This is done by adding a **source_name** field to each model, specifying which source the model should be pulling data from. The value for that field should be the name of the relevant source.

If multiple sources exist and a model doesn't specify which source it should read from, a transfer-time error will be thrown. As such, it's important to add the source_name field to your models before adding more than one source, in order to avoid disturbing existing transfers.

πŸ“˜

Checklist for switching to a multiple sources setup.

  • Add source_name field to existing models.
  • Add a new source through the API endpoint.
  • Add new models that point to the new source.