Implementing a Prequel Destination for your batch-based receiving endpoint
A Batch-based API endpoint is the recommended implementation for most use cases. Contact the Prequel team if you have questions.
A webhook_batch destination delivers multiple records in a single HTTP request.For the API call to register a destination with Prequel, including a complete example, see Create Destination.
Prequel signs every delivery request with the X-Prequel-Webhook-Signature header so your endpoint can validate the request originated from Prequel. Fetch the public key for verification from GET /public/signatures/webhook-public-key. See Webhook headers for the full header list and verification steps.After verifying the request signature, you can optionally read provider_id and load_id from the envelope body to confirm the request belongs to an expected provider and load.
The destination object allows you to customize the contract between data shared by your customers and the shape Prequel provides to your receiving endpoint.Example: Suppose you are importing a users table with string fields id, email, and subscription.Below are four possible shapes for a webhook_batch destination. These are not exhaustive and features can be combined to confirm to your endpoint’s requirements:
Schema as body: Records land in the JSONL artifact in the shape declared by record_schema, with no additional shaping.
Custom body: Use a body template to customize each record. Only supported when format is json.
Routed by provider:uri and/or headers vary at delivery time using the ProviderID delivery-level identifier.
Destination per provider: A separate destination is registered for each provider, with a record_schema that includes provider-specific custom fields.
Set name (string, required) to a unique identifier for this destination within your account.
2
Set the destination type
Set type (string, required) to webhook_batch for batch deliveries.
3
Declare your record schema
Set record_schema (object, required) to a JSON Schema (draft-07) document that declares the fields each record contains. Prequel validates records against this schema before delivery.
4
Configure the request template
Set webhook_batch.request_template (object, required) with the HTTP method (string, defaults to POST), uri (string, required), and optional headers (object). The uri and headers templates support the .Prequel.* namespace.
5
Pick a delivery format
Set webhook_batch.format (string, required) to one of json, csv, or parquet. See Delivery format for what each option produces.
6
Tune optional throughput limits
Optionally set webhook_batch.max_size_per_batch (integer, default 24000), webhook_batch.max_batches_per_minute (integer, default 10), and webhook_batch.max_concurrency (integer, default 1) to control delivery throughput.
Set name (string, required) to a unique identifier for this destination within your account.
2
Set the destination type
Set type (string, required) to webhook_batch for batch deliveries.
3
Declare your record schema
Set record_schema (object, required) to a JSON Schema (draft-07) document that declares the fields each record contains. Prequel validates records against this schema before delivery.
4
Configure the request template with a body template
Set webhook_batch.request_template (object, required) with method, uri, headers, and body. The body template renders once per record while Prequel encodes the JSONL artifact, with access to record fields, the deletion flag, and delivery-level identifiers. The uri and headers templates render once per delivery and have access to delivery-level identifiers only. See Template variables for the full list of available variables. Body templates are only valid when format is json; selecting csv or parquet rejects the destination.
5
Pick a delivery format
Set webhook_batch.format (string, required) to one of json, csv, or parquet. See Delivery format for what each option produces.
6
Tune optional throughput limits
Optionally set webhook_batch.max_size_per_batch (integer, default 24000), webhook_batch.max_batches_per_minute (integer, default 10), and webhook_batch.max_concurrency (integer, default 1) to control delivery throughput.
Set name (string, required) to a unique identifier for this destination within your account.
2
Set the destination type
Set type (string, required) to webhook_batch for batch deliveries.
3
Declare your record schema
Set record_schema (object, required) to a JSON Schema (draft-07) document that declares the fields each record contains. Prequel validates records against this schema before delivery.
4
Route by provider in the request template
Set webhook_batch.request_template (object, required) with method, uri, and optional headers. Use {{.Prequel.ProviderID}} in uri and headers to route by the provider whose data is being delivered. These templates render once per delivery, before any records are selected. See Template variables for the full list of identifiers available.
5
Pick a delivery format
Set webhook_batch.format (string, required) to one of json, csv, or parquet. See Delivery format for what each option produces.
6
Tune optional throughput limits
Optionally set webhook_batch.max_size_per_batch (integer, default 24000), webhook_batch.max_batches_per_minute (integer, default 10), and webhook_batch.max_concurrency (integer, default 1) to control delivery throughput.
Create a separate destination for each provider whose requirements differ. Give each one a name (string, required) that identifies the provider it serves.
2
Set the destination type
Set type (string, required) to webhook_batch for batch deliveries.
3
Declare a provider-specific record schema
Set record_schema (object, required) to a JSON Schema (draft-07) document that declares the fields this provider sends, including any custom fields unique to them. Prequel validates records against this schema before delivery.
4
Configure the request template
Set webhook_batch.request_template (object, required) with method, uri, and optional headers. Because the destination is scoped to a single provider, you can hard-code provider-specific routing into uri and headers without needing template variables.
5
Pick a delivery format
Set webhook_batch.format (string, required) to one of json, csv, or parquet. See Delivery format for what each option produces.
6
Tune optional throughput limits
Optionally set webhook_batch.max_size_per_batch (integer, default 24000), webhook_batch.max_batches_per_minute (integer, default 10), and webhook_batch.max_concurrency (integer, default 1) to control delivery throughput.
Prequel makes one HTTP request per batch. The request body is an envelope with the following fields:
Field
Type
Description
url
string
Presigned URL to the batch file. The URL itself carries a 24-hour TTL set by object storage.
format
string
Format of the file at url. One of json, csv, or parquet.
record_count
integer
Number of records contained in the batch file.
batch_id
string
Unique identifier for this batch.
provider_id
string
Identifier for the provider whose data is being delivered. Omitted when unavailable.
load_id
string
Identifier for the active load job. Omitted when unavailable.
Your endpoint reads the envelope, downloads the file at url, and processes the records inside. The format field on the destination controls what kind of file the URL points to:
Format
File at url
json
JSONL. Each line is one record, optionally shaped by the body template.
Prequel includes the following headers on every delivery request.
Header
Description
X-Prequel-Webhook-Signature
SHA-256 RSA PKCS1 v1.5 signature of the payload. To verify, fetch the public key from GET /public/signatures/webhook-public-key and check the hex-decoded signature against the SHA-256 of <X-Prequel-Webhook-Timestamp>.<raw body> using PKCS1 v1.5.
X-Prequel-Webhook-Timestamp
RFC 3339 timestamp of when the request was sent. Reject stale timestamps to prevent replay attacks.
X-Prequel-Webhook-Digest
SHA-256 hash of the raw request body.
Content-Type
Defaults to application/json unless overridden by the headers template.
