Skip to main content
Prequel has intentional retry, redelivery, and circuit breaking logic designed to protect data integrity in your pipeline and minimize the operational burden of failed deliveries.

​
Retries

Prequel retries delivery to your destination for only 429, 503, and 504 responses that carry a Retry-After header, for a maximum of three attempts per delivery. Every other response, and every network failure, is attempted once. For which status code to return and the error code Prequel records for it, see the Record API or Batch API spec. The Retry-After value (in seconds) sets the wait before the next attempt, capped at 30 seconds.

​
Redelivery

Each load selects every record that has not been delivered yet, plus any record whose content changed since its last delivery. A record that failed to reach your endpoint therefore remains eligible and is delivered again on the next scheduled load, with no backfill required.
Because a record can be delivered more than once across attempts and loads, your endpoint must be idempotent.In your pipeline, deduplicate on the dataset’s primary_key_column. Every delivery also carries an X-Prequel-Idempotency-Key header, which stays the same across retry attempts of the same request.

​
Rejected records

A 5xx means your endpoint could not accept the delivery. A 4xx means something is wrong with the data being delivered, so Prequel treats a 400, 412, 413, or 422 as a rejection. Prequel builds the batch request itself, so on a batch delivery every error status indicates the records were not delivered rather than that any record was bad. For these records, what happens on the next load depends on the delivery type: A record counts as delivered once your endpoint receives it, whether or not your endpoint accepted it. The outcome of an individual record in a batch reaches Prequel only through acknowledgements.

​
Circuit breaking

Retries and redelivery assume a failure will clear on its own. When it will not, such as a broken mapping or an unreachable endpoint, circuit breakers stop the pipeline and report the cause to preserve resources on your Provider’s source and on your destination.
Subscribe a webhook to be alerted when an extract or load fails.

​
Source circuit breaker

Prequel halts an extract from the source, or a load to the destination, when 100 or more of its records fail, which indicates a mapping or schema mismatch at the source. The next scheduled extract or load runs as usual, and trips again if the same records still fail.

​
Destination circuit breaker

Prequel sets is_circuit_broken on the stream when a load fails with a destination-side error, such as an unreachable host, a 403 or 404, or a 5xx response. The failure ends the load even when other records in it were delivered successfully, and cancels the delivery work still queued behind it. A load sends one request per record for record delivery, or one per batch for batch delivery, split by max_size_per_batch. Every record in a batch shares that batch’s outcome, so a halt leaves a load’s records in three states: While the flag is set, extracts continue to run but produce no loads, and manual load triggers are rejected. To resume loads, set it back to false with Update import stream.