Sensitive Trace Webhooks

Overview

As a matter of company policy and compliance, Prequel never stores or retains any of the data it transfers.

Prequel occasionally encounters errors during data transfers, typically due to issues with the source or destination database systems. When these rare errors occur, the trace surfaced by the relevant database driver may occasionally include data included in the transfer. Note that this is entirely outside of Prequel's control.

To uphold its data privacy and security guarantees, Prequel handles errors as follows:

1. In the Prequel application and Prequel API endpoints, Prequel surfaces a coded error message which will never contain sensitive customer data. Raw traces are explicitly redacted from these messages to guarantee that no sensitive data is surfaced.
2. Prequel provides Sensitive Trace Webhooks to enable customers to receive unmodified errors in their own infrastructure standardized for a generic HTTP POST endpoint, providing additional context for debugging errors while keeping sensitive data out of standard logs.

⚠️

Important: These payloads may include sensitive information. Ensure your receiver is authenticated, access-controlled, and monitored according to your policies.

When are sensitive traces sent?

• Error details are sent per-transfer, aggregating errors across models in an array of traces. These are delivered as a single POST request per endpoint configured.
  • Continue reading for the format of sensitive trace payloads.
• If no sensitive traces occur during a transfer, no POST request is sent.
• Sensitive traces are also sent for source and destination connection tests. Sensitive traces are not forwarded for model validation errors.
• One POST request is made per-transfer to each sensitive trace endpoint configured.

Configuring the sensitive trace webhook

The webhook is managed through the sensitive_trace_endpoint resource in Prequel, which supports CRUD operations. See the API reference for complete request/response schemas, status codes, and pagination parameters. Continue reading below for detail on configuring these endpoints.

Template system

You can template the URL, headers, and body of the webhook using Go text/template syntax. The variables below are available:

Create sensitive trace webhook sample request

Request Body:

{
  "sensitive_trace_endpoint": {
    "name": "Production Error Webhook",
    "description": "Sends errors to our internal monitoring system",
    "url": "https://your-domain.com/webhooks/prequel-errors",
    "secret": "your-webhook-secret-token",
    "header_template": "Authorization: Bearer {{.Secret}}\nContent-Type: application/json",
    "body_template": "{\"environment\": \"{{.Environment}}\", \"timestamp\": \"{{.Timestamp}}\", \"errors\": {{.SensitiveTraces}}}"
  }
}

Response:

{
  "sensitive_trace_endpoint": {
    "id": "01234567-89ab-cdef-0123-456789abcdef",
    "name": "Production Error Webhook",
    "description": "Sends errors to our internal monitoring system",
    "url": "https://your-domain.com/webhooks/prequel-errors",
    "header_template": "Authorization: Bearer {{.Secret}}\nContent-Type: application/json",
    "body_template": "{\"environment\": \"{{.Environment}}\", \"timestamp\": \"{{.Timestamp}}\", \"errors\": {{.SensitiveTraces}}}",
    "created_at": "2024-01-15T10:30:00Z",
    "updated_at": "2024-01-15T10:30:00Z"
  }
}

Notes:

• The header_template is plain text with one header per line as Key: Value as shown in the example, not JSON.
• For authorization, deliveries are not signed by Prequel, unlike other Prequel webhooks. You must authenticate using your secret, as shown.

Default behavior

When no custom template is provided for either of the header_template or body_template, Prequel uses the following defaults:

Default Headers

Content-Type: application/json
Authorization: Bearer {Secret}

Default Body

{
  "environment": "prod",
  "sensitive_traces": [
    { "destination_id": "dest-123", "trace": "ERROR: ..." }
  ],
  "destination_id": "dest-123",
  "timestamp": "2024-01-15T10:30:00Z"
}

Sensitive Trace Payloads

Each item in SensitiveTraces is an object that can include:

{
  "destination_id": "dest-123",
  "source_id": "src-456",
  "model_id": "model-789",
  "model_name": "users_table",
  "source_vendor": "postgres",
  "destination_vendor": "snowflake",
  "execution_context": "write_destination",
  "trace": "ERROR: duplicate key value violates unique constraint ..."
}

For connection tests, execution_context will reflect the test (e.g., test_source_connection or test_destination_connection). trace is always present.

If you have any additional questions on configuring Sensitive Trace Webhooks, contact the Prequel team.

Related

• Webhooks Integration
• Logging & Monitoring