> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prequel.co/llms.txt
> Use this file to discover all available pages before exploring further.
# Usage data
> Send your Prequel usage data to your own data warehouse
## Overview
You can receive your Prequel usage data in your own data warehouse, using any of our supported [destination vendors](/export/destinations/overview). Contact Prequel support to enable and configure this feature.
Once you configure your usage data destination, Prequel syncs three tables to your destination: `audit_log`, `destinations_snapshot`, and `model_transfers_log`. By default, this data is refreshed hourly.
## Data format
Prequel syncs three tables to your destination: `audit_log`, `destinations_snapshot`, and `model_transfers_log`.
### `audit_log`
Each row in this table corresponds to one event in the audit log, and is the same data available via the [`/logs` endpoint](/export/api-reference/monitoring-&-logging/list-logs). These logs are intended to be a comprehensive record of all write actions taken on your Prequel account. For more information, including the events included in this table, see the [audit log documentation](/export/monitoring/monitoring).
| Field Name | Data Type | Description |
| ------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `event_id` | `string` | Audit log ID. This ID is the same as the `/logs` endpoint. |
| `occurred_at` | `timestamp` | Timestamp when event occurred, corresponds to `created_at` in logs endpoint, reported in UTC. |
| `event_type` | `string` | The type of event. |
| `resource_id` | `string` | The ID of the resource involved in the event. |
| `actor_id` | `string` | The identifier of the actor who initialized the event. Some historical events have the value `unknown`. |
| `actor_type` | `string` | The type of actor who initialized the event. The possible values are `unknown`, `user`, `api_key`, `recipient`. |
| `current_resource` | `json` | Nullable JSON object representing the current state of the resource. DELETE operations will never have this field populated. Some historical events do not have this field populated. |
| `previous_resource` | `json` | Nullable JSON object representing the previous state of the resource. CREATE operations will never have this field populated. Some historical events do not have this field populated. |
### `destinations_snapshot`
Each entry in this table corresponds to one destination in your Prequel account. Note that this table represents the current state of destinations configured in Prequel. To get a historical record of actions taken on destinations, you can use the audit log data.
| Field Name | Data Type | Description |
| ------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------- |
| `destination_id` | `string` | Destination ID in the Prequel API. |
| `is_enabled` | `bool` | Whether scheduled syncs are enabled on this destination. |
| `updated_at` | `timestamp` | Timestamp when the destination was last updated. |
| `created_at` | `timestamp` | Timestamp corresponding to when the destination was first created. |
| `recipient_id` | `string` | The ID of the recipient the destination is assigned to. |
| `id_in_provider_system` | `string` | The ID in the source provider system, corresponds to the tenant schema in schema tenanting mode. |
| `products` | `json` | JSON array of objects representing assigned products. Each entry has the fields `id` and `name`. |
| `frequency_minutes` | `int` | The frequency at which Prequel's scheduler enqueues transfers to this destination. |
| `enabled_models` | `json` | JSON array representing the enabled models for this destination. Each entry has the fields `id` and `name`. |
| `has_all_models_enabled` | `bool` | Indicates whether the destination inherits all current and future models on its products. |
| `destination_name` | `string` | The name of the destination. |
| `destination_vendor` | `string` | The vendor of the destination. |
| `max_concurrent_transfers` | `int` | Maximum number of simultaneous transfers allowed. |
| `max_concurrent_queries_per_transfer` | `int` | Maximum number of concurrent queries allowed per transfer. |
| `deleted_at` | `timestamp` | Timestamp when the destination was deleted. Null for active destinations. |
### `model_transfers_log`
Each entry in this table corresponds to one completed model transfer within a transfer job. Model transfers that are pending or in progress are not currently included in this table.
| Field Name | Data Type | Description |
| -------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model_transfer_id` | `string` | Model ID concatenated with transfer ID. This ID only appears within Prequel's usage data. |
| `transfer_id` | `string` | The ID of the transfer in the API. |
| `model_id` | `string` | The ID of the model being transferred. |
| `model_transfer_id` | `string` | A unique ID for the model transfer, combining transfer ID and model ID. |
| `model_name` | `string` | The name of the model being transferred. |
| `source_id` | `string` | The ID of the source from which the model was read. |
| `source_name` | `string` | The name of the source from which the model was read. |
| `source_vendor` | `string` | The vendor of the source from which the model was read. |
| `destination_id` | `string` | The ID of the destination to which the model was written. |
| `recipient_id` | `string` | The ID of the recipient for the destination. |
| `id_in_provider_system` | `string` | The ID in the provider system, corresponds to the tenant schema. |
| `destination_name` | `string` | The name of the destination. |
| `destination_vendor` | `string` | The vendor type of the destination. |
| `is_full_refresh` | `bool` | Indicates whether the transfer was a full refresh. |
| `transfer_submitted_at` | `timestamp` | Timestamp when the transfer was submitted. |
| `transfer_started_at` | `timestamp` | Timestamp when the worker began executing the model transfer. |
| `transfer_ended_at` | `timestamp` | Timestamp when the model transfer completed. |
| `transfer_start_window_at` | `timestamp` | Lower bound timestamp for the data transferred. |
| `transfer_end_window_at` | `timestamp` | Upper bound timestamp for the data transferred. |
| `status` | `string` | The status of the model transfer (e.g., `SUCCESS`, `ERROR`, `CANCELLED`). |
| `enqueueing_actor_id` | `string` | The identifier of the actor who enqueued the transfer. |
| `enqueueing_actor_type` | `string` | The type of actor who enqueued the transfer, taking one of the following values: `user`, `api_key`, `service`, or `unknown`. |
| `error_trace` | `string` | Detailed error message in case of a transfer error. |
| `error_code` | `string` | Error code corresponding to the transfer error. |
| `error_blame` | `string` | The party responsible for the error. |
| `error_message` | `string` | Error message for the model transfer. |
| `documentation_url` | `string` | URL to documentation for further information on the error. |
| `transfer_log` | `json` | Transfer log information. |
| `billing_date` | `date` | The date on which Prequel will bill for this transfer. This is the date of the transfer `ended_at` at time zone America/Los\_Angeles. |
| `rows_transferred` | `bigint` | Total number of rows selected from the source for this model during this transfer. |
| `delta_rows_transferred` | `bigint` | Estimated number of rows in this transfer with a `last_modified_at` newer than the latest previously completed transfer for this model (does not include lookback overlap). |
| `volume_transferred_in_mb` | `float` | The memory footprint of the data transferred. |
**Warning**
Deleting a model can cause historical transfer records to lose their associated model and source details.
This issue is fixed for future transfers, but the time period of affected historical data varies by Prequel instance. Please contact support for more information.
Transfers for deleted destinations may not be available in `model_transfers_log` prior to a certain date, depending on your deployment. For more details about transfers for deleted destinations, please contact support.
## Staging usage data
Both production and staging environment data from Prequel is synced. Staging environment data is synced as separate tables to the same destination with a `_staging` suffix:
* `audit_log_staging`
* `destinations_snapshot_staging`
* `model_transfers_log_staging`
## Example queries
```sql title="Rows transferred per day" icon="database" expandable theme={null}
SELECT
DATE(transfer_ended_at) AS transfer_date,
SUM(rows_transferred) AS total_rows_transferred
FROM
`{{your-schema}}.model_transfers_log`
WHERE
destination_id = '{{your-destination-id}}'
AND transfer_ended_at >= CAST(CURRENT_DATE - INTERVAL '1 MONTH' AS TIMESTAMP)
GROUP BY
transfer_date
ORDER BY
transfer_date;
```
```sql title="Latest transfer status" icon="database" expandable theme={null}
WITH LatestStatus AS (
SELECT
model_id,
MAX(transfer_ended_at) AS last_transfer_time,
ARRAY_AGG(status ORDER BY transfer_ended_at DESC LIMIT 1)[OFFSET(0)] AS latest_status
MAX(CASE WHEN is_full_refresh THEN transfer_ended_at ELSE NULL END) AS last_full_refresh_time
FROM
`{{your-schema}}.model_transfers_log`
WHERE
destination_id = '{{your-destination-id}}'
GROUP BY
model_id
)
SELECT
l.model_id,
l.latest_status,
l.last_transfer_time,
l.last_full_refresh_time,
CASE
-- Have we ever triggered a transfer for this model?
WHEN l.latest_status IS NULL THEN false
ELSE true
END AS transfer_has_been_triggered
FROM
LatestStatus l
LEFT JOIN
`{{your-schema}}.destinations_snapshot` d
ON d.destination_id = '{{your-destination-id}}';
```
```sql title="Transfer metrics by billing date" icon="database" expandable theme={null}
SELECT
billing_date,
destination_id,
COUNT(*) AS count_transfers,
COUNTIF(status = 'SUCCESS') AS count_transfers_succeeded,
COUNTIF(status = 'ERROR') AS count_transfers_error,
SUM(COALESCE(rows_transferred, 0)) AS total_rows_transferred,
SUM(COALESCE(volume_transferred_in_mb, 0)) AS total_volume_transferred_mb,
SUM(COALESCE(delta_rows_transferred, 0)) AS total_deltas_transferred,
SUM(TIMESTAMP_DIFF(transfer_ended_at, transfer_started_at, SECOND)) AS total_transfer_time,
SUM(TIMESTAMP_DIFF(transfer_started_at, transfer_submitted_at, SECOND)) AS total_queue_wait_time,
SUM(TIMESTAMP_DIFF(transfer_ended_at, transfer_started_at, SECOND)) AS total_execution_time
FROM
`{{your-schema}}.model_transfers_log`
GROUP BY
billing_date, destination_id
ORDER BY
billing_date DESC;
```
```sql title="Destination update history" icon="database" expandable theme={null}
SELECT
DATE(occurred_at) AS update_date,
occurred_at AS update_timestamp,
actor_id,
actor_type,
resource_id AS destination_id,
previous_resource,
current_resource
FROM
`{{your-schema}}.audit_log`
WHERE
event_type = 'DESTINATION_UPDATE'
AND resource_id = '{{your-destination-id}}'
ORDER BY
occurred_at DESC;
```