Skip to main content

Understanding Prequel data types

Because data from any source can be synced to any destination, and because data types are not necessarily synonymous across data stores, a common mapping must be used to specify the source data type and predict the destination data type. As part of Prequel configuration, the expected data type must be defined ahead of time.

Source type mapping

If you are configuring a source and need to decide what PREQUEL type to use, reference this table.
Scale and Precision limitations for decimal typesIn order to support broad compatibility across destinations, decimal typed values must fit within the DECIMAL(38,19) scale and precision. decimal values transferred to BigQuery will be truncated to DECIMAL(38,9), as this is the maximum supported size.
Precision limitations for floating point typesDue to technical constraints, Prequel is unable to guarantee the absence of a loss of precision for any floating point (eg float) type. For use-cases in which precision is key, such as when transferring financial data, we strongly recommend leveraging fixed precision types instead such as decimal.
Geography type is limited to World Geodetic SystemEvery geometric shape has a spatial reference system associated with it. The geography type is limited to geospatial data that corresponds to an SRID of 4326, which represents spatial data using longitude and latitude coordinates on the Earth’s surface as defined in the WGS84 standard.
Choosing between array, json, and vector
  • The vector Prequel type is used when syncing float-valued arrays (e.g., ML embeddings).
  • For all other array-like data, including text arrays, integer arrays, mixed types, or nested structures convert to json upstream and use the json Prequel data type. json has broader destination support and is the safer default for arbitrary arrays.
Timestamps are synced as epoch valuesAll timestamp values are stored as a UTC epoch (microseconds since Unix epoch), regardless of the source timezone. The source timezone has no bearing on what reaches the destination. How the destination displays that epoch depends on the destination system’s timezone configuration.Athena Iceberg limitations: For Iceberg tables, Athena has two timestamp restrictions:
  • Type: Athena only supports timestamp (without time zone) in DDL operations. If your Athena Iceberg source has columns typed as timestamp with time zone, cast them to timestamp in your source before syncing.
  • Precision: Athena Iceberg only supports millisecond precision (timestamp(3)). If your source has timestamp(6) columns, cast them to timestamp(3) in your source before syncing.
* An asterisk indicates partial support. Ask us about any specific data type limitations.

Destination type mapping

If you are interested in what data type your recipient destination will receive, reference this table. * An asterisk indicates partial or incomplete support. Ask us about any specific data type limitations.

Object storage Parquet types

Apache Parquet defines each column using a Parquet type (the physical storage format, such as INT64 or BYTE_ARRAY) and optional type annotations: convertedType (legacy annotation) and logicalType (modern, richer metadata like TimestampType or DecimalType). Prequel selects these encodings to maximize compatibility across analytic engines (Athena, BigQuery, Spark, Trino, etc.) so your exported files load predictably without requiring engine-specific casts or configuration.