Skip to main content

​
Understanding Prequel data types

Because data can be imported from many different source systems, and because data types are not necessarily synonymous across data stores, Prequel Import normalizes every source column into a common Prequel type. The Prequel type determines how each value is represented once it lands in your datalake and how it is delivered to your destination.

​
Source type mapping

Reference this table to see how each source column type is normalized into a Prequel type.
Timestamps without a time zone are handled as RFC 3339 stringsSource timestamp types that do not carry a time zone are mapped to the Prequel string type rather than timestamp. Assuming a time zone such as UTC would be incorrect and could mislead downstream consumers.This applies to:
  • BigQuery datetime
  • Postgres timestamp and timestamp without time zone
  • Redshift timestamp and timestamp without time zone
  • Snowflake datetime, timestamp, and timestamp_ntz
  • Object storage and SFTP timestamp
These values are preserved as RFC 3339 formatted timestamp strings (e.g., "2023-05-01T14:30:00").To convert these values into actual timestamps, use an expression in the Stream’s table mapping to parse the string and apply the appropriate time zone.

​
Datalake representation

Records are stored in your datalake, and delivered to your destination, as JSON. Each Prequel type has a stable JSON representation.
Decimals are represented as strings to preserve precisionValues of the decimal type are serialized as numeric JSON strings rather than JSON numbers.This prevents the loss of precision that would occur if high precision decimals were represented as floating point numbers.
Precision limitations for floating point and integer typesFloating point values are stored in IEEE 754 binary format, which cannot represent all decimal fractions exactly. To ensure accuracy, we strongly recommend using fixed precision types such as decimal instead.Integer values must fit within a signed 64-bit range. Values above 9223372036854775807 will overflow.